Is 16k available today?

Are there any examples of how to use the new update functionality? I really want to figure out how to have my script update/add any files that were modified since the last load (and no others).

@msh702 @Byron Voorbach {Weaviate} I'm thinking of switching to Weaviate. What would be some pros and cons for my use case over something like Pinecone? Feel free to DM.

Is anyone here using pinecone? I'm a little stuck. I've been using the local json option and am trying to shift to using pinecone for a few reasons. So I have two versions of a script working and am trying to merge them. One using gpt_index with SimpleDirectoryReader and SimpleVectorIndex and the other manually upserting into Pinecone. I looked at the GPT_Index Pinecone example and don't understand how to setup the id_to_text_map 🤔

Looking back at my records it looks like it was about $50 for one full run through of the directory. 😬

Does anybody have ideas for extracting more math and date based information via AI queries?

Say I have a file for each day (2023-01-02, 2023-01-01, etc) and each one has various metadata (Weight:: 145, Readiness Score:: 83, etc). Right now i can get good results for a query like What was my weight on Jan 1 2023? but can't yet do things like What was my average weight for the last week?. Could queries be passed through some sort of natural language processor (wrong term?) for dates and something like a wolfram alpha for math? or? What about awareness of what today is (other than prepending today is {{today's date}} in the prompt, or maybe that's good enough?).

Hello all! Thanks to GPT_Index I've managed to put together a script that queries my extensive personal note collection which is a local directory of about 20k markdown files. Some of which are very long. I work in this folder all day everyday, so there are frequent changes. Currently I would need to rerun the entire indexing (is that the correct term?) when I want to incorporate edits I've made.

So my question is... is there a way to schedule indexing to maybe once per day and only add information for files that have changed? Or even just manually run it but still only add edits? This would make a huge difference in saving time (I have to leave it running overnight for the entire directory) as well as cost 😬.

Excuse me if this is a dumb question, I'm not a programmer and am sort of muddling around figuring this out 🤓

Thank you for making this sort of project accessible to someone like me!