Hi everyone 🙂 I m trying to understand

hey @LarryHudson ! so we have a WeaviateReader which is separate from GPTWeaviateIndex. The WeaviateReader allows you to load data from Weaviate, in the form of Document objects. You can then put those Document objects in any index that you want. Here's an example
https://github.com/jerryjliu/gpt_index/blob/main/examples/data_connectors/WeaviateDemo.ipynb


the GPTWeaviateIndex itself will take in documents, and we'll chunk the documents up into nodes, and store them into Weaviate. So we manage storage in weaviate under the hood there

hopefully the former solves your use case though

Thanks Jerry 🙂 that makes sense. I'm thinking about how I could integrate GPT Index with a CMS like Directus. I had previously integrated Weaviate with Directus, so I thought I would add GPT Index onto that, but it looks like I can set up a simple Flask app and connect directly to Directus.

If you were setting up something from scratch, would there be benefits of setting up Weaviate + GPT Index? Or would you only use the Weaviate data connector if you already have data stored in Weaviate? Sorry for the long question 🙂

if you use GPTWeaviateIndex, you have the advantage that the data is backed by a vector store service (vs. in-memory like our GPTSimpleVectorIndex), so theoretically it'd be much more scalable

Ah ok, that makes sense - thanks! By the way, I suggested in the Weaviate Slack that they should get you on the podcast - excited to hear it's happening 🙂