Find answers from the community

Updated 2 years ago

Deploying

I've got a nice chatbot/server working locally but I've been struggling to deploy it for a few days. It seems like the main issue is my index.json file is very large -- 175MB -- and so it's not compatible with the serverless/lambda pattern generally used by most of the platforms I'd like to deploy on. Does anyone have experience with this or advice on how to deploy apps that use llama_index?
L
b
15 comments
If you have a large index, I'm assuming you've created a vector index with a ton of documents?

To reduce sizes, a good option is to integrate with a vector store/database that will store all the stuff for you (and is also much more memory efficient)

Check out pinecone, qdrant, etc. Llama index integrates with a bunch
Thanks Logan, giving that a try now, will report back with my progress.
Okay, gave it a shot. I have sucessfully read the documents and initalized the weaviate client. However, the

Plain Text
index = GPTWeaviateIndex.from_documents(documents, weaviate_client=client)


line doesn't seem to be working for me.

I would assume this line chunks, embeds, and vectorizes my documents before uploading them to Weaviate.

I believe it is successfully chunking, embedding, and vectorizing them, but I don't believe it's uploading them to Weaviate. The line executes successfully but puts out the following log messages.

Plain Text
sys:1: ResourceWarning: Unclosed socket <zmq.Socket(zmq.PUSH) at 0x12e4398e0>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total LLM token usage: 0 tokens
INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 6929313 tokens
{'error': [{'message': "invalid string property 'relationships' on class 'Gpt_Index_1935153078200784925_Node': not a string, but map[string]interface {}"}]}
{'error': [{'message': "invalid string property 'relationships' on class 'Gpt_Index_1935153078200784925_Node': not a string, but map[string]interface {}"}]}
...
Any idea what might be going on?
Hmmm one step closer!

Also that was quite a bit of tokens... let's switch to a small set of documents to debug if needed lol

What llama_index version are you running on? Seems like a small bug that didn't convert the node relationship attribute to a string πŸ€”
I'll check the source code on this one..
Hmm, if you do pip install --upgrade llama_index, the latest code I can see in the repo should be doing the correct thing and converting to string
Okay, point well taken that I should reduce the number of documents while testing. I'm on 0.5.6
okay, i'll upgrade and try again
What version is the most recent?
Released 3 minutes ago lol
Heck yes, that seems to have worked. Thank you!!!
Nice!! πŸ™πŸ™
Add a reply
Sign up and join the conversation on Discord