LlamaIndex

Log inLog into community

Find answers from the community

Updated 4 months ago

Hey guys super new to llama index open

Hey guys super new to llama index open

At a glance

·

Hey guys, super new to llama index + open ai's integration with it. I have gone through the tutorial and stored the vector data received from Open AI inside an index.json but it created a 5.8 GB file. I can't load this file everytime I need to query it so I wanna move it either a vector database modify my code to bring down the load times. Can someone point me to some sample code or tutorial on how to achieve it?

L

A

24 comments

The code will be dependent on the vector store you use (pinecone, qdrant, etc.)

But there are examples for the llama_index side for every option

https://github.com/jerryjliu/llama_index/tree/main/examples/vector_indices

Thank you. I'll check it out. One more question. How can I stop the query function from falling back to open ai to fetch a response in case it doesn't find it in my data?

Not sure what you mean here 🤔

In the sample code below

Plain Text

 response = index.query(question, response_mode="compact",)

If I ask a question which is not present in my data set, it fetches the answer from open ai. I'd like it to return back null if nothing matches with the questions asked in my dataset.

That's a little trickier. You might have to edit the internal prompts for that to work

Basically, llama index fetches the closest matching node(s) to the query, and then asks the model to use only the context retrieved to answer the query.

You can change the text_qa_prompt and refine_prompt to modify the instructions

https://gpt-index.readthedocs.io/en/latest/how_to/customization/custom_prompts.html

the default prompts are here: https://github.com/jerryjliu/llama_index/blob/main/gpt_index/prompts/default_prompts.py

and default chatgpt prompts here:
https://github.com/jerryjliu/llama_index/blob/main/gpt_index/prompts/chat_prompts.py

Ahhh gotcha. I think that's why when I updated the mode to no_text, it started giving out null because it wasn't sending the exact data to open ai for generating the response. Makes sense now.

Thank you.

I'll look into the files you have linked as well. 🙂

sounds good! 💪

Hey @Logan M thanks for all the help. I am following this guide https://github.com/jerryjliu/llama_index/blob/main/examples/vector_indices/PineconeIndexDemo.ipynb but getting stuck on one point. Was hoping you could help me out.

In this guide, the query is called like this

Plain Text

response = index.query("What did the author do growing up?")

I'd like to do so but I want to call this inside a different file. Which means I can't immediately pass the index I used to create the vectors since it has the documents as well.

Is there a way to do something like the following where I don't have to pass in the documents again?

Plain Text

        index = GPTSimpleVectorIndex.load_from_disk(self.get_index_file_path())

        response = index.query(question, response_mode="compact")

        return response.response

For 3rd party indexes, you can just use an empty array instead of actual documents when constructing the index.

Then, it will use the documents on pinecone

My main issue is since my index looks like the following when I am creating the vectors

Plain Text

        index = GPTPineconeIndex(
            documents,
            pinecone_index=pinecone_index,
            llm_predictor=llm_predictor,
            prompt_helper=prompt_helper,
            metadata_filters=metadata_filters
        )

and I have a lot of files I am creating vectors from, I don't want to do it again

it'll be really expensive and time consuming.

Was hoping to just query the index.

Right, so use [] instead of documents

Then, it will use the documents from your pinecone instance

Gotcha. Thanks again. Sorry for all the noob questions lol. I am really new in all of this and wanna get upto speed really quick.

Yea no worries! 💪

Add a reply

Sign up and join the conversation on Discord