Find answers from the community

Updated 4 months ago

Hey all technical leaders I have a

At a glance
Hey, all technical leaders, I have a problem about using llama_index which is an excellent project.
I want to use an existing data and vector in Weaviate, and I use WeaviateReader("http://localhost:18090") to load data, and use GPTWeaviateIndex to Query.
My problem is that I don't know how to deal with the param of index_struct, is there a method to generate a content of index_struct from a existing data.


The follow is my code:

Plain Text
from llama_index import download_loader, GPTWeaviateIndex
WeaviateReader = download_loader('WeaviateReader')

# There is already data in Weaviate
reader = WeaviateReader("http://localhost:18090")
documents = reader.load_data(
    class_name="Question",
    properties=["answer", "category", "question", "_additional{vector}"],
    separate_documents=True
)

weaviate_index = GPTWeaviateIndex(weaviate_client=reader.client)
weaviate_response = weaviate_index.query("Which article is most relevant to biology?")
print(weaviate_response.response)

  • The following error occurred
ValueError: One of documents or index_struct must be provided.
j
v
M
7 comments
hi - if you're using WeaviateReader to load data, have you considered using other index structures, like GPTSimpleVectorIndex, to query your data?
Wow, this is the project author speaking.πŸ˜ƒ
No, I think WeaviateReader and GPTWeaviateIndex must be used together, I've got the wrong idea.
I will have a try using load data from WeaviateReader and using query of GPTSimpleVectorIndex.

But I have two problems. πŸ˜‚
Question A. These data are all loaded into memory. If the data volume is large, I'm afraid that the memory is not enough.
Question B. If I apply GPTSimpleVectorIndex, how do I control the number and similarity of those returned parameters.
In general, the readers just return Document objects, so they're not bound to the index that you will use to make queries. So if you read a small amount of documents from weaviate, you may make a GPTListIndex in memory and query against that
Hi, I use the param of similarity_top_k solve the Question B.
Attachments
image.png
image.png
Exactly, for vector indices like GPTSimpleVectorIndex.

For list indices, there is no such parameter, since all documents in the index will be used.
@jerryjliu0 @Mikko OK, all of you are very kind. Thanks very much. If there is a questions in the future, I will ask them again.😁
Add a reply
Sign up and join the conversation on Discord