Hey all technical leaders I have a

At a glance

Hey, all technical leaders, I have a problem about using llama_index which is an excellent project.
I want to use an existing data and vector in Weaviate, and I use WeaviateReader("http://localhost:18090") to load data, and use GPTWeaviateIndex to Query.
My problem is that I don't know how to deal with the param of index_struct, is there a method to generate a content of index_struct from a existing data.

The follow is my code:

Plain Text

from llama_index import download_loader, GPTWeaviateIndex
WeaviateReader = download_loader('WeaviateReader')

# There is already data in Weaviate
reader = WeaviateReader("http://localhost:18090")
documents = reader.load_data(
    class_name="Question",
    properties=["answer", "category", "question", "_additional{vector}"],
    separate_documents=True
)

weaviate_index = GPTWeaviateIndex(weaviate_client=reader.client)
weaviate_response = weaviate_index.query("Which article is most relevant to biology?")
print(weaviate_response.response)

The following error occurred

ValueError: One of documents or index_struct must be provided.

7 comments

jjerryjliu0

hi - if you're using WeaviateReader to load data, have you considered using other index structures, like GPTSimpleVectorIndex, to query your data?

vvegaviazhang

Wow, this is the project author speaking.😃
No, I think WeaviateReader and GPTWeaviateIndex must be used together, I've got the wrong idea.

vvegaviazhang

I will have a try using load data from WeaviateReader and using query of GPTSimpleVectorIndex.

But I have two problems. 😂
Question A. These data are all loaded into memory. If the data volume is large, I'm afraid that the memory is not enough.
Question B. If I apply GPTSimpleVectorIndex, how do I control the number and similarity of those returned parameters.

MMikko

In general, the readers just return Document objects, so they're not bound to the index that you will use to make queries. So if you read a small amount of documents from weaviate, you may make a GPTListIndex in memory and query against that

vvegaviazhang

Hi, I use the param of similarity_top_k solve the Question B.

Attachments

MMikko

Exactly, for vector indices like GPTSimpleVectorIndex.

For list indices, there is no such parameter, since all documents in the index will be used.

vvegaviazhang

@jerryjliu0 @Mikko OK, all of you are very kind. Thanks very much. If there is a questions in the future, I will ask them again.😁

Add a reply

Find answers from the community

Hey all technical leaders I have a