Hey guys. I have an issue reading the

At a glance

The community member is having an issue reading the storageContext of a KeywordTableIndex. They created a test case with a file containing a document, but when trying to get the storage, they receive an error about a missing vector_store.json file. The community member also wants to know how to query the value of the "test" ID without using the queryEngine.

The comments suggest that the error message is not an actual error, but just indicates that a new (empty) file is being created. The community member then focuses on the second part of the problem - how to successfully query the keyword storage without using the queryEngine.

The community member tries to create the "index" based on the storageContext, create a "retriever", and find the syntax to retrieve the "test" ID. However, they encounter an error about an unknown index struct type "keyword_table". The comments suggest this is likely a bug in the LlamaIndexTS library.

In the end, the community member decides to use a simple database for the keyword storage/retrieval actions and the LlamaIndex library for the vector part.

Useful resources

AAI made approachable

Hey guys. I have an issue reading the storageContext of a KeywordTableIndex:

As a test case I created ./kwstorage which just stored a file with the following content:

Plain Text

const document = new Document({text: "Hello World", id_: "test"})

If I want to get that storage the same way I get the vector storage

Plain Text

    const storageContext = await storageContextFromDefaults({
        persistDir: "./kwstorage",
    });

I get this error:

Plain Text

No valid data found at path: ./kwstorage/vector_store.json starting new store.

And of course this file is missing, as it is no vector storage.

Also how do I query the value of the id "test" later? In the example the queryEngine is being used. I like to retrieve the raw value

13 comments

LLogan M

I don't think that's an error, it's just saying it's creating an (empty) file right?

AAI made approachable

Ah okay, so I can ignore that. Thank you!
Then I only have my 2nd part of the problem left: How do I succesfully query it without using queryEngine

I found out how to do it for the vector storage, but cannot find the solution for the keyword storage

Plain Text

    const retriever = new VectorIndexRetriever({ index: index, similarityTopK: 3 });
    const results = await retriever.retrieve("AirBnB");

AAI made approachable

So basically what I have issues with:

creating the "index" (basedon the storageContext)
creating the "retriever"
finding out the syntax for the retriever to get id "test"

AAI made approachable

Tried out something like this ...

Plain Text

    const storageContext = await storageContextFromDefaults({
        persistDir: "./kwstorage",
    });
    const index = await KeywordTableIndex.init({
        storageContext: storageContext
    });
    const retriever = index.asRetriever({ mode: KeywordTableRetrieverMode.DEFAULT });
    const result = await retriever.retrieve("test");

LLogan M

hmm the keyword index is pretty new to me. I don't actually maintain the TS package (just the python), but let me see if I can figure it out from the source code lol

LLogan M

So the usage you gave above seems correct

AAI made approachable

Currently it fails with:
Uncaught Error Error: Unknown index struct type: keyword_table

AAI made approachable

Fails here

Plain Text

    const index = await KeywordTableIndex.init({
        storageContext: storageContext
    });

LLogan M

Seems like a bug actually

LLogan M

Right here
https://github.com/run-llama/LlamaIndexTS/blob/b64716d3f7e41e0b8db7421b7150cafd13026843/packages/core/src/indices/BaseIndex.ts#L74

LLogan M

It doesn't check for the keyword_table index struct type

AAI made approachable

Oh ok. Surprised my attempts went into the right direction 😉 (not a real developer)
Still thanks a lot! Guess I will just use a simple database for the keyword storage/retrieval actions and llamaindex for the vector part

LLogan M

Haha you were doing everything correctly!

Yea that sounds good, I will try to make sure this gets fixed by one of our JS devs in the meantime 🙂

Add a reply

Find answers from the community

Hey guys. I have an issue reading the