LlamaIndex
Log in

Find answers from the community

Home
Members
beaverTango
b
beaverTango
Offline, last seen 2 months ago
Joined September 25, 2024
bbeaverTango
·

Cache

I remember RedisCache being an alias for RedisKVStore so since the import here fails in the current version should I just import KV store and use it here:

Plain Text
import IngestionCache
from llama_index.core.ingestion.cache import RedisCache
1 comment
L
bbeaverTango
·

Raptor and BM25 metadata filter

Hey @Logan M @WhiteFang_Jr
8 comments
b
L
bbeaverTango
·

Query Fusion Retriever with metadata filtering fails

I have a redis vector index store and a doc store and an upsatash vector store defined. When I define my retriever such that it filters the nodes by metadata it seems to fail.

Steps to Reproduce


defining a base retriever with metadata filter enabled

Query Fusion Retriever with metadata filtering fails

Plain Text
base_retriever = self.base_index.as_retriever(
                    similarity_top_k=self.similarity_top_k,
                    filters=MetadataFilters(
                        filters=[
                            ExactMatchFilter(key="namespace", value=self.namespace)
                        ]
                    ),
                )
retriever = AutoMergingRetriever(
                    base_retriever, self.storage_context, verbose=verbose
                )
                
bm24_retriever = BM25Retriever.from_defaults(
                docstore=self.docstore, similarity_top_k=self.similarity_top_k
            )

fusion_retriever = QueryFusionRetriever(
                [retriever, bm24_retriever],
                similarity_top_k=self.similarity_top_k,
                num_queries=1,  # set this to 1 to disable query generation
                mode="reciprocal_rerank",
                use_async=True,
                verbose=verbose,
            )


https://github.com/run-llama/llama_index/issues/11391#issue-2153660643
10 comments
b
L
bbeaverTango
·

Common vector store issue

If I create a RAG pipeline with a redis docstore and a redis vector index and I intent to create multiple namespaces for each domain. Can they share the same vector store for indexing?
4 comments
L
b
bbeaverTango
·

Failing to retrieve documents once the

Failing to retrieve documents once the app is reloads storage context
Plain Text
redis_client = redis.Redis(
                    host=self.config.get("REDIS_HOST"),
                    
port=self.config.get("REDIS_PORT"),
                    password=self.config.get("REDIS_PASSWORD"),
                    ssl=True,
                )
docstore = RedisDocumentStore.from_redis_client(
                    redis_client=redis_client,
                    namespace=namespace
                )
storage_context = StorageContext.from_defaults(
                    docstore=self.docstore,
                    index_store=RedisIndexStore.from_redis_client(
                        redis_client=redis_client,
                        namespace=namespace
                    ),
                )

after this I reload the index from the storage context like below:
base_index = load_index_from_storage(storage_context)

when setting this up and loading the index like after I close my app:
Plain Text
try:
 indices = load_indices_from_storage(self.storage_context)
                   for index in indices:
                        print(index.index_id)
                    base_index = load_index_from_storage(storage_context)
                    print("[INFO] Index found at storage")
                except ValueError as e:
                    print("[INFO] No index found at storage")
                    base_index = VectorStoreIndex(
                        nodes=[],               storage_context=storage_context)

and the retrieval is defined as:

Plain Text
base_retriever = base_index.as_retriever(
                similarity_top_k=self.similarity_top_k
            )
            retriever = AutoMergingRetriever(
                base_retriever, self.storage_context, verbose=True
            )
              query_bundle = QueryBundle(query_str=query)
            retrived_nodes = retriever.retrieve(query_bundle)

this outputs empty retrieved nodes for some reason.
49 comments
b
L
bbeaverTango
·

I wanted to initialzie an empty

I wanted to initialzie an empty VectorStoreIndex by providing just a storage_context.

Plain Text
 self.base_index = VectorStoreIndex(
                    nodes=None, storage_context=self.storage_context
                )


But I'm getting an error:

Plain Text
An error occurred: One of nodes, objects, or index_struct must be provided.


Considering I would be inserting the nodes later on what would be the correct way of initialisation and reloading of indices?
20 comments
b
L
bbeaverTango
·

Redis doc store and index store imports

what are the new imports for ?

Plain Text
from llama_index.storage.docstore import RedisDocumentStore
from llama_index.storage.index_store import RedisIndexStore
6 comments
b
W
bbeaverTango
·

How do implement something like Auto

How do implement something like Auto Merging Retriever but using two Supabase VectorStoreIndex .

I have defined two separate vector store indices one for nodes and the other for leaf nodes. How do I retrieve from these nodes such that the output is something like Auto Merging Retriever?
14 comments
b
L