kush2861

entity_extractor = EntityExtractor(prediction_threshold=0.2,label_entities=False, device="cpu")

        node_parser = SentenceSplitter(chunk_overlap=200,chunk_size=2000)

        transformations = [node_parser, entity_extractor]

        documents = SimpleDirectoryReader(input_dir=r"Text_Files").load_data()

        pipeline = IngestionPipeline(transformations=transformations)

        nodes = pipeline.run(documents=documents)

        service_context = ServiceContext.from_defaults(llm=OpenAI(model="gpt-3.5-turbo", temperature=0),embed_model=embed_model)

        index = VectorStoreIndex(nodes, service_context=service_context)

Can I speed up this entity extraction process? It's very slow. Takes about an hour or so for 300 files.

3 comments

kkush2861

When I pass an embedding model to the

When I pass an embedding model to the service context and that service context to query engine, I am just changing embeddings of the query, right? I am loading index form storage.

6 comments

kkush2861

Is there any way to pass retrieved nodes

Is there any way to pass retrieved nodes to the query engine instead of the retriever. Asking this because I want to process the retrieved nodes before querying.

2 comments

kkush2861

I am using two different approaches in

I am using two different approaches in advanced RAG. I want to switch to option 2 in case 1 fails. I am depending on the response string and using an if else construct to switch between the approaches. For example if "I'm sorry but..." in response.response: use_approach_2. Is there any other way to do this?

9 comments

kkush2861

Nodes

I am using a query engine. I want to query in such a manner that once a query is answered, the nodes used to answer that query should be dropped from the index before answering the next query and so on. Is it possible?

10 comments

kkush2861

I am using entity extractor for my data

I am using entity extractor for my data. I created the vector index and persisted locally. now while querying I am not loading the index from persists directory and not creating the nodes again. Can I find out what entities are retrieved for a particular query while doing so?

3 comments

kkush2861

is there distances_from_embeddings

is there distances_from_embeddings calculator in llamaindex like openai. the openai one is removed.

7 comments

kkush2861

how can I use

how can I use CondensePlusContextChatEngine with vision models? I want to pass images as well.

6 comments

kkush2861

Which gpt 4 version does the default

Which gpt 4 version does the default llama index setting point to?

1 comment

kkush2861

Is workflows analogous to lang graph in

Is workflows analogous to lang graph in Langchain?

2 comments

kkush2861

Splade

Is there a SPLADE implementation in Llama Index?

3 comments

kkush2861

Embeddings

Why is it generating embeddings so many times?

1 comment

kkush2861

Is there any reason for chunk sizes to

Is there any reason for chunk sizes to be powers of 2?

7 comments

kkush2861

Anyone getting this error while using

Anyone getting this error while using BM25 Retriever?

Plain Text

ValidationError: 1 validation error for NodeWithScore node Can't instantiate abstract class BaseNode with abstract methods get_content, get_metadata_str, get_type, hash, set_content (type=type_error)

18 comments