Find answers from the community

s
F
Y
a
P
Home
Members
alfredmadere
a
alfredmadere
Offline, last seen last month
Joined September 25, 2024
pipeline is throwing pydantic validation errors when passing in a valid vector store like this:

Plain Text
pg_vector_store = PGVectorStore.from_params(
                    **POSTGRES_SETTINGS.model_dump(exclude_none=True),
                    table_name="embeddings",
                    embed_dim=384,
                )
  pipeline = IngestionPipeline(
    transformations=[
      HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5"),
    ],
    docstore=postgres_docstore,
    vector_store=pg_vector_store,
  )


PGVectorStore inherits from BasePydanticVectorStore and implements all the abstract methods, but i am getting this error:

Plain Text
validation_error = ValidationError(model='IngestionPipeline', errors=[{'loc': ('vector_store',), 'msg': "Can't instantiate abstract class...VectorStore without an implementation for abstract methods 'add', 'client', 'delete', 'query'", 'type': 'type_error'}])

    def __init__(__pydantic_self__, **data: Any) -> None:
        """
        Create a new model by parsing and validating input data from keyword arguments.
    
        Raises ValidationError if the input data cannot be parsed to form a valid model.
        """
        # Uses something other than `self` the first arg to allow "self" as a settable attribute
        values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
        if validation_error:
>           raise validation_error
E           pydantic.v1.error_wrappers.ValidationError: 1 validation error for IngestionPipeline
E           vector_store
E             Can't instantiate abstract class BasePydanticVectorStore without an implementation for abstract methods 'add', 'client', 'delete', 'query' (type=type_error)
28 comments
L
a
Is it possible to specify a node parser at the time of insertion into an index? If not, what is the recommended way of handling the insertion of multiple different document types that require different node parsers?

I have several different node parsers that are optimized for chunking different types of documents and I need to be able to hot-swap them out depending on which type of document I am inserting without rebuilding the index from scratch every time. I am assuming that creating an index is not cheap. What is the recommended way of doing this?
4 comments
L
a
What could be causing trace_map in end_trace to be None? In attempting to add arize-phoenix to privateGPT I'm noticing significantly different behavior than the toy project i created to test phoenix. For some reason the traces never get emmited in privateGPT and it seems to be because OpenInferenceTraceCallbackHandler.end_trace() is not called with trace_map.

I'm struggling to find where end_trace is called from as well
22 comments
L
a
a
alfredmadere
·

Prompt

I'm encountering an issue where my retrieved context doesn't seem to be being sent to the llm correctly and in order to debug I need to inquire about the entire prompt that was sent to the llm (after the chats are transformed and the context is inserted). I basically want a way to see all of the text that gets sent to the llm when i call stream_chat like this:

Plain Text
query_stream = chat_service.stream_chat(
                    messages=all_messages,
                    use_context=True,
                )


The response of chat_service.stream_chat() is of type CompletionGen which only contains a list of the sources. I'd like to keep around a copy of the whole prompt that is sent to the llm for each invocation of stream_chat for debugging purposes.

Does anyone know how this might be done in llama-index without serious modifications to the framework code?

Also if theres some way to know for sure how the nodes + messages got composed into the prompt, that would also be sufficient.
2 comments
s
L
It seems that most google collab examples are broken. Here is the error that's thrown when trying to run the Hugging Face llm example. I get the same error trying to run any of the llm examples. Has anyone else run into this?

https://docs.llamaindex.ai/en/stable/examples/customization/llms/SimpleIndexDemo-Huggingface_stablelm.html

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
llmx 0.0.15a0 requires cohere, which is not installed.

Edit: the working collab examples also throw this pip error.
9 comments
L
a
Pydantic validation error when attempting to subclass ChatResponse from llama_index.core.llms and extend it with a custom pydantic model.

The error is thrown when I add a list of MyModels to ChatResponse via subclassing like so

Plain Text
from pydantic import BaseModel
from llama_index.core.llms import ChatResponse
class MyModel(BaseModel):
  foo: str

class CompletionWithAttributions(ChatResponse):
  attributions: list[MyModel] | None = None


The error is not thrown if I don't subclass from ChatResponse like so
Plain Text
from pydantic import BaseModel
class MyModel(BaseModel):
  foo: str

class CompletionWithAttributions(BaseModel):
  attributions: list[MyModel] | None = None


I can resolve this error by importing BaseModel from llama-index itself like this, but that seems VERY clugy as elsewhere in my app i am importing BaseModel from pydantic as per usual.

Plain Text
from llama_index.core.bridge.pydantic import BaseModel



Anyone experienced anything like this with llama-index? I suspect its due to the handling of pydantic versions within the library.

I am using
llama-index==0.10.51
pydantic==2.6.4
3 comments
L
a
alfredmadere
·

Docs

Given an existing VectorIndex and DocStore (both in postgres). How can I get all of the nodes extracted from a document using llama-index abstractions?

I'm able to get a document like this and confirm in the

Plain Text
   doc = docstore.get_document(llama_id)
    assert doc != None
    doc_content = doc.get_content()
    assert doc_content != None



but when i do this:

Plain Text
docstore_ref_doc_info = docstore.get_all_ref_doc_info()


docstore_ref_doc_info is an empty object

Also i get this error when i do index.ref_doc_info
Plain Text
NotImplementedError: Vector store integrations that store text in the vector store are not supported by ref_doc_info yet.


is there a workaround for this?
21 comments
a
L
Has anyone figured out how to get intelisesnse to index the llama_index package? It seems to be able to import stuff from llama_cloud like TextNode, but it has absolutely no idea what is exported from the llama_index package. Thanks so much for the help, I've spent a rediculous 3 hours on this issue today 🤦‍♂️
24 comments
L
J
a
b
W
Bug report:
There is no way to get token usage from openAI when using stream_chat.
Here's a full discussion of the issue https://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156/17.
33 comments
a
L
Is pydantic v2 incompatible with llama-index? Multiple models such as MetadataFilters inherit from the v1 BaseModel (which cannot be used to compose a v2 model) https://github.com/pydantic/pydantic/issues/7997

E.G

This line exists in the schema file

Plain Text
from llama_index.core.bridge.pydantic import (
    BaseModel
)


This v1 BaseModel is used in the creation of MetadataFilters exported from llama_index.core.vector_stores

Other models that i create use the BaseModel from pydantic v2 and are composed with the existing v1 llama-index models. This throws errors. When creating our own pydantic models should we always use the BaseModel exported from pydantic.v1?

I am trying to create my own pydantic model like this

Plain Text
class ChatBody(BaseModel):
    messages: list[OpenAIMessage]
    use_context: bool = False
    conversation_id: str | None = None
    context_filter: MetadataFilters | None = None
    include_sources: bool = True
    stream: bool = False

Where the BaseModel that ChatBody inherits from is important from pydantic (v2) but MetadataFilters inherits from the BaseModel that is exported from llama_index.core.bridge.pydantic which is a v1 model
2 comments
L
I'm struggling to understand how LangChainLLM() is fundamentally different than other llms like LlamaCPP() or SagemakerLLM(), specifically in that you cannot set messages_to_prompt or completion_to_prompt on LangChainLLM but you can on the others.

After looking through the source code it looks like LangChainLLM is the only one that extends LLM instead of CustomLLM.

My use case is the following:

I'm attempting to use an arbitrary TGI Hugging Face model to do streaming chat as shown below

Plain Text
hf = HuggingFaceTextGenInference(
                    inference_server_url="https://api-inference.huggingface.co/models/HuggingFaceH4/zephyr-7b-beta",
                    max_new_tokens=512,
                    top_k=10,
                    top_p=0.95,
                    typical_p=0.95,
                    temperature=0.01,
                    repetition_penalty=1.03,
                )
                prompt_style = get_prompt_style(settings.huggingface.prompt_style)
                self.llm = LangChainLLM(llm=hf)


Because I'm using an arbitrary model, I need to be able to change the messages_to_prompt function. Is there a way to do this?
51 comments
L
a