Slack

ccmosguy

Hey @Logan M There seems to be something breaking with the phoenix-arize instrument. The folks at at phoenix are telling me to downgrade to llama-inde<=10.19.0, that does not feel right. Can you see our conversation here? https://arize-ai.slack.com/archives/C04R3GXC8HK/p1713806586565339?thread_ts=1713530589.312499&cid=C04R3GXC8HK

Basically, the issue is when i start doing things like this:

Plain Text

set_global_handler("arize_phoenix")
query_engine = vector_index.as_query_engine(
    llm=llm,
    similarity_top_k=20,
    node_postprocessors=[reranker],
    refine_template=PromptTemplate(prompt_str),
    # text_qa_template=PromptTemplate(prompt_str),
)
output = query_engine.query("my query")

the grouping of the spans start breaking. Do you have any thoughts here?

Attachment

17 comments

LLogan M

Theres no issue that I'm aware of.

Do you have the latest version of core and the latest version of the arize libraries?

ccmosguy

I dunno

ccmosguy

@Logan M :

Plain Text

llama-index-core==0.10.30
llama-index-callbacks-arize-phoenix==0.1.5

ccmosguy

it totally breaks on the query engine, however, when I do a QueryPipeline it groups just fine

ccmosguy

i need this to work for query engine

ccmosguy

I feel like what arize folks are saying by reverting back to before 0.10.19 is a dumb idea

ccmosguy

there's been a ton of updates since then

LLogan M

Works fine for me in my testing

Plain Text

from llama_index.core import Document, VectorStoreIndex
from llama_index.postprocessor.colbert_rerank import ColbertRerank

documents = [
    Document(text="1"),
    Document(text="2"),
    Document(text="3"),
    Document(text="4"),
    Document(text="5"),
    Document(text="6"),
]

index = VectorStoreIndex.from_documents(documents)

query_engine=index.as_query_engine(similarity_top_k=6, node_postprocessors=[ColbertRerank(top_n=2)])

response = query_engine.query("3")

Plain Text

llama-index-core==0.10.30
llama-index-callbacks-arize-pheonix==0.1.5
arize-phoenix==3.23.0
openinference-instrumentation-llama-index==1.2.1

Attachment

ccmosguy

@Logan M thanks, just did test I am using:

Plain Text

llama-index-core==0.10.30
llama-index-callbacks-arize-phoenix==0.1.5
arize-phoenix==3.22.0
llama-index-callbacks-arize-phoenix==0.1.5

Plain Text

from dotenv import load_dotenv
from llama_index.core import Document, VectorStoreIndex, set_global_handler
from llama_index.postprocessor.colbert_rerank import ColbertRerank
from llama_index.llms.azure_openai import AzureOpenAI
from llama_index.embeddings.azure_openai import AzureOpenAIEmbedding
import os

load_dotenv("../src/.env")
set_global_handler("arize_phoenix")
embed_model_name = "text-embedding-ada-002"

embed_model = AzureOpenAIEmbedding(
    model=embed_model_name,
    deployment_name="EDAG-TEXT-EMBEDDING-ADA002-DEP1",
    api_key=os.environ["AZURE_OPENAI_KEY"],
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_version="2023-05-15",
)
llm = AzureOpenAI(
    model="gpt-35-turbo",
    deployment_name="EDAG-GPT35-TURBO-1106-Dep1",
    api_key=os.environ["AZURE_OPENAI_KEY"],
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_version="2023-12-01-preview",
    # callback_manager=callback_manager,
)

documents = [
    Document(text="1"),
    Document(text="2"),
    Document(text="3"),
    Document(text="4"),
    Document(text="5"),
    Document(text="6"),
]

index = VectorStoreIndex.from_documents(documents, embed_model=embed_model)

query_engine = index.as_query_engine(
    similarity_top_k=6,
    node_postprocessors=[ColbertRerank(top_n=2)],
    llm=llm,
    embed_model=embed_model,
)

response = query_engine.query("3")

and it still messes up

Attachment

ccmosguy

How did you get this to work then if you did not specify the set_global_handler?

LLogan M

Whoops, that was in another cell above. I did

Plain Text

import phoenix as px

px.launch_app()
import llama_index.core

llama_index.core.set_global_handler("arize_phoenix")

LLogan M

Are you sure that is the right trace in the screenshot? I don't see the query input (3) or output

ccmosguy

I had to refresh the webpage

ccmosguy

and now it is displaying fine

ccmosguy

very weird inconsistent results. btw, I am seeing this:

Plain Text

WARNING:opentelemetry.attributes:Invalid type float32 for attribute 'reranker.output_documents.0.document.score' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
WARNING:opentelemetry.attributes:Invalid type float32 for attribute 'reranker.output_documents.1.document.score' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
WARNING:opentelemetry.attributes:Invalid type float32 for attribute 'reranker.output_documents.2.document.score' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
WARNING:opentelemetry.attributes:Invalid type float32 for attribute 'reranker.output_documents.3.document.score' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types

ccmosguy

does this mean anything to you?

LLogan M

I haven't seen that, seems like the logged score wasn't the expected type from the reranker though

Add a reply

Find answers from the community

Slack