Slack

At a glance

A community member is experiencing issues with the phoenix-arize instrument, where the grouping of spans is breaking when using the query engine. The community members have discussed various versions of the llama-index-core, llama-index-callbacks-arize-phoenix, and arize-phoenix libraries, but have not found a clear solution. One community member suggests that reverting to an older version of the libraries may not be the right approach, as there have been many updates since then. Another community member has provided a working example, but the original community member is still experiencing issues. The community members are also seeing some warnings related to the reranker output, but it's unclear if this is the root cause of the problem.

Useful resources

ccmosguy

Hey @Logan M There seems to be something breaking with the phoenix-arize instrument. The folks at at phoenix are telling me to downgrade to llama-inde<=10.19.0, that does not feel right. Can you see our conversation here? https://arize-ai.slack.com/archives/C04R3GXC8HK/p1713806586565339?thread_ts=1713530589.312499&cid=C04R3GXC8HK

Basically, the issue is when i start doing things like this:

Plain Text

set_global_handler("arize_phoenix")
query_engine = vector_index.as_query_engine(
    llm=llm,
    similarity_top_k=20,
    node_postprocessors=[reranker],
    refine_template=PromptTemplate(prompt_str),
    # text_qa_template=PromptTemplate(prompt_str),
)
output = query_engine.query("my query")

the grouping of the spans start breaking. Do you have any thoughts here?

Attachment

17 comments

LLogan M

Theres no issue that I'm aware of.

Do you have the latest version of core and the latest version of the arize libraries?

ccmosguy

I dunno

ccmosguy

@Logan M :

Plain Text

llama-index-core==0.10.30
llama-index-callbacks-arize-phoenix==0.1.5

ccmosguy

it totally breaks on the query engine, however, when I do a QueryPipeline it groups just fine

ccmosguy

i need this to work for query engine

ccmosguy

I feel like what arize folks are saying by reverting back to before 0.10.19 is a dumb idea

ccmosguy

there's been a ton of updates since then

LLogan M

Works fine for me in my testing

Plain Text

from llama_index.core import Document, VectorStoreIndex
from llama_index.postprocessor.colbert_rerank import ColbertRerank

documents = [
    Document(text="1"),
    Document(text="2"),
    Document(text="3"),
    Document(text="4"),
    Document(text="5"),
    Document(text="6"),
]

index = VectorStoreIndex.from_documents(documents)

query_engine=index.as_query_engine(similarity_top_k=6, node_postprocessors=[ColbertRerank(top_n=2)])

response = query_engine.query("3")

Plain Text

llama-index-core==0.10.30
llama-index-callbacks-arize-pheonix==0.1.5
arize-phoenix==3.23.0
openinference-instrumentation-llama-index==1.2.1

Attachment

ccmosguy

@Logan M thanks, just did test I am using:

Plain Text

llama-index-core==0.10.30
llama-index-callbacks-arize-phoenix==0.1.5
arize-phoenix==3.22.0
llama-index-callbacks-arize-phoenix==0.1.5

Plain Text

from dotenv import load_dotenv
from llama_index.core import Document, VectorStoreIndex, set_global_handler
from llama_index.postprocessor.colbert_rerank import ColbertRerank
from llama_index.llms.azure_openai import AzureOpenAI
from llama_index.embeddings.azure_openai import AzureOpenAIEmbedding
import os

load_dotenv("../src/.env")
set_global_handler("arize_phoenix")
embed_model_name = "text-embedding-ada-002"

embed_model = AzureOpenAIEmbedding(
    model=embed_model_name,
    deployment_name="EDAG-TEXT-EMBEDDING-ADA002-DEP1",
    api_key=os.environ["AZURE_OPENAI_KEY"],
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_version="2023-05-15",
)
llm = AzureOpenAI(
    model="gpt-35-turbo",
    deployment_name="EDAG-GPT35-TURBO-1106-Dep1",
    api_key=os.environ["AZURE_OPENAI_KEY"],
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_version="2023-12-01-preview",
    # callback_manager=callback_manager,
)

documents = [
    Document(text="1"),
    Document(text="2"),
    Document(text="3"),
    Document(text="4"),
    Document(text="5"),
    Document(text="6"),
]

index = VectorStoreIndex.from_documents(documents, embed_model=embed_model)

query_engine = index.as_query_engine(
    similarity_top_k=6,
    node_postprocessors=[ColbertRerank(top_n=2)],
    llm=llm,
    embed_model=embed_model,
)

response = query_engine.query("3")

and it still messes up

Attachment

ccmosguy

How did you get this to work then if you did not specify the set_global_handler?

LLogan M

Whoops, that was in another cell above. I did

Plain Text

import phoenix as px

px.launch_app()
import llama_index.core

llama_index.core.set_global_handler("arize_phoenix")

LLogan M

Are you sure that is the right trace in the screenshot? I don't see the query input (3) or output

ccmosguy

I had to refresh the webpage

ccmosguy

and now it is displaying fine

ccmosguy

very weird inconsistent results. btw, I am seeing this:

Plain Text

WARNING:opentelemetry.attributes:Invalid type float32 for attribute 'reranker.output_documents.0.document.score' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
WARNING:opentelemetry.attributes:Invalid type float32 for attribute 'reranker.output_documents.1.document.score' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
WARNING:opentelemetry.attributes:Invalid type float32 for attribute 'reranker.output_documents.2.document.score' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
WARNING:opentelemetry.attributes:Invalid type float32 for attribute 'reranker.output_documents.3.document.score' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types

ccmosguy

does this mean anything to you?

LLogan M

I haven't seen that, seems like the logged score wasn't the expected type from the reranker though

Add a reply

Find answers from the community

Slack