Find answers from the community

Updated 2 months ago

With the migration from `ServiceContext`

With the migration from ServiceContext to Settings the ContextRelevancyEvaluator in llama_index.core.evaluation.context_relevancy imports ServiceContext in version 10 still. When would this be updated?
L
S
24 comments
Probably just got missed in the transition. There's a few places tbh that this got mixed. I have time now to fix this
Ah actually, it import service context, but does not use it (it entirely optional)
this pattern is in a lot of the codebase
to maintain backwards compatibility
but its pulling from settings

self._llm = llm or llm_from_settings_or_context(Settings, service_context)
Whats the issue with that pattern?
Thanks for the response, yeah I saw that it was optional too and I didn't plan on using it either.
I am following the setup of evaluators as per https://docs.llamaindex.ai/en/latest/examples/evaluation/batch_eval.html#batchevalrunner-running-multiple-evaluations guide
Plain Text
from llama_index.core.evaluation import (
    FaithfulnessEvaluator,
    RelevancyEvaluator,
    CorrectnessEvaluator,
)

I am stuck at this import. How do I import these evaluators otherwise? Change the base code in llama_index.core.evaluation.context_relevancy? How do I do this?
What do you mean change them? What's the error?
Seems to be working now. Though I do get another error message:
Plain Text
ModuleNotFoundError                       Traceback (most recent call last)
Input In [5], in <cell line: 1>()
----> 1 from llama_index.llms.openai import OpenAI
      2 from llama_index.core.evaluation import (
      3     FaithfulnessEvaluator,
      4     RelevancyEvaluator,
      5     CorrectnessEvaluator,
      6 )
      8 from llama_index.core import Settings

File ~\anaconda3\lib\site-packages\llama_index\llms\openai\__init__.py:1, in <module>
----> 1 from llama_index.llms.openai.base import AsyncOpenAI, OpenAI, SyncOpenAI, Tokenizer
      3 __all__ = ["OpenAI", "Tokenizer", "SyncOpenAI", "AsyncOpenAI"]

File ~\anaconda3\lib\site-packages\llama_index\llms\openai\base.py:48, in <module>
     46 from llama_index.core.llms.llm import LLM
     47 from llama_index.core.types import BaseOutputParser, PydanticProgramMode
---> 48 from llama_index.llms.openai.utils import (
     49     create_retry_decorator,
     50     from_openai_message,
     51     from_openai_token_logprobs,
     52     from_openai_completion_logprobs,
     53     is_chat_model,
     54     is_function_calling_model,
     55     openai_modelname_to_contextsize,
     56     resolve_openai_credentials,
     57     to_openai_message_dicts,
     58 )
     60 from openai import AsyncOpenAI, AzureOpenAI
     61 from openai import OpenAI as SyncOpenAI

File ~\anaconda3\lib\site-packages\llama_index\llms\openai\utils.py:24, in <module>
     22 from openai.types.chat.chat_completion_chunk import ChoiceDeltaToolCall
     23 from openai.types.chat.chat_completion_message import ChatCompletionMessage
---> 24 from openai.types.chat.chat_completion_token_logprob import ChatCompletionTokenLogprob
     25 from openai.types.completion_choice import Logprobs
     26 from openai.types.completion import Completion

ModuleNotFoundError: No module named 'openai.types.chat.chat_completion_token_logprob'

Should I just revert to an earlier release?
update your openai client I think
pip install -U openai
I'm now getting an error ModuleNotFoundError: No module named 'llama_index.core.image_retriever'
what code triggers that?
Plain Text
ModuleNotFoundError                       Traceback (most recent call last)
Input In [2], in <cell line: 2>()
      1 from llama_index.llms.openai import OpenAI
----> 2 from llama_index.core.evaluation import (
      3     FaithfulnessEvaluator,
      4     RelevancyEvaluator,
      5     CorrectnessEvaluator,
      6 )
      8 from llama_index.core import Settings
     10 from llama_index import (
     11     Response,
     12     KnowledgeGraphIndex,
   (...)
     16     load_index_from_storage    
     17 )

File ~\anaconda3\lib\site-packages\llama_index\core\evaluation\__init__.py:3, in <module>
      1 """Evaluation modules."""
----> 3 from llama_index.core.evaluation.answer_relevancy import AnswerRelevancyEvaluator
      4 from llama_index.core.evaluation.base import (
      5     BaseEvaluator,
      6     EvaluationResult,
      7 )
      8 from llama_index.core.evaluation.batch_runner import BatchEvalRunner

File ~\anaconda3\lib\site-packages\llama_index\core\evaluation\answer_relevancy.py:8, in <module>
      5 import re
      6 from typing import Any, Callable, Optional, Sequence, Tuple
----> 8 from llama_index.core.indices.service_context import ServiceContext
      9 from llama_index.core.evaluation.base import BaseEvaluator, EvaluationResult
     10 from llama_index.core.llms.llm import LLM

...
importing llama_index.core.evaluation
I feel like make your env is borked? I might try a fresh venv

That import works fine for me

This happens if you updated updated from v0.9.x without fully removing it
yeah I have been using v0.9
yea that'll do it -- the switch to namespaced imports means the old install has to be completely removed, or a completely fresh venv is needed
wouldn't pip uninstall llama-index do the trick or is there something else I need to do to "completely" remove it before installing v.10?
if you already installed v0.10 before that, its already too late (in my experience)
for reals, try a new venv (I also would not recomend anaconda, but thats just me lol, poetry or venv all the way)
yeah its for testing but okay thanks for the advice. I'll use poetry
Add a reply
Sign up and join the conversation on Discord