With the migration from `ServiceContext`

SSimon 📐🛠

With the migration from ServiceContext to Settings the ContextRelevancyEvaluator in llama_index.core.evaluation.context_relevancy imports ServiceContext in version 10 still. When would this be updated?

24 comments

LLogan M

Probably just got missed in the transition. There's a few places tbh that this got mixed. I have time now to fix this

LLogan M

Ah actually, it import service context, but does not use it (it entirely optional)

LLogan M

this pattern is in a lot of the codebase

LLogan M

to maintain backwards compatibility

LLogan M

but its pulling from settings

self._llm = llm or llm_from_settings_or_context(Settings, service_context)

LLogan M

Whats the issue with that pattern?

SSimon 📐🛠

Thanks for the response, yeah I saw that it was optional too and I didn't plan on using it either.
I am following the setup of evaluators as per https://docs.llamaindex.ai/en/latest/examples/evaluation/batch_eval.html#batchevalrunner-running-multiple-evaluations guide

Plain Text

from llama_index.core.evaluation import (
    FaithfulnessEvaluator,
    RelevancyEvaluator,
    CorrectnessEvaluator,
)

I am stuck at this import. How do I import these evaluators otherwise? Change the base code in llama_index.core.evaluation.context_relevancy? How do I do this?

LLogan M

What do you mean change them? What's the error?

SSimon 📐🛠

Seems to be working now. Though I do get another error message:

Plain Text

ModuleNotFoundError                       Traceback (most recent call last)
Input In [5], in <cell line: 1>()
----> 1 from llama_index.llms.openai import OpenAI
      2 from llama_index.core.evaluation import (
      3     FaithfulnessEvaluator,
      4     RelevancyEvaluator,
      5     CorrectnessEvaluator,
      6 )
      8 from llama_index.core import Settings

File ~\anaconda3\lib\site-packages\llama_index\llms\openai\__init__.py:1, in <module>
----> 1 from llama_index.llms.openai.base import AsyncOpenAI, OpenAI, SyncOpenAI, Tokenizer
      3 __all__ = ["OpenAI", "Tokenizer", "SyncOpenAI", "AsyncOpenAI"]

File ~\anaconda3\lib\site-packages\llama_index\llms\openai\base.py:48, in <module>
     46 from llama_index.core.llms.llm import LLM
     47 from llama_index.core.types import BaseOutputParser, PydanticProgramMode
---> 48 from llama_index.llms.openai.utils import (
     49     create_retry_decorator,
     50     from_openai_message,
     51     from_openai_token_logprobs,
     52     from_openai_completion_logprobs,
     53     is_chat_model,
     54     is_function_calling_model,
     55     openai_modelname_to_contextsize,
     56     resolve_openai_credentials,
     57     to_openai_message_dicts,
     58 )
     60 from openai import AsyncOpenAI, AzureOpenAI
     61 from openai import OpenAI as SyncOpenAI

File ~\anaconda3\lib\site-packages\llama_index\llms\openai\utils.py:24, in <module>
     22 from openai.types.chat.chat_completion_chunk import ChoiceDeltaToolCall
     23 from openai.types.chat.chat_completion_message import ChatCompletionMessage
---> 24 from openai.types.chat.chat_completion_token_logprob import ChatCompletionTokenLogprob
     25 from openai.types.completion_choice import Logprobs
     26 from openai.types.completion import Completion

ModuleNotFoundError: No module named 'openai.types.chat.chat_completion_token_logprob'

Should I just revert to an earlier release?

LLogan M

update your openai client I think

LLogan M

pip install -U openai

SSimon 📐🛠

Thanks that worked!

SSimon 📐🛠

I'm now getting an error ModuleNotFoundError: No module named 'llama_index.core.image_retriever'

LLogan M

what code triggers that?

SSimon 📐🛠

Plain Text

ModuleNotFoundError                       Traceback (most recent call last)
Input In [2], in <cell line: 2>()
      1 from llama_index.llms.openai import OpenAI
----> 2 from llama_index.core.evaluation import (
      3     FaithfulnessEvaluator,
      4     RelevancyEvaluator,
      5     CorrectnessEvaluator,
      6 )
      8 from llama_index.core import Settings
     10 from llama_index import (
     11     Response,
     12     KnowledgeGraphIndex,
   (...)
     16     load_index_from_storage    
     17 )

File ~\anaconda3\lib\site-packages\llama_index\core\evaluation\__init__.py:3, in <module>
      1 """Evaluation modules."""
----> 3 from llama_index.core.evaluation.answer_relevancy import AnswerRelevancyEvaluator
      4 from llama_index.core.evaluation.base import (
      5     BaseEvaluator,
      6     EvaluationResult,
      7 )
      8 from llama_index.core.evaluation.batch_runner import BatchEvalRunner

File ~\anaconda3\lib\site-packages\llama_index\core\evaluation\answer_relevancy.py:8, in <module>
      5 import re
      6 from typing import Any, Callable, Optional, Sequence, Tuple
----> 8 from llama_index.core.indices.service_context import ServiceContext
      9 from llama_index.core.evaluation.base import BaseEvaluator, EvaluationResult
     10 from llama_index.core.llms.llm import LLM

...

SSimon 📐🛠

importing llama_index.core.evaluation

LLogan M

I feel like make your env is borked? I might try a fresh venv

That import works fine for me

This happens if you updated updated from v0.9.x without fully removing it

SSimon 📐🛠

yeah I have been using v0.9

SSimon 📐🛠

I am using anaconda

LLogan M

yea that'll do it -- the switch to namespaced imports means the old install has to be completely removed, or a completely fresh venv is needed

SSimon 📐🛠

wouldn't pip uninstall llama-index do the trick or is there something else I need to do to "completely" remove it before installing v.10?

LLogan M

if you already installed v0.10 before that, its already too late (in my experience)

LLogan M

for reals, try a new venv (I also would not recomend anaconda, but thats just me lol, poetry or venv all the way)

SSimon 📐🛠

yeah its for testing but okay thanks for the advice. I'll use poetry

Add a reply

Find answers from the community

With the migration from `ServiceContext`