Find answers from the community

Updated 10 months ago

hello! what is the migration for service

hello! what is the migration for service context for methods like this for 0.10 over?

Plain Text
service_context = ServiceContext.from_defaults(
    llm=llm,
    embed_model=embeddings,
    callback_manager=callback_manager
)
object_index = ObjectIndex.from_objects(
    table_schema_objs,
    table_node_mapping,
    VectorStoreIndex,
    service_context=service_context
)
L
c
14 comments
Plain Text
object_index = ObjectIndex.from_objects(
    table_schema_objs,
    table_node_mapping,
    VectorStoreIndex,
    embed_model=embed_model,
    callback_manager=callback_manager,
)

(note LLM is not here, since it is not actually used in this case)

Or you can set it globally as a default

Plain Text
from llama_index.core import Settings

Settings.callback_manager = callback_manager
Settings.llm = llm
Settings.embed_model = embed_model
I see thanks and when I use

Plain Text
from llama_index.core.indices.struct_store.sql_query import SQLTableRetrieverQueryEngine

query_engine = SQLTableRetrieverQueryEngine(
            sql_database=sql_database,
            table_retriever=object_index.as_retriever(),
            text_to_sql_prompt=DEFAULT_TEXT_TO_SQL_PROMPT,
            response_synthesis_prompt=DEFAULT_RESPONSE_SYNTHESIS_PROMPT,
            embed_model=embeddings,
            callback_manager=callback_manager,
            llm=llm
        )

It fails and says Could not load OpenAI embedding model. If you intended to use OpenAI, please check your OPENAI_API_KEY.
it was working previously with

Plain Text
      query_engine = SQLTableRetrieverQueryEngine(
            sql_database=sql_database,
            table_retriever=object_index.as_retriever(),
            text_to_sql_prompt=DEFAULT_TEXT_TO_SQL_PROMPT,
            response_synthesis_prompt=DEFAULT_RESPONSE_SYNTHESIS_PROMPT,
            service_context=service_context,
        )


I am using azure open ai so i am using it like this. i installed llama-index-llms-azure-openai==0.1.4

Plain Text
from llama_index.llms.azure_openai import AzureOpenAI


def AzureLLMClient():
    config = settings.AZURE_OPENAI_CONFIG
    llm = AzureOpenAI(
        engine=config['engine'],
        model=config['model'],
        temperature=config['temperature'],
        api_key=config['api_key'],
        azure_endpoint=config['azure_endpoint'],
        api_base=config['api_base'],
        api_version=config['api_version'],
        max_retries=config['max_retries'],
        timeout=config['timeout']
    )
    return llm
and this is the embedding model
Plain Text
import os
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from transformers import AutoModel, AutoTokenizer


def EmbeddingModel():
    USE_MODEL_CACHE = os.getenv('USE_MODEL_CACHE')
    if (USE_MODEL_CACHE == "true"):
        tokenizer = AutoTokenizer.from_pretrained(
            'XXX', cache_dir='/build/huggingface_cache')

        model_name = AutoModel.from_pretrained(
            'XXX', cache_dir='/build/huggingface_cache')
        embeddings = HuggingFaceEmbedding(
            model=model_name,
            tokenizer=tokenizer,
        )
    else:
        embeddings = HuggingFaceEmbedding(
            model_name='XXX'
        )
    return embeddings
Do you have the full traceback? From the source code anyways it seems correct
ohhh I see the issue, the embed_model isn't getting passed into the NLSQLRetriever under the hood
try just setting it globally
Plain Text
Settings.llm = llm
Settings.embed_model = embed_model
Will need to make a PR to fix that
(Or you can just use service context for now)
I see. In order to use service context I just install llama index legacy right?
traceback wasn't logging for me will need to debug more
Using Settings helped thanks. If I have multiple files that need this, do I just need to set the following in every file?
Plain Text
Settings.llm = llm
Settings.embed_model = embed_model
If each file runs in a seperate process, then yes. But if each file is running in the same program/process, you only need to do it once
Add a reply
Sign up and join the conversation on Discord