hello! what is the migration for service

hello! what is the migration for service context for methods like this for 0.10 over?

Plain Text

service_context = ServiceContext.from_defaults(
    llm=llm,
    embed_model=embeddings,
    callback_manager=callback_manager
)
object_index = ObjectIndex.from_objects(
    table_schema_objs,
    table_node_mapping,
    VectorStoreIndex,
    service_context=service_context
)

14 comments

LLogan M

Plain Text

object_index = ObjectIndex.from_objects(
    table_schema_objs,
    table_node_mapping,
    VectorStoreIndex,
    embed_model=embed_model,
    callback_manager=callback_manager,
)

(note LLM is not here, since it is not actually used in this case)

Or you can set it globally as a default

Plain Text

from llama_index.core import Settings

Settings.callback_manager = callback_manager
Settings.llm = llm
Settings.embed_model = embed_model

cchantlong

I see thanks and when I use

Plain Text

from llama_index.core.indices.struct_store.sql_query import SQLTableRetrieverQueryEngine

query_engine = SQLTableRetrieverQueryEngine(
            sql_database=sql_database,
            table_retriever=object_index.as_retriever(),
            text_to_sql_prompt=DEFAULT_TEXT_TO_SQL_PROMPT,
            response_synthesis_prompt=DEFAULT_RESPONSE_SYNTHESIS_PROMPT,
            embed_model=embeddings,
            callback_manager=callback_manager,
            llm=llm
        )

It fails and says Could not load OpenAI embedding model. If you intended to use OpenAI, please check your OPENAI_API_KEY.

cchantlong

it was working previously with

Plain Text

      query_engine = SQLTableRetrieverQueryEngine(
            sql_database=sql_database,
            table_retriever=object_index.as_retriever(),
            text_to_sql_prompt=DEFAULT_TEXT_TO_SQL_PROMPT,
            response_synthesis_prompt=DEFAULT_RESPONSE_SYNTHESIS_PROMPT,
            service_context=service_context,
        )

I am using azure open ai so i am using it like this. i installed llama-index-llms-azure-openai==0.1.4

Plain Text

from llama_index.llms.azure_openai import AzureOpenAI


def AzureLLMClient():
    config = settings.AZURE_OPENAI_CONFIG
    llm = AzureOpenAI(
        engine=config['engine'],
        model=config['model'],
        temperature=config['temperature'],
        api_key=config['api_key'],
        azure_endpoint=config['azure_endpoint'],
        api_base=config['api_base'],
        api_version=config['api_version'],
        max_retries=config['max_retries'],
        timeout=config['timeout']
    )
    return llm

cchantlong

and this is the embedding model

Plain Text

import os
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from transformers import AutoModel, AutoTokenizer


def EmbeddingModel():
    USE_MODEL_CACHE = os.getenv('USE_MODEL_CACHE')
    if (USE_MODEL_CACHE == "true"):
        tokenizer = AutoTokenizer.from_pretrained(
            'XXX', cache_dir='/build/huggingface_cache')

        model_name = AutoModel.from_pretrained(
            'XXX', cache_dir='/build/huggingface_cache')
        embeddings = HuggingFaceEmbedding(
            model=model_name,
            tokenizer=tokenizer,
        )
    else:
        embeddings = HuggingFaceEmbedding(
            model_name='XXX'
        )
    return embeddings

LLogan M

Do you have the full traceback? From the source code anyways it seems correct

LLogan M

ohhh I see the issue, the embed_model isn't getting passed into the NLSQLRetriever under the hood

LLogan M

try just setting it globally

LLogan M

Plain Text

Settings.llm = llm
Settings.embed_model = embed_model

LLogan M

Will need to make a PR to fix that

LLogan M

(Or you can just use service context for now)

cchantlong

I see. In order to use service context I just install llama index legacy right?

cchantlong

traceback wasn't logging for me will need to debug more

cchantlong

Using Settings helped thanks. If I have multiple files that need this, do I just need to set the following in every file?

Plain Text

Settings.llm = llm
Settings.embed_model = embed_model

LLogan M

If each file runs in a seperate process, then yes. But if each file is running in the same program/process, you only need to do it once

Add a reply

Find answers from the community

hello! what is the migration for service