OpenAI Platform

aanonymous_lambda

Hello, is there a reason a Llama LLM is loaded up during index creation?

INFO:sentence_transformers.SentenceTransformer:Load pretrained SentenceTransformer: /home/ubuntu/e5-base-v2
Load pretrained SentenceTransformer: /home/ubuntu/e5-base-v2
INFO:sentence_transformers.SentenceTransformer:Use pytorch device: cpu
Use pytorch device: cpu
**
Could not load OpenAI model. Using default LlamaCPP=llama2-13b-chat. If you intended to use OpenAI, please check your OPENAI_API_KEY.
Original error:
No API key found for OpenAI.
Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization.
API keys can be found or created at https://platform.openai.com/account/api-keys

**
llama_model_loader: loaded meta data with 19 key-value pairs and 363 tensors from /tmp/llama_index/models/llama-2-13b-chat.Q4_0.gguf (version GGUF V2 (latest))

5 comments

TTeemu

Is your goal to use OpenAI? The Llama is a fallback

aanonymous_lambda

I eventually plan to use OpenAI for the generation part, not for embeddings. Is Llama the fallback for embeddings? I'm trying to use a custom embedding model.

LLogan M

The service context (which every index includes) initializes an embedding model and LLM

If your openai key isn't present, it defaults to llama.cpp

If you want to disable embeddings, set the llm to None

Plain Text

service_context = ServiceContext.from_defaults(llm=None)
index = VectorStoreIndex.from_documents(documents, service_context=service_context)

note that without an LLM, you will not be able to query the index, only retrieval will work

Plain Text

retreiever = index.as_retriever(similarity_top_k=2)
nodes = retriever.retrieve("Query")

aanonymous_lambda

Thanks. For now, I'm just trying to create an index with embeddings and store it in qdrant. I'll turn off the LLM in the service_context.

Plain Text

service_context = ServiceContext.from_defaults(embed_model=E5Embeddings(), chunk_size=512, chunk_overlap=128)
vector_store = QdrantVectorStore(client=client, collection_name="collection_test")
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(documents, service_context=service_context, storage_context=storage_context)

LLogan M

yea just need to set llm=None then

Add a reply

Find answers from the community

OpenAI Platform