Find answers from the community

Updated 3 months ago

Embedding

Hi everyone. I would like to build a question-answering app that retrieves embeddings from a vector store and uses them as context in the prompt to answer a question. In this app, I am not using OPENAI_API_KEY as my LLM is from Hugging Face Hub. Specifically, I created my LLM instance ("llm") by HuggingFacePipeline, and provided it to the following:
Plain Text
llm_predictor = LLMPredictor(llm=llm)
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor)
index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context) 

However, GPTVectorStoreIndex throws
Plain Text
AuthenticationError: No API key provided.

Could anyone help me implement a vector store index without OPENAI_API_KEY?? (Or is a vector store index necessary to build an app if I am going to have an external vector store like FAISS or Pinecone?) Thank you in advance πŸ™
L
D
8 comments
There are two models in llama index, the LLM (for generating text and answering queries) and the embedding model (for generating embeddings)

Here you've only set the LLM, so the embed model is still defaulting to openai
We support any embeddings offered by langchain, just need to wrap it with our langchain wrapper
@Logan M
Thanks for your kind responses! However, the URL you provided returns "404 Not Found". Could you give me the link?
Lol ya we just refactored the docs
@Logan M Thanks, I can see it now!!
@Logan M
As embedding model, I would like to use a SentenceTransformer (SBERT) model and wonder if I need to change the tokenizer for the embedding part as well. Would you let me know if the following is correct?

  1. For my node parser, I need to set the tokenizer for my SBERT model.
  2. Since my SBERT accepts 128 tokens as input at maximum (i.e., model's max_seq_length is 128), I need to set chunk size == 128.
As a result, my node parser looks like this:
Plain Text
# Node Parser
from transformers import AutoTokenizer
embed_tokenizer = AutoTokenizer.from_pretrained(EMBED_MODEL_ID)
text_splitter = TokenTextSplitter(
    chunk_size=128,
    chunk_overlap=20,
    tokenizer=embed_tokenizer.encode
    )
node_parser = SimpleNodeParser(text_splitter=text_splitter)

Thanks in advance πŸ™
Add a reply
Sign up and join the conversation on Discord