Find answers from the community

Updated 2 months ago

Pinecone embedding

Hey all I am trying to use a non default embedding(trying to use hugging face) with a pinecone index but I keep on getting an error that it does not access an embed param? Here is my code for reference:

gpt_index = pinecone.Index("testing")
embed_model = LangchainEmbedding(HuggingFaceEmbeddings())
index = GPTPineconeIndex(pinecone_index=gpt_index, embedding_model=embed_model)

I am also on llama-index 0.5.8

Would appreciate any help!
L
a
27 comments
You'll want to pass in the embed model in the service context object

Plain Text
from llama_index import ServiceContext

index = GPTPineconeIndex(...., service_context=ServiceContext.from_defaults(embed_model=embed_model))
Ill try it out
Also is there a way to load documents from pinecone and use it as vectorstore at the same time. Main use-case I am trying to do is that I will just load all my documents from pinecone, but I am stuck on generating the query vector and the id_to_text_map, but creating index requires documents.
You can pass an empty array if the documents are already on pinecone

GPTPineconeIndex([], ...)
Is something like this enough: embed_model =
index = GPTPineconeIndex.from_documents([], pinecone_index=gpt_index)
result = index.query("what is life?")
Oh shoot i get an error like KeyError: 'doc_id'
Is gpt_index equal to the pinecone client index? πŸ€”
Full code:

api_key = os.getenv('PTOKEN')
pinecone_environment = os.getenv('PCLOUD')

pinecone.init(api_key=api_key, environment=pinecone_environment)


gpt_index = pinecone.Index("gptindextwo")
embed_model = LangchainEmbedding(HuggingFaceEmbeddings())
index = GPTPineconeIndex([], pinecone_index=gpt_index,)
result = index.query("what is life?")
i don't use hugging face here, but forgot to delete it
Gotcha. And what's the full error/stack trace?
happens in query result
query_result = self._vector_store.query(
python3.10/site-packages/llama_index/vector_stores/pinecone.py"
id = match.metadata["id"]
KeyError: 'id'
Interesting πŸ€” hmm I'll check something
unless maybe my env are incorrect
thanks for helping me!
nope just checked 😦
Yea no worries! Maybe try re-creating the pinecone index? Not sure whats up here haha. Judging from the code, the Metadata from pinecone isn't what we expected https://github.com/jerryjliu/llama_index/blob/main/gpt_index/vector_stores/pinecone.py#L202
I'll give it a go
I'm thinking maybe the documents have to be originally inserted with llama index? πŸ€” then once they are there you can load the index with that empty array.

not totally sure though, I haven't used pinecone yet lol just trying to understand the code πŸ˜…
Add a reply
Sign up and join the conversation on Discord