langchain/base.py at 9833fcfe32eab8b419a...

mmat

Hi folks, a vector store question: do vector stores on GPT index (e.g. GPTSimpleVectorIndex) support having separate embedding functions for docs and queries?

e.g. on https://learn.microsoft.com/en-us/azure/cognitive-services/openai/concepts/models#text-search-embedding, there are different endpoints for embedding a query vs a document

Is it as simple as implementing the langchain.Embeddings (https://github.com/hwchase17/langchain/blob/9833fcfe32eab8b419a6624f02c2536ac4115ed3/langchain/embeddings/base.py) interface?

8 comments

yyourbuddyconner

Yes, you can add embeddings created out-of-band to a document via the embeddings kwarg

Does this answer your question or are you more speaking about implementing this within the gpt_index pipeline?

mmat

Yes, you can add embeddings created out-of-band to a document via the embeddings kwarg

sorry i'm not sure if i understand this part, can you elaborate?

yyourbuddyconner

This might be helpful: https://gpt-index.readthedocs.io/en/latest/how_to/embeddings.html#adding-embeddings-to-document-objects

yyourbuddyconner

The example in there shows how one might use an external model to generate embeddings, as opposed to using the OpenAI embedding model built-in to gpt_index

jjerryjliu0

you can also define custom embedding functions (including plugging in langchain embedding models) here! https://gpt-index.readthedocs.io/en/latest/how_to/embeddings.html#custom-embeddings

yyourbuddyconner

Hey nice

yyourbuddyconner

Adding that to the ol' GPTMeatBrainIndex

mmat

Sweet, this helps. Thanks everyone!

Add a reply

Find answers from the community

langchain/base.py at 9833fcfe32eab8b419a...