hey can we pass headers while

At a glance

The community members are discussing how to pass headers while initializing the Embedding model for the service context, as they want to integrate PortKey. They provide an example using the ChatOpenAI class from LangChain, but are unsure how to do this with the OpenAIEmbedding class from LlamaIndex.

The comments reveal some confusion around PortKey's support for the embeddings endpoint. One community member says PortKey doesn't support it, while another says they do, and that a PR is being opened to fix the issue. Another community member confirms that LlamaIndex's OpenAIEmbeddings class does not currently allow passing headers, and they will raise a PR to address this.

Meanwhile, a community member suggests a simple solution, and provides a link to a PR that is being worked on to add support for passing headers in the OpenAIEmbeddings class. Another community member encourages the maintainers to review the updated PR, as it would be helpful to have this functionality.

Useful resources

SSiddhant Saurabh

hey, can we pass headers while initializing Embedding model for the service context
Background we want to integrate PortKey.
we can do so in from langchain.chat_models import ChatOpenAI
eg

Plain Text

self.llm = ChatOpenAI(
            model=self.model_name,
            temperature=self.temperature,
            max_tokens=self.max_tokens,
            frequency_penalty=self.frequency_penalty,
            top_p=self.top_p,
            headers = {
                <some_header>
            }
        )

        # LLM Predictor
        self.llm_predictor = LLMPredictor(llm=self.llm)

works
so how can we pass the headers in from llama_index.embeddings.openai import OpenAIEmbedding
cc: @Logan M @jerryjliu0 @ravitheja

8 comments

SSiddhant Saurabh

sorry, guys it seems Portkey don't suport Embedding endpoint.
sorry for the trouble

rrohit

hey SIddhant - Portkey does support the embeddings endpoint.

Curious what's a reliable way to pass headers though

SSiddhant Saurabh

now I am confused. 🤔
they said "Portkey support said they are opening a PR to get it fixed. Right now embeddings don’t allow headers."

rrohit

Llamaindex does not allow headers in their OpenAIEmbeddings class. Spoke to @ravitheja today and I'll raise a PR on LLamaIndex

rrohit

Meanwhile, there's a simple solution if you want

rrohit

You can follow the PR here - https://github.com/jerryjliu/llama_index/pull/2527

rrohit

@jerryjliu0 - can you check this out again? Would be awesome if this gets merged as embeddings can then support the extra parameters

rrohit

Updated PR - https://github.com/jerryjliu/llama_index/pull/3248

Add a reply

Find answers from the community

hey can we pass headers while