@Logan M hey, i wanna change the openai

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

@Logan M hey, i wanna change the openai_base_api/url and apikey, how do i change that in llamaindex.ts?

♔

29 comments

LLogan M

No idea for llama-index TS 😆 cc @Emanuel Ferreira maybe?

EEmanuel Ferreira

for azure you can:

https://ts.llamaindex.ai/modules/llms/#azure-openai

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

For just openai?

EEmanuel Ferreira

to understand, what's the reason to change the openai base api?

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

we have a llm that can be accessed through api, it can be accessed using openai lib, we just had to change the base_url to our api url @Emanuel Ferreira

EEmanuel Ferreira

Plain Text

const llm = new OpenAI({
 apiKey: "YOUR_API_KEY_HERE",
 additionalSessionOptions: {
   baseURL: "base_url_here"
 }
})

this should work

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

this way?

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

Attachment

EEmanuel Ferreira

if you want to update all llms then is Settings.llm = new OpenAI({
})

EEmanuel Ferreira

just openai config currently isn't possible

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

im using groq but it's asking for openai api key?

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

Plain Text

throw new Error("Set OpenAI Key in OPENAI_API_KEY env variable"); // Overriding OpenAI package's error message
                  ^

Error: Set OpenAI Key in OPENAI_API_KEY env variable

after specifying the api key

EEmanuel Ferreira

can you share the latest code?

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

@Emanuel Ferreira

Attachment

Screenshot_2024-04-20_at_12.42.12_AM.png

EEmanuel Ferreira

you're instantiating openai, that's why its requiring the api key

EEmanuel Ferreira

not understanding if you want to use groq, why you want to instantiate OpenAI

LLogan M

@Emanuel Ferreira Lots of APIs are openia-compatible. Its very easy in a lot of cases to just change where the openai client is looking (the base url) and use some other LLM API

LlamaIndex (python) supports this by having an OpenAILike LLM class, that makes it very easy to use the openai client with any API that is the same as openai

EEmanuel Ferreira

Yea, on typescript you can override the api key and base url, so it should work as well.

I'm just trying to understand why he's using Groq class, since he can just update the OpenAI settings to use the desired llm API

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

Groq only has few models, we want to use groq for low latency and our other llm for intelligence..

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

finally got the custom api url working, but got a stop at this error..

Attachment

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

@Emanuel Ferreira can you help please?

EEmanuel Ferreira

can you share some code snippet again?

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

it kinda works! but i have few doubts regarding llama index

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

@Logan M what's the purpose of function calls in llama index?? it forcefully uses the functions without deciding...

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

@Logan M @Emanuel Ferreira

Attachment

LLogan M

It does not forcefully use the tools? The tools are just sent to the api, and the api figures it out under the hood

Although here, you've given a function with

a non-deacriptive name
no description

So the model is probably just confused.

Try either adding a docstring or providing a description to from_defaults. And maybe give it a better name?

Plain Text

def give_me_20():
  """Useful for getting the number 20."""
  return 20

tool = FunctionTool.from_defaults(fn=give_me_20)

Here it uses the function name as the name, and the docstring as description. You can also manually override this

FunctionTool.from_defaults(fn=fn, name="...", description="...")

LLogan M

The name and description is SUPER important

♔♔†𝓟𝓘𝓣𝓑𝓤𝓛𝓛 †

ahh very true, but it always used all the functions when many functions are specified maybe out of confusion??

LLogan M

yea, I think so. With no good description, it just makes a choice 🤷‍♂️

Add a reply

Find answers from the community

@Logan M hey, i wanna change the openai_