Find answers from the community

Updated 8 months ago

@Logan M hey, i wanna change the openai_

@Logan M hey, i wanna change the openai_base_api/url and apikey, how do i change that in llamaindex.ts?
L
E
β™”
29 comments
No idea for llama-index TS πŸ˜† cc @Emanuel Ferreira maybe?
to understand, what's the reason to change the openai base api?
we have a llm that can be accessed through api, it can be accessed using openai lib, we just had to change the base_url to our api url @Emanuel Ferreira
Plain Text
const llm = new OpenAI({
 apiKey: "YOUR_API_KEY_HERE",
 additionalSessionOptions: {
   baseURL: "base_url_here"
 }
})


this should work
if you want to update all llms then is Settings.llm = new OpenAI({
})
just openai config currently isn't possible
Plain Text
throw new Error("Set OpenAI Key in OPENAI_API_KEY env variable"); // Overriding OpenAI package's error message
                  ^

Error: Set OpenAI Key in OPENAI_API_KEY env variable

after specifying the api key
can you share the latest code?
you're instantiating openai, that's why its requiring the api key
not understanding if you want to use groq, why you want to instantiate OpenAI
@Emanuel Ferreira Lots of APIs are openia-compatible. Its very easy in a lot of cases to just change where the openai client is looking (the base url) and use some other LLM API

LlamaIndex (python) supports this by having an OpenAILike LLM class, that makes it very easy to use the openai client with any API that is the same as openai
Yea, on typescript you can override the api key and base url, so it should work as well.

I'm just trying to understand why he's using Groq class, since he can just update the OpenAI settings to use the desired llm API
Groq only has few models, we want to use groq for low latency and our other llm for intelligence..
finally got the custom api url working, but got a stop at this error..
Attachment
Screenshot_.png
can you share some code snippet again?
@Logan M what's the purpose of function calls in llama index?? it forcefully uses the functions without deciding...
It does not forcefully use the tools? The tools are just sent to the api, and the api figures it out under the hood

Although here, you've given a function with
  • a non-deacriptive name
  • no description
So the model is probably just confused.

Try either adding a docstring or providing a description to from_defaults. And maybe give it a better name?

Plain Text
def give_me_20():
  """Useful for getting the number 20."""
  return 20

tool = FunctionTool.from_defaults(fn=give_me_20)


Here it uses the function name as the name, and the docstring as description. You can also manually override this

FunctionTool.from_defaults(fn=fn, name="...", description="...")
The name and description is SUPER important
ahh very true, but it always used all the functions when many functions are specified maybe out of confusion??
yea, I think so. With no good description, it just makes a choice πŸ€·β€β™‚οΈ
Add a reply
Sign up and join the conversation on Discord