Find answers from the community

Updated 2 months ago

Anyone know what additional kwargs are

Anyone know what additional kwargs are allowed/expected for the OpenAILike llm? I was hoping to be able to pass parameters such as top p/k, repetition penalty, etc. but it doesn't seem those are supported. I assume since it's just a wrapper for OpenAI, options are limited. Worth creating a custom implementation or is it possible to modify the wrapper?

7 comments

bbin4ry_d3struct0r

How are you instantiating your LLM?

bbin4ry_d3struct0r

I use HuggingFaceLLM() from llama_index.llms.huggingface.

bbin4ry_d3struct0r

I'd see what I can throw in the generate_kwargs argument.

bbin4ry_d3struct0r

Or try passing them directly into HuggingFaceLLM()

bbin4ry_d3struct0r

I cannot find the documentation to exactly which params HuggingFaceLLM() accepts, but here's an example of its use:

bbin4ry_d3struct0r

https://docs.llamaindex.ai/en/stable/examples/customization/llms/SimpleIndexDemo-Huggingface_stablelm.html

LLogan M

Openailike will probably accept those under the additional_kwargs field

Add a reply