anyone have any information on how to

You want to use local llm ?
I guess defining service_context with local llm and embed model should work.

maybe

It may not work as intended due to the llm limitations. Somewhere in the llamaindex docs there is a table of different popular llm models and their capabilities.

Cant find it now

Yeah it wont work as good as OpenAI, Compatibility report on Open source LLM: https://docs.llamaindex.ai/en/stable/module_guides/models/llms.html#open-source-llms

Thx, i was still looking for it