llama_parse/examples/demo_advanced.ipynb...

At a glance

The community member is trying to replicate the LLamaParse example using LM Studio instead of OpenAI. They are encountering an error when using the OpenAI class, and the comments suggest using OpenAILike instead. The community members discuss the import issues and provide a solution to install the llama-index-llms-openai-like package, which seems to resolve the issue.

Useful resources

sscottyakc

Trying to replicate the LLamaParse example here: https://github.com/run-llama/llama_parse/blob/main/examples/demo_advanced.ipynb

But instead of OpenAI trying to use LM Studio on my local machine:
from llama_index.embeddings.huggingface import HuggingFaceEmbedding


embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
llm = OpenAI(
    api_key="NULL",
    api_base="http://localhost:1234/v1",
    temperature=0.2,
)

Settings.llm = llm
Settings.embed_model = embed_model

node_parser = MarkdownElementNodeParser(llm=llm, num_workers=8)

...

sub_query_engine = SubQuestionQueryEngine.from_defaults(
    query_engine_tools=query_engine_tools,
    llm=llm,
    use_async=True,
)

response = sub_query_engine.query(
    "Which fund has the smallest minimum investment size?"
)

The api call is received by the server, but when the response is sent back, Llama Index throws the below error:

ValueError: Expected tool_calls in ai_message.additional_kwargs, but none found.

Anyone experience this?

8 comments

LLogan M

Don't use OpenAI if you aren't using OpenAI, use OpenAILike

Plain Text

pip install llama-index-llms-openai-like

Plain Text

from llama_index.llms.open_like import OpenAILike

llm = OpenAILike(api_key="NULL", api_base="...", is_chat_model=True)

LLogan M

Otherwise, it thinks its a function calling model, and is trying to use the function calling API (which won't exist)

sscottyakc

Thanks! I tried to use the OpenAILike llm with the markdown node parse


llm = OpenAILike(
    api_key="NULL",
    api_base="http://localhost:1234/v1",
    temperature=0.1,
    is_chat_model=False
)

node_parser = MarkdownElementNodeParser(llm=llm, num_workers=8)

But I get the below, when trying to define node_parser:

ValidationError: 1 validation error for MarkdownElementNodeParser
llm
Can't instantiate abstract class LLM with abstract methods achat, acomplete, astream_chat, astream_complete, chat, complete, metadata, stream_chat, stream_complete (type=type_error)

LLogan M

how did you import it?

LLogan M

Seems like there is a mix of legacy and non-legacy imports here

sscottyakc

I had to import OpenAILike from legacy module, I get below when trying to import from the main package.

ModuleNotFoundError: No module named 'llama_index.llms.open_like'

Is there another llama-index version I should be using? I'm on 0.10.27

LLogan M

pip install llama-index-llms-openai-like

sscottyakc

Awesome, seems to be working now. You're the man, thanks!

Add a reply

Find answers from the community

llama_parse/examples/demo_advanced.ipynb...