Summarize flow in screenshot with figure title

Question

Hi anyone experience this woith azuremultimodal chat?
msg = ChatMessage(
role=MessageRole.USER,
blocks=[
TextBlock(text="Summarize this flow in screenshot of application present in image and support the description along with surronding text with a figure title."),
ImageBlock(image=image_document.image_resource.data),
],
)

response = llm.chat(messages=[msg])

Logan M · Answer

oh dang, this might be a bug, lemme look

Logan M · Answer

Hmm, I think something is just out of date on your end -- pip install -U llama-index-llms-openai llama-index-multi-modal-llms-openai llama-index-multi-modal-llms-azure-openai ?

Logan M · Answer

(I should bump the min-deps, thats a lot lol)

Daniya · Answer

This is working with simple AzureOpenAI class but not with AzureMultiModal

Daniya · Answer

Also index.as_query_engine()When I pass llm = azuremultimodallmm its workingbut when I pass multi_modal_llm = azuremultimodallmm its need settings.llm to be set but with the donwgraded version I'm not able to set AzureMultiModal in my settings

Logan M · Answer

Can you run pip freeze | grep llama and show me what you have installed?

Daniya · Answer

assert isinstance(llm, LLM) is false for this AzureMultiModal model class

Logan M · Answer

Did you run the pip install -U llama-index-llms-openai llama-index-multi-modal-llms-openai llama-index-multi-modal-llms-azure-openai command? The multi-modal llm classes are not up to date

Daniya · Answer

ok let me yesterday I was facing the issue so you gove me some downgraded version.

Logan M · Answer

oh 😅 lol

Logan M · Answer

Oh! now I remember -- I'm about to fix your previous issue

Logan M · Answer

https://github.com/run-llama/llama_index/pull/18112

WhiteFang_Jr · Answer

The muli-modal assert one 😅

Find answers from the community

Summarize flow in screenshot with figure title