Multi-Modal

Question

llm = llm or Settings.llm--> 143 assert isinstance(llm, MultiModalLLM) 144 145 return SimpleMultiModalQueryEngine(Hi I'm getting this error with AzureOpenAI llm it was working fine a week ago is there any update or specific version I have to donwgrade to?isinstance(llm, MultiModalLLM) - false

WhiteFang_Jr · Answer

You need to setup Multi-Modal instance for Azure: pip install llama-index-multi-modal-llms-azure-openai

Daniya · Answer

Yeah I checked it trying it now Thanks

Daniya · Answer

Hi so MultiModelVector Index uses clip for both images and text? Do I have to provide embed modal? index = MultiModalVectorStoreIndex(merged_text_nodes+merged_image_nodes) ? here

Logan M · Answer

it uses clip for images only

Logan M · Answer

if you dont provide, its defaulting to clip + openai embeddings

Daniya · Answer

I used AzureOpenAIMultiModal and from docs table I see its a multi modal but I'm still running in to this check issue

Logan M · Answer

Oh whoops, it is not multimodal 😢 Thats my bad, long story, we are halfway through a migration -- try downgrading it pip install -U "llama-index-multi-modal-llms-azure-openai<0.4.0" "llama-index-multi-modal-llms-openai<0.5.0"

Logan M · Answer

Will make a fix in core for this though in the future

Find answers from the community

Multi-Modal