Image

Question

does LI have a solution to managing the size of a page to send the oai image model? to manage resolution vs. tokens?

Logan M · Answer

I think for resolution, you can just set low/high/auto for image details
https://github.com/run-llama/llama_index/blob/af9abd06a456a3745d02379f8afc4b6cab3a3f72/llama-index-integrations/multi_modal_llms/llama-index-multi-modal-llms-openai/llama_index/multi_modal_llms/openai/base.py#L60

I havent checked openais exact api to see if they have more controls than that recently

tarpus · Answer

Will take a look. Glad to see you're in the room. Thank you.

Logan M · Answer

we added a new multimodal node https://github.com/run-llama/llama_index/pull/16962improved multimodal support in chat messages https://github.com/run-llama/llama_index/pull/15969Next up is updating the llms to work with the new chat messagesThen updating the retrievers to work with the new nodesThen updating anything else remaining

tarpus · Answer

my pdf's are all scanned physial documents. so the pages are basically like photos of document pages.

tarpus · Answer

they are a challenge to work with.

Logan M · Answer

Typically the best approach we've seen is using llama parse (or something else) to ocr the page, and sending both the text and image to the llm We have examples doing that 😁

Logan M · Answer

https://github.com/run-llama/llama_parse/blob/main/examples/multimodal/multimodal_rag_slide_deck.ipynb

Find answers from the community

Image