If anyone worked on something similar I

If anyone worked on something similar I can seriously use some insights!

7 comments

I think you could give a try to GPT-4 vision, since it contains machinery images and stuff.
My guess would be that Open source model might not fit in there.

See if this helps you in your adventure: https://docs.llamaindex.ai/en/stable/examples/multi_modal/gpt4v_multi_modal_retrieval.html

GGeoloeG

LLamaParse is not using OCR also? and even more the JSON is said to extract figures?

LLogan M

I would definitely check out llama-parse

gghxsted.

is there a way to use llama-parse with a huggingface embedding model and a huggingface llm to build the index ? in the notebook in the doc it only covers using it with openai models

gghxsted.

here : https://github.com/run-llama/llama_parse/blob/main/examples/demo_advanced.ipynb

LLogan M

Just install the integration you want and use it. For example, I would personally use ollama and huggingface

Plain Text

pip install llama-index-embeddings-huggingface llama-index-llms-ollama

Install and start ollama (ollama serve) and download the LLM to use (ollama pull <model>)

Then:

Plain Text

from llama_index.core import Settings
from llama_index.embeddings.huggingface import HuggingfaceEmbedding
from llama_index.llms.ollama import Ollama

Settings.llm = Ollama(model="<model>", request_timeout=3000.0)
Settings.embed_model = HuggingfaceEmbedding(model_name="BAAI/bge-small-en-v1.5", embed_batch_size=2)

<rest of code below

gghxsted.

thank you so much logan ;))

Add a reply

Find answers from the community

If anyone worked on something similar I