Find answers from the community

Updated 3 months ago

The integration with mlflow (the recent

The integration with mlflow (the recent release) is not fully tested, i am getting issues while i am testing a RAG with external vectors stores (qdrant)
W
p
L
54 comments
Could you share the errors that you faced. That will help to debug the issue
one with llama_index + ollama + mlflow it fails with
TypeError: Ollama.init() got an unexpected keyword argument 'system_prompt'
another thing i tested is with the below
Plain Text
index = VectorStoreIndex.from_documents(documents=docs,storage_context=storage_context,  transformations=Settings.transformations,
                                         show_progress=True)
but in docs its very bare minimum with out any external storage context
which will be of no actual use
all my experiments failed today 😒
I'm pretty sure your ollama integration isn't updated
Okay let me check and update you
@Logan M i am sure there are issue with the new integration release of mlflow.
Attachment
image.png
one of my bug got closed and its fixed in the new release of llama-index==0.10.59 as its breaking on llama-index==0.10.58
now the model tracking will happen but the inference fails with the below error (Stack Trace)
Plain Text
------------- Inference via Llama Index   -------------
Traceback (most recent call last):
  File "/Users/pavanmantha/Pavans/PracticeExamples/DataScience_Practice/mlflow-tutorials/mlflow-llamaindex-playground/serve_model.py", line 34, in <module>
    index = mlflow.llama_index.load_model(model_uri)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pavanmantha/Pavans/PracticeExamples/DataScience_Practice/mlflow-tutorials/mlflow-llamaindex-playground/venv/lib/python3.11/site-packages/mlflow/tracing/provider.py", line 237, in wrapper
    is_func_called, result = True, f(*args, **kwargs)
                                   ^^^^^^^^^^^^^^^^^^
  File "/Users/pavanmantha/Pavans/PracticeExamples/DataScience_Practice/mlflow-tutorials/mlflow-llamaindex-playground/venv/lib/python3.11/site-packages/mlflow/llama_index/__init__.py", line 360, in load_model
    deserialize_settings(settings_path)
  File "/Users/pavanmantha/Pavans/PracticeExamples/DataScience_Practice/mlflow-tutorials/mlflow-llamaindex-playground/venv/lib/python3.11/site-packages/mlflow/llama_index/serialize_objects.py", line 171, in deserialize_settings
    settings_dict = _deserialize_dict_of_objects(path)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pavanmantha/Pavans/PracticeExamples/DataScience_Practice/mlflow-tutorials/mlflow-llamaindex-playground/venv/lib/python3.11/site-packages/mlflow/llama_index/serialize_objects.py", line 117, in _deserialize_dict_of_objects
    output.update({k: dict_to_object(v)})
                      ^^^^^^^^^^^^^^^^^
  File "/Users/pavanmantha/Pavans/PracticeExamples/DataScience_Practice/mlflow-tutorials/mlflow-llamaindex-playground/venv/lib/python3.11/site-packages/mlflow/llama_index/serialize_objects.py", line 105, in dict_to_object
    return object_class.from_dict(kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pavanmantha/Pavans/PracticeExamples/DataScience_Practice/mlflow-tutorials/mlflow-llamaindex-playground/venv/lib/python3.11/site-packages/llama_index/core/schema.py", line 125, in from_dict
    return cls(**data)

^^^^^^^^^^^
TypeError: OllamaEmbedding.__init__() got an unexpected keyword argument 'num_workers'
and i see this could because of the model settings saved in mlflow as below
Plain Text
{
  "_llm": {
    "object_constructor": "llama_index.llms.ollama.base.Ollama",
    "object_kwargs": {
      "system_prompt": null,
      "pydantic_program_mode": "default",
      "base_url": "http://localhost:11434",
      "model": "llama3.1",
      "temperature": 0.75,
      "context_window": 3900,
      "request_timeout": 30,
      "prompt_key": "prompt",
      "json_mode": false,
      "additional_kwargs": {},
      "is_function_calling_model": true
    }
  },
  "_embed_model": {
    "object_constructor": "llama_index.embeddings.ollama.base.OllamaEmbedding",
    "object_kwargs": {
      "model_name": "nomic-embed-text:latest",
      "embed_batch_size": 10,
      "num_workers": null,
      "base_url": "http://localhost:11434",
      "ollama_additional_kwargs": {}
    }
  },
  "_transformations": [
    {
      "object_constructor": "llama_index.core.node_parser.text.sentence.SentenceSplitter",
      "object_kwargs": {
        "include_metadata": true,
        "include_prev_next_rel": true,
        "chunk_size": 512,
        "chunk_overlap": 100,
        "separator": " ",
        "paragraph_separator": "\n\n\n",
        "secondary_chunking_regex": "[^,.;γ€‚οΌŸοΌ]+[,.;γ€‚οΌŸοΌ]?"
      }
    }
  ]
}
observe the embedding setting the num_workers is set as null
@pavanmantha I updated ollama embedding, pip install -U llama-index-embeddings-ollama

Just an issue with ollama embeddings not allowing kwargs in the constructor
okay let me do it now
also what if suppose i have a external vector store configured like qdrant or weaviate ?
this mlflow integration is not working
Yup Finally we solved this Ollama issue this is good
it started working
mlflow developed this completely in their house πŸ˜… tbh I didn't even know they were working on this until like a week ago
Thanks, but what if suppose we want to use the external vector_stores as i asked instead of the default VectorStoreIndex as below
index = VectorStoreIndex.from_documents(documents=docs)
this is funny !!
you undertand the problem right the basic ones is of no use actually (just think of alpha, beta or prod) environments
where we tend to use the best vector engines for their reliability and scalability
this itself is missing
where can i report this ? or will you take this internally ?
@Logan M I opened this bug, with mlflow let me see if they respond.
https://github.com/mlflow/mlflow/issues/12856
@Logan M i got response from mlflow team and the bug is accepted and they are working on it.
Attachment
image.png
@Logan M or @WhiteFang_Jr in the llama_index_workflow i keep getting the below issue, while implementing "RAGWorkflow"
I just ran this RAGWorkFlow: https://docs.llamaindex.ai/en/latest/examples/workflow/rag/ example and it is working fine for me.

Can you share your code to get more clarity on the issue here
Interesting !! when i take out the reranking all works fine something to do with reranking
Just tried the re-ranking example with gpt-4o-mini: https://docs.llamaindex.ai/en/latest/examples/workflow/rag/#rag-workflow-with-reranking working fine for me. Maybe 3.1 llm is not able to re-rank them?
will change to GPT-4o
@WhiteFang_Jr : llama-index workflow are no more a "beta" am i right ?
It is as of now. It got released yesterday only. Based on people's reviews and addition into llama-agent will make it move from beta mode I guess
Fine.. my powerful article is getting ready on this feature
it has lot of potential
by the way where can i provide my feedback if i feel any feature or modification sugestions ?
You are very much welcome here
If you find any issue or feature request you can do that on GitHub as well
thank to @WhiteFang_Jr and @Logan M for helping me in getting my queries resolved as quick as possible
Woah that's super fast!!
Have you posted it on twitter? If so the team can give you a shoutout with your tweet(X).
yes i did it in twitter also
Very cool! Putting this into our content pipeline. Thank you very much for your thoughts on it and trying it out πŸ’ͺ
Add a reply
Sign up and join the conversation on Discord