"Turn off embeddings and just index the markdown" -- that doesn't really make sense. Indexing requires embeddings. That's how basic vector retrieval works 👀
When you launch the app, you'll need to set loop="asyncio" in uvicorn
I guess I misspoke (I used to work with embeddings in gentics in their early emergence). Can't you build a summary index that just returns the markdown docs? My use case doen't really require embeddings. I'm just extracting fairly structured data and forcing it into a schema. llama-extract was working great for this before some of the endpoints went down.
I actually pulled the loaders modules into the non --pro Information Extractor and it seems to work! But can't really tell since the OpenAI embeddings API is going bonkers right now.
loop="asyncio" in uvicorn is a CLI arg? (Sorry, dumb data engineer)
@MindwhY you'd have to change the code to swap the VectorStoreIndex with a summary index -- I'm not 100% familiar enough with the code right now to remember where that is
For the asyncio loop, in main.py you should see this line
uvicorn.run(app="main:app", host=app_host, port=app_port, reload=reload)
Change it to
uvicorn.run(app="main:app", host=app_host, port=app_port, reload=reload, loop="asyncio")
Changing the indexer was easy enough, but still confused on the async issue when trying to llama-parse. There's no main.py in the app and the only references to uvicorn are in the dependency management and a few getLogger() s
It looks related to an issue I had using the Llama-Extract SDK from a Lambda before just switching to calling the API
[Reflex Backend Exception]
Traceback (most recent call last):
File "/usr/local/python/3.11.11/lib/python3.11/site-packages/reflex/state.py", line 1847, in _process_event
events = fn(payload) ^^^^^^^^^^^^^ File "/workspaces/doc-workflow/app/ui/components/upload.py", line 61, in remove_file generate_datasource() File "/workspaces/doc-workflow/app/engine/generate.py", line 24, in generate_datasource documents = get_documents() ^^^^^^^^^^^^^^^ File "/workspaces/doc-workflow/app/engine/loaders/init.py", line 28, in get_documents document = get_file_documents(FileLoaderConfig(loader_config))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/doc-workflow/app/engine/loaders/file.py", line 74, in get_file_documents
raise e
File "/workspaces/doc-workflow/app/engine/loaders/file.py", line 48, in get_file_documents
nest_asyncio.apply()
File "/usr/local/python/3.11.11/lib/python3.11/site-packages/nest_asyncio.py", line 19, in apply
_patch_loop(loop)
File "/usr/local/python/3.11.11/lib/python3.11/site-packages/nest_asyncio.py", line 193, in _patch_loop
raise ValueError('Can't patch loop of type %s' % type(loop))
ValueError: Can't patch loop of type <class 'uvloop.Loop'>
try:
file_extractor = None
if config.use_llama_parse:
# LlamaParse is async first,
# so we need to use nest_asyncio to run it in sync mode
import nest_asyncio
nest_asyncio.apply()
file_extractor = llama_parse_extractor()
reader = SimpleDirectoryReader(
DATA_DIR,
recursive=True,
filename_as_id=True,
raise_on_error=True,
file_extractor=file_extractor,
)
return reader.load_data()
except Exception as e:
hmm I was looking at the create-llama repo and noticed a main.py 🤷♂️ The error is because its calling nest_asyncio.apply(), but that only works if the loop type in fastapi is set to asyncio
Let me create a fresh create-llama example and figure out where that change is..
I just created a fresh create-llama app
npx create-llama@latest --pro
There's a main.py right in the root of the generated project
For example, my app was called my-asyncio-app, so I see my-asyncio-app/main.py in my file system
I add the loop param there
Weird. I've generated a project multiple time and it never gives me a main.py
You ran the exact same as above? npx create-llama@latest --pro ? Even without the pro, it should be the same (assuming you are building the python backend)
Yep. It's super strange. Just ran it again.
what if you don't use --pro ?
wild -- my only guess is somehow you don't have the latest version of create-llama ? Thats the only way it could be different across yours and mine machines
Anyways, there has to be uvicorn mentioned somewhere in the generated code
Especially wild given that both versions run, but only with llama parse set to false.
I'm using @latest so I don't see how I could have a different version
Idk what to tell you lol
Here's my full terminal code, the options I used, and me outputing the main.py
llama-index-py3.10(base) loganmarkewich@Mac ~ % npx create-llama@latest --version
0.3.28
llama-index-py3.10(base) loganmarkewich@Mac ~ % npx create-llama@latest
✔ What is your project named? … my-test-app-42
✔ What app do you want to build? › 🤖 Agentic RAG
✔ What language do you want to use? › Python (FastAPI)
✔ Do you want to use LlamaCloud services? … No / Yes
✔ Please provide your LlamaCloud API key (leave blank to skip): …
✔ Please provide your OpenAI API key (leave blank to skip): …
✔ How would you like to proceed? › Just generate code (~1 sec)
Creating a new LlamaIndex app in /Users/loganmarkewich/my-test-app-42.
...
llama-index-py3.10(base) loganmarkewich@Mac ~ % cat my-test-app-42/main.py
...
if __name__ == "__main__":
app_host = os.getenv("APP_HOST", "0.0.0.0")
app_port = int(os.getenv("APP_PORT", "8000"))
reload = True if environment == "dev" else False
uvicorn.run(app="main:app", host=app_host, port=app_port, reload=reload)(truncated the text that didn't matter)
Does yours also print 0.3.28?
Yep, 0..3.28. Absolutely wild.
Today I spunn up a few diffrent Linux containers, and the result is the same. No main.py, no uvicorn mentions other than dependanceies and loggers
thats wild my guy. Not sure what else to tell you
uvicorn is required to launch any fastapi backend
We could track this down if you really want. How do you launch your app?
poetry run dev ? There's a
run.py file with a
def dev() function. That calls a function
async def start_development_servers() -- eventually, that calls
async def _run_backend()In there, for me anyways, its using poetry to run the
main.py process = await asyncio.create_subprocess_exec(
poetry_executable,
"run",
"python",
"main.py",
env=envs,
)If
main.py does not exist, does this trace end up looking like for you?
No run.py file either. I use poetry run reflex run, but poetry run dev does nothing
In the version from --pro I'm running via dev containers, but I did the non --pro extractor and moved the parts it need over, and run locally, and exact same thing
It's shouldn't run, and yet it does
Can you dumb the run file in the thread?
Ohhhhhh you are using the reflex example!!!
omg that explains so much
Its kind of unique in terms of structure
I wass starting to thing I was crazy
But I can get this concept working then I'll have the space break it into micorserves and bring in other people. But we would be throwing a lot of docs at the API until llama-extrac is stable.
Ok next issue -- I created the extraction app locally with llamacloud/llamaparse enabled, added my llamacloud and openai keys to the .env, ran the steps in the readme, and then tried the app -- it worked well 😅
Steps I took (selected yes for LlamaCloud services)
llama-index-py3.10(base) loganmarkewich@Mac ~ % npx create-llama@latest
✔ What is your project named? … my-reflex-app
✔ What app do you want to build? › 🤖 Information Extractor
✔ Do you want to use LlamaCloud services? … No / Yes
✔ Please provide your LlamaCloud API key (leave blank to skip): …
✔ Please provide your OpenAI API key (leave blank to skip): …
✔ How would you like to proceed? › Just generate code (~1 sec)
Creating a new LlamaIndex app in /Users/loganmarkewich/my-reflex-app.
Initializing Python project with template: reflex
Adding additional dependencies
Adding tools dependencies
Added llama-index-indices-managed-llama-cloud, docx2txt, llama-index-llms-openai, llama-index-embeddings-openai, llama-index-agent-openai to pyproject.toml
Created '.env' file. Please check the settings.
Copying data from path: /Users/loganmarkewich/.npm/_npx/7bfc2205dda2d438/node_modules/create-llama/dist/templates/components/data/101.pdf
Initialized a git repository.
Success! Created my-reflex-app at /Users/loganmarkewich/my-reflex-app
Now have a look at the README.md (file:///Users/loganmarkewich/my-reflex-app/README.md) and learn how to get started.
The issue comes when you flip on llama-parse and drop a file into the upload window
Could it maybe be a Python 3.13 issue? I’v beee noticing those in ML and API libraries.
Guess I’ll blow it up in the morning and see (American in Ukraine, 8:30pm here already)
I DO appreciate your patience and help with this!
(Now that I think about it the —pro version running in dev containers based on the JSON definition, is running 3.11.11)
Maybe it’s my pyenv virtual env
But yea, if I don’t switch llama-parse on it works fine
oh maybe I missed the llama-parse button, let me check again
I did a fresh build produced the same result. It seems a reflex call to uvicorn is the issue, but I'm not sure where in the call stack something would need to change.
Changing the loop type to "asyncio" didn't resolve it
def get_file_documents(config: FileLoaderConfig):
from llama_index.core.readers import SimpleDirectoryReader
try:
file_extractor = None
if config.use_llama_parse:
# LlamaParse is async first,
# so we need to use nest_asyncio to run it in sync mode
import nest_asyncio
nest_asyncio.apply(loop='asyncio')
file_extractor = llama_parse_extractor()
reader = SimpleDirectoryReader(
DATA_DIR,
recursive=True,
filename_as_id=True,
raise_on_error=True,
file_extractor=file_extractor,
)
return reader.load_data()
except Exception as e:
import sys
import traceback
To make this not nest async,
get_file_documents needs to be an async function. Then, you need to call llama-parse directly I think, skipping simple directory reader. Something like
async def get_file_documents(config: FileLoaderConfig):
from llama_index.core.readers import SimpleDirectoryReader
try:
file_extractor = None
if config.use_llama_parse:
file_extractor = llama_parse_extractor()
reader = SimpleDirectoryReader(
DATA_DIR,
recursive=True,
filename_as_id=True,
raise_on_error=True,
file_extractor=file_extractor,
)
return await reader.aload_data()
except Exception as e:But then, you need to find who is calling this function and also update it to be async
There's probably also some way to key it sync, and use asyncio like
return asyncio.get_event_loop.run_until_complete(reader.aload_data())
But unsure
get_documents in init, but it doesn't awaut
async def get_documents() -> List[Document]:
documents = []
config = load_configs()
for loader_type, loader_config in config.items():
logger.info(
f"Loading documents from loader: {loader_type}, config: {loader_config}"
)
match loader_type:
case "file":
document = get_file_documents(FileLoaderConfig(loader_config)) case "web": document = get_web_documents(WebLoaderConfig(loader_config))
case "db":
document = get_db_documents(
configs=[DBLoaderConfig(**cfg) for cfg in loaderconfig] ) case :
raise ValueError(f"Invalid loader type: {loader_type}")
documents.extend(document)
return await documents
Think I just need to just say if pdf
parser = LlamaParse(
result_type="markdown",
verbose=True,
language="en",
ignore_errors=False,
)
else simple directory reader
you can make it await if you want and use async 🤷♂️ Or directly use llama-parse (but then you need the file paths) -- up to you!
Making the whole call stack from the UI worked
