Find answers from the community

Updated 2 months ago

I m trying to index an entire codebase

I'm trying to index an entire codebase (consists of 200+ repositories) with llamaindex and Parsing documents into nodes takes forever. I believe that the splitter is quite slow due to its being serial. Is there any parallelised implementation available?
L
E
o
4 comments
not yet, although that's good feedback. You could parse/thread each repo manually though
Yes. What I've seen is reading the documents is faster in a single-thread but indexing is what's taking a lot of time
Could you please let me know how this is helpful?
Add a reply
Sign up and join the conversation on Discord