I m a front end guy and new to this

I'm a front end guy and new to this stuff.

Plain Text

documents = SimpleDirectoryReader('branch-data',recursive=True).load_data()
index = GPTVectorStoreIndex.from_documents(documents)

retriever = index.as_retriever(retriever_mode='embedding')
query_engine = RetrieverQueryEngine.from_args(retriever, response_mode='tree_summarize')

this code loads my 5MB of text file, took 6 minutes in my macbook pro. And im thinking of using google compute engine to run it much, much faster. Can anyone share what config should i look for? high cpu or gpu?

2 comments

LLogan M

Which part of the code was slow though? I have a feeling just the loading was slow, but that should only happen once in a normal application

If you are looking at different servers though, you probably want a better CPU unless you are running the LLM/Embeddings models yourself

SSenna

Yes, the loading part is slow. My app will rely mostly new data. User will load many times, so even that part is important to be fast.

I will not run the LLM myself at the moment, I'll still use openAI models. Thank you for your response @Logan M

Add a reply

Find answers from the community

I m a front end guy and new to this