Find answers from the community

Home
Members
ToMa1785
T
ToMa1785
Offline, last seen 3 months ago
Joined September 25, 2024
Hi there, is there way using llama-index to summarize a text ? Basically i have list of texts and i want to loop through them to get an overall summary. For example I have texts as follows

I want to have a summarization block before document querying

text = ['paragraph1', 'pargaraph2'.......]
6 comments
T
W
T
ToMa1785
·

Import

Is this import working ? from llama_index.llms import HuggingFaceLLM
i tried to install !pip install llama-index-llms-huggingface
but it still shows some error. I think recently some libraries were migrated
55 comments
k
W
T
Hello community. I am trying to build a RAG application focussing on QA, summarization for research papers and also articles. So far i have experimented with query retrieval for simple text documents. I realize the problem with research papers is that it has many other features such as diagrams, tables, images on top of normal text. I am wondering what is the best way forward
  1. Is it to somehow preprocess the data for example lets say there is a document 1.pdf to something like 1.txt, 1,jpg ,1_table and so on ?
  2. Create seperate nodes - one for text, images, tables and text from webpages ?
That is how i am thinking on a very high level , i guess there would be many intermediate steps. I guess this problem has been already been tackled and was wondering for the best practices here.

I am currently using Mistral-7B-instructv2 basemodel (quantized) with gte embeddings and working in google colab enviroment for now. So totally open source approach . Any tips, notebooks would be highly appreciated.
5 comments
T
L