----------------------
Based on the information provided in the knowledge sources, it seems like you are trying to load files from an S3 bucket and convert them into
Document objects using LlamaIndex.
The
S3Reader class in LlamaIndex provides a method
load_s3_files_as_docs which loads files from S3 and returns a list of
Document objects. Here is the source code for this method:
def load_s3_files_as_docs(self, temp_dir=None) -> List[Document]:
"""Load file(s) from S3."""
loader = self._get_simple_directory_reader()
return loader.load_data()(Source:
LlamaIndex S3Reader Documentation ())
If you are getting a list of dictionaries instead of
Document objects, it's possible that the data loader you are using is not correctly converting the data into
Document objects.
You can manually convert your data into
Document objects using the
Document struct exposed by LlamaIndex. Here is an example of how to do this:
from llama_index.core import Document
text_list = [text1, text2, ...] # replace with your list of texts
documents = [Document(text=t) for t in text_list]
(Source:
LlamaIndex Document Creation Documentation ())
Please replace
text1, text2, ... with your actual texts. This will create a list of
Document objects from your list of texts.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
