Find answers from the community

s
F
Y
a
P
Updated last month

can u save openai embedding Im working

can u save openai embedding. Im working with Langchain and i dont want to keep emedding the same 300pg book over and over. Can i save it to disk like and index some how?
j
K
2 comments
depends on the index you're using, if you're using simple vector index you can do save_to_disk and the entire index + nodes + embeddings will be saved
I was looking at Langchain Demo that pulls sources, from langchain.embeddings.openai import OpenAIEmbeddings from langchain.embeddings.cohere import CohereEmbeddings from langchain.text_splitter import CharacterTextSplitter from langchain.vectorstores.elastic_vector_search import ElasticVectorSearch from langchain.vectorstores import Chroma from langchain.docstore.document import Document from langchain.prompts import PromptTemplate with open('../../state_of_the_union.txt') as f: state_of_the_union = f.read() text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0) texts = text_splitter.split_text(state_of_the_union) embeddings = OpenAIEmbeddings()
I want to save that embedding. Im using the save_to_disk in other parts of the project thats why came here looking for help. Can i save that just as easily as i do indeces
Add a reply
Sign up and join the conversation on Discord