Find answers from the community

Updated 3 months ago

Running the REBEL + wiki KG index,

Running the REBEL + wiki KG index, somehow pytorch error?
Plain Text
File ~/anaconda3/envs/test_cluster2/lib/python3.11/site-packages/transformers/generation/beam_search.py:397, in BeamSearchScorer.finalize(self, input_ids, final_beam_scores, final_beam_tokens, final_beam_indices, max_length, pad_token_id, eos_token_id, beam_indices, decoder_prompt_len)
    395 # fill with hypotheses and eos_token_id if the latter fits in
    396 for i, (hypo, best_idx) in enumerate(zip(best, best_indices)):
--> 397     decoded[i, : sent_lengths[i]] = hypo
    399     if indices is not None:
    400         indices[i, : len(best_idx)] = torch.tensor(best_idx)

RuntimeError: The expanded size of the tensor (200) must match the existing size (38) at non-singleton dimension 0.  Target sizes: [200].  Tensor sizes: [38]
O
L
d
83 comments
here is the error message of llamaindex part
Plain Text
File ~/anaconda3/envs/test_cluster2/lib/python3.11/site-packages/llama_index/indices/base.py:179, in BaseIndex.build_index_from_nodes(self, nodes)
    177 """Build the index from nodes."""
    178 self._docstore.add_documents(nodes, allow_update=True)
--> 179 return self._build_index_from_nodes(nodes)

File ~/anaconda3/envs/test_cluster2/lib/python3.11/site-packages/llama_index/indices/knowledge_graph/base.py:176, in KnowledgeGraphIndex._build_index_from_nodes(self, nodes)
    172 nodes_with_progress = get_tqdm_iterable(
    173     nodes, self._show_progress, "Processing nodes"
    174 )
    175 for n in nodes_with_progress:
--> 176     triplets = self._extract_triplets(
    177         n.get_content(metadata_mode=MetadataMode.LLM)
    178     )
    179     logger.debug(f"> Extracted triplets: {triplets}")
    180     for triplet in triplets:

File ~/anaconda3/envs/test_cluster2/lib/python3.11/site-packages/llama_index/indices/knowledge_graph/base.py:118, in KnowledgeGraphIndex._extract_triplets(self, text)
    116 def _extract_triplets(self, text: str) -> List[Tuple[str, str, str]]:
    117     if self._kg_triplet_extract_fn is not None:
--> 118         return self._kg_triplet_extract_fn(text)
    119     else:
    120         return self._llm_extract_triplets(text)
No idea on this one -- it's just passing the text to the extract fn, and the extract fn is doing the rest

Maybe a text length issue?
let me try again
can't figure out, very odd,
literally copied and pasted from documents, and inputted a pdf, using unstructured reader
I will give up for now, circle back later
@Logan M another problem,
Plain Text
File ~/anaconda3/envs/cluster_test2/lib/python3.11/site-packages/llama_index/indices/query/embedding_utils.py:31, in get_top_k_embeddings(query_embedding, embeddings, similarity_fn, similarity_top_k, embedding_ids, similarity_cutoff)
     29 similarity_heap: List[Tuple[float, Any]] = []
     30 for i, emb in enumerate(embeddings_np):
---> 31     similarity = similarity_fn(query_embedding_np, emb)
     32     if similarity_cutoff is None or similarity > similarity_cutoff:
     33         heapq.heappush(similarity_heap, (similarity, embedding_ids[i]))

File ~/anaconda3/envs/cluster_test2/lib/python3.11/site-packages/llama_index/core/embeddings/base.py:48, in similarity(embedding1, embedding2, mode)
     46     return np.dot(embedding1, embedding2)
     47 else:
---> 48     product = np.dot(embedding1, embedding2)
     49     norm = np.linalg.norm(embedding1) * np.linalg.norm(embedding2)
     50     return product / norm

ValueError: shapes (1536,) and (3072,) not aligned: 1536 (dim 0) != 3072 (dim 0)
so I used the new embedding model, and somehow aquery doesn't work
ah, sorted out, need to add service context
wow, recursive indices are so big
has like 1 G from a few MB of codes
A few MB of text is still a decent amount. Id be curious how many nodes that created (since each node needs a fairly large embedding vector)
lionagi codebase is 1.5MB, its storage is 423MB....
and it has 5863 nodesw
maybe I chunk it too small?
Hmm, I don't think that's too many nodes. Each node is 1536*4bytes (plus whatever size the text of the node is)

When you say recursive indices, what did you setup?
like the small to big pack with code splitter plus the new embedding
@Logan M I am considering adding llama_index as a dependency of lionagi, the query engines are just so handy to have. Question, what features do you guys plan to "keep"? (less likely for me to go back and change everything again?, i mean serviceContext was just removed)
the index object itself should be relatively stable right?
Yes, the VectorStoreIndex, SummaryIndex are pretty set in stone
Same with the ingestion pipeline
ok, what about KGIndex, and I see that llamaHub got moved to llama_index as well, I am wondering why did you guys make this choice
We wanted all integrations in a single repo. Centralize all the github issues and PRs. Centralize all testing and CICD. I quite like the new monorepo setup πŸ™‚
KGIndex is staying for now, there is plans to introduce a LabelledPropertyGraphIndex, which may take the place of KGIndex. But KGIndex will likely stay in the codebase. (this is lower priority at the moment)
I see, yeah monorepo makes sense, idk how you handle so many repos
It was definitely a hurdle. Getting unit tests to work across 400+ packages was fun πŸ™‚
Glad we got it out!
@Logan M @jerryjliu0 question, what do you guys think of the idea of using vector index and pandas query engine to do "unlimited" context length in lionagi. Basically compress the tokens, and use RAG to inject additional context as needed. like we use pd.df for message storing already, this doesn't sound too far fetched
could work yea! pandas query engine can be a tad unreliable, so it might need some TLC
maybe do twice? one index to know what rows to check out, then pandas query fetch that specific row
I noticed pandas query can only do one command at a time, more than that, it's just not doing it
tender love and care haha basically it needs some attention probably in terms of implementation.
will give it a go, starting with just regular query engine
Plain Text
from llama_index.core.llms import ChatMessage
from llama_index.llms.anthropic import Anthropic

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]
resp = Anthropic().chat(messages)
what does ChatMessage do, I checked, it's a BaseModel, with only str method
so can I pass a list of messages instead?
I mean strings
is there a way I can do this without using lower level object?
When using the LLM directly, thats the simplest it gets

llm.complete() accepts a single string if you don't need a system prompt
I ended up doing a parser, is there a place I can set rate limit?
also the new repo is a lot of digging around
I think it just takes some learning curver to figure out where things are. ctrl-f is your friend πŸ™‚
A rate limit for the LLM? or what is causing rate limit issues?
it's just lionagi has this feature to set
rate limit, so i'm wondering is there a place I can find complete response?
it's just a lot to change, I think I should probably just use llamaindex directly
i was like, why don't I just use yours,
hmm I don't think we have any specific rate limiting features at the moment beyond what you can set on the openai client directly.
been meaning to add tenancity and make it configurable, but haven't seen huge demand for it yet
yeah we focus only async, so kinda like as much data throughput as possible, but yeah no demand yet for other models
can I make it into a feature request? : )
Hi, question. once I pip install let's say llama-index-readers-ABC, what's the usage pattern? and which package should I check and install if import is not successful?
from llama_index.readers.ABC import ABCReader ? is it always namd this way? so I can just getattr(llama_index.readers, 'ABCReader')?
It is always named that way πŸ™‚ Or at least it should be -- llamahub should have all the actual module names
ok, it is that I saw download_reader will be depreciated, so wanna update to new syntax
I mainly want a wrapper around all readers so I can have a easier time with pip
or maybe you guys can rename that to install_reader? change the implementation to do pip install?
Actually it is doing pip install lol
Attachment
image.png
Plain Text
def get_llama_index_reader(reader: BaseReader | str = None) -> BaseReader:
    if not isinstance(reader, [str, BaseReader]):
        raise TypeError(f'reader must be a string or BaseReader, not {type(reader)}')

    if isinstance(reader, str):
        if 'index' in reader:
            reader_ = reader.split('index')[-1]
            reader_ = strip_lower(reader_.replace("_", "-"))
            if reader_.startswith('-'):
                reader_ = reader_[1:]
            package = f'llama-index-reader-{reader_}'
            reader = f"{reader_}Reader"
            
            ...
it's I want user to just input the reader name, without giving the full pip install name
is the pip_install function there already does that?
A step before that code above, it converts the Module/Class name to the full pip install string
cool, will this still get depreciated ?
probably someday -- maintaining that mappings.json isn't fun haha
ok, i will try just to parse that myself then, if failed, just ask user to input a reader object directly
Plain Text
packages/llama_index/llms/openai/base.py:36
     29 from llama_index.core.constants import (
     30     DEFAULT_TEMPERATURE,
     31 )
     32 from llama_index.core.llms.callbacks import (
     33     llm_chat_callback,
     34     llm_completion_callback,
     35 )
---> 36 from llama_index.core.llms.generic_utils import (
     37     achat_to_completion_decorator,
     38     acompletion_to_chat_decorator,
     39     astream_chat_to_completion_decorator,
     40     astream_completion_to_chat_decorator,
     41     chat_to_completion_decorator,
     42     completion_to_chat_decorator,
     43     stream_chat_to_completion_decorator,
     44     stream_completion_to_chat_decorator,
     45 )
     46 from llama_index.core.llms.llm import LLM
     47 from llama_index.core.types import BaseOutputParser, PydanticProgramMode

ModuleNotFoundError: No module named 'llama_index.core.llms.generic_utils'
verison is 0.10.8
try pip install -U llama-index-llms-openai llama-index-embeddings-openai llama-index-core
Had a weird circular import issue this morning
thanks, will try
Wow.
Attachment
image.png
why is it 10k LOL we have like 400 max
@Logan M third-party packages, me thinks
@Logan M how do i delete certain nodes of an index loaded from storage
index.delete_ref_doc(ref_doc_id) will delete all nodes created from a particular ref doc

index.delete_nodes(node_ids) will delete nodes by id (but isn't supported by most vector db integrations )
Add a reply
Sign up and join the conversation on Discord