Find answers from the community

Updated 5 months ago

Sentence parsing

At a glance

The community member is having an issue where "partial" sentences are being retrieved, but the rest of the sentence is missing, resulting in a lack of useful context in the final LLM call. The other community members try to help by suggesting potential causes, such as it being a truncation issue or a problem with a custom docx reader. However, they are unable to reproduce the issue consistently and do not have a clear solution. The discussion ends with the community members acknowledging that they are unsure of the root cause of the problem.

The problem I am having is that these "partial" sentences are being retrieved...but missing the rest of the sentence...so then no useful context ends up actually being provided in the final LLM call
L
i
13 comments
That's... super odd πŸ€” it's not just something that's getting cut off in pycharms ide right? do you have a file you can share that reproduces this?
opened in TextEdit just in case
Attachment
Screen_Shot_2023-08-27_at_8.49.22_AM.png
I thought this was maybe some intentional truncating behaviour πŸ˜…
It shouldn't be πŸ˜…

Thanks for the source documents. I'll see if I can reproduce in a bit πŸ‘
hmmm are you sure that's the right syllabus? There's no google maps links in the one sent

Although the node parser looks fine for me, at least on my end. No truncation

Using all defaults except window size, loaded the file with the default docx loader
it's a hyperlink being extracted out by a custom docx reader πŸ™‚

I couldn't reproduce it myself this morning. If I can again, I will let you know. Honestly no clue how I reproduced it for an hour straight yesterday but not today.
And I did confirm it is not the docx reader adding those ellipses
so, no clue. but whatever..
:pepeSus: lol good to know!
Add a reply
Sign up and join the conversation on Discord