Find answers from the community

Updated 8 months ago

llama_index/llama-index-core/llama_index...

does nayone know how to custom models for PII masking
https://github.com/run-llama/llama_index/blob/b430d1757081e1944474bf281cd757e69431cc07/llama-index-core/llama_index/core/postprocessor/pii.py#L96

i can find where to pass the bert model and i dont know how the default defaulted to dbmdz/bert-large-cased-finetuned-conll03-english and revision f2482bf (https://huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english). is being supplse
L
d
5 comments
I think its hardcoded

Plain Text
from transformers import pipeline  # pants: no-infer-dep

ner = pipeline("ner", grouped_entities=True)
This just loads the model huggingface thinks is best for ner
i created a custom node processor
yea that works
I thinks its better than the default
Attachment
image.png
Add a reply
Sign up and join the conversation on Discord