Log in
Log into community
Find answers from the community
View all posts
Related posts
Did this answer your question?
๐
๐
๐
Powered by
Hall
Inactive
Updated 8 months ago
0
Follow
llama_index/llama-index-core/llama_index...
llama_index/llama-index-core/llama_index...
Inactive
0
Follow
d
drewskidang
8 months ago
ยท
does nayone know how to custom models for PII masking
https://github.com/run-llama/llama_index/blob/b430d1757081e1944474bf281cd757e69431cc07/llama-index-core/llama_index/core/postprocessor/pii.py#L96
i can find where to pass the bert model and i dont know how the default defaulted to dbmdz/bert-large-cased-finetuned-conll03-english and revision f2482bf (
https://huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english
). is being supplse
L
d
5 comments
Share
Open in Discord
L
Logan M
8 months ago
I think its hardcoded
Plain Text
Copy
from transformers import pipeline # pants: no-infer-dep ner = pipeline("ner", grouped_entities=True)
L
Logan M
8 months ago
This just loads the model huggingface thinks is best for ner
d
drewskidang
8 months ago
i created a custom node processor
L
Logan M
8 months ago
yea that works
d
drewskidang
8 months ago
I thinks its better than the default
Attachment
Add a reply
Sign up and join the conversation on Discord
Join on Discord