Hi, I'm having an issue with
SummaryExtractor
returning node summaries attached to the incorrect nodes. The node to which the summary is incorrectly attached seems to be related to the
num_workers
value but even at 1 worker, I still get misplaced summaries, my code is:
pipeline = IngestionPipeline(
transformations=[
MyCustomNodeTransformer(),
SummaryExtractor(
llm=llm,
metadata_mode=MetadataMode.NONE,
prompt_template=SUMMARY_EXTRACT_TEMPLATE,
num_workers=1,
summaries=["self"],
),
Settings.embed_model,
]
)
nodes = pipeline.run(
show_progress=True,
nodes=nodes,
num_workers=1,
)