Find answers from the community

Updated 3 months ago

can we increase the numbers of workers

can we increase the numbers of workers for the dataset generator
a
D
3 comments
which dataset generator? I don't believe we've built in multi processing for any of our generators yet
it was the finetune memorize knowledge one
Yeah, I'm pretty sure that doesn't have num_workers argument yet. We def do want to make these dataset generators more efficient in the future tho.
Add a reply
Sign up and join the conversation on Discord