Find answers from the community

Updated 2 years ago

Possibly a noob python question any tips

Possibly a noob python question: any tips on sharing a gpt index object (not too concerned about the type of index) between python processes?

For context, I'm running a flask application, and I'd like to have multiple workers to serve concurrent requests. I'd like them all to access the same instances of gpt indices that i end up generating, rather than building them multiple times.

6 comments

jjerryjliu0

you could try using a weaviate index since its backed by an underlying store.

jjerryjliu0

but before that, are you sure there's no way to share python objects in memory between processes in flask? (haven't looked super deeply, just asking)

yyourbuddyconner

You'd probably have to serialize the object

yyourbuddyconner

But can save the index to disk and load it iirc which might be a good option depending on the use-case

mmat

but before that, are you sure there's no way to share python objects in memory between processes in flask? (haven't looked super deeply, just asking)

haven't looked deeply into this one either, but i know python's multiprocessing library tries to do something like this. It's not perfect and doesn't seem to have the durability guarantee https://stackoverflow.com/questions/65686318/sharing-python-objects-across-multiple-workers

mmat

Yeah i think serializing for multiple python processes to access may work, and if we create an index once and never modify it again (which applies in some settings) we don't have to worry about consistency (i think?)

Add a reply