Automating the Process of Re-running Ingestion Pipelines

The community member is looking for ways to automate the process of re-running ingestion pipelines, particularly for a small enterprise use-case involving data sources like Google Drive, Airtable, Affinity, and Dropbox. The comments suggest that this is a common problem that can be solved in many ways, such as using a job scheduler or a cron tab with Docker. One community member mentions looking into Pathway and getting it to work with Google Drive, but the responses were poor, leading them to consider a vector database approach or a job scheduler.