My /ingest endpoint knew too much about validation, content hashes, PostgreSQL, S3, BullMQ and rollback. I moved these rules into a dedicated Capture Intake module without changing behavior.
I rebuilt the worker so that one capture produces multiple token chunks, each chunk is embedded individually, and storage in Qdrant remains completely isolated.
I wanted to make the path from the extension to storage visible. With Bull Board and Dozzle, I have queue status and container logs directly in the browser and can identify failures much faster.
POST /ingest blocked the extension until embedding generation and the Qdrant upsert finished. BullMQ and Redis make ingestion asynchronous: an immediate 202, background processing and status queries through GET /captures/:id/status.
Describe your workflow. Together, we will identify the right next step.
Discuss your workflow