Capture Intake as a separate module with less orchestration in the Express endpoint

Contents

• Why I revisited the ingest endpoint
• Where the route knew too much
• Which rules had to stay the same
• The new Capture Intake module
• Why the refactoring helps in practice
• What deliberately did not change
• After testing
• All articles in the series

Why I revisited the ingest endpoint

The capture flow worked. The endpoint accepted the request, wrote the record to PostgreSQL, uploaded the image to S3, queued the job in BullMQ and returned 202. Functionally, that was fine.

Reading the code, I still noticed that routes/ingest.ts knew too much about accepting a capture. The route validated the payload, calculated the content hash, normalized image sizes, handled duplicates, orchestrated the upload and performed rollback on failure. That was too much responsibility in one file for an Express endpoint.

An endpoint like this is convenient at first. Later, it becomes expensive. Every additional rule ends up in the same place. Every new failure case extends the same orchestration. Every change to persistence sends you back to the route.

Where the route knew too much

Before refactoring, the relevant path looked roughly like this:

router.post('/', async (req: Request, res: Response) => {
  const payload = req.body as PlatformAnalysisPayload;
  const userId = req.auth!.userId;
  payload.capturedBy.userId = userId;

  const captureId = crypto.randomUUID();
  const embeddingText = buildEmbeddingText(payload);
  const contentHash = computeContentHash(embeddingText);

  await ensureUser(userId);
  await insertCapture(buildInsertCaptureInput(payload, { id: captureId, userId, contentHash }));

  const blobName = await uploadImage(payload.image.base64, payload.image.mimeType, userId);
  await updateCaptureBlobName(captureId, blobName);
  await ingestionQueue.add('ingest', { captureId, userId }, { jobId: captureId });

  res.status(202).json({ captureId });
});

The code was not wrong. It was in the wrong place. The route was no longer just an HTTP adapter. It already contained much of a domain module.

File size is not the only problem. The real problem is knowledge. Once a route needs to know the order in which PostgreSQL, S3 and BullMQ must be called, the domain rule is no longer concentrated in one place. It has moved to the edge of the system.

Which rules had to stay the same

I did not want the refactoring to introduce hidden behavior changes. Four rules had to remain exactly the same.

1. A duplicate capture still returns 409 with DUPLICATE_CAPTURE.
2. Failed S3 uploads or queue handoffs still trigger complete cleanup.
3. The quota counter remains best effort and does not block a successful capture.
4. JWT validation stays in the auth middleware path rather than moving into the new module.

The second point was particularly important. A refactoring can look clean when you extract cleanup logic. It is only genuinely clean if that logic still enforces the same consistency rules afterward.

The new Capture Intake module

I moved the entire capture-acceptance process into backend/src/services/capture-intake.ts. The route is now thin and only translates HTTP into a module call.

router.post('/', async (req: Request, res: Response) => {
  const payload = req.body as PlatformAnalysisPayload;

  try {
    const { captureId } = await acceptCapture({
      payload,
      userId: req.auth!.userId,
    });

    res.status(202).json({ captureId });
  } catch (error: unknown) {
    if (error instanceof InvalidCapturePayloadError) {
      res.status(400).json({ error: error.message });
      return;
    }

    if (error instanceof DuplicateCaptureError) {
      res.status(409).json({
        error: error.message,
        code: 'DUPLICATE_CAPTURE',
        captureId: error.captureId,
      });
      return;
    }

    const message = error instanceof Error ? error.message : 'Unknown error';
    res.status(500).json({ error: message });
  }
});

The domain rule now lives in acceptCapture():

export async function acceptCapture({
  payload,
  userId,
}: CaptureIntakeInput): Promise<CaptureIntakeResult> {
  validateCapturePayload(payload);

  const captureId = crypto.randomUUID();
  const embeddingText = buildEmbeddingText(capturePayload);
  const contentHash = computeContentHash(embeddingText);

  await ensureUser(userId);
  await insertCapture(...);

  const blobName = await uploadImage(...);
  await updateCaptureBlobName(captureId, blobName);
  await ingestionQueue.add('ingest', { captureId, userId }, { jobId: captureId });

  incrementCaptures(userId).catch(...);

  return { captureId };
}

The important difference is not that there is less code. Capture acceptance now has a named home. If I change its behavior later, I no longer have to search a route handler for scattered persistence logic. I go directly to the module responsible for it.

Why the refactoring helps in practice

For me, the change has three concrete effects.

First, the Express route stays readable. If I only want to understand how HTTP errors become status codes, that is exactly what I see, without half the persistence path alongside it.

Second, the capture path no longer depends on Express. If I later call the same process from another entry point, such as an import or replay path, I do not need request and response objects.

Third, rollback now lives where the risk originates. PostgreSQL, S3 and BullMQ are called in one coherent process. The same place knows how to clean up if one of those steps fails. That is where I want to look when something goes wrong.

What deliberately did not change

I did not make the module more general than it needs to be today. It is not a universal import path. It is not an abstract pipeline engine. It is not a new layer for everything related to ingestion.

The module serves exactly one current caller, POST /ingest. That is enough for this step. The value does not come from maximum flexibility. It comes from keeping one concrete rule together in one place.

I also deliberately preserved the existing order. First, the capture record is created. Next, the image is uploaded. Then the blob name is persisted. Finally, the job enters the queue. Behavior stays stable for the running stack.

After testing

After the change, I clicked through the capture flow again. The request still returned 202, the record was stored and the worker started as before. That was exactly what I wanted to see.

That was the most important point of this refactoring for me. The code became clearer without silently changing the production path.

All articles in the series

1. Vision and system overview: Chrome Extension, RAG architecture and project background: Read article
2. Building a RAG system: Qdrant, embeddings and cosine similarity in TypeScript: Read article
3. AI provider abstraction: Ollama vs. OpenAI, interface design and no vendor lock-in: Read article
4. Chrome Extension MV3: Three isolated runtime contexts, message passing and Strategy Pattern: Read article
5. Docker Compose strategy: The override pattern, from local development to Azure: Read article
6. Ollama locally vs. Docker: The decision and its consequences: Read article
7. Ollama auto-pull entrypoint: Automated model setup at container startup: Read article
8. tsconfig and Vite: Node16 vs. bundler, and why Vite has its own rules: Read article
9. Load complete Instagram captions with MutationObserver: Read article
10. Chrome Extension foundation with health indicator and retry queue: Read article
11. Phase 2 features: Shadow DOM overlay, Tailwind v4 and duplicate detection: Read article
12. Race condition in platform detection: How a UI event breaks Instagram detection: Read article
13. Extracting PostId in two Instagram layouts: querySelector vs. ancestor traversal: Read article
14. Capture a complete Instagram carousel with MutationObserver: Lazy loading, observer-before-click and timeout fallback: Read article
15. Notes and tags when saving screenshots: Read article
16. Instagram keyboard shortcuts block Chrome Extension input: Read article
17. Lowercase normalization and duplicate detection in tag input: Read article
18. Zitadel Login V2 in Docker Compose with three hidden errors: Read article
19. PKCE OAuth in a Chrome MV3 extension: Read article
20. React frontend with react-oidc-context and Zitadel: Read article
21. Vite build-time environment variables in Docker: Read article
22. Event-driven ingestion with BullMQ and Redis: Read article
23. MinIO instead of Azurite: S3-compatible object storage locally and on Hetzner: Read article
24. access_token, id_token and the Userinfo endpoint: Which belongs where: Read article
25. Qdrant multi-tenancy with one collection per user: Read article
26. When backend and frontend know different types: Read article
27. Removing Zitadel bootstrap: Host header bug and manual setup: Read article
28. Backend code review with six problems fixed before launch: Read article
29. Traefik instead of NGINX: Reverse proxy for a growing Docker Compose stack: Read article
30. Two-layer rate limiting: Traefik and express-rate-limit with Redis: Read article
31. Implementing GDPR Article 17 correctly: Promise.allSettled and export batching: Read article
32. Embedding model lock-in: Why mxbai-embed-large is a permanent production decision: Read article
33. Docker volumes in production: Named volumes, bind mounts and the Hetzner volume approach: Read article
34. Two vulnerabilities before launch: Redis without authentication and an exposed Qdrant admin port: Read article
35. Traefik as the single entry point in the Docker Compose stack: Read article
36. Connecting Zitadel behind Traefik with issuer, JWKS and Login V2: Read article
37. Fixing the frontend when the nginx healthcheck fails on localhost: Read article
38. Observability for my Docker Compose stack with Bull Board and Dozzle: Read article
39. Open the Qdrant dashboard securely with local Traefik and an SSH tunnel: Read article
40. Diagnosis of chunking failures despite tokenization: Read article
41. Why I set the chunk size to 1500 tokens: Read article
42. Splitting the embedding workflow into clear steps: Read article
43. Bringing chunking, storage and search back together: Read article
44. Capture Intake as a separate module with less orchestration in the Express endpoint (this article)

Are you working on an Express backend where an endpoint is gradually taking on too much persistence and rollback knowledge? Let us assess it together.