MailHub Delivery Webhooks Design

Goal

Let each user push terminal email delivery results (sent, bounced, failed) to external business systems via signed HTTPS callbacks, with an observable SQLite-backed delivery queue (retry, logs, manual replay). Account-level endpoints are the default; domains may define override endpoints.

Product Decisions

Item	Choice
Primary use case	Notify business systems of delivery outcomes
Events (MVP)	Terminal only: `sent`, `bounced`, `failed`
Scope	Account default + optional domain override
Reliability	In-process worker + SQLite queue, exponential backoff, delivery log, manual replay
Endpoints	Multiple URLs per account and per domain; each subscribes to a subset of events
Stack fit	Node app + SQLite; no Redis / external broker

Non-Goals (MVP)

Intermediate events (queued, deferred)
Per-recipient split payloads beyond fields already on send_events
Workflow builders / conditional routing UI
Guaranteed exactly-once delivery (at-least-once with idempotent delivery keys)
Redis / BullMQ / separate worker service
Webhook for DNS health or admin audit events

Architecture

send_event reaches terminal status
        │
        ▼
 enqueueWebhookDeliveries(sendEvent)
   resolve targets (domain override vs account)
   insert webhook_deliveries (pending), idempotent
        │
        ▼
 webhook worker (interval inside app process)
   claim due rows → HTTP POST signed body
   success | schedule retry | mark dead

Trigger points

Whenever send_events.status becomes or is set to sent | bounced | failed:

updateSendEventDelivery (Postfix tracker) when computed nextStatus is terminal and previous status was not the same terminal status.
Direct failure/success writes from mailer / submission / API send paths that set terminal status.

Enqueue is synchronous DB insert only (must not await outbound HTTP). Worker performs HTTP.

Process model

Worker starts with HTTP server lifecycle (e.g. after listen).
Poll interval: ~5–15s (constant).
Concurrency: small fixed pool (e.g. 2–3 in-flight POSTs) to protect local Node and remote endpoints.
On process restart, pending/retryable rows remain in SQLite and resume via next_attempt_at.

Resolution Rules

Given userId, domainId, eventType ∈ {sent, bounced, failed}:

Load enabled webhooks for this user where domain_id = domainId and events includes eventType.
If any such domain-scoped webhooks exist → use only those (override account).
Else load enabled account-scoped webhooks (domain_id IS NULL) that subscribe to eventType.
Skip disabled or unsubscribed endpoints.
For each selected webhook, insert a delivery row if not already present for
(webhook_id, send_event_id, event_type).

Data Model

`webhooks`

Column	Type	Notes
id	INTEGER PK
user_id	INTEGER NOT NULL	FK users
domain_id	INTEGER NULL	NULL = account-level; else domain override
name	TEXT NOT NULL	Display label
url	TEXT NOT NULL	HTTPS endpoint
secret_ciphertext	TEXT NOT NULL	Encrypted, same pattern as DNS/SMTP secrets
secret_prefix	TEXT NOT NULL	For UI display
events_json	TEXT NOT NULL	JSON array subset of sent/bounced/failed
enabled	TEXT/INTEGER	boolean
created_at / updated_at	TEXT	ISO

Indexes: (user_id), (user_id, domain_id).

`webhook_deliveries`

Column	Type	Notes
id	INTEGER PK	Public id may be exposed as `whd_{id}`
webhook_id	INTEGER NOT NULL
user_id	INTEGER NOT NULL	Denormalized for isolation queries
send_event_id	INTEGER NOT NULL
event_type	TEXT NOT NULL	sent \| bounced \| failed
payload_json	TEXT NOT NULL	Exact body bytes basis (JSON text)
status	TEXT NOT NULL	pending \| success \| failed \| dead
attempt_count	INTEGER NOT NULL DEFAULT 0
next_attempt_at	TEXT NOT NULL
last_attempt_at	TEXT
response_status	INTEGER
response_body_preview	TEXT	Truncated (~1–2KB)
error	TEXT
created_at	TEXT

Constraints / indexes:

UNIQUE (webhook_id, send_event_id, event_type) for enqueue idempotency
(status, next_attempt_at) for worker
(user_id, created_at DESC) for UI logs

On webhook delete: CASCADE or mark deliveries orphaned—prefer ON DELETE CASCADE for simplicity.

Retry Policy

Attempt after first failure	Delay (approx.)
1	1 minute
2	5 minutes
3	30 minutes
4	2 hours
5	6 hours
6+	12 hours

Max attempts: 8 (including first try). Then status = dead.
Success: HTTP 2xx.
Manual replay: set status=pending, attempt_count=0, next_attempt_at=now, clear last error (keep historical response fields or clear—prefer clear error + allow new attempts; do not change payload_json so signature body stays consistent for that delivery identity; optional: create a new delivery row for replay to preserve history—MVP: mutate same row for simplicity).

HTTP Contract

Request

Method: POST
Headers:
- Content-Type: application/json
- User-Agent: MailHub-Webhook/1.0
- X-MailHub-Signature: t=<unix_seconds>,v1=<hex_hmac>
- X-MailHub-Event: email.sent|email.bounced|email.failed
- X-MailHub-Delivery: whd_<id>
Timeout: 10 seconds
Redirects: do not follow

Body

{
  "id": "whd_42",
  "type": "email.sent",
  "created_at": "2026-07-09T12:00:00.000Z",
  "data": {
    "message_id": "mh-42",
    "send_event_id": 42,
    "queue_id": "A1B2C3D4E5",
    "status": "sent",
    "domain": "example.com",
    "from": "noreply@example.com",
    "to": ["user@example.com"],
    "subject": "Hello",
    "detail": "optional status detail",
    "delivered_at": "2026-07-09T12:00:01.000Z"
  }
}

Internal status	`type`
sent	`email.sent`
bounced	`email.bounced`
failed	`email.failed`

Signature

Signed string: {t}.{raw_body} where raw_body is the exact JSON string POSTed.

v1 = hex(HMAC_SHA256(secret, signed_string))

Receivers should reject if |now - t| > 300 seconds (document 5-minute skew).

Secret: generated on create / rotate; returned once in API response; stored encrypted; list UI shows prefix only.

SSRF protections

Default: only https: URLs.
Optional env flag (e.g. WEBHOOK_ALLOW_HTTP_LOCAL=1) for http://127.0.0.1 / localhost in development.
After DNS resolve, block private, loopback, link-local, and metadata IPs (document exceptions for local flag).
Reject non-HTTP(S) schemes.

Admin API

All routes require authenticated session (or existing API auth pattern used by other user-scoped CRUD). Resources always filtered by user_id.

Method	Path	Behavior
GET	`/api/webhooks`	List; query `domainId` optional
POST	`/api/webhooks`	Create; body: name, url, events[], domainId?, enabled; response includes `secret` once
PATCH	`/api/webhooks/:id`	Update name, url, events, enabled, domainId (not secret)
POST	`/api/webhooks/:id/rotate-secret`	New secret, return once
DELETE	`/api/webhooks/:id`	Delete
POST	`/api/webhooks/:id/test`	Enqueue synthetic delivery for first subscribed event or `sent`
GET	`/api/webhook-deliveries`	List with filters: status, webhookId, eventType, limit
POST	`/api/webhook-deliveries/:id/replay`	Replay failed/dead/success (re-queue)

Validation: URL format + SSRF check on create/update; events non-empty subset of allowed three; domainId must belong to user when set.

UI

Global Webhooks page (replace Placeholder)

Table: name, scope (Account / domain name), URL (truncated), events tags, enabled switch, last delivery status snippet
Create / edit drawer
After create / rotate: modal with full secret + copy (warning: shown once)
Actions: test, open deliveries filtered by webhook, delete
Deliveries panel or sub-view: status, event, attempts, HTTP code, error, replay button

Domain detail → Webhooks tab

Same CRUD scoped to current domainId (override endpoints)
Short help text: “If this domain has any enabled endpoints for an event, account-level endpoints for that event are skipped.”

Reuse existing design system: PageHeader, SectionCard, StatusPill, CodeBlock for secret / sample payload docs.

Code Modules

Module	Responsibility
`src/webhook-model.js`	Pure: event map, resolve list, payload build, sign, backoff, URL validation helpers
`src/webhook-dispatcher.js`	Worker loop, fetch, SSRF resolve checks, status updates
`src/db.js`	Schema, CRUD, enqueue helper
`src/server.js`	Routes; call enqueue after terminal status transitions
`src/pages/Webhooks.tsx` (+ small components)	Global UI
Domain detail tab	Wire domain-scoped list

Testing

Unit: resolve rules (domain override vs account), signature stable, backoff schedule, idempotent enqueue key, event type mapping
DB: CRUD isolation by user; unique delivery constraint
Dispatcher: mock fetch—2xx success; 500 retries; timeout; SSRF rejected URL never fetched
Server API: auth boundary; secret not returned on list

Delivery Order

Schema + db CRUD + model pure functions + tests
Enqueue hooks on terminal status
Dispatcher worker + SSRF
REST API
Global Webhooks UI
Domain detail tab
i18n + build gate

Success Criteria

Terminal status change creates at most one delivery row per webhook endpoint
Domain override suppresses account endpoints for that event
Failed POSTs retry with backoff and appear in UI
Manual replay works
Secrets encrypted at rest; shown once on create/rotate
npm test / npm run build pass

Open Follow-ups (out of MVP)

queued / deferred events
Multiple concurrent endpoints without override (always merge account+domain)
Dead-letter alerts / email to owner
Signing key versioning (v2=)
OpenAPI doc page for receivers

2026-07-09-mailhub-webhooks-design.md 10 KB History Raw