# MailHub Delivery Webhooks Design

## Goal

Let each user push **terminal email delivery results** (`sent`, `bounced`, `failed`) to external business systems via signed HTTPS callbacks, with an observable SQLite-backed delivery queue (retry, logs, manual replay). Account-level endpoints are the default; domains may define override endpoints.

## Product Decisions

| Item | Choice |
|------|--------|
| Primary use case | Notify business systems of delivery outcomes |
| Events (MVP) | Terminal only: `sent`, `bounced`, `failed` |
| Scope | Account default + optional domain override |
| Reliability | In-process worker + SQLite queue, exponential backoff, delivery log, manual replay |
| Endpoints | Multiple URLs per account and per domain; each subscribes to a subset of events |
| Stack fit | Node app + SQLite; no Redis / external broker |

## Non-Goals (MVP)

- Intermediate events (`queued`, `deferred`)
- Per-recipient split payloads beyond fields already on `send_events`
- Workflow builders / conditional routing UI
- Guaranteed exactly-once delivery (at-least-once with idempotent delivery keys)
- Redis / BullMQ / separate worker service
- Webhook for DNS health or admin audit events

## Architecture

```text
send_event reaches terminal status
        │
        ▼
 enqueueWebhookDeliveries(sendEvent)
   resolve targets (domain override vs account)
   insert webhook_deliveries (pending), idempotent
        │
        ▼
 webhook worker (interval inside app process)
   claim due rows → HTTP POST signed body
   success | schedule retry | mark dead
```

### Trigger points

Whenever `send_events.status` becomes or is set to `sent` | `bounced` | `failed`:

1. `updateSendEventDelivery` (Postfix tracker) when computed `nextStatus` is terminal **and** previous status was not the same terminal status.
2. Direct failure/success writes from mailer / submission / API send paths that set terminal status.

Enqueue is **synchronous DB insert only** (must not await outbound HTTP). Worker performs HTTP.

### Process model

- Worker starts with HTTP server lifecycle (e.g. after listen).
- Poll interval: ~5–15s (constant).
- Concurrency: small fixed pool (e.g. 2–3 in-flight POSTs) to protect local Node and remote endpoints.
- On process restart, pending/retryable rows remain in SQLite and resume via `next_attempt_at`.

## Resolution Rules

Given `userId`, `domainId`, `eventType` ∈ {sent, bounced, failed}:

1. Load **enabled** webhooks for this user where `domain_id = domainId` and `events` includes `eventType`.
2. If any such domain-scoped webhooks exist → **use only those** (override account).
3. Else load **enabled** account-scoped webhooks (`domain_id IS NULL`) that subscribe to `eventType`.
4. Skip disabled or unsubscribed endpoints.
5. For each selected webhook, insert a delivery row if not already present for  
   `(webhook_id, send_event_id, event_type)`.

## Data Model

### `webhooks`

| Column | Type | Notes |
|--------|------|--------|
| id | INTEGER PK | |
| user_id | INTEGER NOT NULL | FK users |
| domain_id | INTEGER NULL | NULL = account-level; else domain override |
| name | TEXT NOT NULL | Display label |
| url | TEXT NOT NULL | HTTPS endpoint |
| secret_ciphertext | TEXT NOT NULL | Encrypted, same pattern as DNS/SMTP secrets |
| secret_prefix | TEXT NOT NULL | For UI display |
| events_json | TEXT NOT NULL | JSON array subset of sent/bounced/failed |
| enabled | TEXT/INTEGER | boolean |
| created_at / updated_at | TEXT | ISO |

Indexes: `(user_id)`, `(user_id, domain_id)`.

### `webhook_deliveries`

| Column | Type | Notes |
|--------|------|--------|
| id | INTEGER PK | Public id may be exposed as `whd_{id}` |
| webhook_id | INTEGER NOT NULL | |
| user_id | INTEGER NOT NULL | Denormalized for isolation queries |
| send_event_id | INTEGER NOT NULL | |
| event_type | TEXT NOT NULL | sent \| bounced \| failed |
| payload_json | TEXT NOT NULL | Exact body bytes basis (JSON text) |
| status | TEXT NOT NULL | pending \| success \| failed \| dead |
| attempt_count | INTEGER NOT NULL DEFAULT 0 | |
| next_attempt_at | TEXT NOT NULL | |
| last_attempt_at | TEXT | |
| response_status | INTEGER | |
| response_body_preview | TEXT | Truncated (~1–2KB) |
| error | TEXT | |
| created_at | TEXT | |

Constraints / indexes:

- **UNIQUE** `(webhook_id, send_event_id, event_type)` for enqueue idempotency
- `(status, next_attempt_at)` for worker
- `(user_id, created_at DESC)` for UI logs

On webhook delete: CASCADE or mark deliveries orphaned—prefer **ON DELETE CASCADE** for simplicity.

## Retry Policy

| Attempt after first failure | Delay (approx.) |
|-----------------------------|-----------------|
| 1 | 1 minute |
| 2 | 5 minutes |
| 3 | 30 minutes |
| 4 | 2 hours |
| 5 | 6 hours |
| 6+ | 12 hours |

- Max attempts: **8** (including first try). Then `status = dead`.
- Success: HTTP **2xx**.
- Manual **replay**: set `status=pending`, `attempt_count=0`, `next_attempt_at=now`, clear last error (keep historical response fields or clear—prefer clear error + allow new attempts; do not change `payload_json` so signature body stays consistent for that delivery identity; optional: create a new delivery row for replay to preserve history—**MVP: mutate same row for simplicity**).

## HTTP Contract

### Request

- Method: `POST`
- Headers:
  - `Content-Type: application/json`
  - `User-Agent: MailHub-Webhook/1.0`
  - `X-MailHub-Signature: t=<unix_seconds>,v1=<hex_hmac>`
  - `X-MailHub-Event: email.sent|email.bounced|email.failed`
  - `X-MailHub-Delivery: whd_<id>`
- Timeout: 10 seconds
- Redirects: do not follow

### Body

```json
{
  "id": "whd_42",
  "type": "email.sent",
  "created_at": "2026-07-09T12:00:00.000Z",
  "data": {
    "message_id": "mh-42",
    "send_event_id": 42,
    "queue_id": "A1B2C3D4E5",
    "status": "sent",
    "domain": "example.com",
    "from": "noreply@example.com",
    "to": ["user@example.com"],
    "subject": "Hello",
    "detail": "optional status detail",
    "delivered_at": "2026-07-09T12:00:01.000Z"
  }
}
```

| Internal status | `type` |
|-----------------|--------|
| sent | `email.sent` |
| bounced | `email.bounced` |
| failed | `email.failed` |

### Signature

Signed string: `{t}.{raw_body}` where `raw_body` is the exact JSON string POSTed.

```text
v1 = hex(HMAC_SHA256(secret, signed_string))
```

Receivers should reject if `|now - t| > 300` seconds (document 5-minute skew).

Secret: generated on create / rotate; returned **once** in API response; stored encrypted; list UI shows prefix only.

### SSRF protections

- Default: only `https:` URLs.
- Optional env flag (e.g. `WEBHOOK_ALLOW_HTTP_LOCAL=1`) for `http://127.0.0.1` / localhost in development.
- After DNS resolve, block private, loopback, link-local, and metadata IPs (document exceptions for local flag).
- Reject non-HTTP(S) schemes.

## Admin API

All routes require authenticated session (or existing API auth pattern used by other user-scoped CRUD). Resources always filtered by `user_id`.

| Method | Path | Behavior |
|--------|------|----------|
| GET | `/api/webhooks` | List; query `domainId` optional |
| POST | `/api/webhooks` | Create; body: name, url, events[], domainId?, enabled; response includes `secret` once |
| PATCH | `/api/webhooks/:id` | Update name, url, events, enabled, domainId (not secret) |
| POST | `/api/webhooks/:id/rotate-secret` | New secret, return once |
| DELETE | `/api/webhooks/:id` | Delete |
| POST | `/api/webhooks/:id/test` | Enqueue synthetic delivery for first subscribed event or `sent` |
| GET | `/api/webhook-deliveries` | List with filters: status, webhookId, eventType, limit |
| POST | `/api/webhook-deliveries/:id/replay` | Replay failed/dead/success (re-queue) |

Validation: URL format + SSRF check on create/update; events non-empty subset of allowed three; domainId must belong to user when set.

## UI

### Global Webhooks page (replace Placeholder)

- Table: name, scope (Account / domain name), URL (truncated), events tags, enabled switch, last delivery status snippet
- Create / edit drawer
- After create / rotate: modal with full secret + copy (warning: shown once)
- Actions: test, open deliveries filtered by webhook, delete
- Deliveries panel or sub-view: status, event, attempts, HTTP code, error, replay button

### Domain detail → Webhooks tab

- Same CRUD scoped to current `domainId` (override endpoints)
- Short help text: “If this domain has any enabled endpoints for an event, account-level endpoints for that event are skipped.”

Reuse existing design system: PageHeader, SectionCard, StatusPill, CodeBlock for secret / sample payload docs.

## Code Modules

| Module | Responsibility |
|--------|----------------|
| `src/webhook-model.js` | Pure: event map, resolve list, payload build, sign, backoff, URL validation helpers |
| `src/webhook-dispatcher.js` | Worker loop, fetch, SSRF resolve checks, status updates |
| `src/db.js` | Schema, CRUD, enqueue helper |
| `src/server.js` | Routes; call enqueue after terminal status transitions |
| `src/pages/Webhooks.tsx` (+ small components) | Global UI |
| Domain detail tab | Wire domain-scoped list |

## Testing

- Unit: resolve rules (domain override vs account), signature stable, backoff schedule, idempotent enqueue key, event type mapping
- DB: CRUD isolation by user; unique delivery constraint
- Dispatcher: mock fetch—2xx success; 500 retries; timeout; SSRF rejected URL never fetched
- Server API: auth boundary; secret not returned on list

## Delivery Order

1. Schema + db CRUD + model pure functions + tests  
2. Enqueue hooks on terminal status  
3. Dispatcher worker + SSRF  
4. REST API  
5. Global Webhooks UI  
6. Domain detail tab  
7. i18n + build gate  

## Success Criteria

- Terminal status change creates at most one delivery row per webhook endpoint  
- Domain override suppresses account endpoints for that event  
- Failed POSTs retry with backoff and appear in UI  
- Manual replay works  
- Secrets encrypted at rest; shown once on create/rotate  
- `npm test` / `npm run build` pass  

## Open Follow-ups (out of MVP)

- `queued` / `deferred` events  
- Multiple concurrent endpoints without override (always merge account+domain)  
- Dead-letter alerts / email to owner  
- Signing key versioning (`v2=`)  
- OpenAPI doc page for receivers