docs: add delivery webhooks implementation plan

Task breakdown for model, SQLite queue, worker, REST API, and admin UI
aligned with the approved webhook design spec.

AI-Co-Authored-By: Grok

docs/superpowers/plans/2026-07-09-mailhub-webhooks.md

@@ -0,0 +1,506 @@
+# MailHub Delivery Webhooks Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Implement signed HTTPS webhooks for terminal email events (`sent` / `bounced` / `failed`) with account default + domain override, multi-URL subscriptions, and an observable SQLite delivery queue (claim, retry, logs, replay).
+
+**Architecture:** Pure helpers in `webhook-model.js`; CRUD + enqueue in `db.js`; HTTP worker in `webhook-dispatcher.js` started from `server.js`; REST under `/api/webhooks` and `/api/webhook-deliveries`; React pages replace Placeholder and domain Webhooks tab. Secrets use existing `encryptSecret` / `decryptSecret`.
+
+**Tech Stack:** Node.js ESM, SQLite (`node:sqlite`), React + Ant Design admin UI, `node:test`
+
+**Spec:** `docs/superpowers/specs/2026-07-09-mailhub-webhooks-design.md`
+
+---
+
+## File map
+
+| File | Responsibility |
+|------|----------------|
+| Create `src/webhook-model.js` | Event map, resolve targets, payload, HMAC sign, backoff, URL precheck helpers (pure) |
+| Create `src/webhook-dispatcher.js` | Worker loop, claim, SSRF-safe fetch, status updates |
+| Create `test/webhook-model.test.js` | Pure model unit tests |
+| Create `test/webhook-dispatcher.test.js` | Dispatcher with mocked fetch |
+| Modify `src/db.js` | Tables, webhook CRUD, enqueue, claim, replay, list deliveries |
+| Modify `src/server.js` | API routes; start worker; ensure enqueue after terminal writes if not fully inside db |
+| Create `src/pages/Webhooks.tsx` | Global webhooks + deliveries UI |
+| Modify `src/frontend/App.tsx` | Wire Webhooks page, load webhooks data |
+| Modify `src/pages/Domains/DomainDetail.tsx` | Real Webhooks tab |
+| Modify `src/frontend/types.ts` | Webhook / delivery types |
+| Modify `src/frontend/services/api.ts` | Client methods |
+| Modify `src/frontend/i18n/index.js` | zh/en strings |
+| Modify `test/frontend-i18n.test.js` | New keys if asserted |
+| Create `test/server-webhooks-api.test.js` | API auth + CRUD smoke (follow `server-admin-api.test.js` patterns) |
+
+---
+
+### Task 1: Pure webhook model
+
+**Files:**
+- Create: `src/webhook-model.js`
+- Create: `test/webhook-model.test.js`
+
+- [ ] **Step 1: Write failing tests**
+
+```js
+// test/webhook-model.test.js
+import assert from 'node:assert/strict';
+import { test } from 'node:test';
+import {
+  TERMINAL_WEBHOOK_EVENTS,
+  eventTypeForStatus,
+  resolveWebhooksForEvent,
+  buildWebhookPayload,
+  signWebhookBody,
+  nextBackoffMs,
+  isTerminalWebhookStatus
+} from '../src/webhook-model.js';
+
+test('maps terminal statuses to email.* types', () => {
+  assert.equal(eventTypeForStatus('sent'), 'email.sent');
+  assert.equal(eventTypeForStatus('bounced'), 'email.bounced');
+  assert.equal(eventTypeForStatus('failed'), 'email.failed');
+  assert.equal(eventTypeForStatus('queued'), null);
+});
+
+test('domain webhooks override account for the same event', () => {
+  const account = [
+    { id: 1, domainId: null, enabled: true, events: ['sent', 'failed'] },
+    { id: 2, domainId: null, enabled: true, events: ['bounced'] }
+  ];
+  const domain = [
+    { id: 3, domainId: 9, enabled: true, events: ['sent'] }
+  ];
+  const resolved = resolveWebhooksForEvent({
+    accountWebhooks: account,
+    domainWebhooks: domain,
+    eventType: 'sent'
+  });
+  assert.deepEqual(resolved.map((w) => w.id), [3]);
+});
+
+test('falls back to account when domain has no matching enabled subscription', () => {
+  const resolved = resolveWebhooksForEvent({
+    accountWebhooks: [{ id: 1, domainId: null, enabled: true, events: ['failed'] }],
+    domainWebhooks: [{ id: 3, domainId: 9, enabled: true, events: ['sent'] }],
+    eventType: 'failed'
+  });
+  assert.deepEqual(resolved.map((w) => w.id), [1]);
+});
+
+test('signs body with Stripe-style t and v1', () => {
+  const body = '{"id":"whd_1"}';
+  const header = signWebhookBody(body, 'secret', 1_700_000_000);
+  assert.equal(header.startsWith('t=1700000000,v1='), true);
+  assert.match(header, /^t=\d+,v1=[0-9a-f]{64}$/);
+});
+
+test('backoff grows then caps', () => {
+  assert.ok(nextBackoffMs(1) < nextBackoffMs(2));
+  assert.equal(nextBackoffMs(10), nextBackoffMs(20));
+});
+```
+
+- [ ] **Step 2: Run tests — expect FAIL**
+
+Run: `node --test test/webhook-model.test.js`
+
+- [ ] **Step 3: Implement `src/webhook-model.js`**
+
+Export at least:
+
+```js
+export const TERMINAL_WEBHOOK_EVENTS = ['sent', 'bounced', 'failed'];
+export const MAX_WEBHOOK_ATTEMPTS = 8;
+export const WEBHOOK_LEASE_MS = 2 * 60 * 1000;
+
+export function isTerminalWebhookStatus(status) {
+  return TERMINAL_WEBHOOK_EVENTS.includes(status);
+}
+
+export function eventTypeForStatus(status) {
+  if (status === 'sent') return 'email.sent';
+  if (status === 'bounced') return 'email.bounced';
+  if (status === 'failed') return 'email.failed';
+  return null;
+}
+
+/** @param {{ accountWebhooks: any[]; domainWebhooks: any[]; eventType: string }} input */
+export function resolveWebhooksForEvent({ accountWebhooks, domainWebhooks, eventType }) {
+  const matches = (list) => (list || []).filter(
+    (w) => w.enabled !== false && w.enabled !== 'false' && Array.isArray(w.events) && w.events.includes(eventType)
+  );
+  const domainHits = matches(domainWebhooks);
+  if (domainHits.length) return domainHits;
+  return matches(accountWebhooks);
+}
+
+export function buildWebhookPayload({ deliveryId, eventType, createdAt, sendEvent, test = false }) {
+  const status = sendEvent.status;
+  return {
+    id: `whd_${deliveryId}`,
+    type: eventTypeForStatus(status) || eventType,
+    created_at: createdAt,
+    data: {
+      ...(test ? { test: true } : {}),
+      message_id: test ? 'mh-test' : `mh-${sendEvent.id}`,
+      send_event_id: sendEvent.id,
+      queue_id: sendEvent.queueId || '',
+      status,
+      domain: sendEvent.domain || '',
+      from: sendEvent.sender || '',
+      to: sendEvent.recipients || [],
+      subject: sendEvent.subject || '',
+      detail: sendEvent.detail || '',
+      delivered_at: sendEvent.deliveredAt || null
+    }
+  };
+}
+
+export function signWebhookBody(rawBody, secret, unixSeconds = Math.floor(Date.now() / 1000)) {
+  const crypto = awaitImportOrRequireCrypto(); // use import crypto from 'node:crypto' at top
+  const signed = `${unixSeconds}.${rawBody}`;
+  const v1 = crypto.createHmac('sha256', secret).update(signed).digest('hex');
+  return `t=${unixSeconds},v1=${v1}`;
+}
+
+/** attemptCount after increment for failed path; attempt 1 → 60s, … cap 12h */
+export function nextBackoffMs(attemptCount) {
+  const table = [60_000, 300_000, 1_800_000, 7_200_000, 21_600_000, 43_200_000];
+  const index = Math.max(0, Math.min(table.length - 1, attemptCount - 1));
+  return table[index];
+}
+```
+
+Use top-level `import crypto from 'node:crypto'`. Fix any pseudocode above to real ESM.
+
+Also export `parseWebhookEventsJson`, `normalizeWebhookEvents(input)` validating non-empty subset of three events.
+
+- [ ] **Step 4: Run tests — PASS**
+
+Run: `node --test test/webhook-model.test.js`
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add src/webhook-model.js test/webhook-model.test.js
+git commit -m "feat(webhooks): add pure model for events, resolve, sign, backoff"
+```
+
+---
+
+### Task 2: Schema + DB CRUD + enqueue
+
+**Files:**
+- Modify: `src/db.js`
+- Create: `test/webhook-db.test.js` (or extend `test/db.test.js` if preferred — prefer dedicated file)
+
+- [ ] **Step 1: Failing tests for isolation and enqueue idempotency**
+
+```js
+// test/webhook-db.test.js — follow openTempDb / createUser patterns from test/db.test.js
+test('creates webhooks per user and lists by domain scope', () => { /* ... */ });
+test('enqueueWebhookDeliveries is idempotent per webhook+event+send_event', () => { /* ... */ });
+test('domain override skips account webhooks for that event', () => { /* ... */ });
+test('secret not returned on list; create returns plaintext once', () => { /* ... */ });
+```
+
+- [ ] **Step 2: Run — FAIL**
+
+- [ ] **Step 3: Schema in `initDb` / migrations block**
+
+```sql
+CREATE TABLE IF NOT EXISTS webhooks (
+  id INTEGER PRIMARY KEY AUTOINCREMENT,
+  user_id INTEGER NOT NULL,
+  domain_id INTEGER,
+  name TEXT NOT NULL,
+  url TEXT NOT NULL,
+  secret_ciphertext TEXT NOT NULL,
+  secret_prefix TEXT NOT NULL,
+  events_json TEXT NOT NULL,
+  enabled TEXT NOT NULL DEFAULT 'true',
+  created_at TEXT NOT NULL,
+  updated_at TEXT NOT NULL,
+  FOREIGN KEY(user_id) REFERENCES users(id) ON DELETE CASCADE
+);
+
+CREATE TABLE IF NOT EXISTS webhook_deliveries (
+  id INTEGER PRIMARY KEY AUTOINCREMENT,
+  webhook_id INTEGER NOT NULL,
+  user_id INTEGER NOT NULL,
+  send_event_id INTEGER NOT NULL,
+  event_type TEXT NOT NULL,
+  payload_json TEXT NOT NULL,
+  status TEXT NOT NULL,
+  attempt_count INTEGER NOT NULL DEFAULT 0,
+  next_attempt_at TEXT NOT NULL,
+  last_attempt_at TEXT,
+  response_status INTEGER,
+  response_body_preview TEXT NOT NULL DEFAULT '',
+  error TEXT NOT NULL DEFAULT '',
+  created_at TEXT NOT NULL,
+  FOREIGN KEY(webhook_id) REFERENCES webhooks(id) ON DELETE CASCADE,
+  UNIQUE(webhook_id, send_event_id, event_type)
+);
+
+CREATE INDEX IF NOT EXISTS idx_webhooks_user_id ON webhooks(user_id);
+CREATE INDEX IF NOT EXISTS idx_webhooks_user_domain ON webhooks(user_id, domain_id);
+CREATE INDEX IF NOT EXISTS idx_webhook_deliveries_status_next ON webhook_deliveries(status, next_attempt_at);
+CREATE INDEX IF NOT EXISTS idx_webhook_deliveries_user_created ON webhook_deliveries(user_id, created_at);
+```
+
+No FK on `send_event_id` (test sentinel `0`).
+
+- [ ] **Step 4: Implement API functions**
+
+- `listWebhooks(userId, { domainId }?)`
+- `createWebhook(userId, { name, url, events, domainId, enabled })` → returns row + `secret` plaintext once; store `encryptSecret(secret)`, prefix first 8 chars
+- `updateWebhook(userId, id, patch)`
+- `rotateWebhookSecret(userId, id)` → new secret once
+- `deleteWebhook(userId, id)`
+- `enqueueWebhookDeliveries(sendEvent)` — load account + domain webhooks, resolve, for each insert in **one transaction**: insert pending with temporary payload `'{}'`, then update `payload_json` from `buildWebhookPayload({ deliveryId: id, ... })` before commit. Use `INSERT OR IGNORE` or catch unique to keep idempotent.
+- `claimWebhookDeliveries(limit)` — transaction: select pending due, set processing + lease `next_attempt_at`
+- `reapExpiredWebhookProcessing()`
+- `completeWebhookDeliverySuccess(id, { responseStatus, bodyPreview })`
+- `completeWebhookDeliveryFailure(id, { responseStatus, bodyPreview, error })` — increment already done or do inside; set pending+backoff or dead
+- `listWebhookDeliveries(userId, filters)`
+- `replayWebhookDelivery(userId, id)` — reject if processing
+- `enqueueWebhookTestDelivery(userId, webhookId)` — send_event_id 0, reuse unique key via reset-to-pending
+
+Wire **`enqueueWebhookDeliveries`** at end of:
+
+1. `logSendEvent` if `isTerminalWebhookStatus(event.status)` after insert (pass full event with id)
+2. `updateSendEventDelivery` if `nextStatus` terminal and `nextStatus !== row.status`
+
+Keep enqueue try/catch logged so webhook failure never breaks mail path.
+
+- [ ] **Step 5: Tests PASS + commit**
+
+```bash
+git add src/db.js test/webhook-db.test.js
+git commit -m "feat(webhooks): add schema, CRUD, and terminal enqueue hooks"
+```
+
+---
+
+### Task 3: Dispatcher worker + SSRF
+
+**Files:**
+- Create: `src/webhook-dispatcher.js`
+- Create: `test/webhook-dispatcher.test.js`
+- Modify: `src/server.js` (start/stop worker on listen)
+
+- [ ] **Step 1: Tests with injectable `fetch` and clock**
+
+```js
+test('posts signed body and marks success on 2xx', async () => { /* mock fetch ok */ });
+test('schedules retry on 500', async () => { /* */ });
+test('marks dead after max attempts', async () => { /* */ });
+test('rejects private IP targets without calling fetch', async () => { /* */ });
+```
+
+- [ ] **Step 2: Implement dispatcher**
+
+```js
+// Core loop
+export function startWebhookWorker({ intervalMs = 10_000, batchSize = 3, fetchImpl = fetch } = {}) { ... }
+export function stopWebhookWorker() { ... }
+export async function processWebhookBatch({ fetchImpl, batchSize } = {}) {
+  reapExpiredWebhookProcessing();
+  const rows = claimWebhookDeliveries(batchSize);
+  for (const row of rows) {
+    await deliverOne(row, fetchImpl);
+  }
+}
+```
+
+`deliverOne`:
+
+1. Load webhook + decrypt secret; if missing → dead/error
+2. Validate URL (https; optional `WEBHOOK_ALLOW_HTTP_LOCAL`)
+3. DNS lookup / block private ranges (use `dns.promises.lookup` + IP checks; for hostnames resolving to private, fail closed)
+4. `fetch(url, { method: 'POST', headers: { 'Content-Type': 'application/json', 'User-Agent': 'MailHub-Webhook/1.0', 'X-MailHub-Signature': sign..., 'X-MailHub-Event': type, 'X-MailHub-Delivery': id }, body: payload_json, signal: AbortSignal.timeout(10_000), redirect: 'manual' })`
+5. Success / failure complete helpers
+
+- [ ] **Step 3: Start worker from server listen path** (find existing `server.listen` / bootstrap)
+
+- [ ] **Step 4: Tests PASS + commit**
+
+```bash
+git add src/webhook-dispatcher.js test/webhook-dispatcher.test.js src/server.js
+git commit -m "feat(webhooks): add delivery worker with claim and SSRF guards"
+```
+
+---
+
+### Task 4: REST API
+
+**Files:**
+- Modify: `src/server.js`
+- Create: `test/server-webhooks-api.test.js`
+
+- [ ] **Step 1: Tests** — session auth required; user A cannot see user B webhooks; create returns secret; list omits secret; test + replay endpoints
+
+Follow patterns in `test/server-admin-api.test.js` (login cookie, temp data dir).
+
+- [ ] **Step 2: Routes** (session user required, same as other user APIs)
+
+```
+GET    /api/webhooks
+POST   /api/webhooks
+PATCH  /api/webhooks/:id
+POST   /api/webhooks/:id/rotate-secret
+DELETE /api/webhooks/:id
+POST   /api/webhooks/:id/test
+GET    /api/webhook-deliveries
+POST   /api/webhook-deliveries/:id/replay
+```
+
+Validate body; map errors to 400.
+
+- [ ] **Step 3: Tests PASS + commit**
+
+```bash
+git add src/server.js test/server-webhooks-api.test.js
+git commit -m "feat(webhooks): expose management and delivery APIs"
+```
+
+---
+
+### Task 5: Frontend API + types + i18n
+
+**Files:**
+- Modify: `src/frontend/types.ts`
+- Modify: `src/frontend/services/api.ts`
+- Modify: `src/frontend/i18n/index.js`
+- Modify: `test/frontend-i18n.test.js` (if new keys asserted)
+
+- [ ] **Step 1: Types**
+
+```ts
+export type WebhookEvent = 'sent' | 'bounced' | 'failed';
+export interface Webhook {
+  id: number;
+  userId: number;
+  domainId: number | null;
+  name: string;
+  url: string;
+  secretPrefix: string;
+  events: WebhookEvent[];
+  enabled: boolean;
+  createdAt: string;
+  updatedAt: string;
+  secret?: string; // only on create/rotate responses
+}
+export interface WebhookDelivery {
+  id: number;
+  webhookId: number;
+  sendEventId: number;
+  eventType: string;
+  status: 'pending' | 'processing' | 'success' | 'dead';
+  attemptCount: number;
+  // ...
+}
+```
+
+- [ ] **Step 2: api.ts methods** mirroring REST
+
+- [ ] **Step 3: i18n keys** for page chrome (zh-CN + en-US): titles, events labels, empty states, secret warning, replay, test
+
+- [ ] **Step 4: Commit**
+
+```bash
+git add src/frontend/types.ts src/frontend/services/api.ts src/frontend/i18n/index.js test/frontend-i18n.test.js
+git commit -m "feat(webhooks): add frontend types, API client, and i18n"
+```
+
+---
+
+### Task 6: Global Webhooks page + App wiring
+
+**Files:**
+- Create: `src/pages/Webhooks.tsx`
+- Modify: `src/frontend/App.tsx`
+- Optionally: small components under `src/components/webhook/`
+
+- [ ] **Step 1: Build `Webhooks.tsx`**
+
+Use PageHeader, SectionCard, StatusPill, Table, Drawer/Form, Modal for secret once, Popconfirm delete.
+
+Sections:
+
+1. Endpoints table + create button  
+2. Deliveries table (filter by webhook) + replay  
+
+Handlers call `api.*` and refresh list. Toggle enabled via PATCH.
+
+- [ ] **Step 2: App.tsx**
+
+- Import Webhooks page  
+- Replace Placeholder for `webhooks` view  
+- Load webhooks (and optionally deliveries) in `loadAll` or lazy on view enter  
+
+- [ ] **Step 3: `npx tsc --noEmit` PASS**
+
+- [ ] **Step 4: Commit**
+
+```bash
+git add src/pages/Webhooks.tsx src/frontend/App.tsx src/components/webhook
+git commit -m "feat(webhooks): add global webhooks management UI"
+```
+
+---
+
+### Task 7: Domain detail Webhooks tab
+
+**Files:**
+- Modify: `src/pages/Domains/DomainDetail.tsx`
+- Possibly reuse list component with `domainId` prop from Task 6
+
+- [ ] **Step 1: Replace Placeholder tab with filtered Webhooks panel** (`domainId={domain.id}`) + help Alert about override semantics
+
+- [ ] **Step 2: tsc PASS + commit**
+
+```bash
+git add src/pages/Domains/DomainDetail.tsx src/pages/Webhooks.tsx
+git commit -m "feat(webhooks): wire domain-scoped webhook overrides in detail tab"
+```
+
+---
+
+### Task 8: Full gate
+
+**Files:** polish only if needed
+
+- [ ] **Step 1:** `npm test` — all pass  
+- [ ] **Step 2:** `npm run build` — pass; commit built `public/` assets if hashes change  
+- [ ] **Step 3:** Manual checklist  
+  - Create account webhook, copy secret  
+  - Send test  
+  - Force fail (bad URL) → pending retry fields  
+  - Replay  
+  - Domain override skips account for same event  
+- [ ] **Step 4:** Final commit if assets/docs changed  
+
+```bash
+git add -A
+git commit -m "chore(webhooks): build assets after webhook feature"
+```
+
+---
+
+## Notes
+
+- Do not block mail send on webhook errors.  
+- Never log full secrets.  
+- Align field naming with existing API camelCase JSON helpers in `db.js` / server.  
+- Follow `encryptSecret` usage from DNS credentials.  
+- After implementation, use @superpowers:verification-before-completion before claiming done.  
+- Deploy only after user requests per Agents.md.
+
+## Execution handoff
+
+Plan complete. Choose:
+
+1. **Subagent-Driven (recommended)** — fresh subagent per task + review  
+2. **Inline Execution** — this session with checkpoints