disclosure-bureau/CHANGELOG.md

Changelog · Disclosure Bureau

All notable changes to this project go here. Newest on top.

[Unreleased]

W1 — Observability + resilience + Meili autocomplete

2026-05-23 · systems-atelier engagement trace 794f00ba

• Studio container fixed (carry-over from W0) — root cause was Next.js standalone binding to the container hostname only. The docker healthcheck (fetch 127.0.0.1:3000/api/profile) looped on ECONNREFUSED, the service never went healthy, and Traefik returned 404 because the upstream wasn't responding. Fix: HOSTNAME: 0.0.0.0 in the studio env. Studio now healthy, basic auth from W0-F3 enforces correctly (no-auth → 401, valid creds → 307), and Let's Encrypt issued a real cert for studio.disclosure.top once the route started responding.
• TD#10 · PG pool max — PG_POOL_MAX=20 (was hard-coded 5) configurable via .env; default raised for prod. Files: docker-compose.yml, .env.
• W1-F8 · CLAUDE_CODE_OAUTH_TOKEN gated — only injected into the web service when explicitly set in CLAUDE_CODE_OAUTH_TOKEN_FOR_WEB. Default empty since CHAT_PROVIDER=openrouter does not need it. Reduces blast radius if web container is compromised. Files: docker-compose.yml, .env.
• TD#30 · Subprocess timeout configurable — CLAUDE_CODE_TIMEOUT_MS env now controls the claude -p subprocess timeout (default 90s, matches prior hard-coded value). Files: web/lib/chat/claude-code.ts.
• TD#23 · OpenRouter retry + circuit breaker — fetchOpenRouter() wraps every call with: retry up to OPENROUTER_RETRY_MAX (default 2) on 408 / 425 / 429 / 500 / 502 / 503 / 504 and network errors, with exponential backoff and Retry-After honored; in-memory circuit breaker trips when PRIMARY fails CB_THRESHOLD times (default 3) within CB_WINDOW_MS (60s), promoting FALLBACK for CB_COOLDOWN_MS (2 min). Both sendOnce and openrouterStreamCall go through it. Files: web/lib/chat/openrouter.ts.
• TD#6 · Structured logging with pino — web/lib/logger.ts provides a JSON logger (NDJSON in prod, pretty in dev) plus withRequest() helper for correlation-id-bound child loggers. Edge runtime falls back to a console adapter. Middleware now mints a correlation_id for every request, stamps the response header (x-correlation-id), and emits one structured http_request line per /api/* call with method, path, status, and duration. messages/route.ts switched to the new logger. Files: web/lib/logger.ts, web/middleware.ts, web/app/api/sessions/[id]/messages/route.ts, web/package.json.
• Meilisearch indexer + /api/search/autocomplete + UI — the previously idle Meili instance now backs typo-tolerant prefix search. Indexer script scripts/maintain/60_meili_index.py ingests documents (canonical_title + collection) and is-searchable chunks (content_pt + content_en + meta). The new /api/search/autocomplete?q=... route hits both indexes in parallel with a 2s abort and returns a merged payload. SearchAutocomplete React component drops a debounced dropdown under the /search input. Median latency in production: 5–8ms. Files: scripts/maintain/60_meili_index.py, web/app/api/search/autocomplete/route.ts, web/components/search-autocomplete.tsx, web/components/search-panel.tsx.

Verified on disclosure.top (2026-05-23T20:30Z):

• /api/admin/{batch,indexer,stats} → 404 ✓ (W0 still holds)
• studio.disclosure.top no-auth → 401 · admin:<DASHBOARD_PASSWORD> → 307 ✓
• Let's Encrypt cert issued for studio.disclosure.top ✓
• Autocomplete q=Roswell → 8 chunks in 8ms; q=Sandia → 1 doc + 8 chunks in 8ms; q=1947 → 5 docs + 8 chunks in 6ms ✓
• x-correlation-id header present on /api/search/hybrid response (e.g. c48b7cc761dac172) ✓
• 18 513 searchable chunks indexed into Meili ✓
• OpenRouter retry/breaker present (7 references in source) ✓

Deferred to W1.2 / W2 (need user-in-loop steps):

• Glitchtip self-host — needs DNS for glitchtip.disclosure.top, initial signup-as-superuser, project DSN copied to .env. Logger and middleware are already feeding the data; SDK wiring is one PR.
• Forgejo Actions self-host CI — Forgejo server + runner bootstrap, initial admin account, repo migration / mirror. Recommend a separate session because of the depth of setup.

W0 — Hardening (security + reproducibility)

2026-05-23 · systems-atelier engagement trace 794f00ba-7cb6-4b90-a48e-23ebd02d1f44

• F1 · Auth gate em /api/admin/* — middleware now matches /api/admin too; non-admin (including anonymous) gets HTTP 404. Verified: curl on /api/admin/{batch,indexer,stats} returns 404 publicly. Files: web/middleware.ts.
• F2 · Imgproxy filesystem root tightened — IMGPROXY_LOCAL_FILESYSTEM_ROOT moved from / (entire VPS root) to /var/lib/storage (Storage backend mount only). Reduces blast radius of any future imgproxy CVE. Files: infra/disclosure-stack/docker-compose.yml.
• F3 · Studio basic auth label — replaced the dead-end basicauth.usersfile=/dev/null with a real bcrypt-hashed credential (DASHBOARD_USERNAME / DASHBOARD_PASSWORD from .env) and wired the middleware into the router via disclosure-studio.middlewares= disclosure-studio-auth@docker. Caveat: the Studio container itself has a pre-existing instability (restarts in a Next.js loop, status unhealthy) so the front-end currently returns 404 from Traefik. When Studio is stabilized (queue for W1), the basic auth will kick in. Files: infra/disclosure-stack/docker-compose.yml.
• F4 · RLS on public.relations — ENABLE ROW LEVEL SECURITY + public SELECT policy + GRANT SELECT TO anon, authenticated. Aligns with every other public table. Files: infra/supabase/migrations/0003_w0_hardening.sql.
• TD#2 · is_searchable folded into canonical migrations — the column, reclassification rules, partial index, and the updated hybrid_search_chunks RPC (BM25 + dense, both filtered by is_searchable) are now in migration 0003_w0_hardening.sql. A clean bootstrap on a fresh VPS produces a searchable database without any scripts/maintain/47-48 post-hoc patches. Files: infra/supabase/migrations/0003_w0_hardening.sql.

Verified on disclosure.top (2026-05-23T19:30Z):

• /api/admin/batch → HTTP 404 ✓
• /api/admin/indexer → HTTP 404 ✓
• /api/admin/stats → HTTP 404 ✓
• pg_class.relrowsecurity = t for chunks, documents, entities, entity_mentions, relations ✓
• is_searchable distribution: 18 513 searchable / 10 046 not-searchable (35% of corpus deduplicated from results) ✓
• /api/search/hybrid?q=Roswell → HTTP 200, 10 hits, first c0527 ✓
• Studio: Traefik labels in place; container itself unhealthy (separate issue, deferred to W1) ⚠

Notes for clean-install reproducibility:

• 0003_w0_hardening.sql MUST be applied as supabase_admin, not postgres, because public.chunks / .entities / .relations are owned by supabase_admin. The migration file documents this in its header.