Commit graph

5 commits

Author SHA1 Message Date
Luiz Gustavo
dd75a67964 W3.8: Investigation Bureau complete — Poirot, Taleb, Tetlock, Case-Writer
Some checks are pending
CI / Web — typecheck + lint + build (push) Waiting to run
CI / Scripts — Python smoke (push) Waiting to run
CI / Web — npm audit (push) Waiting to run
CI / Retrieval — golden set (Recall@5 + MRR) (push) Waiting to run
Brings the bureau from 4 → 8 detectives. All eight run as Bun + claude-CLI
subprocesses against the same Supabase + investigation_jobs LISTEN/NOTIFY
queue, sharing search.ts hybridSearch and writer-side validators that
gate writes against schema + FK.

New detectives:

  Poirot (witness_analysis)
    - prompts/poirot.md — credibility / access / bias / corroboration /
      verdict; uses entity_mentions JOIN chunks to pull 12 chunks per
      person; resolves corroboration_refs chunk_ids defensively (accepts
      bare cNNNN even when the model emits pNNN/cNNNN).
    - INSERT into public.witnesses with W-NNNN naming.
    - Tone: purple (#9b5de5).

  Taleb (outlier_scan)
    - prompts/taleb.md — "surprise is relative to a model"; at most 3
      outliers; each requires explicit dominant_model + why_surprising +
      what_it_implies; fan-out into public.gaps with scope.kind="outlier".
    - Same unscoped-fallback as Dupin (Pass 1 with doc_id, Pass 2 widens
      to corpus if hits < 3).
    - Tone: yellow (#ffd23f).

  Tetlock (calibrate_hypothesis)
    - prompts/tetlock.md — honest Bayesian update; emits new_posterior +
      Δ + recommended_action ∈ {keep, downgrade, upgrade, supersede}.
    - write_calibration UPDATEs public.hypotheses + APPENDS a
      "## Calibration history" section to the H-NNNN.md case file
      (calibration is append-only — each datapoint matters). Posterior
      band auto-corrected to match Tetlock thresholds.
    - NO_NEW_EVIDENCE sentinel handled; pure 'keep' with |Δ|<0.005 only
      touches updated_at + reviewed_by.
    - Tone: teal (#26d4cc).

  Case-Writer (case_report)
    - prompts/case-writer.md — Dr. Watson assembles all artefacts
      (E-NNNN, H-NNNN, R-NNNN, W-NNNN, G-NNNN) into a five-act narrative.
      ILIKE filter on topic; doc_id optional scope.
    - Larger budget cap (≥ $0.50) + longer timeout for prose generation.
    - Writes case/reports/<slug>.md with frontmatter (topic + counts);
      no DB table for v0.
    - New page /c/[slug] renders the report via MarkdownBody + stat chips.
    - Tone: gold (#e0c080).

Hardening across the bureau:
  - Sentinel parsing now accepts backticked AND prose-trailing forms
    (Holmes NO_HYPOTHESES, Dupin NO_CONTRADICTIONS, Schneier
    INSUFFICIENT_HYPOTHESIS, Poirot INSUFFICIENT_TESTIMONY, Taleb
    NO_OUTLIERS, Tetlock NO_NEW_EVIDENCE, Case-Writer
    INSUFFICIENT_ARTEFACTS). Avoids the failure mode where the model
    refuses honestly but the runtime treated it as a parse error
    (observed live with Poirot+Hoover identifying the DIRECTOR
    false-positive disambiguation issue in entity_mentions).

Chat tool extensions (web/lib/chat/tools.ts):
  - request_investigation now accepts 7 kinds. Each routes to its
    detective with appropriate validation (hypothesis_id regex,
    person_id kebab-case, topic non-empty, doc_id for evidence_chain).
  - ETA per kind: Holmes/Dupin 60s, Poirot 45s, Schneier/Tetlock 30s,
    Taleb 50s, Case-Writer 180s (longer prose), Locard 30×n_chunks.

UI integration:
  - chat-bubble inline card paints each detective in its tone color.
  - /jobs/[id] page header swaps name/subtitle/tone per detective;
    question label adapts ("Topic" / "Hypothesis under attack" /
    "Witness under analysis" / "Topic to outlier-scan" / "Hypothesis
    under recalibration" / "Case to assemble").
  - job-status-poller renders: case-report link card (gold), outlier
    cards (yellow), witness cards (purple) — alongside existing
    hypothesis, evidence, contradiction cards.
  - /api/jobs/[id] hydrates witnesses (JOIN entities for canonical_name)
    + gaps (with scope JSONB).
  - /c/[slug] page reads /data/ufo/case/reports/<slug>.md and renders
    with MarkdownBody, frontmatter parsed for stat chips.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 22:11:39 -03:00
Luiz Gustavo
857dd771d2 W3.8: Schneier red-team detective + /h/[hypothesisId] dossier page
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 33s
CI / Scripts — Python smoke (push) Failing after 7s
CI / Web — npm audit (push) Failing after 38s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Adds the fourth AI detective in the Investigation Bureau runtime: Bruce
Schneier, who attacks an existing hypothesis as a red-team operator.

Runtime:
  - prompts/schneier.md — discipline (don't disprove, just attack;
    structured output with hidden_assumptions, failure_modes,
    alternative_explanations, recommended_tests, verdict_one_sentence;
    severity ∈ {low, medium, high}; emit INSUFFICIENT_HYPOTHESIS when
    the input is too thin)
  - src/detectives/schneier.ts — reads the hypothesis row + evidence
    chain (joined via evidence_refs FK), feeds Claude with the
    arguments + verbatim quotes, parses strict JSON object
  - src/tools/write_red_team_review.ts — UPDATEs hypotheses.reviewed_by
    + updated_at; APPENDS (or replaces if re-reviewed) a structured
    "## Red-team review (Schneier · X severity)" section to
    case/hypotheses/H-NNNN.md. Caps each list at 5 entries × 240 chars,
    validates verdict ≤ 280 chars.
  - orchestrator: new `red_team_review` kind dispatching to runSchneier

Chat + UI:
  - request_investigation gains kind=red_team_review + hypothesis_id arg
    (validated against H-NNNN regex); detective auto-resolves to schneier
  - chat-bubble inline card paints Schneier in red (#ff3344)
  - /jobs/[id] page swaps title/subtitle/tone per detective; the
    "Question" label becomes "Hypothesis under attack" for red_team_review

New /h/[hypothesisId] page (hypothesis dossier):
  - Server-rendered from public.hypotheses + public.evidence (joined
    via evidence_refs FK + chunk lookup)
  - Header: ID + creator + reviewer (highlighted when Schneier has
    visited), position as headline, question subtitle, Tetlock band
  - Prior + posterior bars with Δ-delta indicator
  - Argument grid: argument_for (green) vs argument_against (pink)
    side-by-side with [[wiki-link]] auto-linking to source chunks
  - Evidence chain: each E-NNNN with Grade A/B/C badge, verbatim
    blockquote, link to source page
  - Red-team review panel: parses the markdown section in the case
    file (severity badge, verdict, 4 bullet panels for
    hidden_assumptions / failure_modes / alternative_explanations /
    recommended_tests). Empty state when not yet reviewed.

RedTeamRequestButton client component + POST /api/h/[id]/red-team —
authenticated user can trigger Schneier in one click; UI swaps to
"acompanhar" link to /jobs/[id] once queued.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 21:48:12 -03:00
Luiz Gustavo
5ac53cb3e2 W3.7: Dupin contradiction-scan detective + UI integration
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 39s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Failing after 37s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Adds the third AI detective in the Investigation Bureau runtime: C. Auguste
Dupin, who scans a corpus shortlist for pairs (or small groups) of chunks
that cannot both be true under any ordinary reading.

Runtime:
  - prompts/dupin.md — discipline (no contradiction without ≥2 distinct
    chunk_ids; reject same-vocabulary near-misses; FEW high-confidence
    over MANY weak ones; emit `NO_CONTRADICTIONS` when corpus is silent)
  - src/detectives/dupin.ts — hybridSearch with k=18 (more chunks than
    Holmes because contradictions emerge from comparing dispersed
    claims), strict JSON-array parsing, AT MOST 3 contradictions per call
  - src/tools/write_contradiction.ts — validates topic + ≥2 positions
    drawn from ≥2 distinct chunks, resolves chunk_pk via DB lookup
    (rejects positions citing unknown chunks), INSERTs into
    public.contradictions + writes case/contradictions/R-NNNN.md
  - orchestrator: new `contradiction_scan` kind dispatching to runDupin;
    payload { topic, doc_id?, lang?, context_chunks? }

Chat + UI:
  - request_investigation gains kind=contradiction_scan + topic arg;
    triggered detective auto-resolves to dupin
  - chat-bubble inline card renders dupin in orange (#ff8a4d) to
    distinguish from holmes (cyan) and locard (green)
  - /jobs/[id] page swaps title + subtitle + tone per detective;
    "Question" label becomes "Topic" for contradiction_scan
  - /api/jobs/[id] hydrates public.contradictions when outputs[] surfaces
    contradiction_ids
  - job-status-poller renders ContradictionCard: topic + N positions
    (verbatim statements quoted, stance label optional, link to source
    chunk) + optional notes panel, with resolution_status badge
    (open/resolved/irreconcilable)

R-NNNN shares the contradiction_id_seq slot with relation per
CLAUDE.md naming — same conceptual class (a connection between two
pieces of evidence in tension).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 21:34:04 -03:00
Luiz Gustavo
4d4c02a8e1 W3.5: Holmes hypothesis tournament detective
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 34s
CI / Scripts — Python smoke (push) Failing after 3s
CI / Web — npm audit (push) Failing after 29s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 3s
Adds the second AI detective in the Investigation Bureau runtime: Sherlock
Holmes, who builds 2-3 rival hypotheses with calibrated priors + posteriors
against a corpus shortlist.

Pipeline:
  1. hybridSearch() grounds Holmes with 8-15 chunks via the same
     hybrid_search_chunks RPC the web uses (BM25 + dense + RRF). Default
     max_dense_dist=0.55 (runtime favors recall over precision; web's
     /api/search/hybrid stays at 0.40 for chat).
  2. claude-sonnet-4-6 emits a strict JSON array with position +
     argument_for + argument_against + prior + posterior + confidence_band
     + evidence_refs. Citations use [[doc-id/pNNN#cNNNN]] wiki-links.
  3. writeHypothesis() validates posterior ∈ [0,1], auto-corrects the
     Tetlock band from the posterior (high ≥0.90, medium 0.60-0.89,
     low 0.30-0.59, speculation <0.30), checks evidence_refs FK against
     public.evidence, INSERTs into public.hypotheses + writes
     case/hypotheses/H-NNNN.md.

Discipline guarantees (prompts/holmes.md):
  - posteriors across rivals sum to ≈1.0
  - no claim without chunk citation
  - prefer lower band when ambiguous (anti-inflation)
  - declarative one-sentence position, no hedging
  - emit `NO_HYPOTHESES` when corpus is silent (refuses to fabricate)

Smoke test (Sandia green fireballs 1948-49):
  - H-0001 prior 0.5 → posterior 0.2 (speculation): natural meteoric
  - H-0002 prior 0.3 → posterior 0.4 (low): classified weapons / tests
  - H-0003 prior 0.2 → posterior 0.4 (low): genuinely unidentified
  Bayesian update visible: "natural meteoric" prior dropped 60%; both
  rivals climbed. 4 unique chunk citations across the 3 hypotheses.

orchestrator dispatches `hypothesis_tournament` kind via runHolmes;
job marked `failed` if all rivals error, `complete` otherwise.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 21:19:43 -03:00
Luiz Gustavo
189a771cbe W3.1-W3.4: Investigation Bureau foundation — migrations, runtime, Locard
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 38s
CI / Scripts — Python smoke (push) Failing after 3s
CI / Web — npm audit (push) Failing after 33s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Migrations:
- 0004_investigation_bureau.sql: 7 new tables (investigation_jobs + evidence,
  hypotheses, contradictions, witnesses, gaps, residual_uncertainties), id
  sequences, pg_notify trigger on investigation_jobs, RLS read-only public,
  investigator role with least-privilege grants (no service_role).
- 0005_investigator_write_policies.sql: fixup adding RLS INSERT/UPDATE
  policies bound to investigator + service_role + postgres (RLS with only a
  SELECT policy was silently blocking the worker's claim UPDATE).

investigator-runtime/ (new Bun + TS container):
- src/main.ts: LISTEN/NOTIFY poller, claim-with-SKIP-LOCKED, drain pool,
  healthcheck file, graceful SIGTERM shutdown.
- src/orchestrator.ts: chief-detective dispatch (evidence_chain → Locard).
  Marks job failed when all per-item outputs error; surfaces first errors.
- src/lib/{env,pg,audit,ids,claude}.ts: typed config (gate #8), pool +
  dedicated LISTEN client, NDJSON audit, sequence allocator (E-NNNN etc),
  claude -p subprocess with quota detection (api_error_status=429).
- src/tools/write_evidence.ts: schema-validate (grade A/B/C custody steps),
  resolve chunk_pk via FK, verify verbatim_excerpt actually appears in
  chunk content, INSERT + render case/evidence/E-NNNN.md + audit.
- src/detectives/locard.ts: load chunk → call Claude with locard.md system
  prompt → parse strict JSON → call writeEvidence locally.
- Dockerfile installs `claude` CLI (OAuth) at build time.

Compose:
- new `investigator` service builds from investigator-runtime/, connects
  with low-privilege role, mounts case/ RW and wiki/+raw/ RO, 512m mem cap.

Web:
- /api/admin/investigate/test (POST+GET) gated by middleware (W0-F1).
  POST creates a job, GET polls status. For W3.6 it becomes the chat tool.

End-to-end smoke: INSERT job → pg_notify → claim → Locard dispatch →
claude subprocess invoked. Auth works (CLI v2.1.150). Currently quota
exhausted (weekly limit · resets 3pm UTC) — pipeline catches the typed
isQuota error, marks job failed with surfaced reason. Architecture proven;
quota reset enables real evidence creation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 19:49:33 -03:00