W3.6: chat request_investigation tool + /jobs/[id] case-file viewer
Closes the loop between the chat UI and the Investigation Bureau runtime.
Chat tool (web/lib/chat/tools.ts):
- request_investigation { kind, question, doc_id?, chunks?, claim? }
INSERTs a row in public.investigation_jobs and returns
{ job_id, kind, status, eta_seconds, status_url, detective }.
- kind=hypothesis_tournament → Holmes (1 question → 2-3 rival hypotheses)
- kind=evidence_chain → Locard (1 doc → grade-A/B/C evidence with chain
of custody, default top-5 anomaly chunks)
- Plumbed user.email through ToolHandlerContext so triggered_by audits
the requesting user.
Public job viewer:
- GET /api/jobs/[id] joins investigation_jobs → public.evidence +
public.hypotheses for the IDs surfaced in outputs[]. Returns one
payload the page can render without n+1 round-trips. Strips
triggered_by from the response (it carries the user's email).
- app/jobs/[id]/page.tsx server-renders the case-file shell:
detective lore header (Holmes blue or Locard green), question chip,
scope chip with link back to the document.
- components/job-status-poller.tsx client island that polls every 3 s
while non-terminal, then once on terminal to hydrate evidence +
hypotheses. Renders:
· Phase tracker (queued → running → complete | failed)
· Hypothesis cards w/ prior + posterior bars + Δ delta indicator
+ Tetlock band badge (high/medium/low/speculation)
· Argument-for / argument-against with [[wiki-link]] auto-linking
to /d/<doc>/p<NNN>#<cNNNN>
· Evidence cards w/ Grade A/B/C badge + verbatim blockquote +
bbox crop preview via /api/crop + custody-steps disclosure
· Empty/in-flight panel ("os detetives estão lendo o corpus")
· Failure panel surfacing error + partial outputs
Inline chat-bubble card (components/chat-bubble.tsx):
- ToolTrace.richRender recognises request_investigation results and
renders a detective banner with status + ETA + link to /jobs/[id]
(target=_blank). Error case renders a red strip with the message.
UX flow now: user asks Sherlock a question → request_investigation
queues the job → chat card shows "🔎 Holmes · hypothesis_tournament ·
ETA ~60s" → user clicks → /jobs/<id> live-updates → 60 s later, 2-3
rival hypotheses + their arguments + chunk citations are rendered with
Bayesian update visible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:26:18 +00:00
|
|
|
/**
|
|
|
|
|
* GET /api/jobs/[id] — public read of an investigation_jobs row.
|
|
|
|
|
*
|
|
|
|
|
* Hydrates the outputs[] payload by joining to public.evidence / public.hypotheses
|
|
|
|
|
* so the /jobs/[id] page can render evidence cards + hypothesis cards without
|
|
|
|
|
* n+1 round-trips.
|
|
|
|
|
*
|
|
|
|
|
* No auth required (read-only): anyone with a job_id can see status. This
|
|
|
|
|
* matches the chat tool's UX: the chat reveals the URL, the user opens it,
|
|
|
|
|
* the page renders. No PII is exposed beyond what the user already typed
|
|
|
|
|
* (triggered_by carries their email — we strip it).
|
|
|
|
|
*/
|
|
|
|
|
import { NextResponse } from "next/server";
|
|
|
|
|
import { pgQuery } from "@/lib/retrieval/db";
|
|
|
|
|
|
|
|
|
|
export const runtime = "nodejs";
|
|
|
|
|
export const dynamic = "force-dynamic";
|
|
|
|
|
|
|
|
|
|
interface JobRow {
|
|
|
|
|
job_id: string;
|
|
|
|
|
kind: string;
|
|
|
|
|
payload: Record<string, unknown> | null;
|
|
|
|
|
status: string;
|
|
|
|
|
worker_id: string | null;
|
|
|
|
|
started_at: string | null;
|
|
|
|
|
finished_at: string | null;
|
|
|
|
|
outputs: unknown;
|
|
|
|
|
error: string | null;
|
|
|
|
|
created_at: string;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
interface EvidenceRow {
|
|
|
|
|
evidence_id: string;
|
|
|
|
|
grade: string | null;
|
|
|
|
|
source_page_id: string;
|
|
|
|
|
doc_id: string | null;
|
|
|
|
|
page: number | null;
|
|
|
|
|
chunk_id: string | null;
|
|
|
|
|
verbatim_excerpt: string | null;
|
|
|
|
|
custody_steps: unknown;
|
|
|
|
|
bbox: unknown;
|
|
|
|
|
confidence_band: string | null;
|
|
|
|
|
related_hypotheses: unknown;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
interface HypothesisRow {
|
|
|
|
|
hypothesis_id: string;
|
|
|
|
|
question: string | null;
|
|
|
|
|
position: string | null;
|
|
|
|
|
argument_for: string | null;
|
|
|
|
|
argument_against: string | null;
|
|
|
|
|
prior: number | null;
|
|
|
|
|
posterior: number | null;
|
|
|
|
|
confidence_band: string | null;
|
|
|
|
|
status: string | null;
|
|
|
|
|
evidence_refs: unknown;
|
|
|
|
|
}
|
|
|
|
|
|
W3.7: Dupin contradiction-scan detective + UI integration
Adds the third AI detective in the Investigation Bureau runtime: C. Auguste
Dupin, who scans a corpus shortlist for pairs (or small groups) of chunks
that cannot both be true under any ordinary reading.
Runtime:
- prompts/dupin.md — discipline (no contradiction without ≥2 distinct
chunk_ids; reject same-vocabulary near-misses; FEW high-confidence
over MANY weak ones; emit `NO_CONTRADICTIONS` when corpus is silent)
- src/detectives/dupin.ts — hybridSearch with k=18 (more chunks than
Holmes because contradictions emerge from comparing dispersed
claims), strict JSON-array parsing, AT MOST 3 contradictions per call
- src/tools/write_contradiction.ts — validates topic + ≥2 positions
drawn from ≥2 distinct chunks, resolves chunk_pk via DB lookup
(rejects positions citing unknown chunks), INSERTs into
public.contradictions + writes case/contradictions/R-NNNN.md
- orchestrator: new `contradiction_scan` kind dispatching to runDupin;
payload { topic, doc_id?, lang?, context_chunks? }
Chat + UI:
- request_investigation gains kind=contradiction_scan + topic arg;
triggered detective auto-resolves to dupin
- chat-bubble inline card renders dupin in orange (#ff8a4d) to
distinguish from holmes (cyan) and locard (green)
- /jobs/[id] page swaps title + subtitle + tone per detective;
"Question" label becomes "Topic" for contradiction_scan
- /api/jobs/[id] hydrates public.contradictions when outputs[] surfaces
contradiction_ids
- job-status-poller renders ContradictionCard: topic + N positions
(verbatim statements quoted, stance label optional, link to source
chunk) + optional notes panel, with resolution_status badge
(open/resolved/irreconcilable)
R-NNNN shares the contradiction_id_seq slot with relation per
CLAUDE.md naming — same conceptual class (a connection between two
pieces of evidence in tension).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:34:04 +00:00
|
|
|
interface ContradictionRow {
|
|
|
|
|
contradiction_id: string;
|
|
|
|
|
topic: string;
|
|
|
|
|
chunks: unknown;
|
|
|
|
|
resolution_status: string | null;
|
|
|
|
|
notes: string | null;
|
|
|
|
|
detected_by: string | null;
|
|
|
|
|
}
|
|
|
|
|
|
W3.8: Investigation Bureau complete — Poirot, Taleb, Tetlock, Case-Writer
Brings the bureau from 4 → 8 detectives. All eight run as Bun + claude-CLI
subprocesses against the same Supabase + investigation_jobs LISTEN/NOTIFY
queue, sharing search.ts hybridSearch and writer-side validators that
gate writes against schema + FK.
New detectives:
Poirot (witness_analysis)
- prompts/poirot.md — credibility / access / bias / corroboration /
verdict; uses entity_mentions JOIN chunks to pull 12 chunks per
person; resolves corroboration_refs chunk_ids defensively (accepts
bare cNNNN even when the model emits pNNN/cNNNN).
- INSERT into public.witnesses with W-NNNN naming.
- Tone: purple (#9b5de5).
Taleb (outlier_scan)
- prompts/taleb.md — "surprise is relative to a model"; at most 3
outliers; each requires explicit dominant_model + why_surprising +
what_it_implies; fan-out into public.gaps with scope.kind="outlier".
- Same unscoped-fallback as Dupin (Pass 1 with doc_id, Pass 2 widens
to corpus if hits < 3).
- Tone: yellow (#ffd23f).
Tetlock (calibrate_hypothesis)
- prompts/tetlock.md — honest Bayesian update; emits new_posterior +
Δ + recommended_action ∈ {keep, downgrade, upgrade, supersede}.
- write_calibration UPDATEs public.hypotheses + APPENDS a
"## Calibration history" section to the H-NNNN.md case file
(calibration is append-only — each datapoint matters). Posterior
band auto-corrected to match Tetlock thresholds.
- NO_NEW_EVIDENCE sentinel handled; pure 'keep' with |Δ|<0.005 only
touches updated_at + reviewed_by.
- Tone: teal (#26d4cc).
Case-Writer (case_report)
- prompts/case-writer.md — Dr. Watson assembles all artefacts
(E-NNNN, H-NNNN, R-NNNN, W-NNNN, G-NNNN) into a five-act narrative.
ILIKE filter on topic; doc_id optional scope.
- Larger budget cap (≥ $0.50) + longer timeout for prose generation.
- Writes case/reports/<slug>.md with frontmatter (topic + counts);
no DB table for v0.
- New page /c/[slug] renders the report via MarkdownBody + stat chips.
- Tone: gold (#e0c080).
Hardening across the bureau:
- Sentinel parsing now accepts backticked AND prose-trailing forms
(Holmes NO_HYPOTHESES, Dupin NO_CONTRADICTIONS, Schneier
INSUFFICIENT_HYPOTHESIS, Poirot INSUFFICIENT_TESTIMONY, Taleb
NO_OUTLIERS, Tetlock NO_NEW_EVIDENCE, Case-Writer
INSUFFICIENT_ARTEFACTS). Avoids the failure mode where the model
refuses honestly but the runtime treated it as a parse error
(observed live with Poirot+Hoover identifying the DIRECTOR
false-positive disambiguation issue in entity_mentions).
Chat tool extensions (web/lib/chat/tools.ts):
- request_investigation now accepts 7 kinds. Each routes to its
detective with appropriate validation (hypothesis_id regex,
person_id kebab-case, topic non-empty, doc_id for evidence_chain).
- ETA per kind: Holmes/Dupin 60s, Poirot 45s, Schneier/Tetlock 30s,
Taleb 50s, Case-Writer 180s (longer prose), Locard 30×n_chunks.
UI integration:
- chat-bubble inline card paints each detective in its tone color.
- /jobs/[id] page header swaps name/subtitle/tone per detective;
question label adapts ("Topic" / "Hypothesis under attack" /
"Witness under analysis" / "Topic to outlier-scan" / "Hypothesis
under recalibration" / "Case to assemble").
- job-status-poller renders: case-report link card (gold), outlier
cards (yellow), witness cards (purple) — alongside existing
hypothesis, evidence, contradiction cards.
- /api/jobs/[id] hydrates witnesses (JOIN entities for canonical_name)
+ gaps (with scope JSONB).
- /c/[slug] page reads /data/ufo/case/reports/<slug>.md and renders
with MarkdownBody, frontmatter parsed for stat chips.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 01:11:39 +00:00
|
|
|
interface WitnessRow {
|
|
|
|
|
witness_id: string;
|
|
|
|
|
canonical_name: string | null;
|
|
|
|
|
entity_id: string | null;
|
|
|
|
|
credibility: string | null;
|
|
|
|
|
access_to_event: string | null;
|
|
|
|
|
bias_notes: string | null;
|
|
|
|
|
corroboration_refs: unknown;
|
|
|
|
|
verdict: string | null;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
interface GapRow {
|
|
|
|
|
gap_id: string;
|
|
|
|
|
description: string;
|
|
|
|
|
scope: unknown;
|
|
|
|
|
suggested_next_move: string | null;
|
|
|
|
|
status: string;
|
|
|
|
|
created_by: string;
|
|
|
|
|
}
|
|
|
|
|
|
W3.6: chat request_investigation tool + /jobs/[id] case-file viewer
Closes the loop between the chat UI and the Investigation Bureau runtime.
Chat tool (web/lib/chat/tools.ts):
- request_investigation { kind, question, doc_id?, chunks?, claim? }
INSERTs a row in public.investigation_jobs and returns
{ job_id, kind, status, eta_seconds, status_url, detective }.
- kind=hypothesis_tournament → Holmes (1 question → 2-3 rival hypotheses)
- kind=evidence_chain → Locard (1 doc → grade-A/B/C evidence with chain
of custody, default top-5 anomaly chunks)
- Plumbed user.email through ToolHandlerContext so triggered_by audits
the requesting user.
Public job viewer:
- GET /api/jobs/[id] joins investigation_jobs → public.evidence +
public.hypotheses for the IDs surfaced in outputs[]. Returns one
payload the page can render without n+1 round-trips. Strips
triggered_by from the response (it carries the user's email).
- app/jobs/[id]/page.tsx server-renders the case-file shell:
detective lore header (Holmes blue or Locard green), question chip,
scope chip with link back to the document.
- components/job-status-poller.tsx client island that polls every 3 s
while non-terminal, then once on terminal to hydrate evidence +
hypotheses. Renders:
· Phase tracker (queued → running → complete | failed)
· Hypothesis cards w/ prior + posterior bars + Δ delta indicator
+ Tetlock band badge (high/medium/low/speculation)
· Argument-for / argument-against with [[wiki-link]] auto-linking
to /d/<doc>/p<NNN>#<cNNNN>
· Evidence cards w/ Grade A/B/C badge + verbatim blockquote +
bbox crop preview via /api/crop + custody-steps disclosure
· Empty/in-flight panel ("os detetives estão lendo o corpus")
· Failure panel surfacing error + partial outputs
Inline chat-bubble card (components/chat-bubble.tsx):
- ToolTrace.richRender recognises request_investigation results and
renders a detective banner with status + ETA + link to /jobs/[id]
(target=_blank). Error case renders a red strip with the message.
UX flow now: user asks Sherlock a question → request_investigation
queues the job → chat card shows "🔎 Holmes · hypothesis_tournament ·
ETA ~60s" → user clicks → /jobs/<id> live-updates → 60 s later, 2-3
rival hypotheses + their arguments + chunk citations are rendered with
Bayesian update visible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:26:18 +00:00
|
|
|
function durationMs(started: string | null, finished: string | null, created: string): number | null {
|
|
|
|
|
const a = started ? new Date(started).getTime() : null;
|
|
|
|
|
const b = finished ? new Date(finished).getTime() : null;
|
|
|
|
|
if (a !== null && b !== null) return b - a;
|
|
|
|
|
if (a !== null) return Date.now() - a;
|
|
|
|
|
return Date.now() - new Date(created).getTime();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
export async function GET(
|
|
|
|
|
_request: Request,
|
|
|
|
|
ctx: { params: Promise<{ id: string }> },
|
|
|
|
|
) {
|
|
|
|
|
const { id } = await ctx.params;
|
|
|
|
|
if (!/^[0-9a-f-]{36}$/i.test(id)) {
|
|
|
|
|
return NextResponse.json({ error: "bad_job_id" }, { status: 400 });
|
|
|
|
|
}
|
|
|
|
|
try {
|
|
|
|
|
const rows = await pgQuery<JobRow>(
|
|
|
|
|
`SELECT job_id, kind, payload, status, worker_id, started_at, finished_at,
|
|
|
|
|
outputs, error, created_at
|
|
|
|
|
FROM public.investigation_jobs WHERE job_id = $1`,
|
|
|
|
|
[id],
|
|
|
|
|
);
|
|
|
|
|
const job = rows[0];
|
|
|
|
|
if (!job) return NextResponse.json({ error: "not_found" }, { status: 404 });
|
|
|
|
|
|
|
|
|
|
// Collect IDs surfaced in outputs[] for hydration.
|
|
|
|
|
const evidenceIds: string[] = [];
|
|
|
|
|
const hypothesisIds: string[] = [];
|
W3.7: Dupin contradiction-scan detective + UI integration
Adds the third AI detective in the Investigation Bureau runtime: C. Auguste
Dupin, who scans a corpus shortlist for pairs (or small groups) of chunks
that cannot both be true under any ordinary reading.
Runtime:
- prompts/dupin.md — discipline (no contradiction without ≥2 distinct
chunk_ids; reject same-vocabulary near-misses; FEW high-confidence
over MANY weak ones; emit `NO_CONTRADICTIONS` when corpus is silent)
- src/detectives/dupin.ts — hybridSearch with k=18 (more chunks than
Holmes because contradictions emerge from comparing dispersed
claims), strict JSON-array parsing, AT MOST 3 contradictions per call
- src/tools/write_contradiction.ts — validates topic + ≥2 positions
drawn from ≥2 distinct chunks, resolves chunk_pk via DB lookup
(rejects positions citing unknown chunks), INSERTs into
public.contradictions + writes case/contradictions/R-NNNN.md
- orchestrator: new `contradiction_scan` kind dispatching to runDupin;
payload { topic, doc_id?, lang?, context_chunks? }
Chat + UI:
- request_investigation gains kind=contradiction_scan + topic arg;
triggered detective auto-resolves to dupin
- chat-bubble inline card renders dupin in orange (#ff8a4d) to
distinguish from holmes (cyan) and locard (green)
- /jobs/[id] page swaps title + subtitle + tone per detective;
"Question" label becomes "Topic" for contradiction_scan
- /api/jobs/[id] hydrates public.contradictions when outputs[] surfaces
contradiction_ids
- job-status-poller renders ContradictionCard: topic + N positions
(verbatim statements quoted, stance label optional, link to source
chunk) + optional notes panel, with resolution_status badge
(open/resolved/irreconcilable)
R-NNNN shares the contradiction_id_seq slot with relation per
CLAUDE.md naming — same conceptual class (a connection between two
pieces of evidence in tension).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:34:04 +00:00
|
|
|
const contradictionIds: string[] = [];
|
W3.8: Investigation Bureau complete — Poirot, Taleb, Tetlock, Case-Writer
Brings the bureau from 4 → 8 detectives. All eight run as Bun + claude-CLI
subprocesses against the same Supabase + investigation_jobs LISTEN/NOTIFY
queue, sharing search.ts hybridSearch and writer-side validators that
gate writes against schema + FK.
New detectives:
Poirot (witness_analysis)
- prompts/poirot.md — credibility / access / bias / corroboration /
verdict; uses entity_mentions JOIN chunks to pull 12 chunks per
person; resolves corroboration_refs chunk_ids defensively (accepts
bare cNNNN even when the model emits pNNN/cNNNN).
- INSERT into public.witnesses with W-NNNN naming.
- Tone: purple (#9b5de5).
Taleb (outlier_scan)
- prompts/taleb.md — "surprise is relative to a model"; at most 3
outliers; each requires explicit dominant_model + why_surprising +
what_it_implies; fan-out into public.gaps with scope.kind="outlier".
- Same unscoped-fallback as Dupin (Pass 1 with doc_id, Pass 2 widens
to corpus if hits < 3).
- Tone: yellow (#ffd23f).
Tetlock (calibrate_hypothesis)
- prompts/tetlock.md — honest Bayesian update; emits new_posterior +
Δ + recommended_action ∈ {keep, downgrade, upgrade, supersede}.
- write_calibration UPDATEs public.hypotheses + APPENDS a
"## Calibration history" section to the H-NNNN.md case file
(calibration is append-only — each datapoint matters). Posterior
band auto-corrected to match Tetlock thresholds.
- NO_NEW_EVIDENCE sentinel handled; pure 'keep' with |Δ|<0.005 only
touches updated_at + reviewed_by.
- Tone: teal (#26d4cc).
Case-Writer (case_report)
- prompts/case-writer.md — Dr. Watson assembles all artefacts
(E-NNNN, H-NNNN, R-NNNN, W-NNNN, G-NNNN) into a five-act narrative.
ILIKE filter on topic; doc_id optional scope.
- Larger budget cap (≥ $0.50) + longer timeout for prose generation.
- Writes case/reports/<slug>.md with frontmatter (topic + counts);
no DB table for v0.
- New page /c/[slug] renders the report via MarkdownBody + stat chips.
- Tone: gold (#e0c080).
Hardening across the bureau:
- Sentinel parsing now accepts backticked AND prose-trailing forms
(Holmes NO_HYPOTHESES, Dupin NO_CONTRADICTIONS, Schneier
INSUFFICIENT_HYPOTHESIS, Poirot INSUFFICIENT_TESTIMONY, Taleb
NO_OUTLIERS, Tetlock NO_NEW_EVIDENCE, Case-Writer
INSUFFICIENT_ARTEFACTS). Avoids the failure mode where the model
refuses honestly but the runtime treated it as a parse error
(observed live with Poirot+Hoover identifying the DIRECTOR
false-positive disambiguation issue in entity_mentions).
Chat tool extensions (web/lib/chat/tools.ts):
- request_investigation now accepts 7 kinds. Each routes to its
detective with appropriate validation (hypothesis_id regex,
person_id kebab-case, topic non-empty, doc_id for evidence_chain).
- ETA per kind: Holmes/Dupin 60s, Poirot 45s, Schneier/Tetlock 30s,
Taleb 50s, Case-Writer 180s (longer prose), Locard 30×n_chunks.
UI integration:
- chat-bubble inline card paints each detective in its tone color.
- /jobs/[id] page header swaps name/subtitle/tone per detective;
question label adapts ("Topic" / "Hypothesis under attack" /
"Witness under analysis" / "Topic to outlier-scan" / "Hypothesis
under recalibration" / "Case to assemble").
- job-status-poller renders: case-report link card (gold), outlier
cards (yellow), witness cards (purple) — alongside existing
hypothesis, evidence, contradiction cards.
- /api/jobs/[id] hydrates witnesses (JOIN entities for canonical_name)
+ gaps (with scope JSONB).
- /c/[slug] page reads /data/ufo/case/reports/<slug>.md and renders
with MarkdownBody, frontmatter parsed for stat chips.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 01:11:39 +00:00
|
|
|
const witnessIds: string[] = [];
|
|
|
|
|
const gapIds: string[] = [];
|
W3.6: chat request_investigation tool + /jobs/[id] case-file viewer
Closes the loop between the chat UI and the Investigation Bureau runtime.
Chat tool (web/lib/chat/tools.ts):
- request_investigation { kind, question, doc_id?, chunks?, claim? }
INSERTs a row in public.investigation_jobs and returns
{ job_id, kind, status, eta_seconds, status_url, detective }.
- kind=hypothesis_tournament → Holmes (1 question → 2-3 rival hypotheses)
- kind=evidence_chain → Locard (1 doc → grade-A/B/C evidence with chain
of custody, default top-5 anomaly chunks)
- Plumbed user.email through ToolHandlerContext so triggered_by audits
the requesting user.
Public job viewer:
- GET /api/jobs/[id] joins investigation_jobs → public.evidence +
public.hypotheses for the IDs surfaced in outputs[]. Returns one
payload the page can render without n+1 round-trips. Strips
triggered_by from the response (it carries the user's email).
- app/jobs/[id]/page.tsx server-renders the case-file shell:
detective lore header (Holmes blue or Locard green), question chip,
scope chip with link back to the document.
- components/job-status-poller.tsx client island that polls every 3 s
while non-terminal, then once on terminal to hydrate evidence +
hypotheses. Renders:
· Phase tracker (queued → running → complete | failed)
· Hypothesis cards w/ prior + posterior bars + Δ delta indicator
+ Tetlock band badge (high/medium/low/speculation)
· Argument-for / argument-against with [[wiki-link]] auto-linking
to /d/<doc>/p<NNN>#<cNNNN>
· Evidence cards w/ Grade A/B/C badge + verbatim blockquote +
bbox crop preview via /api/crop + custody-steps disclosure
· Empty/in-flight panel ("os detetives estão lendo o corpus")
· Failure panel surfacing error + partial outputs
Inline chat-bubble card (components/chat-bubble.tsx):
- ToolTrace.richRender recognises request_investigation results and
renders a detective banner with status + ETA + link to /jobs/[id]
(target=_blank). Error case renders a red strip with the message.
UX flow now: user asks Sherlock a question → request_investigation
queues the job → chat card shows "🔎 Holmes · hypothesis_tournament ·
ETA ~60s" → user clicks → /jobs/<id> live-updates → 60 s later, 2-3
rival hypotheses + their arguments + chunk citations are rendered with
Bayesian update visible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:26:18 +00:00
|
|
|
if (Array.isArray(job.outputs)) {
|
|
|
|
|
for (const o of job.outputs as Array<Record<string, unknown>>) {
|
|
|
|
|
if (typeof o.evidence_id === "string") evidenceIds.push(o.evidence_id);
|
|
|
|
|
if (typeof o.hypothesis_id === "string") hypothesisIds.push(o.hypothesis_id);
|
W3.7: Dupin contradiction-scan detective + UI integration
Adds the third AI detective in the Investigation Bureau runtime: C. Auguste
Dupin, who scans a corpus shortlist for pairs (or small groups) of chunks
that cannot both be true under any ordinary reading.
Runtime:
- prompts/dupin.md — discipline (no contradiction without ≥2 distinct
chunk_ids; reject same-vocabulary near-misses; FEW high-confidence
over MANY weak ones; emit `NO_CONTRADICTIONS` when corpus is silent)
- src/detectives/dupin.ts — hybridSearch with k=18 (more chunks than
Holmes because contradictions emerge from comparing dispersed
claims), strict JSON-array parsing, AT MOST 3 contradictions per call
- src/tools/write_contradiction.ts — validates topic + ≥2 positions
drawn from ≥2 distinct chunks, resolves chunk_pk via DB lookup
(rejects positions citing unknown chunks), INSERTs into
public.contradictions + writes case/contradictions/R-NNNN.md
- orchestrator: new `contradiction_scan` kind dispatching to runDupin;
payload { topic, doc_id?, lang?, context_chunks? }
Chat + UI:
- request_investigation gains kind=contradiction_scan + topic arg;
triggered detective auto-resolves to dupin
- chat-bubble inline card renders dupin in orange (#ff8a4d) to
distinguish from holmes (cyan) and locard (green)
- /jobs/[id] page swaps title + subtitle + tone per detective;
"Question" label becomes "Topic" for contradiction_scan
- /api/jobs/[id] hydrates public.contradictions when outputs[] surfaces
contradiction_ids
- job-status-poller renders ContradictionCard: topic + N positions
(verbatim statements quoted, stance label optional, link to source
chunk) + optional notes panel, with resolution_status badge
(open/resolved/irreconcilable)
R-NNNN shares the contradiction_id_seq slot with relation per
CLAUDE.md naming — same conceptual class (a connection between two
pieces of evidence in tension).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:34:04 +00:00
|
|
|
if (typeof o.contradiction_id === "string") contradictionIds.push(o.contradiction_id);
|
W3.8: Investigation Bureau complete — Poirot, Taleb, Tetlock, Case-Writer
Brings the bureau from 4 → 8 detectives. All eight run as Bun + claude-CLI
subprocesses against the same Supabase + investigation_jobs LISTEN/NOTIFY
queue, sharing search.ts hybridSearch and writer-side validators that
gate writes against schema + FK.
New detectives:
Poirot (witness_analysis)
- prompts/poirot.md — credibility / access / bias / corroboration /
verdict; uses entity_mentions JOIN chunks to pull 12 chunks per
person; resolves corroboration_refs chunk_ids defensively (accepts
bare cNNNN even when the model emits pNNN/cNNNN).
- INSERT into public.witnesses with W-NNNN naming.
- Tone: purple (#9b5de5).
Taleb (outlier_scan)
- prompts/taleb.md — "surprise is relative to a model"; at most 3
outliers; each requires explicit dominant_model + why_surprising +
what_it_implies; fan-out into public.gaps with scope.kind="outlier".
- Same unscoped-fallback as Dupin (Pass 1 with doc_id, Pass 2 widens
to corpus if hits < 3).
- Tone: yellow (#ffd23f).
Tetlock (calibrate_hypothesis)
- prompts/tetlock.md — honest Bayesian update; emits new_posterior +
Δ + recommended_action ∈ {keep, downgrade, upgrade, supersede}.
- write_calibration UPDATEs public.hypotheses + APPENDS a
"## Calibration history" section to the H-NNNN.md case file
(calibration is append-only — each datapoint matters). Posterior
band auto-corrected to match Tetlock thresholds.
- NO_NEW_EVIDENCE sentinel handled; pure 'keep' with |Δ|<0.005 only
touches updated_at + reviewed_by.
- Tone: teal (#26d4cc).
Case-Writer (case_report)
- prompts/case-writer.md — Dr. Watson assembles all artefacts
(E-NNNN, H-NNNN, R-NNNN, W-NNNN, G-NNNN) into a five-act narrative.
ILIKE filter on topic; doc_id optional scope.
- Larger budget cap (≥ $0.50) + longer timeout for prose generation.
- Writes case/reports/<slug>.md with frontmatter (topic + counts);
no DB table for v0.
- New page /c/[slug] renders the report via MarkdownBody + stat chips.
- Tone: gold (#e0c080).
Hardening across the bureau:
- Sentinel parsing now accepts backticked AND prose-trailing forms
(Holmes NO_HYPOTHESES, Dupin NO_CONTRADICTIONS, Schneier
INSUFFICIENT_HYPOTHESIS, Poirot INSUFFICIENT_TESTIMONY, Taleb
NO_OUTLIERS, Tetlock NO_NEW_EVIDENCE, Case-Writer
INSUFFICIENT_ARTEFACTS). Avoids the failure mode where the model
refuses honestly but the runtime treated it as a parse error
(observed live with Poirot+Hoover identifying the DIRECTOR
false-positive disambiguation issue in entity_mentions).
Chat tool extensions (web/lib/chat/tools.ts):
- request_investigation now accepts 7 kinds. Each routes to its
detective with appropriate validation (hypothesis_id regex,
person_id kebab-case, topic non-empty, doc_id for evidence_chain).
- ETA per kind: Holmes/Dupin 60s, Poirot 45s, Schneier/Tetlock 30s,
Taleb 50s, Case-Writer 180s (longer prose), Locard 30×n_chunks.
UI integration:
- chat-bubble inline card paints each detective in its tone color.
- /jobs/[id] page header swaps name/subtitle/tone per detective;
question label adapts ("Topic" / "Hypothesis under attack" /
"Witness under analysis" / "Topic to outlier-scan" / "Hypothesis
under recalibration" / "Case to assemble").
- job-status-poller renders: case-report link card (gold), outlier
cards (yellow), witness cards (purple) — alongside existing
hypothesis, evidence, contradiction cards.
- /api/jobs/[id] hydrates witnesses (JOIN entities for canonical_name)
+ gaps (with scope JSONB).
- /c/[slug] page reads /data/ufo/case/reports/<slug>.md and renders
with MarkdownBody, frontmatter parsed for stat chips.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 01:11:39 +00:00
|
|
|
if (typeof o.witness_id === "string") witnessIds.push(o.witness_id);
|
|
|
|
|
if (typeof o.gap_id === "string") gapIds.push(o.gap_id);
|
W3.6: chat request_investigation tool + /jobs/[id] case-file viewer
Closes the loop between the chat UI and the Investigation Bureau runtime.
Chat tool (web/lib/chat/tools.ts):
- request_investigation { kind, question, doc_id?, chunks?, claim? }
INSERTs a row in public.investigation_jobs and returns
{ job_id, kind, status, eta_seconds, status_url, detective }.
- kind=hypothesis_tournament → Holmes (1 question → 2-3 rival hypotheses)
- kind=evidence_chain → Locard (1 doc → grade-A/B/C evidence with chain
of custody, default top-5 anomaly chunks)
- Plumbed user.email through ToolHandlerContext so triggered_by audits
the requesting user.
Public job viewer:
- GET /api/jobs/[id] joins investigation_jobs → public.evidence +
public.hypotheses for the IDs surfaced in outputs[]. Returns one
payload the page can render without n+1 round-trips. Strips
triggered_by from the response (it carries the user's email).
- app/jobs/[id]/page.tsx server-renders the case-file shell:
detective lore header (Holmes blue or Locard green), question chip,
scope chip with link back to the document.
- components/job-status-poller.tsx client island that polls every 3 s
while non-terminal, then once on terminal to hydrate evidence +
hypotheses. Renders:
· Phase tracker (queued → running → complete | failed)
· Hypothesis cards w/ prior + posterior bars + Δ delta indicator
+ Tetlock band badge (high/medium/low/speculation)
· Argument-for / argument-against with [[wiki-link]] auto-linking
to /d/<doc>/p<NNN>#<cNNNN>
· Evidence cards w/ Grade A/B/C badge + verbatim blockquote +
bbox crop preview via /api/crop + custody-steps disclosure
· Empty/in-flight panel ("os detetives estão lendo o corpus")
· Failure panel surfacing error + partial outputs
Inline chat-bubble card (components/chat-bubble.tsx):
- ToolTrace.richRender recognises request_investigation results and
renders a detective banner with status + ETA + link to /jobs/[id]
(target=_blank). Error case renders a red strip with the message.
UX flow now: user asks Sherlock a question → request_investigation
queues the job → chat card shows "🔎 Holmes · hypothesis_tournament ·
ETA ~60s" → user clicks → /jobs/<id> live-updates → 60 s later, 2-3
rival hypotheses + their arguments + chunk citations are rendered with
Bayesian update visible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:26:18 +00:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
W3.8: Investigation Bureau complete — Poirot, Taleb, Tetlock, Case-Writer
Brings the bureau from 4 → 8 detectives. All eight run as Bun + claude-CLI
subprocesses against the same Supabase + investigation_jobs LISTEN/NOTIFY
queue, sharing search.ts hybridSearch and writer-side validators that
gate writes against schema + FK.
New detectives:
Poirot (witness_analysis)
- prompts/poirot.md — credibility / access / bias / corroboration /
verdict; uses entity_mentions JOIN chunks to pull 12 chunks per
person; resolves corroboration_refs chunk_ids defensively (accepts
bare cNNNN even when the model emits pNNN/cNNNN).
- INSERT into public.witnesses with W-NNNN naming.
- Tone: purple (#9b5de5).
Taleb (outlier_scan)
- prompts/taleb.md — "surprise is relative to a model"; at most 3
outliers; each requires explicit dominant_model + why_surprising +
what_it_implies; fan-out into public.gaps with scope.kind="outlier".
- Same unscoped-fallback as Dupin (Pass 1 with doc_id, Pass 2 widens
to corpus if hits < 3).
- Tone: yellow (#ffd23f).
Tetlock (calibrate_hypothesis)
- prompts/tetlock.md — honest Bayesian update; emits new_posterior +
Δ + recommended_action ∈ {keep, downgrade, upgrade, supersede}.
- write_calibration UPDATEs public.hypotheses + APPENDS a
"## Calibration history" section to the H-NNNN.md case file
(calibration is append-only — each datapoint matters). Posterior
band auto-corrected to match Tetlock thresholds.
- NO_NEW_EVIDENCE sentinel handled; pure 'keep' with |Δ|<0.005 only
touches updated_at + reviewed_by.
- Tone: teal (#26d4cc).
Case-Writer (case_report)
- prompts/case-writer.md — Dr. Watson assembles all artefacts
(E-NNNN, H-NNNN, R-NNNN, W-NNNN, G-NNNN) into a five-act narrative.
ILIKE filter on topic; doc_id optional scope.
- Larger budget cap (≥ $0.50) + longer timeout for prose generation.
- Writes case/reports/<slug>.md with frontmatter (topic + counts);
no DB table for v0.
- New page /c/[slug] renders the report via MarkdownBody + stat chips.
- Tone: gold (#e0c080).
Hardening across the bureau:
- Sentinel parsing now accepts backticked AND prose-trailing forms
(Holmes NO_HYPOTHESES, Dupin NO_CONTRADICTIONS, Schneier
INSUFFICIENT_HYPOTHESIS, Poirot INSUFFICIENT_TESTIMONY, Taleb
NO_OUTLIERS, Tetlock NO_NEW_EVIDENCE, Case-Writer
INSUFFICIENT_ARTEFACTS). Avoids the failure mode where the model
refuses honestly but the runtime treated it as a parse error
(observed live with Poirot+Hoover identifying the DIRECTOR
false-positive disambiguation issue in entity_mentions).
Chat tool extensions (web/lib/chat/tools.ts):
- request_investigation now accepts 7 kinds. Each routes to its
detective with appropriate validation (hypothesis_id regex,
person_id kebab-case, topic non-empty, doc_id for evidence_chain).
- ETA per kind: Holmes/Dupin 60s, Poirot 45s, Schneier/Tetlock 30s,
Taleb 50s, Case-Writer 180s (longer prose), Locard 30×n_chunks.
UI integration:
- chat-bubble inline card paints each detective in its tone color.
- /jobs/[id] page header swaps name/subtitle/tone per detective;
question label adapts ("Topic" / "Hypothesis under attack" /
"Witness under analysis" / "Topic to outlier-scan" / "Hypothesis
under recalibration" / "Case to assemble").
- job-status-poller renders: case-report link card (gold), outlier
cards (yellow), witness cards (purple) — alongside existing
hypothesis, evidence, contradiction cards.
- /api/jobs/[id] hydrates witnesses (JOIN entities for canonical_name)
+ gaps (with scope JSONB).
- /c/[slug] page reads /data/ufo/case/reports/<slug>.md and renders
with MarkdownBody, frontmatter parsed for stat chips.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 01:11:39 +00:00
|
|
|
const [evidence, hypotheses, contradictions, witnesses, gaps] = await Promise.all([
|
W3.6: chat request_investigation tool + /jobs/[id] case-file viewer
Closes the loop between the chat UI and the Investigation Bureau runtime.
Chat tool (web/lib/chat/tools.ts):
- request_investigation { kind, question, doc_id?, chunks?, claim? }
INSERTs a row in public.investigation_jobs and returns
{ job_id, kind, status, eta_seconds, status_url, detective }.
- kind=hypothesis_tournament → Holmes (1 question → 2-3 rival hypotheses)
- kind=evidence_chain → Locard (1 doc → grade-A/B/C evidence with chain
of custody, default top-5 anomaly chunks)
- Plumbed user.email through ToolHandlerContext so triggered_by audits
the requesting user.
Public job viewer:
- GET /api/jobs/[id] joins investigation_jobs → public.evidence +
public.hypotheses for the IDs surfaced in outputs[]. Returns one
payload the page can render without n+1 round-trips. Strips
triggered_by from the response (it carries the user's email).
- app/jobs/[id]/page.tsx server-renders the case-file shell:
detective lore header (Holmes blue or Locard green), question chip,
scope chip with link back to the document.
- components/job-status-poller.tsx client island that polls every 3 s
while non-terminal, then once on terminal to hydrate evidence +
hypotheses. Renders:
· Phase tracker (queued → running → complete | failed)
· Hypothesis cards w/ prior + posterior bars + Δ delta indicator
+ Tetlock band badge (high/medium/low/speculation)
· Argument-for / argument-against with [[wiki-link]] auto-linking
to /d/<doc>/p<NNN>#<cNNNN>
· Evidence cards w/ Grade A/B/C badge + verbatim blockquote +
bbox crop preview via /api/crop + custody-steps disclosure
· Empty/in-flight panel ("os detetives estão lendo o corpus")
· Failure panel surfacing error + partial outputs
Inline chat-bubble card (components/chat-bubble.tsx):
- ToolTrace.richRender recognises request_investigation results and
renders a detective banner with status + ETA + link to /jobs/[id]
(target=_blank). Error case renders a red strip with the message.
UX flow now: user asks Sherlock a question → request_investigation
queues the job → chat card shows "🔎 Holmes · hypothesis_tournament ·
ETA ~60s" → user clicks → /jobs/<id> live-updates → 60 s later, 2-3
rival hypotheses + their arguments + chunk citations are rendered with
Bayesian update visible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:26:18 +00:00
|
|
|
evidenceIds.length > 0
|
|
|
|
|
? pgQuery<EvidenceRow>(
|
|
|
|
|
`SELECT e.evidence_id, e.grade, e.source_page_id,
|
|
|
|
|
split_part(e.source_page_id, '/p', 1) AS doc_id,
|
|
|
|
|
NULLIF(split_part(e.source_page_id, '/p', 2), '')::int AS page,
|
|
|
|
|
c.chunk_id, e.verbatim_excerpt, e.custody_steps, e.bbox,
|
|
|
|
|
e.confidence_band, e.related_hypotheses
|
|
|
|
|
FROM public.evidence e
|
|
|
|
|
LEFT JOIN public.chunks c ON c.chunk_pk = e.source_chunk_pk
|
|
|
|
|
WHERE e.evidence_id = ANY($1::text[])
|
|
|
|
|
ORDER BY e.evidence_id`,
|
|
|
|
|
[evidenceIds],
|
|
|
|
|
)
|
|
|
|
|
: Promise.resolve([] as EvidenceRow[]),
|
|
|
|
|
hypothesisIds.length > 0
|
|
|
|
|
? pgQuery<HypothesisRow>(
|
|
|
|
|
`SELECT hypothesis_id, question, position, argument_for, argument_against,
|
|
|
|
|
prior, posterior, confidence_band, status, evidence_refs
|
|
|
|
|
FROM public.hypotheses
|
|
|
|
|
WHERE hypothesis_id = ANY($1::text[])
|
|
|
|
|
ORDER BY hypothesis_id`,
|
|
|
|
|
[hypothesisIds],
|
|
|
|
|
)
|
|
|
|
|
: Promise.resolve([] as HypothesisRow[]),
|
W3.7: Dupin contradiction-scan detective + UI integration
Adds the third AI detective in the Investigation Bureau runtime: C. Auguste
Dupin, who scans a corpus shortlist for pairs (or small groups) of chunks
that cannot both be true under any ordinary reading.
Runtime:
- prompts/dupin.md — discipline (no contradiction without ≥2 distinct
chunk_ids; reject same-vocabulary near-misses; FEW high-confidence
over MANY weak ones; emit `NO_CONTRADICTIONS` when corpus is silent)
- src/detectives/dupin.ts — hybridSearch with k=18 (more chunks than
Holmes because contradictions emerge from comparing dispersed
claims), strict JSON-array parsing, AT MOST 3 contradictions per call
- src/tools/write_contradiction.ts — validates topic + ≥2 positions
drawn from ≥2 distinct chunks, resolves chunk_pk via DB lookup
(rejects positions citing unknown chunks), INSERTs into
public.contradictions + writes case/contradictions/R-NNNN.md
- orchestrator: new `contradiction_scan` kind dispatching to runDupin;
payload { topic, doc_id?, lang?, context_chunks? }
Chat + UI:
- request_investigation gains kind=contradiction_scan + topic arg;
triggered detective auto-resolves to dupin
- chat-bubble inline card renders dupin in orange (#ff8a4d) to
distinguish from holmes (cyan) and locard (green)
- /jobs/[id] page swaps title + subtitle + tone per detective;
"Question" label becomes "Topic" for contradiction_scan
- /api/jobs/[id] hydrates public.contradictions when outputs[] surfaces
contradiction_ids
- job-status-poller renders ContradictionCard: topic + N positions
(verbatim statements quoted, stance label optional, link to source
chunk) + optional notes panel, with resolution_status badge
(open/resolved/irreconcilable)
R-NNNN shares the contradiction_id_seq slot with relation per
CLAUDE.md naming — same conceptual class (a connection between two
pieces of evidence in tension).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:34:04 +00:00
|
|
|
contradictionIds.length > 0
|
|
|
|
|
? pgQuery<ContradictionRow>(
|
|
|
|
|
`SELECT contradiction_id, topic, chunks, resolution_status, notes, detected_by
|
|
|
|
|
FROM public.contradictions
|
|
|
|
|
WHERE contradiction_id = ANY($1::text[])
|
|
|
|
|
ORDER BY contradiction_id`,
|
|
|
|
|
[contradictionIds],
|
|
|
|
|
)
|
|
|
|
|
: Promise.resolve([] as ContradictionRow[]),
|
W3.8: Investigation Bureau complete — Poirot, Taleb, Tetlock, Case-Writer
Brings the bureau from 4 → 8 detectives. All eight run as Bun + claude-CLI
subprocesses against the same Supabase + investigation_jobs LISTEN/NOTIFY
queue, sharing search.ts hybridSearch and writer-side validators that
gate writes against schema + FK.
New detectives:
Poirot (witness_analysis)
- prompts/poirot.md — credibility / access / bias / corroboration /
verdict; uses entity_mentions JOIN chunks to pull 12 chunks per
person; resolves corroboration_refs chunk_ids defensively (accepts
bare cNNNN even when the model emits pNNN/cNNNN).
- INSERT into public.witnesses with W-NNNN naming.
- Tone: purple (#9b5de5).
Taleb (outlier_scan)
- prompts/taleb.md — "surprise is relative to a model"; at most 3
outliers; each requires explicit dominant_model + why_surprising +
what_it_implies; fan-out into public.gaps with scope.kind="outlier".
- Same unscoped-fallback as Dupin (Pass 1 with doc_id, Pass 2 widens
to corpus if hits < 3).
- Tone: yellow (#ffd23f).
Tetlock (calibrate_hypothesis)
- prompts/tetlock.md — honest Bayesian update; emits new_posterior +
Δ + recommended_action ∈ {keep, downgrade, upgrade, supersede}.
- write_calibration UPDATEs public.hypotheses + APPENDS a
"## Calibration history" section to the H-NNNN.md case file
(calibration is append-only — each datapoint matters). Posterior
band auto-corrected to match Tetlock thresholds.
- NO_NEW_EVIDENCE sentinel handled; pure 'keep' with |Δ|<0.005 only
touches updated_at + reviewed_by.
- Tone: teal (#26d4cc).
Case-Writer (case_report)
- prompts/case-writer.md — Dr. Watson assembles all artefacts
(E-NNNN, H-NNNN, R-NNNN, W-NNNN, G-NNNN) into a five-act narrative.
ILIKE filter on topic; doc_id optional scope.
- Larger budget cap (≥ $0.50) + longer timeout for prose generation.
- Writes case/reports/<slug>.md with frontmatter (topic + counts);
no DB table for v0.
- New page /c/[slug] renders the report via MarkdownBody + stat chips.
- Tone: gold (#e0c080).
Hardening across the bureau:
- Sentinel parsing now accepts backticked AND prose-trailing forms
(Holmes NO_HYPOTHESES, Dupin NO_CONTRADICTIONS, Schneier
INSUFFICIENT_HYPOTHESIS, Poirot INSUFFICIENT_TESTIMONY, Taleb
NO_OUTLIERS, Tetlock NO_NEW_EVIDENCE, Case-Writer
INSUFFICIENT_ARTEFACTS). Avoids the failure mode where the model
refuses honestly but the runtime treated it as a parse error
(observed live with Poirot+Hoover identifying the DIRECTOR
false-positive disambiguation issue in entity_mentions).
Chat tool extensions (web/lib/chat/tools.ts):
- request_investigation now accepts 7 kinds. Each routes to its
detective with appropriate validation (hypothesis_id regex,
person_id kebab-case, topic non-empty, doc_id for evidence_chain).
- ETA per kind: Holmes/Dupin 60s, Poirot 45s, Schneier/Tetlock 30s,
Taleb 50s, Case-Writer 180s (longer prose), Locard 30×n_chunks.
UI integration:
- chat-bubble inline card paints each detective in its tone color.
- /jobs/[id] page header swaps name/subtitle/tone per detective;
question label adapts ("Topic" / "Hypothesis under attack" /
"Witness under analysis" / "Topic to outlier-scan" / "Hypothesis
under recalibration" / "Case to assemble").
- job-status-poller renders: case-report link card (gold), outlier
cards (yellow), witness cards (purple) — alongside existing
hypothesis, evidence, contradiction cards.
- /api/jobs/[id] hydrates witnesses (JOIN entities for canonical_name)
+ gaps (with scope JSONB).
- /c/[slug] page reads /data/ufo/case/reports/<slug>.md and renders
with MarkdownBody, frontmatter parsed for stat chips.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 01:11:39 +00:00
|
|
|
witnessIds.length > 0
|
|
|
|
|
? pgQuery<WitnessRow>(
|
|
|
|
|
`SELECT w.witness_id, e.canonical_name, e.entity_id,
|
|
|
|
|
w.credibility, w.access_to_event, w.bias_notes,
|
|
|
|
|
w.corroboration_refs, w.verdict
|
|
|
|
|
FROM public.witnesses w
|
|
|
|
|
LEFT JOIN public.entities e ON e.entity_pk = w.person_entity_pk
|
|
|
|
|
WHERE w.witness_id = ANY($1::text[])
|
|
|
|
|
ORDER BY w.witness_id`,
|
|
|
|
|
[witnessIds],
|
|
|
|
|
)
|
|
|
|
|
: Promise.resolve([] as WitnessRow[]),
|
|
|
|
|
gapIds.length > 0
|
|
|
|
|
? pgQuery<GapRow>(
|
|
|
|
|
`SELECT gap_id, description, scope, suggested_next_move, status, created_by
|
|
|
|
|
FROM public.gaps WHERE gap_id = ANY($1::text[]) ORDER BY gap_id`,
|
|
|
|
|
[gapIds],
|
|
|
|
|
)
|
|
|
|
|
: Promise.resolve([] as GapRow[]),
|
W3.6: chat request_investigation tool + /jobs/[id] case-file viewer
Closes the loop between the chat UI and the Investigation Bureau runtime.
Chat tool (web/lib/chat/tools.ts):
- request_investigation { kind, question, doc_id?, chunks?, claim? }
INSERTs a row in public.investigation_jobs and returns
{ job_id, kind, status, eta_seconds, status_url, detective }.
- kind=hypothesis_tournament → Holmes (1 question → 2-3 rival hypotheses)
- kind=evidence_chain → Locard (1 doc → grade-A/B/C evidence with chain
of custody, default top-5 anomaly chunks)
- Plumbed user.email through ToolHandlerContext so triggered_by audits
the requesting user.
Public job viewer:
- GET /api/jobs/[id] joins investigation_jobs → public.evidence +
public.hypotheses for the IDs surfaced in outputs[]. Returns one
payload the page can render without n+1 round-trips. Strips
triggered_by from the response (it carries the user's email).
- app/jobs/[id]/page.tsx server-renders the case-file shell:
detective lore header (Holmes blue or Locard green), question chip,
scope chip with link back to the document.
- components/job-status-poller.tsx client island that polls every 3 s
while non-terminal, then once on terminal to hydrate evidence +
hypotheses. Renders:
· Phase tracker (queued → running → complete | failed)
· Hypothesis cards w/ prior + posterior bars + Δ delta indicator
+ Tetlock band badge (high/medium/low/speculation)
· Argument-for / argument-against with [[wiki-link]] auto-linking
to /d/<doc>/p<NNN>#<cNNNN>
· Evidence cards w/ Grade A/B/C badge + verbatim blockquote +
bbox crop preview via /api/crop + custody-steps disclosure
· Empty/in-flight panel ("os detetives estão lendo o corpus")
· Failure panel surfacing error + partial outputs
Inline chat-bubble card (components/chat-bubble.tsx):
- ToolTrace.richRender recognises request_investigation results and
renders a detective banner with status + ETA + link to /jobs/[id]
(target=_blank). Error case renders a red strip with the message.
UX flow now: user asks Sherlock a question → request_investigation
queues the job → chat card shows "🔎 Holmes · hypothesis_tournament ·
ETA ~60s" → user clicks → /jobs/<id> live-updates → 60 s later, 2-3
rival hypotheses + their arguments + chunk citations are rendered with
Bayesian update visible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:26:18 +00:00
|
|
|
]);
|
|
|
|
|
|
|
|
|
|
return NextResponse.json({
|
|
|
|
|
job_id: job.job_id,
|
|
|
|
|
kind: job.kind,
|
|
|
|
|
payload: job.payload,
|
|
|
|
|
status: job.status,
|
|
|
|
|
worker_id: job.worker_id,
|
|
|
|
|
started_at: job.started_at,
|
|
|
|
|
finished_at: job.finished_at,
|
|
|
|
|
created_at: job.created_at,
|
|
|
|
|
duration_ms: durationMs(job.started_at, job.finished_at, job.created_at),
|
|
|
|
|
error: job.error,
|
|
|
|
|
outputs: Array.isArray(job.outputs) ? job.outputs : [],
|
|
|
|
|
evidence,
|
|
|
|
|
hypotheses,
|
W3.7: Dupin contradiction-scan detective + UI integration
Adds the third AI detective in the Investigation Bureau runtime: C. Auguste
Dupin, who scans a corpus shortlist for pairs (or small groups) of chunks
that cannot both be true under any ordinary reading.
Runtime:
- prompts/dupin.md — discipline (no contradiction without ≥2 distinct
chunk_ids; reject same-vocabulary near-misses; FEW high-confidence
over MANY weak ones; emit `NO_CONTRADICTIONS` when corpus is silent)
- src/detectives/dupin.ts — hybridSearch with k=18 (more chunks than
Holmes because contradictions emerge from comparing dispersed
claims), strict JSON-array parsing, AT MOST 3 contradictions per call
- src/tools/write_contradiction.ts — validates topic + ≥2 positions
drawn from ≥2 distinct chunks, resolves chunk_pk via DB lookup
(rejects positions citing unknown chunks), INSERTs into
public.contradictions + writes case/contradictions/R-NNNN.md
- orchestrator: new `contradiction_scan` kind dispatching to runDupin;
payload { topic, doc_id?, lang?, context_chunks? }
Chat + UI:
- request_investigation gains kind=contradiction_scan + topic arg;
triggered detective auto-resolves to dupin
- chat-bubble inline card renders dupin in orange (#ff8a4d) to
distinguish from holmes (cyan) and locard (green)
- /jobs/[id] page swaps title + subtitle + tone per detective;
"Question" label becomes "Topic" for contradiction_scan
- /api/jobs/[id] hydrates public.contradictions when outputs[] surfaces
contradiction_ids
- job-status-poller renders ContradictionCard: topic + N positions
(verbatim statements quoted, stance label optional, link to source
chunk) + optional notes panel, with resolution_status badge
(open/resolved/irreconcilable)
R-NNNN shares the contradiction_id_seq slot with relation per
CLAUDE.md naming — same conceptual class (a connection between two
pieces of evidence in tension).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:34:04 +00:00
|
|
|
contradictions,
|
W3.8: Investigation Bureau complete — Poirot, Taleb, Tetlock, Case-Writer
Brings the bureau from 4 → 8 detectives. All eight run as Bun + claude-CLI
subprocesses against the same Supabase + investigation_jobs LISTEN/NOTIFY
queue, sharing search.ts hybridSearch and writer-side validators that
gate writes against schema + FK.
New detectives:
Poirot (witness_analysis)
- prompts/poirot.md — credibility / access / bias / corroboration /
verdict; uses entity_mentions JOIN chunks to pull 12 chunks per
person; resolves corroboration_refs chunk_ids defensively (accepts
bare cNNNN even when the model emits pNNN/cNNNN).
- INSERT into public.witnesses with W-NNNN naming.
- Tone: purple (#9b5de5).
Taleb (outlier_scan)
- prompts/taleb.md — "surprise is relative to a model"; at most 3
outliers; each requires explicit dominant_model + why_surprising +
what_it_implies; fan-out into public.gaps with scope.kind="outlier".
- Same unscoped-fallback as Dupin (Pass 1 with doc_id, Pass 2 widens
to corpus if hits < 3).
- Tone: yellow (#ffd23f).
Tetlock (calibrate_hypothesis)
- prompts/tetlock.md — honest Bayesian update; emits new_posterior +
Δ + recommended_action ∈ {keep, downgrade, upgrade, supersede}.
- write_calibration UPDATEs public.hypotheses + APPENDS a
"## Calibration history" section to the H-NNNN.md case file
(calibration is append-only — each datapoint matters). Posterior
band auto-corrected to match Tetlock thresholds.
- NO_NEW_EVIDENCE sentinel handled; pure 'keep' with |Δ|<0.005 only
touches updated_at + reviewed_by.
- Tone: teal (#26d4cc).
Case-Writer (case_report)
- prompts/case-writer.md — Dr. Watson assembles all artefacts
(E-NNNN, H-NNNN, R-NNNN, W-NNNN, G-NNNN) into a five-act narrative.
ILIKE filter on topic; doc_id optional scope.
- Larger budget cap (≥ $0.50) + longer timeout for prose generation.
- Writes case/reports/<slug>.md with frontmatter (topic + counts);
no DB table for v0.
- New page /c/[slug] renders the report via MarkdownBody + stat chips.
- Tone: gold (#e0c080).
Hardening across the bureau:
- Sentinel parsing now accepts backticked AND prose-trailing forms
(Holmes NO_HYPOTHESES, Dupin NO_CONTRADICTIONS, Schneier
INSUFFICIENT_HYPOTHESIS, Poirot INSUFFICIENT_TESTIMONY, Taleb
NO_OUTLIERS, Tetlock NO_NEW_EVIDENCE, Case-Writer
INSUFFICIENT_ARTEFACTS). Avoids the failure mode where the model
refuses honestly but the runtime treated it as a parse error
(observed live with Poirot+Hoover identifying the DIRECTOR
false-positive disambiguation issue in entity_mentions).
Chat tool extensions (web/lib/chat/tools.ts):
- request_investigation now accepts 7 kinds. Each routes to its
detective with appropriate validation (hypothesis_id regex,
person_id kebab-case, topic non-empty, doc_id for evidence_chain).
- ETA per kind: Holmes/Dupin 60s, Poirot 45s, Schneier/Tetlock 30s,
Taleb 50s, Case-Writer 180s (longer prose), Locard 30×n_chunks.
UI integration:
- chat-bubble inline card paints each detective in its tone color.
- /jobs/[id] page header swaps name/subtitle/tone per detective;
question label adapts ("Topic" / "Hypothesis under attack" /
"Witness under analysis" / "Topic to outlier-scan" / "Hypothesis
under recalibration" / "Case to assemble").
- job-status-poller renders: case-report link card (gold), outlier
cards (yellow), witness cards (purple) — alongside existing
hypothesis, evidence, contradiction cards.
- /api/jobs/[id] hydrates witnesses (JOIN entities for canonical_name)
+ gaps (with scope JSONB).
- /c/[slug] page reads /data/ufo/case/reports/<slug>.md and renders
with MarkdownBody, frontmatter parsed for stat chips.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 01:11:39 +00:00
|
|
|
witnesses,
|
|
|
|
|
gaps,
|
W3.6: chat request_investigation tool + /jobs/[id] case-file viewer
Closes the loop between the chat UI and the Investigation Bureau runtime.
Chat tool (web/lib/chat/tools.ts):
- request_investigation { kind, question, doc_id?, chunks?, claim? }
INSERTs a row in public.investigation_jobs and returns
{ job_id, kind, status, eta_seconds, status_url, detective }.
- kind=hypothesis_tournament → Holmes (1 question → 2-3 rival hypotheses)
- kind=evidence_chain → Locard (1 doc → grade-A/B/C evidence with chain
of custody, default top-5 anomaly chunks)
- Plumbed user.email through ToolHandlerContext so triggered_by audits
the requesting user.
Public job viewer:
- GET /api/jobs/[id] joins investigation_jobs → public.evidence +
public.hypotheses for the IDs surfaced in outputs[]. Returns one
payload the page can render without n+1 round-trips. Strips
triggered_by from the response (it carries the user's email).
- app/jobs/[id]/page.tsx server-renders the case-file shell:
detective lore header (Holmes blue or Locard green), question chip,
scope chip with link back to the document.
- components/job-status-poller.tsx client island that polls every 3 s
while non-terminal, then once on terminal to hydrate evidence +
hypotheses. Renders:
· Phase tracker (queued → running → complete | failed)
· Hypothesis cards w/ prior + posterior bars + Δ delta indicator
+ Tetlock band badge (high/medium/low/speculation)
· Argument-for / argument-against with [[wiki-link]] auto-linking
to /d/<doc>/p<NNN>#<cNNNN>
· Evidence cards w/ Grade A/B/C badge + verbatim blockquote +
bbox crop preview via /api/crop + custody-steps disclosure
· Empty/in-flight panel ("os detetives estão lendo o corpus")
· Failure panel surfacing error + partial outputs
Inline chat-bubble card (components/chat-bubble.tsx):
- ToolTrace.richRender recognises request_investigation results and
renders a detective banner with status + ETA + link to /jobs/[id]
(target=_blank). Error case renders a red strip with the message.
UX flow now: user asks Sherlock a question → request_investigation
queues the job → chat card shows "🔎 Holmes · hypothesis_tournament ·
ETA ~60s" → user clicks → /jobs/<id> live-updates → 60 s later, 2-3
rival hypotheses + their arguments + chunk citations are rendered with
Bayesian update visible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:26:18 +00:00
|
|
|
});
|
|
|
|
|
} catch (e) {
|
|
|
|
|
return NextResponse.json({ error: "db_unavailable", message: (e as Error).message }, { status: 503 });
|
|
|
|
|
}
|
|
|
|
|
}
|