W3.9: surface the Investigation Bureau on the homepage + /bureau hub
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 40s
CI / Scripts — Python smoke (push) Failing after 3s
CI / Web — npm audit (push) Failing after 31s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s

Closes a UX gap the user surfaced: W3.5-3.8 built 8 detectives, 4 new
URL endpoints (/jobs/[id], /h/[id], /c/[slug], /api/h/[id]/red-team)
and a chat tool, but the homepage was unchanged — the bureau was
invisible unless you knew the URL or asked the chat to invoke
request_investigation.

Homepage (web/app/page.tsx):
  - Title `▍ war.gov/ufo — Investigative Wiki` → `▍ The Disclosure Bureau`
  - Subtitle expanded from "Holmes · Poirot · Dupin · Locard" to all 8
    detectives (Holmes · Locard · Dupin · Schneier · Poirot · Taleb ·
    Tetlock · Case-Writer)
  - New `🔎 bureau` topbar link (gold, between graph/stats and batch)
  - BureauSnapshot inserted right after the header

BureauSnapshot (web/components/bureau-snapshot.tsx) — server component:
  - 8 detective tiles with role labels (each in its tone color)
  - 6 clickable counters (evidence / hypotheses / contradictions /
    witnesses / outliers / case reports) — anchor to /bureau#section
  - 6 "recent artefacts" columns surfacing the last 3-4 of each kind:
    hypotheses with prior→posterior + band + ↳reviewed_by marker,
    contradictions with topic + resolution_status, evidence with
    Grade badge + verbatim quote, outliers with title + scope.kind,
    witness analyses with canonical_name + credibility + verdict,
    case reports with slug + link to /c/<slug>
  - "Recent jobs" strip linking to /jobs/[id] color-coded by status
  - Reports read from /data/ufo/case/reports/ via fs.readdir + stat,
    sorted by mtime — no DB round-trip needed for that section

/bureau (web/app/bureau/page.tsx) — full hub:
  - Header with full counts
  - 7 sections (anchored to homepage counter links): Case reports,
    Hypotheses, Evidence, Contradictions, Outliers, Witnesses,
    Recent jobs table — each rendering up to 100 rows
  - Reports section parses frontmatter from each .md to surface topic
    + n_hypotheses + n_evidence on the card

Runtime fixes batched in:
  - Poirot: coerce entity_pk via Number() — node-postgres returns
    BIGINT as string by default; writer's Number.isFinite() rejected
    it as "person_entity_pk required" (j-edgar-hoover retry path)
  - Tetlock: write_calibration rationale cap 600 → 1200 chars. Prompt
    still asks ≤ 600 but a 2× slack beats failing the job on honest
    analysis. Observed live: Tetlock emitted ~620 chars on H-0003 and
    the writer rejected the entire calibration.
  - Case-Writer: Promise.all of 5 queries × max_parallel=2 jobs
    demanded up to 10 connections against the investigator role's
    rolconnlimit=4 → "too many connections for role investigator".
    Sequentialized — the LLM call is the hot path, not these queries.

Smoke results visible now on the homepage:
  - 3 hypotheses (H-0001/2/3) about green fireballs origin
  - 3 contradictions (R-0001/2/3) about color, geographic confinement,
    exclusive-green vs multicolored
  - 2 evidence cards (E-0002/3) Grade B
  - 3 outliers (G-0001/2/3) — including Taleb's deliberate
    meteor-shower-camouflage flag
  - 1 case report at /c/green-fireballs-sandia (Watson 13.4 KB,
    five-act narrative, fully cited)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Luiz Gustavo 2026-05-23 22:41:28 -03:00
parent dd75a67964
commit 67185ff518
6 changed files with 734 additions and 56 deletions

View file

@ -223,58 +223,59 @@ export async function runCaseWriter(task: CaseWriterTask): Promise<
const filter = `%${topic.toLowerCase()}%`; const filter = `%${topic.toLowerCase()}%`;
const docIdFilter = task.doc_id ?? null; const docIdFilter = task.doc_id ?? null;
// Pull artefacts. For evidence/contradictions we use the doc_id when set; // Pull artefacts SEQUENTIALLY. The investigator role has rolconnlimit=4 and
// otherwise widen by topic substring on the artefact's text fields. // pool.max=4; Promise.all of 5 queries × max_parallel=2 jobs would demand
const [evidence, hypotheses, contradictions, witnesses, gaps] = await Promise.all([ // 10 simultaneous connections and the role rejects "too many connections".
query<EvidenceRow>( // Sequential gather still finishes well under a second — case-writer's hot
docIdFilter // path is the LLM call, not these queries.
? `SELECT e.evidence_id, e.grade, e.source_page_id, e.verbatim_excerpt, const evidence = await query<EvidenceRow>(
e.confidence_band, e.related_hypotheses docIdFilter
FROM public.evidence e ? `SELECT e.evidence_id, e.grade, e.source_page_id, e.verbatim_excerpt,
WHERE e.source_page_id LIKE $1 || '/%' e.confidence_band, e.related_hypotheses
ORDER BY e.evidence_id LIMIT 20` FROM public.evidence e
: `SELECT e.evidence_id, e.grade, e.source_page_id, e.verbatim_excerpt, WHERE e.source_page_id LIKE $1 || '/%'
e.confidence_band, e.related_hypotheses ORDER BY e.evidence_id LIMIT 20`
FROM public.evidence e : `SELECT e.evidence_id, e.grade, e.source_page_id, e.verbatim_excerpt,
WHERE LOWER(e.verbatim_excerpt) LIKE $1 e.confidence_band, e.related_hypotheses
ORDER BY e.evidence_id LIMIT 20`, FROM public.evidence e
[docIdFilter ?? filter], WHERE LOWER(e.verbatim_excerpt) LIKE $1
), ORDER BY e.evidence_id LIMIT 20`,
query<HypothesisRow>( [docIdFilter ?? filter],
`SELECT hypothesis_id, question, position, argument_for, argument_against, );
prior, posterior, confidence_band, status, reviewed_by const hypotheses = await query<HypothesisRow>(
FROM public.hypotheses `SELECT hypothesis_id, question, position, argument_for, argument_against,
WHERE LOWER(question) LIKE $1 OR LOWER(position) LIKE $1 prior, posterior, confidence_band, status, reviewed_by
ORDER BY hypothesis_id LIMIT 12`, FROM public.hypotheses
[filter], WHERE LOWER(question) LIKE $1 OR LOWER(position) LIKE $1
), ORDER BY hypothesis_id LIMIT 12`,
query<ContradictionRow>( [filter],
`SELECT contradiction_id, topic, chunks, resolution_status, notes );
FROM public.contradictions const contradictions = await query<ContradictionRow>(
WHERE LOWER(topic) LIKE $1 `SELECT contradiction_id, topic, chunks, resolution_status, notes
ORDER BY contradiction_id LIMIT 8`, FROM public.contradictions
[filter], WHERE LOWER(topic) LIKE $1
), ORDER BY contradiction_id LIMIT 8`,
query<WitnessRow>( [filter],
`SELECT w.witness_id, e.canonical_name, w.credibility, w.verdict, );
w.access_to_event, w.bias_notes const witnesses = await query<WitnessRow>(
FROM public.witnesses w `SELECT w.witness_id, e.canonical_name, w.credibility, w.verdict,
LEFT JOIN public.entities e ON e.entity_pk = w.person_entity_pk w.access_to_event, w.bias_notes
WHERE LOWER(COALESCE(w.verdict,'')) LIKE $1 FROM public.witnesses w
OR LOWER(COALESCE(e.canonical_name,'')) LIKE $1 LEFT JOIN public.entities e ON e.entity_pk = w.person_entity_pk
ORDER BY w.witness_id LIMIT 8`, WHERE LOWER(COALESCE(w.verdict,'')) LIKE $1
[filter], OR LOWER(COALESCE(e.canonical_name,'')) LIKE $1
), ORDER BY w.witness_id LIMIT 8`,
query<GapRow>( [filter],
`SELECT gap_id, description, scope, suggested_next_move, status );
FROM public.gaps const gaps = await query<GapRow>(
WHERE LOWER(description) LIKE $1 `SELECT gap_id, description, scope, suggested_next_move, status
OR LOWER(COALESCE(scope->>'title','')) LIKE $1 FROM public.gaps
OR LOWER(COALESCE(scope->>'why_surprising','')) LIKE $1 WHERE LOWER(description) LIKE $1
ORDER BY gap_id LIMIT 8`, OR LOWER(COALESCE(scope->>'title','')) LIKE $1
[filter], OR LOWER(COALESCE(scope->>'why_surprising','')) LIKE $1
), ORDER BY gap_id LIMIT 8`,
]); [filter],
);
await audit({ await audit({
event: "case_writer_grounded", event: "case_writer_grounded",

View file

@ -149,7 +149,9 @@ export async function runPoirot(task: PoirotTask): Promise<
); );
if (!row) return { skipped: true, reason: "person_not_found" }; if (!row) return { skipped: true, reason: "person_not_found" };
if (row.entity_class !== "person") return { skipped: true, reason: "entity_is_not_person" }; if (row.entity_class !== "person") return { skipped: true, reason: "entity_is_not_person" };
entity_pk = row.entity_pk; // node-postgres returns BIGINT as a string by default; coerce so writer's
// Number.isFinite() check passes.
entity_pk = Number(row.entity_pk);
canonical_name = row.canonical_name; canonical_name = row.canonical_name;
aliases = row.aliases ?? []; aliases = row.aliases ?? [];
} else if (entity_pk !== null) { } else if (entity_pk !== null) {

View file

@ -86,7 +86,10 @@ export async function writeCalibration(
// Force the band to match the posterior — Tetlock can mis-label. // Force the band to match the posterior — Tetlock can mis-label.
body.new_confidence_band = expectedBand; body.new_confidence_band = expectedBand;
if (!body.rationale?.trim()) throw new Error("rationale required"); if (!body.rationale?.trim()) throw new Error("rationale required");
if (body.rationale.length > 600) throw new Error(`rationale too long`); // Soft cap: 1200 chars. Tetlock often writes 600-800 of substantive
// reasoning + chunk citations; the prompt asks for ≤ 600 but a 2× slack
// beats failing the job on an honest analysis.
if (body.rationale.length > 1200) throw new Error(`rationale too long (${body.rationale.length} > 1200)`);
const action = body.recommended_action; const action = body.recommended_action;
if (!["keep", "downgrade", "upgrade", "supersede"].includes(action)) { if (!["keep", "downgrade", "upgrade", "supersede"].includes(action)) {

320
web/app/bureau/page.tsx Normal file
View file

@ -0,0 +1,320 @@
/**
* /bureau Investigation Bureau hub.
*
* Full listing of every artefact: evidence, hypotheses, contradictions,
* witnesses, outliers, case reports, recent jobs. Each anchor-section
* matches the hash-links from the homepage's counter row.
*/
import Link from "next/link";
import { pgQuery } from "@/lib/retrieval/db";
import { AuthBar } from "@/components/auth-bar";
export const runtime = "nodejs";
export const dynamic = "force-dynamic";
interface EvidenceRow {
evidence_id: string;
grade: string;
verbatim_excerpt: string;
source_page_id: string;
confidence_band: string | null;
}
interface HypothesisRow {
hypothesis_id: string;
question: string;
position: string;
prior: number | string | null;
posterior: number | string | null;
confidence_band: string | null;
status: string;
reviewed_by: string | null;
created_at: string;
}
interface ContradictionRow {
contradiction_id: string;
topic: string;
resolution_status: string;
chunks: unknown;
}
interface GapRow {
gap_id: string;
description: string;
scope: unknown;
status: string;
suggested_next_move: string | null;
}
interface WitnessRow {
witness_id: string;
canonical_name: string | null;
entity_id: string | null;
credibility: string | null;
verdict: string | null;
}
interface JobRow {
job_id: string;
kind: string;
status: string;
payload: Record<string, unknown> | null;
created_at: string;
finished_at: string | null;
triggered_by: string | null;
}
const BAND_TONE: Record<string, string> = {
high: "text-[#06d6a0] border-[#06d6a0]",
medium: "text-[#3fde6a] border-[#3fde6a]",
low: "text-[#ffa500] border-[#ffa500]",
speculation: "text-[#ff6ec7] border-[#ff6ec7]",
};
const GRADE_TONE: Record<string, string> = {
A: "text-[#06d6a0] border-[#06d6a0]",
B: "text-[#3fde6a] border-[#3fde6a]",
C: "text-[#ffa500] border-[#ffa500]",
};
export default async function BureauPage() {
// All artefacts. Server component — single round per query, no n+1.
const [hyp, ev, ctr, gap, wit, jobs] = await Promise.all([
pgQuery<HypothesisRow>(
`SELECT hypothesis_id, question, position, prior, posterior, confidence_band,
status, reviewed_by, created_at
FROM public.hypotheses ORDER BY created_at DESC LIMIT 100`,
).catch(() => []),
pgQuery<EvidenceRow>(
`SELECT evidence_id, grade, verbatim_excerpt, source_page_id, confidence_band
FROM public.evidence ORDER BY created_at DESC LIMIT 100`,
).catch(() => []),
pgQuery<ContradictionRow>(
`SELECT contradiction_id, topic, resolution_status, chunks
FROM public.contradictions ORDER BY created_at DESC LIMIT 100`,
).catch(() => []),
pgQuery<GapRow>(
`SELECT gap_id, description, scope, status, suggested_next_move
FROM public.gaps ORDER BY created_at DESC LIMIT 100`,
).catch(() => []),
pgQuery<WitnessRow>(
`SELECT w.witness_id, e.canonical_name, e.entity_id, w.credibility, w.verdict
FROM public.witnesses w
LEFT JOIN public.entities e ON e.entity_pk = w.person_entity_pk
ORDER BY w.created_at DESC LIMIT 100`,
).catch(() => []),
pgQuery<JobRow>(
`SELECT job_id, kind, status, payload, created_at, finished_at, triggered_by
FROM public.investigation_jobs ORDER BY created_at DESC LIMIT 25`,
).catch(() => []),
]);
const { readdir, stat, readFile } = await import("node:fs/promises");
const path = await import("node:path");
const reportsDir = path.join(process.env.CASE_ROOT || "/data/ufo/case", "reports");
let reports: Array<{ slug: string; topic: string; mtimeMs: number; n_evidence: number; n_hypotheses: number }> = [];
try {
const files = await readdir(reportsDir);
const items = await Promise.all(
files.filter((f) => f.endsWith(".md")).map(async (f) => {
const full = path.join(reportsDir, f);
const st = await stat(full);
const head = (await readFile(full, "utf-8")).slice(0, 2000);
const topicMatch = head.match(/topic:\s*"([^"]+)"/);
const eMatch = head.match(/n_evidence:\s*(\d+)/);
const hMatch = head.match(/n_hypotheses:\s*(\d+)/);
return {
slug: f.replace(/\.md$/, ""),
topic: topicMatch?.[1] ?? f,
mtimeMs: st.mtimeMs,
n_evidence: Number(eMatch?.[1] ?? 0),
n_hypotheses: Number(hMatch?.[1] ?? 0),
};
}),
);
reports = items.sort((a, b) => b.mtimeMs - a.mtimeMs);
} catch { /* fine */ }
return (
<div className="min-h-screen bg-[#0a0e1a] text-[#e7ecf3]">
<AuthBar />
<div className="mx-auto max-w-6xl px-4 py-8 pt-16">
<div className="text-[11px] text-[#5a6678] font-mono mb-2">
<Link href="/" className="hover:text-[#7fdbff]">disclosure.top</Link>
<span className="mx-1">/</span>
<span className="text-[#e0c080]">bureau</span>
</div>
<header className="mb-8 border-b border-[rgba(224,192,128,0.32)] pb-4">
<h1 className="font-mono text-3xl text-[#e0c080]"> The Investigation Bureau</h1>
<p className="text-[#8896aa] text-sm mt-1">
Live case folder · {ev.length} evidence · {hyp.length} hypotheses · {ctr.length} contradictions ·{" "}
{wit.length} witnesses · {gap.length} outliers · {reports.length} reports
</p>
</header>
{/* Case reports */}
<Section id="reports" title="Case reports" color="text-[#e0c080]">
{reports.length === 0 ? <Empty /> : reports.map((r) => (
<Link key={r.slug} href={`/c/${r.slug}`} className="block rounded border border-[rgba(224,192,128,0.18)] bg-[#0d1220] p-3 hover:bg-[rgba(224,192,128,0.04)] mb-2">
<div className="text-[10px] font-mono text-[#5a6678]">/c/{r.slug}</div>
<div className="text-[14px] text-[#e7ecf3] font-medium mt-1">{r.topic}</div>
<div className="text-[10px] font-mono text-[#5a6678] mt-1">
{r.n_hypotheses} hypotheses · {r.n_evidence} evidence
</div>
</Link>
))}
</Section>
{/* Hypotheses */}
<Section id="hypotheses" title="Hypotheses" color="text-[#7fdbff]">
{hyp.length === 0 ? <Empty /> : hyp.map((h) => {
const post = h.posterior !== null ? Number(h.posterior) : null;
const prior = h.prior !== null ? Number(h.prior) : null;
const delta = post !== null && prior !== null ? post - prior : null;
const bandTone = (h.confidence_band && BAND_TONE[h.confidence_band]) || "text-[#9aa6b8] border-[#9aa6b8]";
return (
<Link key={h.hypothesis_id} href={`/h/${h.hypothesis_id}`}
className="block rounded border border-[rgba(127,219,255,0.18)] bg-[#0d1220] p-3 hover:bg-[rgba(127,219,255,0.04)] mb-2">
<div className="flex items-baseline justify-between gap-2 mb-1 flex-wrap">
<span className="text-[10px] font-mono text-[#5a6678]">{h.hypothesis_id} · {h.status}</span>
<div className="flex items-center gap-1">
{h.confidence_band && (
<span className={`px-1.5 py-0.5 rounded text-[10px] font-mono uppercase border ${bandTone}`}>{h.confidence_band}</span>
)}
{h.reviewed_by && (
<span className="text-[9px] font-mono text-[#ff3344]"> {h.reviewed_by}</span>
)}
</div>
</div>
<div className="text-[13px] text-[#e7ecf3] leading-snug">{h.position}</div>
<div className="text-[10px] font-mono text-[#5a6678] mt-1">
prior {prior?.toFixed(2) ?? "—"} posterior {post?.toFixed(2) ?? "—"}
{delta !== null && <span className={delta > 0 ? " text-[#06d6a0]" : delta < 0 ? " text-[#ff6ec7]" : ""}> · Δ {delta >= 0 ? "+" : ""}{delta.toFixed(3)}</span>}
</div>
</Link>
);
})}
</Section>
{/* Evidence */}
<Section id="evidence" title="Evidence" color="text-[#06d6a0]">
{ev.length === 0 ? <Empty /> : ev.map((e) => {
const tone = (e.grade && GRADE_TONE[e.grade]) || "text-[#9aa6b8] border-[#9aa6b8]";
return (
<div key={e.evidence_id} className="rounded border border-[rgba(6,214,160,0.18)] bg-[#0d1220] p-3 mb-2">
<div className="flex items-baseline justify-between gap-2 mb-1">
<span className="text-[10px] font-mono text-[#5a6678]">{e.evidence_id} · {e.source_page_id}</span>
<span className={`px-1.5 py-0.5 rounded text-[10px] font-mono uppercase border ${tone}`}>Grade {e.grade}</span>
</div>
<blockquote className="text-[12px] text-[#cbd2dd] italic border-l-2 border-[#06d6a0] pl-2 mt-1">
{e.verbatim_excerpt}
</blockquote>
</div>
);
})}
</Section>
{/* Contradictions */}
<Section id="contradictions" title="Contradictions" color="text-[#ff8a4d]">
{ctr.length === 0 ? <Empty /> : ctr.map((c) => {
const n = Array.isArray(c.chunks) ? c.chunks.length : 0;
return (
<div key={c.contradiction_id} className="rounded border border-[rgba(255,138,77,0.18)] bg-[#0d1220] p-3 mb-2">
<div className="flex items-baseline justify-between gap-2 mb-1">
<span className="text-[10px] font-mono text-[#5a6678]">{c.contradiction_id}</span>
<span className="text-[10px] font-mono text-[#9aa6b8]">{n} positions · {c.resolution_status}</span>
</div>
<div className="text-[13px] text-[#e7ecf3] leading-snug">{c.topic}</div>
</div>
);
})}
</Section>
{/* Outliers */}
<Section id="outliers" title="Outliers" color="text-[#ffd23f]">
{gap.length === 0 ? <Empty /> : gap.map((g) => {
const s = (g.scope ?? {}) as Record<string, unknown>;
const title = (s.title as string) || g.description;
const isOutlier = s.kind === "outlier";
return (
<div key={g.gap_id} className="rounded border border-[rgba(255,210,63,0.25)] bg-[#0d1220] p-3 mb-2">
<div className="flex items-baseline justify-between gap-2 mb-1">
<span className="text-[10px] font-mono text-[#5a6678]">{g.gap_id}{isOutlier && " · outlier"}</span>
<span className="text-[10px] font-mono text-[#9aa6b8]">{g.status}</span>
</div>
<div className="text-[13px] text-[#e7ecf3] leading-snug">{title}</div>
{s.why_surprising !== undefined && (
<div className="text-[11px] text-[#cbd2dd] mt-1 leading-relaxed">{String(s.why_surprising)}</div>
)}
{g.suggested_next_move && (
<div className="text-[10px] font-mono text-[#06d6a0] mt-1"> {g.suggested_next_move}</div>
)}
</div>
);
})}
</Section>
{/* Witnesses */}
<Section id="witnesses" title="Witness analyses" color="text-[#9b5de5]">
{wit.length === 0 ? <Empty /> : wit.map((w) => (
<div key={w.witness_id} className="rounded border border-[rgba(155,93,229,0.18)] bg-[#0d1220] p-3 mb-2">
<div className="flex items-baseline justify-between gap-2 mb-1">
<span className="text-[10px] font-mono text-[#5a6678]">{w.witness_id}</span>
<span className="text-[10px] font-mono text-[#9b5de5]">{w.credibility ?? "—"}</span>
</div>
<div className="text-[13px] text-[#e7ecf3] font-medium">
{w.entity_id ? <Link href={`/e/people/${w.entity_id}`} className="hover:underline">{w.canonical_name ?? w.entity_id}</Link> : (w.canonical_name ?? "—")}
</div>
{w.verdict && <blockquote className="text-[12px] text-[#cbd2dd] italic mt-1 border-l-2 border-[#9b5de5] pl-2">{w.verdict}</blockquote>}
</div>
))}
</Section>
{/* Recent jobs */}
<Section id="jobs" title="Recent investigation jobs" color="text-[#9aa6b8]">
{jobs.length === 0 ? <Empty /> : (
<div className="rounded border border-[rgba(127,219,255,0.08)] bg-[#0d1220] overflow-hidden">
<table className="w-full text-[11px] font-mono">
<thead className="text-[#5a6678] border-b border-[rgba(127,219,255,0.1)]">
<tr>
<th className="text-left px-3 py-2">job</th>
<th className="text-left px-3 py-2">kind</th>
<th className="text-left px-3 py-2">status</th>
<th className="text-left px-3 py-2">created</th>
</tr>
</thead>
<tbody>
{jobs.map((j) => (
<tr key={j.job_id} className="border-t border-[rgba(127,219,255,0.04)] hover:bg-[rgba(127,219,255,0.03)]">
<td className="px-3 py-2">
<Link href={`/jobs/${j.job_id}`} className="text-[#7fdbff] hover:underline">{j.job_id.slice(0, 8)}</Link>
</td>
<td className="px-3 py-2 text-[#cbd2dd]">{j.kind}</td>
<td className={`px-3 py-2 ${
j.status === "complete" ? "text-[#06d6a0]" :
j.status === "failed" ? "text-[#ff6ec7]" :
j.status === "running" ? "text-[#ffd23f]" :
"text-[#9aa6b8]"
}`}>{j.status}</td>
<td className="px-3 py-2 text-[#5a6678]">{new Date(j.created_at).toLocaleString("pt-BR")}</td>
</tr>
))}
</tbody>
</table>
</div>
)}
</Section>
</div>
</div>
);
}
function Section({ id, title, color, children }: { id: string; title: string; color: string; children: React.ReactNode }) {
return (
<section id={id} className="mb-10 scroll-mt-20">
<h2 className={`font-mono text-xs uppercase tracking-[0.18em] ${color} mb-3`}>// {title}</h2>
<div>{children}</div>
</section>
);
}
function Empty() {
return <div className="text-[11px] font-mono text-[#5a6678]">_(none yet)_</div>;
}

View file

@ -6,6 +6,7 @@ import { BatchProgressBanner } from "@/components/batch-progress-banner";
import { getLocale } from "@/components/locale-toggle"; import { getLocale } from "@/components/locale-toggle";
import { summarize, pickPitch } from "@/lib/doc-summary"; import { summarize, pickPitch } from "@/lib/doc-summary";
import { DocListFilters } from "@/components/doc-list-filters"; import { DocListFilters } from "@/components/doc-list-filters";
import { BureauSnapshot } from "@/components/bureau-snapshot";
// Read wiki/ filesystem at request time, not build time. // Read wiki/ filesystem at request time, not build time.
export const dynamic = "force-dynamic"; export const dynamic = "force-dynamic";
@ -63,6 +64,13 @@ export default async function Home() {
> >
📊 stats 📊 stats
</Link> </Link>
<Link
href="/bureau"
className="font-mono text-xs px-3 py-1.5 border border-[#e0c080] text-[#e0c080] hover:bg-[rgba(224,192,128,0.10)] rounded"
title="Investigation Bureau — detectives, hypotheses, evidence, contradictions, outliers, case reports"
>
🔎 bureau
</Link>
<Link <Link
href="/admin/batch" href="/admin/batch"
className="font-mono text-xs px-3 py-1.5 border border-[rgba(0,255,156,0.30)] text-[#00ff9c] hover:bg-[rgba(0,255,156,0.10)] rounded" className="font-mono text-xs px-3 py-1.5 border border-[rgba(0,255,156,0.30)] text-[#00ff9c] hover:bg-[rgba(0,255,156,0.10)] rounded"
@ -74,14 +82,16 @@ export default async function Home() {
</div> </div>
</div> </div>
<h1 className="font-mono text-3xl text-[#00ff9c] mb-2"> <h1 className="font-mono text-3xl text-[#00ff9c] mb-2">
war.gov/ufo Investigative Wiki The Disclosure Bureau
</h1> </h1>
<p className="text-[#8896aa] text-sm"> <p className="text-[#8896aa] text-sm">
{docs.length} declassified documents · {docs.reduce((s, d) => s + d.pages, 0)} pages · {docs.length} declassified documents · {docs.reduce((s, d) => s + d.pages, 0)} pages ·
AI-cataloged by the Investigation Bureau (Holmes · Poirot · Dupin · Locard) investigated by 8 AI detectives (Holmes · Locard · Dupin · Schneier · Poirot · Taleb · Tetlock · Case-Writer)
</p> </p>
</header> </header>
<BureauSnapshot />
<BatchProgressBanner /> <BatchProgressBanner />
<DocListFilters docs={docs} /> <DocListFilters docs={docs} />

View file

@ -0,0 +1,342 @@
/**
* BureauSnapshot homepage panel surfacing the Investigation Bureau's
* live artefact stack.
*
* Reads counts + the 3 most-recent items from each table directly via pg.
* Server component renders straight into the homepage shell, no client
* hydration cost.
*/
import Link from "next/link";
import { pgQuery } from "@/lib/retrieval/db";
interface CountRow { c: string }
interface RecentEvidence { evidence_id: string; grade: string; verbatim_excerpt: string; source_page_id: string; confidence_band: string | null }
interface RecentHypothesis { hypothesis_id: string; position: string; posterior: number | string | null; confidence_band: string | null; reviewed_by: string | null }
interface RecentContradiction { contradiction_id: string; topic: string; resolution_status: string }
interface RecentGap { gap_id: string; description: string; scope: unknown }
interface RecentWitness { witness_id: string; canonical_name: string | null; credibility: string | null; verdict: string | null }
interface RecentJob { job_id: string; kind: string; status: string; created_at: string; payload: Record<string, unknown> | null }
const DETECTIVES = [
{ slug: "holmes", name: "Holmes", role: "Hypothesis tournament", tone: "text-[#7fdbff] border-[#7fdbff]" },
{ slug: "locard", name: "Locard", role: "Evidence chain", tone: "text-[#06d6a0] border-[#06d6a0]" },
{ slug: "dupin", name: "Dupin", role: "Contradiction scan", tone: "text-[#ff8a4d] border-[#ff8a4d]" },
{ slug: "schneier", name: "Schneier", role: "Red-team review", tone: "text-[#ff3344] border-[#ff3344]" },
{ slug: "poirot", name: "Poirot", role: "Witness analysis", tone: "text-[#9b5de5] border-[#9b5de5]" },
{ slug: "taleb", name: "Taleb", role: "Outlier hunter", tone: "text-[#ffd23f] border-[#ffd23f]" },
{ slug: "tetlock", name: "Tetlock", role: "Posterior calibration", tone: "text-[#26d4cc] border-[#26d4cc]" },
{ slug: "case-writer", name: "Case-Writer", role: "Five-act narrative", tone: "text-[#e0c080] border-[#e0c080]" },
] as const;
const BAND_TONE: Record<string, string> = {
high: "text-[#06d6a0]",
medium: "text-[#3fde6a]",
low: "text-[#ffa500]",
speculation: "text-[#ff6ec7]",
};
const GRADE_TONE: Record<string, string> = {
A: "text-[#06d6a0] border-[#06d6a0]",
B: "text-[#3fde6a] border-[#3fde6a]",
C: "text-[#ffa500] border-[#ffa500]",
};
async function loadSnapshot() {
// Counts come from a single round-trip via UNION ALL.
const counts = await pgQuery<CountRow & { k: string }>(
`SELECT 'evidence' AS k, COUNT(*)::text AS c FROM public.evidence
UNION ALL SELECT 'hypotheses', COUNT(*)::text FROM public.hypotheses
UNION ALL SELECT 'contradictions', COUNT(*)::text FROM public.contradictions
UNION ALL SELECT 'witnesses', COUNT(*)::text FROM public.witnesses
UNION ALL SELECT 'gaps', COUNT(*)::text FROM public.gaps
UNION ALL SELECT 'open_hyp', COUNT(*)::text FROM public.hypotheses WHERE status='open'
UNION ALL SELECT 'jobs_running', COUNT(*)::text FROM public.investigation_jobs WHERE status IN ('queued','running')
UNION ALL SELECT 'jobs_complete_24h', COUNT(*)::text FROM public.investigation_jobs WHERE status='complete' AND finished_at > NOW() - INTERVAL '24 hours'`,
).catch(() => [] as Array<CountRow & { k: string }>);
const byKey: Record<string, number> = {};
for (const r of counts) byKey[r.k] = Number(r.c);
const [hyp, ev, ctr, gap, wit, jobs] = await Promise.all([
pgQuery<RecentHypothesis>(
`SELECT hypothesis_id, position, posterior, confidence_band, reviewed_by
FROM public.hypotheses ORDER BY created_at DESC LIMIT 4`,
).catch(() => []),
pgQuery<RecentEvidence>(
`SELECT evidence_id, grade, verbatim_excerpt, source_page_id, confidence_band
FROM public.evidence ORDER BY created_at DESC LIMIT 3`,
).catch(() => []),
pgQuery<RecentContradiction>(
`SELECT contradiction_id, topic, resolution_status
FROM public.contradictions ORDER BY created_at DESC LIMIT 3`,
).catch(() => []),
pgQuery<RecentGap>(
`SELECT gap_id, description, scope FROM public.gaps ORDER BY created_at DESC LIMIT 3`,
).catch(() => []),
pgQuery<RecentWitness>(
`SELECT w.witness_id, e.canonical_name, w.credibility, w.verdict
FROM public.witnesses w
LEFT JOIN public.entities e ON e.entity_pk = w.person_entity_pk
ORDER BY w.created_at DESC LIMIT 3`,
).catch(() => []),
pgQuery<RecentJob>(
`SELECT job_id, kind, status, created_at, payload
FROM public.investigation_jobs ORDER BY created_at DESC LIMIT 6`,
).catch(() => []),
]);
// Case reports live on disk. Read the reports dir.
const { readdir, stat } = await import("node:fs/promises");
const path = await import("node:path");
const reportsDir = path.join(process.env.CASE_ROOT || "/data/ufo/case", "reports");
let reports: Array<{ slug: string; mtimeMs: number }> = [];
try {
const files = await readdir(reportsDir);
const items = await Promise.all(
files.filter((f) => f.endsWith(".md")).map(async (f) => {
const st = await stat(path.join(reportsDir, f));
return { slug: f.replace(/\.md$/, ""), mtimeMs: st.mtimeMs };
}),
);
reports = items.sort((a, b) => b.mtimeMs - a.mtimeMs).slice(0, 4);
} catch { /* no reports yet — fine */ }
return { counts: byKey, hyp, ev, ctr, gap, wit, jobs, reports };
}
export async function BureauSnapshot() {
const snap = await loadSnapshot().catch(() => null);
if (!snap) {
return (
<section className="mb-10 border border-dashed border-[rgba(127,219,255,0.18)] rounded-lg p-4 text-[11px] text-[#9aa6b8] font-mono">
Investigation Bureau snapshot unavailable (db unreachable).
</section>
);
}
const { counts, hyp, ev, ctr, gap, wit, jobs, reports } = snap;
const totalArtefacts = (counts.evidence ?? 0) + (counts.hypotheses ?? 0) + (counts.contradictions ?? 0)
+ (counts.witnesses ?? 0) + (counts.gaps ?? 0) + reports.length;
const running = counts.jobs_running ?? 0;
return (
<section className="mb-10">
{/* Section heading */}
<div className="flex items-baseline justify-between mb-4 flex-wrap gap-2">
<div>
<h2 className="font-mono text-lg text-[#e7ecf3]">
<span className="text-[#7fdbff]"></span> The Investigation Bureau
</h2>
<p className="text-[11px] text-[#5a6678] font-mono mt-1">
8 AI detectives · {totalArtefacts} artefacts in the case folder
{running > 0 && <span className="text-[#ffd23f]"> · {running} job{running === 1 ? "" : "s"} in progress</span>}
</p>
</div>
<Link
href="/bureau"
className="font-mono text-xs px-3 py-1.5 border border-[#e0c080] text-[#e0c080] hover:bg-[rgba(224,192,128,0.10)] rounded"
>
open the bureau
</Link>
</div>
{/* Detective tiles */}
<div className="grid grid-cols-2 md:grid-cols-4 gap-2 mb-5">
{DETECTIVES.map((d) => (
<div
key={d.slug}
className={`border ${d.tone} bg-[#0d1220] rounded p-2 text-[11px] font-mono leading-snug`}
>
<div className="flex items-baseline justify-between">
<span className="font-semibold">{d.name}</span>
<span className="text-[9px] text-[#5a6678] uppercase">{d.slug}</span>
</div>
<div className="text-[10px] text-[#9aa6b8] mt-0.5">{d.role}</div>
</div>
))}
</div>
{/* Counter row */}
<div className="grid grid-cols-3 md:grid-cols-6 gap-2 mb-5 text-center">
<Counter href="/bureau#evidence" label="evidence" value={counts.evidence ?? 0} color="text-[#06d6a0]" />
<Counter href="/bureau#hypotheses" label="hypotheses" value={counts.hypotheses ?? 0} color="text-[#7fdbff]" />
<Counter href="/bureau#contradictions" label="contradictions" value={counts.contradictions ?? 0} color="text-[#ff8a4d]" />
<Counter href="/bureau#witnesses" label="witnesses" value={counts.witnesses ?? 0} color="text-[#9b5de5]" />
<Counter href="/bureau#outliers" label="outliers" value={counts.gaps ?? 0} color="text-[#ffd23f]" />
<Counter href="/bureau#reports" label="case reports" value={reports.length} color="text-[#e0c080]" />
</div>
{/* Recent artefacts grid: hypotheses + contradictions + outliers + reports */}
<div className="grid md:grid-cols-2 gap-3 mb-3">
{hyp.length > 0 && (
<ArtefactColumn
title="Recent hypotheses"
color="text-[#7fdbff]"
border="border-[rgba(127,219,255,0.18)]"
>
{hyp.map((h) => {
const post = h.posterior !== null ? Number(h.posterior) : null;
const bandTone = (h.confidence_band && BAND_TONE[h.confidence_band]) || "text-[#9aa6b8]";
return (
<Link key={h.hypothesis_id} href={`/h/${h.hypothesis_id}`} className="block py-1.5 border-t border-[rgba(127,219,255,0.08)] first:border-t-0 hover:bg-[rgba(127,219,255,0.04)]">
<div className="flex items-baseline justify-between gap-2">
<span className="text-[10px] font-mono text-[#5a6678]">{h.hypothesis_id}</span>
<span className={`text-[10px] font-mono ${bandTone}`}>
{h.confidence_band ?? "—"}{post !== null && ` · ${post.toFixed(2)}`}
</span>
</div>
<div className="text-[12px] text-[#cbd2dd] leading-snug mt-0.5">{h.position}</div>
{h.reviewed_by && (
<div className="text-[9px] font-mono text-[#ff3344] mt-0.5"> reviewed by {h.reviewed_by}</div>
)}
</Link>
);
})}
</ArtefactColumn>
)}
{ctr.length > 0 && (
<ArtefactColumn
title="Recent contradictions"
color="text-[#ff8a4d]"
border="border-[rgba(255,138,77,0.18)]"
>
{ctr.map((c) => (
<div key={c.contradiction_id} className="py-1.5 border-t border-[rgba(255,138,77,0.08)] first:border-t-0">
<div className="flex items-baseline justify-between gap-2">
<span className="text-[10px] font-mono text-[#5a6678]">{c.contradiction_id}</span>
<span className="text-[10px] font-mono text-[#9aa6b8]">{c.resolution_status}</span>
</div>
<div className="text-[12px] text-[#cbd2dd] leading-snug mt-0.5">{c.topic}</div>
</div>
))}
</ArtefactColumn>
)}
{ev.length > 0 && (
<ArtefactColumn
title="Recent evidence"
color="text-[#06d6a0]"
border="border-[rgba(6,214,160,0.18)]"
>
{ev.map((e) => {
const tone = (e.grade && GRADE_TONE[e.grade]) || "text-[#9aa6b8]";
return (
<div key={e.evidence_id} className="py-1.5 border-t border-[rgba(6,214,160,0.08)] first:border-t-0">
<div className="flex items-baseline justify-between gap-2">
<span className="text-[10px] font-mono text-[#5a6678]">{e.evidence_id}</span>
<span className={`text-[10px] font-mono ${tone} px-1.5 border rounded`}>Grade {e.grade}</span>
</div>
<blockquote className="text-[12px] text-[#cbd2dd] italic mt-0.5 leading-snug border-l-2 border-[#06d6a0] pl-2">
{(e.verbatim_excerpt || "").slice(0, 140)}
</blockquote>
</div>
);
})}
</ArtefactColumn>
)}
{gap.length > 0 && (
<ArtefactColumn
title="Recent outliers"
color="text-[#ffd23f]"
border="border-[rgba(255,210,63,0.25)]"
>
{gap.map((g) => {
const s = (g.scope ?? {}) as Record<string, unknown>;
const title = (s.title as string) || g.description;
return (
<div key={g.gap_id} className="py-1.5 border-t border-[rgba(255,210,63,0.08)] first:border-t-0">
<div className="flex items-baseline justify-between gap-2">
<span className="text-[10px] font-mono text-[#5a6678]">{g.gap_id}</span>
{s.kind === "outlier" && (
<span className="text-[10px] font-mono text-[#ffd23f]">outlier</span>
)}
</div>
<div className="text-[12px] text-[#cbd2dd] leading-snug mt-0.5">{title}</div>
</div>
);
})}
</ArtefactColumn>
)}
{wit.length > 0 && (
<ArtefactColumn
title="Witness analyses"
color="text-[#9b5de5]"
border="border-[rgba(155,93,229,0.18)]"
>
{wit.map((w) => (
<div key={w.witness_id} className="py-1.5 border-t border-[rgba(155,93,229,0.08)] first:border-t-0">
<div className="flex items-baseline justify-between gap-2">
<span className="text-[10px] font-mono text-[#5a6678]">{w.witness_id}</span>
<span className="text-[10px] font-mono text-[#9b5de5]">{w.credibility ?? "—"}</span>
</div>
<div className="text-[12px] text-[#cbd2dd] leading-snug mt-0.5 font-semibold">{w.canonical_name ?? "—"}</div>
{w.verdict && <div className="text-[11px] text-[#9aa6b8] mt-0.5 italic">{w.verdict.slice(0, 200)}</div>}
</div>
))}
</ArtefactColumn>
)}
{reports.length > 0 && (
<ArtefactColumn
title="Case reports"
color="text-[#e0c080]"
border="border-[rgba(224,192,128,0.25)]"
>
{reports.map((r) => (
<Link key={r.slug} href={`/c/${r.slug}`} className="block py-1.5 border-t border-[rgba(224,192,128,0.08)] first:border-t-0 hover:bg-[rgba(224,192,128,0.04)]">
<div className="text-[10px] font-mono text-[#5a6678]">case-report</div>
<div className="text-[12px] text-[#e0c080] leading-snug mt-0.5 font-medium">/c/{r.slug}</div>
<div className="text-[10px] font-mono text-[#5a6678] mt-0.5">Watson five-act narrative </div>
</Link>
))}
</ArtefactColumn>
)}
</div>
{/* Job activity strip */}
{jobs.length > 0 && (
<div className="text-[10px] font-mono text-[#5a6678] mt-2">
Recent jobs:{" "}
{jobs.slice(0, 4).map((j, i) => (
<span key={j.job_id}>
{i > 0 && <span className="text-[#3a4250]"> · </span>}
<Link
href={`/jobs/${j.job_id}`}
className={`hover:underline ${
j.status === "complete" ? "text-[#06d6a0]" :
j.status === "failed" ? "text-[#ff6ec7]" :
j.status === "running" ? "text-[#ffd23f]" :
"text-[#9aa6b8]"
}`}
>
{j.kind}/{j.status}
</Link>
</span>
))}
</div>
)}
</section>
);
}
function Counter({ label, value, color, href }: { label: string; value: number; color: string; href: string }) {
return (
<Link href={href} className="block rounded border border-[rgba(127,219,255,0.08)] bg-[#0d1220] py-2 hover:border-[rgba(127,219,255,0.20)]">
<div className={`text-2xl font-mono font-bold ${color}`}>{value}</div>
<div className="text-[10px] font-mono text-[#5a6678] uppercase tracking-wider mt-0.5">{label}</div>
</Link>
);
}
function ArtefactColumn({
title, color, border, children,
}: { title: string; color: string; border: string; children: React.ReactNode }) {
return (
<div className={`rounded-lg border ${border} bg-[#0d1220] p-3`}>
<div className={`text-[10px] font-mono uppercase tracking-wider ${color} mb-1`}>{title}</div>
<div>{children}</div>
</div>
);
}