Commit graph

34 commits

Author SHA1 Message Date
Luiz Gustavo
4fb3e12e15 ux(bureau): search + agency/decade filters + sort, URL-synced
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 35s
CI / Scripts — Python smoke (push) Failing after 6s
CI / Web — npm audit (push) Failing after 39s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 6s
The audit flagged /bureau as the hardest page to navigate: 80+ case cards
in a flat grid with no way to narrow them down. Now the reader gets a
filterable browser.

- New lib/cases.ts (server) — shared loader for the case library; pulls
  cases off disk, derives the originating agency from the slug prefix
  (DoW, FBI, NASA, CIA, DoE, ODNI, DoD, Other) and the incident year via
  a tight 19xx/20xx regex that requires non-digit boundaries so embedded
  ID numbers don't get mistaken for years (e.g. doc-2070-... no longer
  pretends to be a 2070 incident).
- New CaseLibraryBrowser (client) — search input (matches title + opening
  in either language), agency chips with live counts, decade chips, sort
  dropdown (Most recent / Year ↑ / Year ↓ / A–Z), result count, and a
  clear-filters affordance. All four state pieces sync to the URL so a
  filtered view is bookmarkable.
- /bureau page rewritten: magazine hero ("89 declassified cases,
  narrated.") + dynamic case count + lead, then the browser.
- Cards in the browser carry an eyebrow ("YEAR · AGENCY") so the reader
  sees provenance at a glance.
- CaseLibrary (homepage) refactored to consume the shared loader; behaviour
  on / unchanged.

Verified live: agency=FBI filter narrows to 23 cases; sort=year-desc places
modern cases first; chip counts reflect filtered population.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 01:21:39 -03:00
Luiz Gustavo
babffda882 ux(search): redesign /search from dev-tool to magazine
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 36s
CI / Scripts — Python smoke (push) Failing after 6s
CI / Web — npm audit (push) Failing after 44s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 13s
The audit's most ostensive gap: /search rendered as a green-on-black
terminal panel with a tech-jargon eyebrow ("HYBRID SEARCH · BM25 + BGE-M3
DENSE + CROSS-ENCODER RERANK"), no global navbar, and a floating locale
toggle disconnected from the rest of the site. Readers landing here saw
a different product.

Page (app/search/page.tsx):
- Render SiteHeader at the top (same primary nav as home; "Search" now
  picks up the gold active-page indicator).
- Magazine hero: eyebrow + serif headline ("Find a passage, a witness, a
  date." / "Encontre uma passagem, uma testemunha, uma data.") + lead.
- generateMetadata() reads getLocale() so the tab title localises too.

Panel (search-panel.tsx):
- Drop matrix-green palette; switch to magazine gold/cream.
- Rounded input with magnifier icon, gold solid submit pill.
- Plain-language placeholder, no developer terms.
- "Advanced filters" toggle: passage-type + exact doc-id behind a
  collapsible — default state is one clean search box.
- Localise passage-type labels (paragraph → parágrafo, heading → título,
  classification_marking → marcação de classificação, etc).
- Result card palette switched to gold/cream + clearer page · type ·
  classification line; raw retrieval score removed from the UI (debug
  noise for users).
- Locale toggle inside the form gone — the navbar carries it now.

Follow-ups noted (not in this commit):
- SearchAutocomplete dropdown still uses the old terminal palette.
- Navbar wraps on narrow mobile widths and the rightmost links overflow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 17:58:41 -03:00
Luiz Gustavo
2ea350e283 ux(nav): unify navbar palette, localize, fix locale-aware metadata
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 36s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 37s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 5s
Visual audit found several issues in the navbar / header layer; this pass
addresses the cluster.

- AuthBar: localize sign in/out + dev-disabled message; switch matrix-green
  (#00ff9c) to gold (#e0c080) to match the magazine palette.
- NavLink (new): client component reading usePathname to mark the active
  section with aria-current="page" and a gold underline+fill, so the user
  knows where they are inside the entity hub.
- Locale toggle: bump font to 12px, widen the pill, lift the active state's
  contrast. Hard reload on switch (window.location.reload) instead of
  router.refresh — that is what makes the <title>, <html lang> and og:locale
  actually update on toggle (router.refresh kept stale layout metadata).
- Root layout: convert static `export const metadata` to generateMetadata()
  so title/description/OG/twitter all read the cookie per request. Drop the
  duplicate floating LocaleToggle at bottom-left; the navbar carries it now.
- BureauNav: drop the cyan-on-home / gold-on-bureau split (palette mismatch
  with SiteHeader); single gold tone; localize "home"/"case files" + aria
  labels; mount AuthBar inside the bar so sub-pages get a sign-in surface.
- Pages using BureauNav: drop the standalone <AuthBar /> below it (now
  duplicated). Affected: /bureau, /c/[slug], /h/[hypothesisId], /jobs/[id].
- Lift nav-link rest text from #9aa6b8 to #cbd2dd for WCAG-AA on near-black.

Verified live: title/lang/og:locale switch with cookie; active link gold
indicator on /sightings; "Entrar"/"Sign in" localizes; no double AuthBar.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 17:50:10 -03:00
Luiz Gustavo
2e59b01f1f fix(i18n): render case files in one language per locale
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 37s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 27s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 5s
Case bodies hold EN and PT-BR scenes interleaved (## I. EN, ## I. PT, ...),
but the reader rendered the whole body, so an EN visitor saw every scene
twice — once in English, once in Portuguese — and vice-versa.

Add pickLocaleBody(): split by section-number occurrence (first = EN, second
= PT-BR, the narrator's mandated order), with language sniffing only as a
fallback for unpaired/unnumbered sections, and full-body fallback when a body
can't be split (English-only legacy files). Wire it into the case reader
(body, lead, OG description) and the homepage card openings.

Also fix extractBody: it discarded the English H1 by treating the second
(PT-BR) H1 as "the first real heading". Future narrations keep both titles.

Verified live: d44 and doc-65 render EN-only under locale=en and PT-only
under locale=pt-br. Bulk-checked 89 files: 80 clean, 8 structurally clean
(source-language verbatim quotes retained per spec), 1 English-only legacy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 11:57:46 -03:00
Luiz Gustavo
e45f1c04d0 fix(seo): use body H1 as case-file <title> and OG title
generateMetadata still derived the page <title>/OG title from the generic
auto frontmatter topic ("Dow Uap D44 …"). Search engines, link cards, and
AI crawlers show that string, so it must be the human magazine headline the
narrator wrote as the body H1. Mirror the page component's H1 extraction.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 04:09:15 -03:00
Luiz Gustavo
c1260cfa68 W6.1: batch-narrate all 122 documents + magazine titles from body H1
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 31s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 36s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 3s
User wants every case in the archive documented in the narrated format.
Queued a case_report for all 122 documents (slug = doc_id, scoped to that
doc's chunks). The LISTEN worker drains 2 at a time (~5h, ~$18 OAuth).
5 reports already existed; ~122 new jobs queued, skipping any already
queued/running/complete by slug.

Title rendering: the batch seeds `topic` from the auto-derived
canonical_title ("Doc 18 100754 General 1946 7 Vol 2"), which is ugly as a
headline. The narrator writes a proper magazine H1 in the body, so:
  - /c/[slug]: extract the two body H1s (EN + PT-BR), use the locale one
    as the headline, strip both leading H1s from the rendered body so the
    title isn't duplicated. Falls back to frontmatter topic.
  - case-library.tsx + featured-case.tsx: same H1 extraction for card
    titles, so /bureau and the homepage show real titles not "Doc 18…".
  - featured-case: pin the flagship (green-fireballs-narrative, which has
    a curated hero painting) to the featured slot instead of most-recent,
    so the batch's random completions don't take over the homepage hero.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 03:54:35 -03:00
Luiz Gustavo
3e88a64933 W5.8: language toggle in the navbar (whole-site PT-BR / EN)
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 1m16s
CI / Scripts — Python smoke (push) Failing after 36s
CI / Web — npm audit (push) Failing after 28s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
User: "quero que o usuário possa escolher o idioma na navbar, e então leia
o site todo em seu idioma."

The locale infrastructure already existed (cookie + getLocale() + every
page renders bilingual). The toggle only lived as a subtle floating
control bottom-left. This surfaces it in the navbars where users expect
it, and confirms whole-site language switching.

  locale-toggle-client.tsx
    - New variant="navbar": gold-amber pill matching the magazine theme
      (PT | EN, active = solid gold). Retro green variant kept for the
      floating fallback.
    - Guard: clicking the already-active language is a no-op.
    - aria-pressed on each button for a11y.

  locale-toggle.tsx: LocaleToggle gains variant passthrough.

  site-header.tsx: <LocaleToggle variant="navbar"> added to the main nav
    (homepage + /sightings /witnesses /objects /locations /operations
    /documents).

  bureau-nav.tsx: optional `locale` prop renders the same toggle pushed
    to the right (ml-auto) on /bureau, /c/[slug], /h/[id], /jobs/[id].
    All four callers now pass locale (getLocale added to h + jobs pages).

Cookie-based: switching sets `locale` cookie + router.refresh(); the
server re-renders every component through getLocale(), so the entire
page — headers, hero, summaries, case narratives, entity cards — flips
language in one click. The floating bottom-left toggle stays as a global
fallback for pages without a navbar (/d, /e, /search, /graph).

Also this session: Sun-Tzu smoke succeeded on the refreshed
CLAUDE_CODE_OAUTH_TOKEN — brief B-0001 written for the green fireballs
("bolas de fogo verdes confinadas ao corredor nuclear... cor espectral
e trajetória quase horizontal que os próprios analistas consideraram
irreconciliáveis com meteoros naturais", 3 pillars). Bulk entity
enrichment resumed in the background.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 22:54:37 -03:00
Luiz Gustavo
df82d40a96 W5.7 (Phase 3E): perf + a11y + OG images
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 36s
CI / Scripts — Python smoke (push) Failing after 6s
CI / Web — npm audit (push) Failing after 42s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 5s
A11y:
  - Skip-link in <body> (focus-visible only) that jumps to #main.
    Bilingual ("Skip to content" / "Pular para o conteúdo").
  - <main id="main"> landmark wrapping the homepage body.
  - prefers-reduced-motion media query disables the hover-scale + image
    transitions for users with vestibular sensitivity.
  - Skip-link styled with high contrast (gold-on-dark) + outline on
    keyboard focus.

Performance:
  - HeroBanner background image: fetchPriority="high" + explicit
    width/height (1600x900) for zero CLS.
  - FeaturedCase image: fetchPriority="high" + 1280x720 to prevent
    layout shift while the 2.7MB painting loads.
  - IconicCases tiles already have loading="lazy".
  - prose blockquote: overflow-wrap:anywhere so verbatim quotes don't
    bust the mobile viewport.

Open Graph:
  - app/layout.tsx default OG image set to the green-fireballs
    painting (any page without its own image card inherits this).
  - app/c/[slug] OG image is the case's editorial illustration when
    one exists. WhatsApp, Twitter, Telegram, Slack, ChatGPT search
    all pull this when the link is shared. 2000x1125 for the
    "summary_large_image" twitter card.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 16:55:35 -03:00
Luiz Gustavo
1687120b7f W5.6 (Phase 3D): Iconic Cases — curated rail on homepage
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 36s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Failing after 42s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 6s
8 hand-picked UFO incidents that any enthusiast recognises:
  - Kenneth Arnold 1947 (the genesis — "flying saucers" coined)
  - Roswell 1947 (the original Army press release)
  - Maury Island 1947 (Puget Sound, slag drop, FBI plane crash)
  - Mantell 1948 (first known UFO casualty)
  - Rendlesham Forest 1980 (USAF security police, deputy commander tape)
  - Phoenix Lights 1997 (V-formation across Arizona, governor's reversal)
  - Nimitz Tic-Tac 2004 (USS Nimitz F/A-18, gun-cam released 2017)
  - Green Fireballs Sandia 1948 (copper salts, nuclear-site overflights)

Each case has:
  - Bilingual hand-written editorial blurb (PT-BR + EN, no LLM)
  - Painterly editorial hero illustration at 2K
  - Year + tag chips (military / civilian / modern / early / naval /
    aviation / mass-sighting)
  - Link to /e/events/<id> when indexed in corpus, /search?q=... when not,
    /c/<slug> for the green-fireballs narrative case

Components:
  lib/iconic-cases.ts — IconicCase type + ICONIC_CASES editorial list
  components/iconic-cases.tsx — magazine grid: first two as wide hero pair,
    rest as 3-up tiles below. Hover scale on images, gradient overlays,
    aspect-16/11 hero + 16/10 compact, lazy-loaded.
  app/page.tsx — inserted between <FeaturedCase /> and <PortalGrid />

4 new hero illustrations generated this session:
  - iconic-nimitz-tic-tac-2004.png       (Nano Banana Pro)
  - iconic-phoenix-lights-1997.png       (Nano Banana Pro)
  - iconic-rendlesham-forest-1980.png    (gpt-image-1.5 via Codex)
  - iconic-mantell-1948.png              (Nano Banana Pro)
Per user direction: mixed generators (Nano Banana primary, Codex
co-pilot) so the homepage has stylistic variety while keeping the
painterly editorial register.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 16:50:52 -03:00
Luiz Gustavo
2ac42b99a7 W5.5 (Phase 3C): Sun-Tzu strategist feeder + entity hero illustrations
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 33s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 24s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 3s
Sun-Tzu (silent backend) — builds the strongest pro-anomaly brief the
corpus supports for any topic. Bilingual JSON: thesis + 2-4 pillars
(each with claim + citation-backed support) + honest residual
unexplained clause. NEVER surfaced reader-facing.

  Migration 0009 (apply as supabase_admin):
    public.pro_anomaly_briefs  brief_pk BIGSERIAL PK
                               brief_id B-NNNN unique
                               topic + topic_pt_br
                               thesis + thesis_pt_br
                               pillars JSONB
                               unexplained + unexplained_pt_br
                               doc_id, job_id, created_by, created_at
    + brief_id_seq sequence
    + GIN trigram indexes on topic + topic_pt_br
    + RLS policies (investigator INSERT, public SELECT)
    + GRANTs on seq + table to investigator

  prompts/sun-tzu.md
    "Adversarial strategist who plays the pro-disclosure side with the
    same rigour a red-team plays skeptic" — single thesis, 2-4 pillars,
    honest residual. Every claim cites a chunk. No fabrication from
    training-time knowledge. Output INTERNAL — case-writer pulls it.
    Bilingual mandatory. NO_STRONG_CASE sentinel when corpus is thin.

  detectives/sun_tzu.ts
    Grounds with hybridSearch top 18 chunks, calls Sonnet, parses
    JSON strict, calls writeProAnomalyBrief.

  tools/write_pro_anomaly_brief.ts
    Validates 2-4 pillars with bilingual claim+support, requires at
    least one [[wiki-link]] citation per pillar, INSERTs.

  orchestrator: new kind "anomaly_brief" dispatches Sun-Tzu.

Case-writer integration (detectives/case_writer.ts):
  - Pulls most recent matching brief via ILIKE on topic or doc_id.
  - Renders brief as a separate prompt section labelled
    "Strategic brief (internal — do NOT cite or attribute)".
  - Instructs the narrator to weave the thesis as a quiet through-
    line, use pillar facts in scenes, let the unexplained clause
    inform the closing paragraph. Forbidden to name "the analyst",
    say "a brief argues", or use the words "thesis"/"pillar"
    explicitly. Translate it into prose.

Entity hero illustrations:
  - 3 painterly editorial illustrations generated via Nano Banana
    Pro at 2K, stored under /data/disclosure/processing/case-art/:
    * EV-1947-06-24-kenneth-arnold-sighting.png — cockpit POV of
      Arnold in a CallAir A-2 over Mount Rainier, 9 chevron disc
      objects in formation, 1947 Life-magazine register.
    * EV-1947-07-08-roswell-incident.png — debris field in NM
      desert, USAAF officer in 1947 uniform examining foil
      fragments, period staff car.
    * EV-1947-06-21-maury-island-incident.png — wooden patrol
      boat on Puget Sound, 6 doughnut craft hovering, one
      shedding glowing slag, Harold Dahl + son + dog watching.
  - app/e/[cls]/[id]/page.tsx: full-bleed editorial hero replaces
    the old gradient header card when an illustration exists for
    that entity_id. Title sits over the painting with gradient
    overlay. "Ilustração editorial" chip in the top-right.

Quota note: Claude OAuth still rate-limited as of this commit, so
Sun-Tzu hasn't been smoke-tested in production. Code is shipped and
ready; first brief will land when the weekly quota refreshes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 16:41:20 -03:00
Luiz Gustavo
8283237f87 W5.4 followup: hero illustration on /c/[slug] + sitemap fix
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 34s
CI / Scripts — Python smoke (push) Failing after 6s
CI / Web — npm audit (push) Failing after 40s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Hero illustration:
  - Painterly 16:9 editorial illustration generated via Nano Banana Pro
    for the featured case (green-fireballs-narrative): late-1940s
    desert night, vivid emerald fireball over silhouetted Sandia mesas,
    1948-era state-police sedan parked on US 66 shoulder with an
    officer in period uniform looking up, faint green glow on his
    face. Sandia Base 5 Miles roadsign. New Yorker-cover painterly
    register, NOT photorealistic, NOT sci-fi.
  - Stored at /data/disclosure/processing/case-art/<slug>.png, served
    through the existing /api/static/processing/ route. 2.7MB at 2K.
  - components/featured-case.tsx: prefers the illustration over the
    declassified-page thumbnail when present. Tags it "Editorial
    illustration" / "Ilustração editorial" so the reader knows it's
    not a photograph.
  - app/c/[slug]/page.tsx: full-bleed editorial hero at the top of
    the article when an illustration exists for the slug. Title sits
    on the image with gradient overlay; "Ilustração editorial" chip
    in the top-right corner labels the art honestly. When no
    illustration exists the page falls back to the plain title header.

Sitemap fix:
  - Added export const dynamic = "force-dynamic" + revalidate = 3600
    to app/sitemap.ts. Without these Next.js statically generated the
    sitemap at build time, when the DB and case-files volume were
    unreachable from the build container — which is why production
    was serving only the 9 static URLs instead of ~3000.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 16:16:20 -03:00
Luiz Gustavo
70b2fe687f W5.4 (Phase 3B): sitemap + robots + Article schema + magazine reading view
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 31s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 27s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 5s
GEO/SEO surface area:

  app/robots.ts (new — Next.js dynamic robots)
    Explicitly ALLOWS major AI crawlers: GPTBot, OAI-SearchBot,
    ChatGPT-User, ClaudeBot, Claude-Web, anthropic-ai, PerplexityBot,
    Perplexity-User, Google-Extended, Applebot-Extended, CCBot,
    DuckAssistBot, YouBot, Bytespider, Amazonbot. The site exists to
    be cited by LLMs answering UAP/UFO questions — we want them in.
    /api/admin/, /admin/, /auth/ disallowed for everyone.

  app/sitemap.ts (new — Next.js dynamic sitemap)
    Lists 9 top-level routes + every /d/<doc> + every /c/<slug> from
    the filesystem + up to 500 entity URLs per class
    (event, person, uap_object, location, organization),
    sorted with summary-enriched entities first. ~3000 URLs total at
    current corpus size. lastModified honours summary_generated_at so
    crawlers re-index when entities are re-enriched.

  app/c/[slug]/page.tsx (rewritten — magazine reading view)
    - generateMetadata: per-case title, description (auto-extracted
      from the locale-preferred lead paragraph), canonical URL,
      hreflang alternate, OpenGraph article type with publishedTime,
      Twitter card.
    - JSON-LD Article schema embedded at end of page: schema.org
      Article + Organization publisher + inLanguage + isAccessibleForFree.
      This is what makes the case appear as a citable source in
      Google AI Overviews / Perplexity / ChatGPT search.
    - Reading view rewritten: display-serif headline (Fraunces), italic
      blockquotes with gold accent, prose-typography styling, no more
      detective stats line, no more "written by case-writer@detective"
      attribution. Locale-aware: PT-BR pulls topic_pt_br + lead in PT,
      English mirror.

  tailwind.config.ts
    + @tailwindcss/typography plugin
    + font-display family wired to var(--font-display) (Fraunces)

  package.json
    + "@tailwindcss/typography" devDependency

Phase 3A note: bulk entity enrichment hit Claude OAuth weekly quota mid-run.
6 events + 3 uap_objects landed bilingual summaries before the quota
exhausted. UI gracefully splits enriched vs bare entities so /sightings
shows the magazine-grade cards (Kenneth Arnold 1947, Roswell, Maury Island,
Joseph Perry 1960 lunar photo, Civil Defense Director 1966, etc.) on top
of a compact table of the rest. Re-run when quota refreshes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 16:09:50 -03:00
Luiz Gustavo
c40d1b58a0 W5.1 hotfix: Fraunces must be variable when using axes (next/font)
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 39s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Failing after 32s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 3s
2026-05-24 14:11:11 -03:00
Luiz Gustavo
ab4fe2a334 W5.1: enthusiast pivot — strip detective surfacing, magazine homepage
Some checks failed
CI / Retrieval — golden set (Recall@5 + MRR) (push) Waiting to run
CI / Web — typecheck + lint + build (push) Failing after 39s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Has been cancelled
User explicit: "1 bilhão de entusiastas pelo mundo ovni" — site is for the
UFO-curious public, not for skeptics. The 8-detective scaffolding becomes
invisible plumbing; the reader sees stories about what was observed.

Reader-facing changes:

  New homepage (web/app/page.tsx)
    - SiteHeader: magazine-style top nav (no detective tiles)
    - HeroBanner: full-bleed editorial opener with declassified-page art
      background, display-serif headline, live stats row (122 docs,
      2047 events, 1861 witnesses, 867 craft catalogued)
    - FeaturedCase: cover-story treatment of the most recent case_report,
      uses a real document page as hero image, links to /c/[slug]
    - PortalGrid: 6 thematic doorways into the archive — Sightings,
      Witnesses, Craft, Hot spots, Programs, Documents — each tile shows
      a real entity count and short editorial blurb
    - GreatestHits: top 9 most-cited events from the corpus
      (Kenneth Arnold 1947, Mantell 1948, …) as a magazine grid
    - Doc list kept but reframed as "the primary record"

  New sub-pages (5)
    - /sightings → events (2047), magazine grid
    - /witnesses → people (1861), compact table
    - /objects   → uap_objects (867), magazine grid
    - /locations → locations (1757), compact table
    - /operations → organizations (1596), compact table
    - /documents → full doc list with thumbnails (mirrors homepage section
      for direct deep-link)
    All share <EntityListPage> shell with per-page i18n + JSON-LD ItemList

  Stripped detective surfacing
    - /jobs/[id]: "Sherlock Holmes / Dr. Watson" → "Investigation in progress"
    - chat-bubble: detective-named card → neutral "Investigação em andamento"
    - quick-launch: 7-kind detective dropdown → single "investigar um caso"
      input (kind=case_report hardcoded)
    - /bureau: rewritten as the case-file library (no artefact dumps)

Typography + design
  - Fraunces variable serif loaded for display headings
    (`.font-display` class)
  - Gold-amber accent (#e0c080) unified as the brand colour
  - Asymmetric magazine grids (1+2+3 column, generous whitespace)
  - Hover micro-interactions (image scale on featured case, translateX
    on portal arrows)

SEO + GEO
  - layout.tsx metadataBase + title.template + per-route Metadata exports
  - Organization JSON-LD on root layout
  - WebSite + SearchAction JSON-LD on homepage
  - CollectionPage + ItemList JSON-LD on every entity list page
  - openGraph + twitter cards, pt-BR primary + en-US alternate
  - ai:purpose meta tag for Generative Engine Optimization — declares
    the site as a citation-linked primary-source archive
  - robots: index + follow with large image preview

The detectives themselves remain alive in the backend (runtime, DB, audit
log), but the reader never sees "Holmes / Sun-Tzu / Watson" in the UI. The
next phase will reorient case-writer to write as a single best-seller voice
synthesising all the internal sources.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 14:09:46 -03:00
Luiz Gustavo
24f12a27f4 W4.1+W4.2: anti-AI-tics house style + bureau nav (back/home everywhere)
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 31s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Failing after 31s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Two complaints in one wave:

(W4.1) User: "Não pode ter vícios de IA como uso excessivo de '-' que a IA
coloca geralmente no lugar de vírgulas por exemplo. Isso deve fazer parte
do prompt geral."

  - New prompts/_house-style.md banning the 9 most common AI prose tells
    in both EN and PT-BR:
      1. Em dashes as comma replacements (—)
      2. Rule-of-three lists ("concrete, rigorous, and grounded")
      3. Conjunctive openers ("Moreover", "Notably", "Ademais")
      4. Superficial -ing analyses ("marking a shift", "destacando")
      5. Inflated symbolism + AI vocab (tapestry, navigate, delve,
         underscore, robust, multifaceted, marco histórico, ...)
      6. Negative parallelisms ("Not just X but Y")
      7. Vague attribution ("Some scholars say...")
      8. Summary closers ("In summary...", "Em suma...")
      9. Hedging fluff ("It's important to note...")
    Verbatim chunk quotes are explicitly exempt; preserve as-is.
  - claude.ts callClaude() lazily loads _house-style.md once per process
    and PREPENDS it to every detective's system prompt:
        composedSystem = houseStyle + "---" + detective.systemPrompt
    This means all 7 detectives + future ones get the rules without any
    per-prompt change.

(W4.2) User: "Quando entra em uma página da investigação não tem como
voltar! UX terrível!"

  - New <BureauNav> sticky topbar with explicit "← home" + "🔎 bureau"
    buttons + clickable breadcrumb trail. Always visible at the top of
    every bureau page so the user can escape in one click.
  - Wired into /bureau, /h/[hypothesisId], /c/[slug], /jobs/[id]. Each
    page passes its sensible parent crumb (/bureau#hypotheses,
    /bureau#reports, /bureau#jobs).
  - Replaces the previous plain-text "disclosure.top / hypothesis /
    H-0004" line which had no visual affordance.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 13:27:58 -03:00
Luiz Gustavo
7826710051 W4: bilingual EN + PT-BR Investigation Bureau (CLAUDE.md §3 contract)
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 41s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Failing after 26s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
User flagged that the bureau was emitting English-only output, violating
the project's bilingual rule. Every narrative field now ships in both
languages: stored in sibling DB columns + rendered as adjacent markdown
sections per CLAUDE.md §3.

Migration 0007 (apply as supabase_admin):
  - public.hypotheses    +question_pt_br, +position_pt_br,
                         +argument_for_pt_br, +argument_against_pt_br
  - public.contradictions +topic_pt_br, +notes_pt_br
  - public.witnesses     +access_to_event_pt_br, +bias_notes_pt_br,
                         +verdict_pt_br
  - public.gaps          +description_pt_br, +suggested_next_move_pt_br
  - public.evidence: unchanged (verbatim_excerpt stays source-language)
  - JSONB siblings inside contradictions.chunks + gaps.scope handled at
    runtime (statement_pt_br, title_pt_br, dominant_model_pt_br,
    why_surprising_pt_br, what_it_implies_pt_br).

Detective prompts (all 7) rewritten with explicit bilingual JSON contract:
  - Output protocol section names every EN field + its _pt_br sibling
  - "Bilingual is mandatory" warning in the task instruction
  - Sentinel skip-states unchanged (NO_HYPOTHESES, NO_CONTRADICTIONS,
    INSUFFICIENT_TESTIMONY, INSUFFICIENT_HYPOTHESIS, NO_OUTLIERS,
    NO_NEW_EVIDENCE, INSUFFICIENT_ARTEFACTS)
  - Schneier: parallel arrays — hidden_assumptions[i] matches
    hidden_assumptions_pt_br[i], lengths must match
  - Case-Writer: interleaved §1 (EN) / §1 (PT-BR) per act in the body

Writer-side validation (all 7 tools):
  - Reject INSERT if PT-BR sibling missing when EN field is set
  - Persist both languages atomically in one INSERT (no half-updates)
  - Markdown renderers write adjacent EN+PT-BR sections in case files
    (## Argument for (EN) followed by ## Argumento a favor (PT-BR), etc.)

Detective parse layer (all 7 detectives):
  - Coerce both keys from JSON output
  - "incomplete_bilingual_*" skip reason when either side missing
  - Defensive: PT-BR fields trimmed + length-capped same as EN

Orchestrator propagates question_pt_br + topic_pt_br through job payload
to runHolmes / runCaseWriter, mirroring the chat-tool entry point.

Web (UI):
  - /api/jobs/[id] hydrates _pt_br siblings from pg
  - job-status-poller HypothesisCard: PT-BR primary, EN in <details>
    fallback when both exist
  - ContradictionCard: PT-BR statement primary + secondary EN quote
  - WitnessCard: PT-BR verdict primary + secondary EN quote, panels in PT
  - GapCard: PT-BR title/why/implies primary
  - /bureau hub: SELECTs both columns, renders PT-BR primary
  - /h/[id]: ArgumentPanel renders PT-BR primary with collapsible EN
    fallback when both exist
  - BureauSnapshot homepage: position_pt_br / topic_pt_br / verdict_pt_br
    primary
  - DocBureauPanel /d/[doc]: same primary-PT-BR pattern
  - New web/lib/i18n/pick.ts helper (unused yet by chat/agents — kept
    for future locale-driven switching when both languages are equally
    full; current rule is PT-BR-first since the user is brasileiro)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:02:59 -03:00
Luiz Gustavo
d4a2e4f51e W3.10: clickable detective tiles + quick-launch form + doc bureau panel
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 37s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 40s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Builds on top of W3.9 to turn the homepage Bureau from a read-only
dashboard into a working command center.

UI improvements (web/components/bureau-snapshot.tsx):
  - Detective tiles are now <Link>s — each navigates to its primary
    artefact section in /bureau (Holmes→#hypotheses, Locard→#evidence,
    Dupin→#contradictions, Schneier→#hypotheses, Poirot→#witnesses,
    Taleb→#outliers, Tetlock→#hypotheses, Case-Writer→#reports). Hover
    bg matches the detective's tone color.
  - <QuickLaunch /> form inserted right under the tiles.

New <QuickLaunch /> client component:
  - Detective dropdown (7 active kinds; evidence_chain not yet exposed
    here since it needs a doc_id better picked from the doc page).
  - Single input swaps placeholder + aria-label by kind: question for
    Holmes, topic for Dupin/Taleb/Case-Writer, hypothesis_id for
    Schneier/Tetlock, person_id for Poirot.
  - Submits to POST /api/bureau/launch and redirects to /jobs/[id]
    via the next.js router.
  - Loading state ("queueing…") + error display inline.

POST /api/bureau/launch (web/app/api/bureau/launch/route.ts):
  - Same 8-kind validator as the chat tool's request_investigation.
  - Auth required when Supabase is configured (triggered_by = user:email).
  - Returns { job_id, kind, detective, status_url, eta_seconds }.

DocBureauPanel on /d/[docId] (web/components/doc-bureau-panel.tsx):
  - Server component inserted between the doc header and
    AnomalyHighlights.
  - Surfaces every bureau artefact that touches the doc:
    · Evidence whose source_page_id starts with docId/p
    · Hypotheses citing any of those evidence_ids
    · Contradictions whose chunks[] has any item with this doc_id
    · Gaps/outliers with scope.doc_id == docId
    · Case reports whose markdown body references docId (filesystem scan)
  - Empty state shows "Investigation Bureau — untouched" with a CTA
    linking back to the homepage to launch the first investigation.
  - When non-empty, header counts total artefacts + links to /bureau
    for the full view.

Metadata (web/app/layout.tsx):
  - description rewritten from "Investigative wiki of the US Department
    of War UAP/UFO archive (war.gov/ufo)" to one that names the bureau
    + the 8 detectives. Affects SERP previews + social-card defaults.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 23:33:00 -03:00
Luiz Gustavo
67185ff518 W3.9: surface the Investigation Bureau on the homepage + /bureau hub
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 40s
CI / Scripts — Python smoke (push) Failing after 3s
CI / Web — npm audit (push) Failing after 31s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Closes a UX gap the user surfaced: W3.5-3.8 built 8 detectives, 4 new
URL endpoints (/jobs/[id], /h/[id], /c/[slug], /api/h/[id]/red-team)
and a chat tool, but the homepage was unchanged — the bureau was
invisible unless you knew the URL or asked the chat to invoke
request_investigation.

Homepage (web/app/page.tsx):
  - Title `▍ war.gov/ufo — Investigative Wiki` → `▍ The Disclosure Bureau`
  - Subtitle expanded from "Holmes · Poirot · Dupin · Locard" to all 8
    detectives (Holmes · Locard · Dupin · Schneier · Poirot · Taleb ·
    Tetlock · Case-Writer)
  - New `🔎 bureau` topbar link (gold, between graph/stats and batch)
  - BureauSnapshot inserted right after the header

BureauSnapshot (web/components/bureau-snapshot.tsx) — server component:
  - 8 detective tiles with role labels (each in its tone color)
  - 6 clickable counters (evidence / hypotheses / contradictions /
    witnesses / outliers / case reports) — anchor to /bureau#section
  - 6 "recent artefacts" columns surfacing the last 3-4 of each kind:
    hypotheses with prior→posterior + band + ↳reviewed_by marker,
    contradictions with topic + resolution_status, evidence with
    Grade badge + verbatim quote, outliers with title + scope.kind,
    witness analyses with canonical_name + credibility + verdict,
    case reports with slug + link to /c/<slug>
  - "Recent jobs" strip linking to /jobs/[id] color-coded by status
  - Reports read from /data/ufo/case/reports/ via fs.readdir + stat,
    sorted by mtime — no DB round-trip needed for that section

/bureau (web/app/bureau/page.tsx) — full hub:
  - Header with full counts
  - 7 sections (anchored to homepage counter links): Case reports,
    Hypotheses, Evidence, Contradictions, Outliers, Witnesses,
    Recent jobs table — each rendering up to 100 rows
  - Reports section parses frontmatter from each .md to surface topic
    + n_hypotheses + n_evidence on the card

Runtime fixes batched in:
  - Poirot: coerce entity_pk via Number() — node-postgres returns
    BIGINT as string by default; writer's Number.isFinite() rejected
    it as "person_entity_pk required" (j-edgar-hoover retry path)
  - Tetlock: write_calibration rationale cap 600 → 1200 chars. Prompt
    still asks ≤ 600 but a 2× slack beats failing the job on honest
    analysis. Observed live: Tetlock emitted ~620 chars on H-0003 and
    the writer rejected the entire calibration.
  - Case-Writer: Promise.all of 5 queries × max_parallel=2 jobs
    demanded up to 10 connections against the investigator role's
    rolconnlimit=4 → "too many connections for role investigator".
    Sequentialized — the LLM call is the hot path, not these queries.

Smoke results visible now on the homepage:
  - 3 hypotheses (H-0001/2/3) about green fireballs origin
  - 3 contradictions (R-0001/2/3) about color, geographic confinement,
    exclusive-green vs multicolored
  - 2 evidence cards (E-0002/3) Grade B
  - 3 outliers (G-0001/2/3) — including Taleb's deliberate
    meteor-shower-camouflage flag
  - 1 case report at /c/green-fireballs-sandia (Watson 13.4 KB,
    five-act narrative, fully cited)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 22:41:28 -03:00
Luiz Gustavo
dd75a67964 W3.8: Investigation Bureau complete — Poirot, Taleb, Tetlock, Case-Writer
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 45s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 40s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 3s
Brings the bureau from 4 → 8 detectives. All eight run as Bun + claude-CLI
subprocesses against the same Supabase + investigation_jobs LISTEN/NOTIFY
queue, sharing search.ts hybridSearch and writer-side validators that
gate writes against schema + FK.

New detectives:

  Poirot (witness_analysis)
    - prompts/poirot.md — credibility / access / bias / corroboration /
      verdict; uses entity_mentions JOIN chunks to pull 12 chunks per
      person; resolves corroboration_refs chunk_ids defensively (accepts
      bare cNNNN even when the model emits pNNN/cNNNN).
    - INSERT into public.witnesses with W-NNNN naming.
    - Tone: purple (#9b5de5).

  Taleb (outlier_scan)
    - prompts/taleb.md — "surprise is relative to a model"; at most 3
      outliers; each requires explicit dominant_model + why_surprising +
      what_it_implies; fan-out into public.gaps with scope.kind="outlier".
    - Same unscoped-fallback as Dupin (Pass 1 with doc_id, Pass 2 widens
      to corpus if hits < 3).
    - Tone: yellow (#ffd23f).

  Tetlock (calibrate_hypothesis)
    - prompts/tetlock.md — honest Bayesian update; emits new_posterior +
      Δ + recommended_action ∈ {keep, downgrade, upgrade, supersede}.
    - write_calibration UPDATEs public.hypotheses + APPENDS a
      "## Calibration history" section to the H-NNNN.md case file
      (calibration is append-only — each datapoint matters). Posterior
      band auto-corrected to match Tetlock thresholds.
    - NO_NEW_EVIDENCE sentinel handled; pure 'keep' with |Δ|<0.005 only
      touches updated_at + reviewed_by.
    - Tone: teal (#26d4cc).

  Case-Writer (case_report)
    - prompts/case-writer.md — Dr. Watson assembles all artefacts
      (E-NNNN, H-NNNN, R-NNNN, W-NNNN, G-NNNN) into a five-act narrative.
      ILIKE filter on topic; doc_id optional scope.
    - Larger budget cap (≥ $0.50) + longer timeout for prose generation.
    - Writes case/reports/<slug>.md with frontmatter (topic + counts);
      no DB table for v0.
    - New page /c/[slug] renders the report via MarkdownBody + stat chips.
    - Tone: gold (#e0c080).

Hardening across the bureau:
  - Sentinel parsing now accepts backticked AND prose-trailing forms
    (Holmes NO_HYPOTHESES, Dupin NO_CONTRADICTIONS, Schneier
    INSUFFICIENT_HYPOTHESIS, Poirot INSUFFICIENT_TESTIMONY, Taleb
    NO_OUTLIERS, Tetlock NO_NEW_EVIDENCE, Case-Writer
    INSUFFICIENT_ARTEFACTS). Avoids the failure mode where the model
    refuses honestly but the runtime treated it as a parse error
    (observed live with Poirot+Hoover identifying the DIRECTOR
    false-positive disambiguation issue in entity_mentions).

Chat tool extensions (web/lib/chat/tools.ts):
  - request_investigation now accepts 7 kinds. Each routes to its
    detective with appropriate validation (hypothesis_id regex,
    person_id kebab-case, topic non-empty, doc_id for evidence_chain).
  - ETA per kind: Holmes/Dupin 60s, Poirot 45s, Schneier/Tetlock 30s,
    Taleb 50s, Case-Writer 180s (longer prose), Locard 30×n_chunks.

UI integration:
  - chat-bubble inline card paints each detective in its tone color.
  - /jobs/[id] page header swaps name/subtitle/tone per detective;
    question label adapts ("Topic" / "Hypothesis under attack" /
    "Witness under analysis" / "Topic to outlier-scan" / "Hypothesis
    under recalibration" / "Case to assemble").
  - job-status-poller renders: case-report link card (gold), outlier
    cards (yellow), witness cards (purple) — alongside existing
    hypothesis, evidence, contradiction cards.
  - /api/jobs/[id] hydrates witnesses (JOIN entities for canonical_name)
    + gaps (with scope JSONB).
  - /c/[slug] page reads /data/ufo/case/reports/<slug>.md and renders
    with MarkdownBody, frontmatter parsed for stat chips.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 22:11:39 -03:00
Luiz Gustavo
857dd771d2 W3.8: Schneier red-team detective + /h/[hypothesisId] dossier page
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 33s
CI / Scripts — Python smoke (push) Failing after 7s
CI / Web — npm audit (push) Failing after 38s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Adds the fourth AI detective in the Investigation Bureau runtime: Bruce
Schneier, who attacks an existing hypothesis as a red-team operator.

Runtime:
  - prompts/schneier.md — discipline (don't disprove, just attack;
    structured output with hidden_assumptions, failure_modes,
    alternative_explanations, recommended_tests, verdict_one_sentence;
    severity ∈ {low, medium, high}; emit INSUFFICIENT_HYPOTHESIS when
    the input is too thin)
  - src/detectives/schneier.ts — reads the hypothesis row + evidence
    chain (joined via evidence_refs FK), feeds Claude with the
    arguments + verbatim quotes, parses strict JSON object
  - src/tools/write_red_team_review.ts — UPDATEs hypotheses.reviewed_by
    + updated_at; APPENDS (or replaces if re-reviewed) a structured
    "## Red-team review (Schneier · X severity)" section to
    case/hypotheses/H-NNNN.md. Caps each list at 5 entries × 240 chars,
    validates verdict ≤ 280 chars.
  - orchestrator: new `red_team_review` kind dispatching to runSchneier

Chat + UI:
  - request_investigation gains kind=red_team_review + hypothesis_id arg
    (validated against H-NNNN regex); detective auto-resolves to schneier
  - chat-bubble inline card paints Schneier in red (#ff3344)
  - /jobs/[id] page swaps title/subtitle/tone per detective; the
    "Question" label becomes "Hypothesis under attack" for red_team_review

New /h/[hypothesisId] page (hypothesis dossier):
  - Server-rendered from public.hypotheses + public.evidence (joined
    via evidence_refs FK + chunk lookup)
  - Header: ID + creator + reviewer (highlighted when Schneier has
    visited), position as headline, question subtitle, Tetlock band
  - Prior + posterior bars with Δ-delta indicator
  - Argument grid: argument_for (green) vs argument_against (pink)
    side-by-side with [[wiki-link]] auto-linking to source chunks
  - Evidence chain: each E-NNNN with Grade A/B/C badge, verbatim
    blockquote, link to source page
  - Red-team review panel: parses the markdown section in the case
    file (severity badge, verdict, 4 bullet panels for
    hidden_assumptions / failure_modes / alternative_explanations /
    recommended_tests). Empty state when not yet reviewed.

RedTeamRequestButton client component + POST /api/h/[id]/red-team —
authenticated user can trigger Schneier in one click; UI swaps to
"acompanhar" link to /jobs/[id] once queued.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 21:48:12 -03:00
Luiz Gustavo
5ac53cb3e2 W3.7: Dupin contradiction-scan detective + UI integration
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 39s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Failing after 37s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Adds the third AI detective in the Investigation Bureau runtime: C. Auguste
Dupin, who scans a corpus shortlist for pairs (or small groups) of chunks
that cannot both be true under any ordinary reading.

Runtime:
  - prompts/dupin.md — discipline (no contradiction without ≥2 distinct
    chunk_ids; reject same-vocabulary near-misses; FEW high-confidence
    over MANY weak ones; emit `NO_CONTRADICTIONS` when corpus is silent)
  - src/detectives/dupin.ts — hybridSearch with k=18 (more chunks than
    Holmes because contradictions emerge from comparing dispersed
    claims), strict JSON-array parsing, AT MOST 3 contradictions per call
  - src/tools/write_contradiction.ts — validates topic + ≥2 positions
    drawn from ≥2 distinct chunks, resolves chunk_pk via DB lookup
    (rejects positions citing unknown chunks), INSERTs into
    public.contradictions + writes case/contradictions/R-NNNN.md
  - orchestrator: new `contradiction_scan` kind dispatching to runDupin;
    payload { topic, doc_id?, lang?, context_chunks? }

Chat + UI:
  - request_investigation gains kind=contradiction_scan + topic arg;
    triggered detective auto-resolves to dupin
  - chat-bubble inline card renders dupin in orange (#ff8a4d) to
    distinguish from holmes (cyan) and locard (green)
  - /jobs/[id] page swaps title + subtitle + tone per detective;
    "Question" label becomes "Topic" for contradiction_scan
  - /api/jobs/[id] hydrates public.contradictions when outputs[] surfaces
    contradiction_ids
  - job-status-poller renders ContradictionCard: topic + N positions
    (verbatim statements quoted, stance label optional, link to source
    chunk) + optional notes panel, with resolution_status badge
    (open/resolved/irreconcilable)

R-NNNN shares the contradiction_id_seq slot with relation per
CLAUDE.md naming — same conceptual class (a connection between two
pieces of evidence in tension).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 21:34:04 -03:00
Luiz Gustavo
b76e81e4b3 W3.6: chat request_investigation tool + /jobs/[id] case-file viewer
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 44s
CI / Scripts — Python smoke (push) Failing after 3s
CI / Web — npm audit (push) Failing after 43s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 7s
Closes the loop between the chat UI and the Investigation Bureau runtime.

Chat tool (web/lib/chat/tools.ts):
  - request_investigation { kind, question, doc_id?, chunks?, claim? }
    INSERTs a row in public.investigation_jobs and returns
    { job_id, kind, status, eta_seconds, status_url, detective }.
  - kind=hypothesis_tournament → Holmes (1 question → 2-3 rival hypotheses)
  - kind=evidence_chain → Locard (1 doc → grade-A/B/C evidence with chain
    of custody, default top-5 anomaly chunks)
  - Plumbed user.email through ToolHandlerContext so triggered_by audits
    the requesting user.

Public job viewer:
  - GET /api/jobs/[id] joins investigation_jobs → public.evidence +
    public.hypotheses for the IDs surfaced in outputs[]. Returns one
    payload the page can render without n+1 round-trips. Strips
    triggered_by from the response (it carries the user's email).
  - app/jobs/[id]/page.tsx server-renders the case-file shell:
    detective lore header (Holmes blue or Locard green), question chip,
    scope chip with link back to the document.
  - components/job-status-poller.tsx client island that polls every 3 s
    while non-terminal, then once on terminal to hydrate evidence +
    hypotheses. Renders:
      · Phase tracker (queued → running → complete | failed)
      · Hypothesis cards w/ prior + posterior bars + Δ delta indicator
        + Tetlock band badge (high/medium/low/speculation)
      · Argument-for / argument-against with [[wiki-link]] auto-linking
        to /d/<doc>/p<NNN>#<cNNNN>
      · Evidence cards w/ Grade A/B/C badge + verbatim blockquote +
        bbox crop preview via /api/crop + custody-steps disclosure
      · Empty/in-flight panel ("os detetives estão lendo o corpus")
      · Failure panel surfacing error + partial outputs

Inline chat-bubble card (components/chat-bubble.tsx):
  - ToolTrace.richRender recognises request_investigation results and
    renders a detective banner with status + ETA + link to /jobs/[id]
    (target=_blank). Error case renders a red strip with the message.

UX flow now: user asks Sherlock a question → request_investigation
queues the job → chat card shows "🔎 Holmes · hypothesis_tournament ·
ETA ~60s" → user clicks → /jobs/<id> live-updates → 60 s later, 2-3
rival hypotheses + their arguments + chunk citations are rendered with
Bayesian update visible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 21:26:18 -03:00
Luiz Gustavo
189a771cbe W3.1-W3.4: Investigation Bureau foundation — migrations, runtime, Locard
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 38s
CI / Scripts — Python smoke (push) Failing after 3s
CI / Web — npm audit (push) Failing after 33s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Migrations:
- 0004_investigation_bureau.sql: 7 new tables (investigation_jobs + evidence,
  hypotheses, contradictions, witnesses, gaps, residual_uncertainties), id
  sequences, pg_notify trigger on investigation_jobs, RLS read-only public,
  investigator role with least-privilege grants (no service_role).
- 0005_investigator_write_policies.sql: fixup adding RLS INSERT/UPDATE
  policies bound to investigator + service_role + postgres (RLS with only a
  SELECT policy was silently blocking the worker's claim UPDATE).

investigator-runtime/ (new Bun + TS container):
- src/main.ts: LISTEN/NOTIFY poller, claim-with-SKIP-LOCKED, drain pool,
  healthcheck file, graceful SIGTERM shutdown.
- src/orchestrator.ts: chief-detective dispatch (evidence_chain → Locard).
  Marks job failed when all per-item outputs error; surfaces first errors.
- src/lib/{env,pg,audit,ids,claude}.ts: typed config (gate #8), pool +
  dedicated LISTEN client, NDJSON audit, sequence allocator (E-NNNN etc),
  claude -p subprocess with quota detection (api_error_status=429).
- src/tools/write_evidence.ts: schema-validate (grade A/B/C custody steps),
  resolve chunk_pk via FK, verify verbatim_excerpt actually appears in
  chunk content, INSERT + render case/evidence/E-NNNN.md + audit.
- src/detectives/locard.ts: load chunk → call Claude with locard.md system
  prompt → parse strict JSON → call writeEvidence locally.
- Dockerfile installs `claude` CLI (OAuth) at build time.

Compose:
- new `investigator` service builds from investigator-runtime/, connects
  with low-privilege role, mounts case/ RW and wiki/+raw/ RO, 512m mem cap.

Web:
- /api/admin/investigate/test (POST+GET) gated by middleware (W0-F1).
  POST creates a job, GET polls status. For W3.6 it becomes the chat tool.

End-to-end smoke: INSERT job → pg_notify → claim → Locard dispatch →
claude subprocess invoked. Auth works (CLI v2.1.150). Currently quota
exhausted (weekly limit · resets 3pm UTC) — pipeline catches the typed
isQuota error, marks job failed with surfaced reason. Architecture proven;
quota reset enables real evidence creation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 19:49:33 -03:00
Luiz Gustavo
eaf282c535 W2: rerank opt-in, analyze_image_region tool, RAG eval, graph cleanup, ADRs
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 40s
CI / Scripts — Python smoke (push) Failing after 3s
CI / Web — npm audit (push) Failing after 29s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 3s
- TD#8 hybrid.ts: rerank_strategy {always|when_top_k_gt|never} + threshold
  (default skips rerank for top_k ≤ 15; chat tool uses threshold 10)
- O11 vision.ts + tools.ts: analyze_image_region tool — sharp-crops the
  bbox, claude CLI reads the temp PNG via Read tool, Sonnet vision answers
- TD#12 /graph: SigmaGraph replaces ForceGraphCanvas; react-force-graph-2d
  uninstalled (-37 transitive deps); force-graph-canvas.tsx deleted
- TD#27 messages/route.ts gatherContext slice sizes via CTX_* env vars
- TD#22 tests/rag/: golden.yaml (15 queries) + run.py (Recall@k + MRR +
  negative-pass rate) + baseline.json + CI job in .forgejo/workflows/ci.yml
- docs/adrs/: ADR-001..005 published from systems-atelier deliverables

Verified live on disclosure.top: top_k=5 path skips rerank (6.7s embed-only,
was 12-15s with rerank); rerank=always still available on demand.
First RAG baseline: Recall@5 = 0.2083, MRR = 0.25, Negative pass = 1.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 19:20:09 -03:00
Luiz Gustavo
55cac8a395 W0+W1+W1.2: security hardening, observability, autocomplete, glitchtip, forgejo CI
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 1m30s
CI / Scripts — Python smoke (push) Failing after 32s
CI / Web — npm audit (push) Failing after 37s
W0 — security hardening (5 fixes verified live on disclosure.top)
- middleware: gate /api/admin/* same as /admin/* (F1)
- imgproxy: tighten LOCAL_FILESYSTEM_ROOT from / to /var/lib/storage (F2)
- studio: real basic-auth label (bcrypt hash, middleware reference) (F3)
- relations: ENABLE ROW LEVEL SECURITY + public SELECT policy (F4)
- migration 0003: fold is_searchable + hybrid_search update into canonical (TD#2)

W1 — observability + resilience + autocomplete
- studio: HOSTNAME=0.0.0.0 so Next.js binds on loopback for healthcheck
- compose: PG_POOL_MAX=20, CLAUDE_CODE_OAUTH_TOKEN gated by separate env
- claude-code.ts: subprocess timeout configurable (CLAUDE_CODE_TIMEOUT_MS)
- openrouter.ts: retry with exponential backoff + Retry-After + in-memory
  circuit breaker (promotes FALLBACK after CB_THRESHOLD failures)
- lib/logger.ts: pino logger (NDJSON prod / pretty dev) + withRequest helper
- middleware: mints correlation_id, stamps x-correlation-id response header,
  emits structured http_request log per /api/* call
- messages/route.ts: switch to structured logger
- 60_meili_index.py: push documents + chunks into Meilisearch
- /api/search/autocomplete: parallel meili search (docs + chunks), 5-8ms p50
- search-autocomplete.tsx: debounced dropdown wired into search-panel

W1.2 — Glitchtip + Forgejo self-hosted
- compose: glitchtip-redis + glitchtip-web + glitchtip-worker (v4.2)
- compose: forgejo + forgejo-runner (server v9, runner v6) with group_add=988
- @sentry/nextjs SDK wired (instrumentation.ts + sentry.{client,server}.config.ts)
- /api/admin/throw smoke endpoint (gated by W0-F1 middleware)
- Synthetic event ingestion verified at glitchtip.disclosure.top
- forgejo.disclosure.top up, repo discadmin/disclosure-bureau created,
  runner registered (labels: ubuntu-latest, docker)
- .forgejo/workflows/ci.yml: typecheck + lint + build + npm audit + python
  syntax + compose validation

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 18:18:42 -03:00
Luiz Gustavo
e75ca5eda2 add clean LLM reading version of documents (the core goal)
Scanned docs are messy — duplicate transcriptions (typed + handwritten),
two classification variants of the same narrative, OCR noise, repeated
banners. The doc page showed raw chunks, so everything appeared twice.

40_reading_version.py generates ONE clean, deduplicated, well-structured
bilingual Markdown reading version per doc (Sonnet): merges duplicate versions
without losing unique lines, drops page furniture, formats transcripts as
dialogue. Faithful — invents nothing; redactions kept as markers.

/d/[docId] now defaults to a "📖 leitura" tab rendering this clean version,
with "🔍 trechos · scan original" preserving the faithful per-chunk + per-page
scan view. reading.md lives in raw/<doc>--subagent/ alongside the chunks.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 17:23:36 -03:00
Luiz Gustavo
fe19bb9c57 add page↔document navigation + DB repopulation tooling
Doc page (/d/[docId]/[page]) gains prev/next navigation bars (top + bottom):
within a doc it steps page-by-page; at the first/last page it jumps to the
previous/next document. Replaces the disabled-at-boundary links.

Indexer tooling for the VPS repopulation:
- 30-index-chunks-to-db.py: add --no-embed (fast BM25-only index; vectors
  backfilled separately) so the app is usable in minutes, not hours of CPU
  embedding.
- 57_load_relations_from_json.py: load typed relations into public.relations
  from reextract structured fields (deterministic ids, no fuzzy guessing).
- 58_backfill_embeddings.py: async pass to fill chunks.embedding (NULL rows)
  via the embed-service.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 14:28:14 -03:00
Luiz Gustavo
a7e9dce6d2 rebuild entity layer from Sonnet-vision reextract pipeline
Add reextract pipeline (scripts/reextract/) that rebuilds doc-level entity
JSON from Sonnet-vision chunks via Opus, replacing the noisy per-page
extraction. Add synthesize scripts to regenerate wiki/entities from the 116
_reextract.json (30), aggregate missing page.md from chunks (31), and reprocess
805 pages the doc-rebuilder agent dropped on context overflow (32). Add
maintain scripts 43-56 for chunk-page sync, dedup, generic-entity marking, and
typed relation extraction.

Web: wire relations API + entity-relations component; entity/timeline/doc
pages consume the rebuilt layer.

Note: raw/, processing/, wiki/ remain gitignored (bulk data managed
separately); the 116 reextract JSONs and 7,798 rebuilt entity files live on
disk only. The 27 curated anchor events under wiki/entities/events/ are
preserved.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 12:20:24 -03:00
guto
291748df63 sanitize entities: single YAML source of truth, signal_strength badge
The corpus had two parallel reverse-reference signals: the wiki/pages
entities_extracted blocks (Haiku page-level) and public.entity_mentions
(Sonnet chunk-level, ILIKE-matched). The entity page only consulted the
DB, so it showed "0 menções" for thousands of entities that were anchored
in pages or in cross-entity links the DB never indexed.

Resolved by collapsing all signals into the YAML frontmatter, which is
now the single runtime source for entity metadata.

scripts/maintain/42_sync_entity_stats.py walks every entity and writes:

  mentioned_in:        [...]        # consolidated page refs
  total_mentions:      max(db, pages)
  documents_count:     max(db_docs, distinct page docs)
  signal_sources:
    db_chunks:         int
    page_refs:         int
    cross_refs:        int
  signal_strength:     strong | weak | orphan | unverified
  referenced_by:       [[class/id]]  # cross-entity backlinks

Outgoing wikilinks (e.g. OBJ.observed_in_event → EV) count toward the
entity's own cross_refs so anchored-but-not-mentioned entities don't
register as orphan.

OBJ canonical names like "7m long, 1.3m high, two rocket motors,
smooth flow, rotary drive null UAP (OBJ-EV1945-PEYERLSHOTDOWN-01)"
are rewritten to "Peyerl shot down UAP" derived from observed_in_event,
preserving the full description as an alias. --fix-obj-names did this
for every OBJ-* with >80 char canonical_name.

Default behaviour is conservative: --archive-only-junk archives only
single/double-char names and pure-numeric noise. Everything else stays
on disk with signal_strength marked, so the user can review later.

web/lib/retrieval/entity-pages.ts swapped from db-first to yaml-first.
The /e/[cls]/[id] page now reads counts straight from YAML and renders
a "força do sinal" badge with the per-source breakdown. Orphan entities
get a banner explaining they have no cross-references.

DB is still queried for ONE thing: the chunk text for preview cards on
the entity page, so we don't re-parse 21k markdown files on every render.

First-pass result: 9020 strong / 14520 weak / 10814 orphan; OBJ-EV1945-
PEYERLSHOTDOWN-01 now reads "Peyerl shot down UAP · fraca · 1 backlink"
in the live UI.
2026-05-18 19:49:31 -03:00
guto
9889308bf4 fix chat: force synthesis pass + fix ambiguous-column trigger
Two bugs combined to make the chat reply with only cards and no prose:

1. SQL trigger rollup_session_stats was failing with "column reference
   total_cost_usd is ambiguous" because the UPDATE on public.profiles had
   a FROM public.chat_sessions clause and both tables expose that column.
   Persistence of every user message died at this point — sessions were
   created in the DB but had message_count=0 forever. Applied SQL fix
   that qualifies columns with p./s. aliases (production DB updated;
   ALTER FUNCTION run live, not yet codified in a migration file).

2. The free-tier model (nemotron-3-super:free) spent all 5 tool-loop
   turns on hybrid_search calls and never wrote any prose, returning
   content_len=0. Added a forced-synthesis pass in openrouter.ts: when
   the loop exits with empty assembledText but the model did call tools,
   we send ONE final turn with tools omitted from the request payload
   and a user message instructing the model to answer in 3-8 sentences
   citing chunks. openrouterStreamCall now accepts a `withTools` opt
   so the synthesis call can disable tool calling entirely.

Verified end-to-end with the actual user query "O que os astronautas
viram? Quem foi que viu?" on /d/nasa-uap-d6-apollo-17-...:
- content_len: 0 → 947 chars (real synthesis citing Schmitt)
- artifacts: 44 preserved
- assistant message persisted with tool_calls + citations columns
2026-05-18 15:39:46 -03:00
guto
d5f6e6030a fix png-numbering: re-convert 34 zero-based docs + crop fallback
34 of 116 docs were generated with 0-based PNG numbering (p-000.png …
p-008.png) but the Sonnet chunks reference 1-based page numbers in their
YAML frontmatter (page: 9 means the 9th sheet of paper). The /api/crop
handler built p-009.png and got a 500, the browser's Next/Image surfaced
400, and the chunk rendered as a black box on screen.

Fixes:
- web/app/api/crop/route.ts: try p-NNN.png first, fall back to
  p-(NNN-1).png if the 1-based file is missing. Cheap insurance for any
  doc that comes in with the old convention.
- scripts/01-convert-pdfs.sh: previously printf '%03d' "$num" with $num
  starting at 0 (e.g. "008") raised "invalid number" because Bash
  parsed it as octal. Wrap with $((10#$num)) to force decimal — this
  was silently corrupting page sequences and producing gaps like p-001
  ... p-008, p-011 (missing p-009/p-010).
- All 34 affected docs re-converted from PDFs with the patched script;
  every directory now has continuous 1-based PNGs.
- /processing/png/ rsync'd to VPS, web redeployed.

Smoke: /api/crop?doc=doc-341-…&page=9&… now returns 200 image/webp
instead of 500. Tested in browser: chunk c0026 (diagram, p9) renders
the real engineering diagram.
2026-05-18 11:45:40 -03:00
guto
7d13f93393 ship: synthesize 158 entities, AG-UI artifacts, chat persistence, auth flow
Fase 3 onda 2 — entity synthesis at scale:
- scripts/synthesize/20_entity_summary.py: queries DB for entities with
  total_mentions ≥ threshold + top-K verbatim chunk snippets via
  entity_mentions JOIN, prompts Sonnet (Holmes-Watson voice, bilingual),
  writes narrative_summary EN+PT-BR + summary_status=synthesized.
  Ran on 187 candidates (mentions ≥ 20) → 158 OK · 1 err · 29 skipped (no
  snippets). Combined with anchor curation: 20 curated + 158 synthesized
  = 178 entities with real narrative (vs 0 a day ago).

Fase 4 — chat with typed artifacts + persistence:
- lib/chat/agui.ts: AG-UI v1 typed Artifact union (citation, crop_image,
  entity_card, evidence_card, hypothesis_card, case_card, navigation_offer)
  alongside the existing event types.
- lib/chat/tools.ts + openrouter.ts: hybrid_search emits up to 6
  citation + crop_image artifacts per query. Provider collects them and
  returns in done.artifacts so the route can persist.
- api/sessions/[id]/messages: persist artifacts to messages.citations.
- components/chat-bubble.tsx: ArtifactCard renders inline cards (citation,
  crop_image, entity_card, navigation_offer) for streamed and persisted
  messages. activeId now persisted in localStorage so navigation between
  pages keeps the same conversation. New sessions are lazy (only when user
  has zero). loadMessages hydrates tools + artifacts from server. CRUD UI:
  rename (✎) + archive (🗑) buttons per session in the list.

Home search:
- doc-list-filters: input now fires hybrid_search (rerank=0 for speed)
  in parallel with the local title filter; chunk hits render above the doc
  grid with snippet + score + classification.
- api/search/hybrid: accept ?rerank=0 to skip the cross-encoder (1.3s vs 60s).

Auth flow:
- infra: SMTP_HOST=mail.spacemail.com:587 + DMARC published; mail now lands
  in inbox. GOTRUE_MAILER_AUTOCONFIRM=false (real email verification).
- kong.yml: proxy /auth/callback on api.disclosure.top → web:3000 so PKCE
  email links don't 404 at the gateway.
- web/app/auth/callback: handle both ?code= (OAuth) and ?token=&type=
  (PKCE); redirect to the public site host before verifyOtp so the session
  cookie lands on the right domain.

Audit deliverables:
- .nirvana/outputs/disclosure-bureau/.../systems-atelier/: 5 docs (code
  analysis, tech debt, discovery brief, system arch, 5 ADRs) authored by
  sa-principal that produced this roadmap. Kept in-tree for traceability.
2026-05-18 03:52:59 -03:00
guto
4459bd17e4 phase-0: kill stubs, ship 20 curated anchor events, configure SMTP
- scripts/03-dedup-entities.py: stop emitting placeholder narrative ("Stub. Will
  be enriched in Phase 7"); write summary_status=none + null fields instead.
- scripts/maintain/41_strip_stubs.py: idempotent migration that cleaned the
  22,096 entity .md files (now zero stub strings in wiki/).
- scripts/synthesize/01_anchor_events.py: curated 20 anchor UAP events
  (Roswell, Nimitz Tic-Tac, Phoenix Lights, Operação Prato, AATIP, etc.) with
  bilingual Holmes-Watson narrative via claude -p --model sonnet
  (CLAUDE_CODE_OAUTH_TOKEN). All summary_status=curated, confidence=high.
- web/api/timeline + timeline-view: filter narrative-less events by default,
  render "curado" badge for hand-vetted ones, drop the date display alone.
- CLAUDE-schema-full.md: document the summary_status enum and the four states.
- docker-compose.yml: SMTP_HOST=mail.spacemail.com configured;
  GOTRUE_MAILER_AUTOCONFIRM flipped to false (real email confirmation working).
- .nirvana/outputs/.../systems-atelier/: 5 deliverables of the architecture
  audit that produced this roadmap.
2026-05-18 00:44:17 -03:00
guto
19d0678e55 baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00