Commit graph

32 commits

Author SHA1 Message Date
Luiz Gustavo
df05b21dff ux(nav): SiteHeader on every sub-page; BureauNav back to pure breadcrumb
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 37s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 37s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 6s
Two related navbar issues from the audit:
- Sub-pages on desktop only had the breadcrumb pills — readers on a case
  file couldn't get to /sightings or /witnesses without going home first.
- Entity-list pages (/sightings, /witnesses, …) stacked SiteHeader AND
  BureauNav and BOTH carried a locale toggle + auth pill, surfacing every
  control twice.

This pass unifies the chrome:

- BureauNav slimmed: only the ← home pill, the section pill, and the
  breadcrumb trail. Locale toggle, auth pill and mobile hamburger removed
  — SiteHeader owns those.
- SiteHeader added above BureauNav on /bureau, /c/[slug], /h/[id], and
  /jobs/[id]. The four pages that previously skipped it now have the full
  global nav on desktop and a single hamburger on mobile.
- Entity-list pages (which already stacked the two) lose their duplicate
  controls; verified live: 1 toggle + 1 sign-in on /sightings, /bureau,
  /c/d44; 1 hamburger on mobile case files.

No regressions on /documents or the home (both already followed the
SiteHeader-only or stacked-correctly pattern).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 01:41:30 -03:00
Luiz Gustavo
0a4c319a14 ux(search): repaint autocomplete dropdown to magazine palette
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 47s
CI / Scripts — Python smoke (push) Failing after 9s
CI / Web — npm audit (push) Failing after 34s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 6s
The /search input opened a green/cyan terminal-style dropdown leftover from
the dev-tool era — the only piece of the search experience still off-theme
after the page redesign.

- Switch matrix-green (#00ff9c) and cyan (#7fdbff) for gold (#e0c080) and
  cream throughout the dropdown.
- Replace ASCII "" with a Zap lucide icon and reorder the header so the
  Suggestions label leads.
- Localise "documentos" / "trechos" / "autocomplete" via a locale prop
  threaded from SearchPanel; counts pluralise per language.
- Document title now uses the display serif at 15px (was monospace small);
  passage excerpts in #cbd2dd to match the rest of body copy.
- Passage row reorder: page · type · 🛸 · doc-id (was chunk_id first);
  page number gets gold accent so the reader can scan by page.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 01:34:15 -03:00
Luiz Gustavo
4fb3e12e15 ux(bureau): search + agency/decade filters + sort, URL-synced
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 35s
CI / Scripts — Python smoke (push) Failing after 6s
CI / Web — npm audit (push) Failing after 39s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 6s
The audit flagged /bureau as the hardest page to navigate: 80+ case cards
in a flat grid with no way to narrow them down. Now the reader gets a
filterable browser.

- New lib/cases.ts (server) — shared loader for the case library; pulls
  cases off disk, derives the originating agency from the slug prefix
  (DoW, FBI, NASA, CIA, DoE, ODNI, DoD, Other) and the incident year via
  a tight 19xx/20xx regex that requires non-digit boundaries so embedded
  ID numbers don't get mistaken for years (e.g. doc-2070-... no longer
  pretends to be a 2070 incident).
- New CaseLibraryBrowser (client) — search input (matches title + opening
  in either language), agency chips with live counts, decade chips, sort
  dropdown (Most recent / Year ↑ / Year ↓ / A–Z), result count, and a
  clear-filters affordance. All four state pieces sync to the URL so a
  filtered view is bookmarkable.
- /bureau page rewritten: magazine hero ("89 declassified cases,
  narrated.") + dynamic case count + lead, then the browser.
- Cards in the browser carry an eyebrow ("YEAR · AGENCY") so the reader
  sees provenance at a glance.
- CaseLibrary (homepage) refactored to consume the shared loader; behaviour
  on / unchanged.

Verified live: agency=FBI filter narrows to 23 cases; sort=year-desc places
modern cases first; chip counts reflect filtered population.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 01:21:39 -03:00
Luiz Gustavo
067ce7bee6 ux(nav): mobile hamburger + slide-in drawer for narrow viewports
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 1m23s
CI / Scripts — Python smoke (push) Failing after 41s
CI / Web — npm audit (push) Failing after 37s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 5s
The mobile audit caught the primary nav overflowing past the viewport edge
("Programs" was clipped on the right at 390px), and sub-pages on mobile had
no access to the entity hub at all because BureauNav doesn't carry it.

- New MobileNav (client) — hamburger button that opens a right-side slide-in
  panel containing the full primary nav, locale toggle and auth pill. ESC,
  backdrop click, link click, and the close button all dismiss; body scroll
  locks while open; focus jumps to the close button so screen readers
  announce the dialog and keyboard users can dismiss in one keystroke.
- SiteHeader — hide desktop nav <md, show hamburger <md. Brand and trigger
  are the only two elements in the row on mobile, so the overflow stops.
- BureauNav — same treatment: keep the breadcrumb pills, hide the desktop
  right cluster <md, surface the hamburger <md. Means a case-file reader on
  mobile can now jump to /sightings, /witnesses etc without going home
  first.
- MobileNav imports LocaleToggleClient directly (not the server-component
  LocaleToggle wrapper, which would fail Next.js's "client cannot import
  server" rule and break the build).

Verified live at 390×844: drawer opens with all 7 links stacked + toggle +
Sign in; main nav row no longer overflows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 01:09:45 -03:00
Luiz Gustavo
babffda882 ux(search): redesign /search from dev-tool to magazine
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 36s
CI / Scripts — Python smoke (push) Failing after 6s
CI / Web — npm audit (push) Failing after 44s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 13s
The audit's most ostensive gap: /search rendered as a green-on-black
terminal panel with a tech-jargon eyebrow ("HYBRID SEARCH · BM25 + BGE-M3
DENSE + CROSS-ENCODER RERANK"), no global navbar, and a floating locale
toggle disconnected from the rest of the site. Readers landing here saw
a different product.

Page (app/search/page.tsx):
- Render SiteHeader at the top (same primary nav as home; "Search" now
  picks up the gold active-page indicator).
- Magazine hero: eyebrow + serif headline ("Find a passage, a witness, a
  date." / "Encontre uma passagem, uma testemunha, uma data.") + lead.
- generateMetadata() reads getLocale() so the tab title localises too.

Panel (search-panel.tsx):
- Drop matrix-green palette; switch to magazine gold/cream.
- Rounded input with magnifier icon, gold solid submit pill.
- Plain-language placeholder, no developer terms.
- "Advanced filters" toggle: passage-type + exact doc-id behind a
  collapsible — default state is one clean search box.
- Localise passage-type labels (paragraph → parágrafo, heading → título,
  classification_marking → marcação de classificação, etc).
- Result card palette switched to gold/cream + clearer page · type ·
  classification line; raw retrieval score removed from the UI (debug
  noise for users).
- Locale toggle inside the form gone — the navbar carries it now.

Follow-ups noted (not in this commit):
- SearchAutocomplete dropdown still uses the old terminal palette.
- Navbar wraps on narrow mobile widths and the rightmost links overflow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 17:58:41 -03:00
Luiz Gustavo
2ea350e283 ux(nav): unify navbar palette, localize, fix locale-aware metadata
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 36s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 37s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 5s
Visual audit found several issues in the navbar / header layer; this pass
addresses the cluster.

- AuthBar: localize sign in/out + dev-disabled message; switch matrix-green
  (#00ff9c) to gold (#e0c080) to match the magazine palette.
- NavLink (new): client component reading usePathname to mark the active
  section with aria-current="page" and a gold underline+fill, so the user
  knows where they are inside the entity hub.
- Locale toggle: bump font to 12px, widen the pill, lift the active state's
  contrast. Hard reload on switch (window.location.reload) instead of
  router.refresh — that is what makes the <title>, <html lang> and og:locale
  actually update on toggle (router.refresh kept stale layout metadata).
- Root layout: convert static `export const metadata` to generateMetadata()
  so title/description/OG/twitter all read the cookie per request. Drop the
  duplicate floating LocaleToggle at bottom-left; the navbar carries it now.
- BureauNav: drop the cyan-on-home / gold-on-bureau split (palette mismatch
  with SiteHeader); single gold tone; localize "home"/"case files" + aria
  labels; mount AuthBar inside the bar so sub-pages get a sign-in surface.
- Pages using BureauNav: drop the standalone <AuthBar /> below it (now
  duplicated). Affected: /bureau, /c/[slug], /h/[hypothesisId], /jobs/[id].
- Lift nav-link rest text from #9aa6b8 to #cbd2dd for WCAG-AA on near-black.

Verified live: title/lang/og:locale switch with cookie; active link gold
indicator on /sightings; "Entrar"/"Sign in" localizes; no double AuthBar.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 17:50:10 -03:00
Luiz Gustavo
2e59b01f1f fix(i18n): render case files in one language per locale
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 37s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 27s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 5s
Case bodies hold EN and PT-BR scenes interleaved (## I. EN, ## I. PT, ...),
but the reader rendered the whole body, so an EN visitor saw every scene
twice — once in English, once in Portuguese — and vice-versa.

Add pickLocaleBody(): split by section-number occurrence (first = EN, second
= PT-BR, the narrator's mandated order), with language sniffing only as a
fallback for unpaired/unnumbered sections, and full-body fallback when a body
can't be split (English-only legacy files). Wire it into the case reader
(body, lead, OG description) and the homepage card openings.

Also fix extractBody: it discarded the English H1 by treating the second
(PT-BR) H1 as "the first real heading". Future narrations keep both titles.

Verified live: d44 and doc-65 render EN-only under locale=en and PT-only
under locale=pt-br. Bulk-checked 89 files: 80 clean, 8 structurally clean
(source-language verbatim quotes retained per spec), 1 English-only legacy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 11:57:46 -03:00
Luiz Gustavo
c1260cfa68 W6.1: batch-narrate all 122 documents + magazine titles from body H1
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 31s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 36s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 3s
User wants every case in the archive documented in the narrated format.
Queued a case_report for all 122 documents (slug = doc_id, scoped to that
doc's chunks). The LISTEN worker drains 2 at a time (~5h, ~$18 OAuth).
5 reports already existed; ~122 new jobs queued, skipping any already
queued/running/complete by slug.

Title rendering: the batch seeds `topic` from the auto-derived
canonical_title ("Doc 18 100754 General 1946 7 Vol 2"), which is ugly as a
headline. The narrator writes a proper magazine H1 in the body, so:
  - /c/[slug]: extract the two body H1s (EN + PT-BR), use the locale one
    as the headline, strip both leading H1s from the rendered body so the
    title isn't duplicated. Falls back to frontmatter topic.
  - case-library.tsx + featured-case.tsx: same H1 extraction for card
    titles, so /bureau and the homepage show real titles not "Doc 18…".
  - featured-case: pin the flagship (green-fireballs-narrative, which has
    a curated hero painting) to the featured slot instead of most-recent,
    so the batch's random completions don't take over the homepage hero.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 03:54:35 -03:00
Luiz Gustavo
3e88a64933 W5.8: language toggle in the navbar (whole-site PT-BR / EN)
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 1m16s
CI / Scripts — Python smoke (push) Failing after 36s
CI / Web — npm audit (push) Failing after 28s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
User: "quero que o usuário possa escolher o idioma na navbar, e então leia
o site todo em seu idioma."

The locale infrastructure already existed (cookie + getLocale() + every
page renders bilingual). The toggle only lived as a subtle floating
control bottom-left. This surfaces it in the navbars where users expect
it, and confirms whole-site language switching.

  locale-toggle-client.tsx
    - New variant="navbar": gold-amber pill matching the magazine theme
      (PT | EN, active = solid gold). Retro green variant kept for the
      floating fallback.
    - Guard: clicking the already-active language is a no-op.
    - aria-pressed on each button for a11y.

  locale-toggle.tsx: LocaleToggle gains variant passthrough.

  site-header.tsx: <LocaleToggle variant="navbar"> added to the main nav
    (homepage + /sightings /witnesses /objects /locations /operations
    /documents).

  bureau-nav.tsx: optional `locale` prop renders the same toggle pushed
    to the right (ml-auto) on /bureau, /c/[slug], /h/[id], /jobs/[id].
    All four callers now pass locale (getLocale added to h + jobs pages).

Cookie-based: switching sets `locale` cookie + router.refresh(); the
server re-renders every component through getLocale(), so the entire
page — headers, hero, summaries, case narratives, entity cards — flips
language in one click. The floating bottom-left toggle stays as a global
fallback for pages without a navbar (/d, /e, /search, /graph).

Also this session: Sun-Tzu smoke succeeded on the refreshed
CLAUDE_CODE_OAUTH_TOKEN — brief B-0001 written for the green fireballs
("bolas de fogo verdes confinadas ao corredor nuclear... cor espectral
e trajetória quase horizontal que os próprios analistas consideraram
irreconciliáveis com meteoros naturais", 3 pillars). Bulk entity
enrichment resumed in the background.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 22:54:37 -03:00
Luiz Gustavo
df82d40a96 W5.7 (Phase 3E): perf + a11y + OG images
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 36s
CI / Scripts — Python smoke (push) Failing after 6s
CI / Web — npm audit (push) Failing after 42s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 5s
A11y:
  - Skip-link in <body> (focus-visible only) that jumps to #main.
    Bilingual ("Skip to content" / "Pular para o conteúdo").
  - <main id="main"> landmark wrapping the homepage body.
  - prefers-reduced-motion media query disables the hover-scale + image
    transitions for users with vestibular sensitivity.
  - Skip-link styled with high contrast (gold-on-dark) + outline on
    keyboard focus.

Performance:
  - HeroBanner background image: fetchPriority="high" + explicit
    width/height (1600x900) for zero CLS.
  - FeaturedCase image: fetchPriority="high" + 1280x720 to prevent
    layout shift while the 2.7MB painting loads.
  - IconicCases tiles already have loading="lazy".
  - prose blockquote: overflow-wrap:anywhere so verbatim quotes don't
    bust the mobile viewport.

Open Graph:
  - app/layout.tsx default OG image set to the green-fireballs
    painting (any page without its own image card inherits this).
  - app/c/[slug] OG image is the case's editorial illustration when
    one exists. WhatsApp, Twitter, Telegram, Slack, ChatGPT search
    all pull this when the link is shared. 2000x1125 for the
    "summary_large_image" twitter card.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 16:55:35 -03:00
Luiz Gustavo
1687120b7f W5.6 (Phase 3D): Iconic Cases — curated rail on homepage
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 36s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Failing after 42s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 6s
8 hand-picked UFO incidents that any enthusiast recognises:
  - Kenneth Arnold 1947 (the genesis — "flying saucers" coined)
  - Roswell 1947 (the original Army press release)
  - Maury Island 1947 (Puget Sound, slag drop, FBI plane crash)
  - Mantell 1948 (first known UFO casualty)
  - Rendlesham Forest 1980 (USAF security police, deputy commander tape)
  - Phoenix Lights 1997 (V-formation across Arizona, governor's reversal)
  - Nimitz Tic-Tac 2004 (USS Nimitz F/A-18, gun-cam released 2017)
  - Green Fireballs Sandia 1948 (copper salts, nuclear-site overflights)

Each case has:
  - Bilingual hand-written editorial blurb (PT-BR + EN, no LLM)
  - Painterly editorial hero illustration at 2K
  - Year + tag chips (military / civilian / modern / early / naval /
    aviation / mass-sighting)
  - Link to /e/events/<id> when indexed in corpus, /search?q=... when not,
    /c/<slug> for the green-fireballs narrative case

Components:
  lib/iconic-cases.ts — IconicCase type + ICONIC_CASES editorial list
  components/iconic-cases.tsx — magazine grid: first two as wide hero pair,
    rest as 3-up tiles below. Hover scale on images, gradient overlays,
    aspect-16/11 hero + 16/10 compact, lazy-loaded.
  app/page.tsx — inserted between <FeaturedCase /> and <PortalGrid />

4 new hero illustrations generated this session:
  - iconic-nimitz-tic-tac-2004.png       (Nano Banana Pro)
  - iconic-phoenix-lights-1997.png       (Nano Banana Pro)
  - iconic-rendlesham-forest-1980.png    (gpt-image-1.5 via Codex)
  - iconic-mantell-1948.png              (Nano Banana Pro)
Per user direction: mixed generators (Nano Banana primary, Codex
co-pilot) so the homepage has stylistic variety while keeping the
painterly editorial register.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 16:50:52 -03:00
Luiz Gustavo
8283237f87 W5.4 followup: hero illustration on /c/[slug] + sitemap fix
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 34s
CI / Scripts — Python smoke (push) Failing after 6s
CI / Web — npm audit (push) Failing after 40s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Hero illustration:
  - Painterly 16:9 editorial illustration generated via Nano Banana Pro
    for the featured case (green-fireballs-narrative): late-1940s
    desert night, vivid emerald fireball over silhouetted Sandia mesas,
    1948-era state-police sedan parked on US 66 shoulder with an
    officer in period uniform looking up, faint green glow on his
    face. Sandia Base 5 Miles roadsign. New Yorker-cover painterly
    register, NOT photorealistic, NOT sci-fi.
  - Stored at /data/disclosure/processing/case-art/<slug>.png, served
    through the existing /api/static/processing/ route. 2.7MB at 2K.
  - components/featured-case.tsx: prefers the illustration over the
    declassified-page thumbnail when present. Tags it "Editorial
    illustration" / "Ilustração editorial" so the reader knows it's
    not a photograph.
  - app/c/[slug]/page.tsx: full-bleed editorial hero at the top of
    the article when an illustration exists for the slug. Title sits
    on the image with gradient overlay; "Ilustração editorial" chip
    in the top-right corner labels the art honestly. When no
    illustration exists the page falls back to the plain title header.

Sitemap fix:
  - Added export const dynamic = "force-dynamic" + revalidate = 3600
    to app/sitemap.ts. Without these Next.js statically generated the
    sitemap at build time, when the DB and case-files volume were
    unreachable from the build container — which is why production
    was serving only the 9 static URLs instead of ~3000.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 16:16:20 -03:00
Luiz Gustavo
f2b7b116ce W5.3 (Phase 3A): entity summaries — sub-pages get magazine-grade prose
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 45s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Failing after 41s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 3s
Today /sightings, /witnesses, /objects, /locations and /operations show
a name + mention count and nothing else. After this each row carries a
60-100 word bilingual narrative summary written from the chunks where
the entity actually appears.

Migration 0008 (apply as supabase_admin):
  public.entities  +summary_en TEXT
                   +summary_pt_br TEXT
                   +summary_generated_at TIMESTAMPTZ
                   +summary_model TEXT
                   +summary_status TEXT
                     CHECK ('pending'|'ai_generated'|'curated'|'refused')
  + index on summary_status
  + GRANT UPDATE (summary_*) ON entities TO investigator
  + new policy entities_investigator_update_summary (RLS UPDATE for
    investigator role)

Enrichment script (investigator-runtime/scripts/enrich_entity_summaries.ts):
  - Per-class config (chunk_k, min_mentions, max_per_class)
  - Path A: entity_mentions JOIN chunks (high-precision linker)
  - Path B (fallback): hybridSearch on canonical_name + aliases when
    entity_mentions returns zero. This is what unlocked Kenneth Arnold
    and similar entities — their wiki YAML has high total_mentions
    counted from frontmatter mentioned_in[], but the entity_mentions
    extractor was silent because the matches came from the wiki text,
    not the OCR chunks.
  - Sonnet 4.6 via OAuth Max, ~$0.04 per entity, ~$10 for the full
    260-entity bulk run.
  - INSUFFICIENT skip when chunks can't sustain a 60-word summary —
    refused entries get summary_status='refused' so they're not retried.

UI uplift:
  - lib/retrieval/entity-pages.ts: getEntityCore now prefers the DB
    summary (ai_generated or curated) over wiki YAML narrative.
  - components/entity-list-page.tsx:
    * SELECT now pulls summary_en, summary_pt_br, summary_status
    * Sorted with summary-enriched rows first (so the magazine grid
      lands on quality content immediately)
    * MagazineGrid: 4-line summary preview replaces aliases line
    * CompactGrid: enriched rows render as full editorial cards,
      bare rows fall back to a compact table below

Smoke results:
  - Kenneth Arnold sighting: "On June 24, 1947, pilot Kenneth Arnold
    reported sighting unidentified objects over the Pacific Northwest,
    and the account spread worldwide. It set off a run of similar
    reports: County Commissioner Crankes saw comparable objects after
    Arnold's account reached the press, and United Airlines pilot
    Emil H. Smith spotted flying discs on July 4 during a routine
    flight out of Boise, Idaho..."
  - Roswell Incident: includes Colonel Corso's 1997 book + the 1995
    GAO finding that radio messages from Oct 46–Feb 47 were destroyed
    + Senator Strom Thurmond's foreword. Real magazine-grade content.

Background bulk run kicked off across all 5 classes (event,
uap_object, person, location, organization) — populating live as
the homepage rebuilds.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 15:37:01 -03:00
Luiz Gustavo
b6fc9dc84e W5.1 hotfix: page PNGs are named p-NNN.png not pNNN.png
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 42s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 34s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 5s
2026-05-24 14:14:53 -03:00
Luiz Gustavo
ab4fe2a334 W5.1: enthusiast pivot — strip detective surfacing, magazine homepage
Some checks failed
CI / Retrieval — golden set (Recall@5 + MRR) (push) Waiting to run
CI / Web — typecheck + lint + build (push) Failing after 39s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Has been cancelled
User explicit: "1 bilhão de entusiastas pelo mundo ovni" — site is for the
UFO-curious public, not for skeptics. The 8-detective scaffolding becomes
invisible plumbing; the reader sees stories about what was observed.

Reader-facing changes:

  New homepage (web/app/page.tsx)
    - SiteHeader: magazine-style top nav (no detective tiles)
    - HeroBanner: full-bleed editorial opener with declassified-page art
      background, display-serif headline, live stats row (122 docs,
      2047 events, 1861 witnesses, 867 craft catalogued)
    - FeaturedCase: cover-story treatment of the most recent case_report,
      uses a real document page as hero image, links to /c/[slug]
    - PortalGrid: 6 thematic doorways into the archive — Sightings,
      Witnesses, Craft, Hot spots, Programs, Documents — each tile shows
      a real entity count and short editorial blurb
    - GreatestHits: top 9 most-cited events from the corpus
      (Kenneth Arnold 1947, Mantell 1948, …) as a magazine grid
    - Doc list kept but reframed as "the primary record"

  New sub-pages (5)
    - /sightings → events (2047), magazine grid
    - /witnesses → people (1861), compact table
    - /objects   → uap_objects (867), magazine grid
    - /locations → locations (1757), compact table
    - /operations → organizations (1596), compact table
    - /documents → full doc list with thumbnails (mirrors homepage section
      for direct deep-link)
    All share <EntityListPage> shell with per-page i18n + JSON-LD ItemList

  Stripped detective surfacing
    - /jobs/[id]: "Sherlock Holmes / Dr. Watson" → "Investigation in progress"
    - chat-bubble: detective-named card → neutral "Investigação em andamento"
    - quick-launch: 7-kind detective dropdown → single "investigar um caso"
      input (kind=case_report hardcoded)
    - /bureau: rewritten as the case-file library (no artefact dumps)

Typography + design
  - Fraunces variable serif loaded for display headings
    (`.font-display` class)
  - Gold-amber accent (#e0c080) unified as the brand colour
  - Asymmetric magazine grids (1+2+3 column, generous whitespace)
  - Hover micro-interactions (image scale on featured case, translateX
    on portal arrows)

SEO + GEO
  - layout.tsx metadataBase + title.template + per-route Metadata exports
  - Organization JSON-LD on root layout
  - WebSite + SearchAction JSON-LD on homepage
  - CollectionPage + ItemList JSON-LD on every entity list page
  - openGraph + twitter cards, pt-BR primary + en-US alternate
  - ai:purpose meta tag for Generative Engine Optimization — declares
    the site as a citation-linked primary-source archive
  - robots: index + follow with large image preview

The detectives themselves remain alive in the backend (runtime, DB, audit
log), but the reader never sees "Holmes / Sun-Tzu / Watson" in the UI. The
next phase will reorient case-writer to write as a single best-seller voice
synthesising all the internal sources.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 14:09:46 -03:00
Luiz Gustavo
24f12a27f4 W4.1+W4.2: anti-AI-tics house style + bureau nav (back/home everywhere)
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 31s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Failing after 31s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Two complaints in one wave:

(W4.1) User: "Não pode ter vícios de IA como uso excessivo de '-' que a IA
coloca geralmente no lugar de vírgulas por exemplo. Isso deve fazer parte
do prompt geral."

  - New prompts/_house-style.md banning the 9 most common AI prose tells
    in both EN and PT-BR:
      1. Em dashes as comma replacements (—)
      2. Rule-of-three lists ("concrete, rigorous, and grounded")
      3. Conjunctive openers ("Moreover", "Notably", "Ademais")
      4. Superficial -ing analyses ("marking a shift", "destacando")
      5. Inflated symbolism + AI vocab (tapestry, navigate, delve,
         underscore, robust, multifaceted, marco histórico, ...)
      6. Negative parallelisms ("Not just X but Y")
      7. Vague attribution ("Some scholars say...")
      8. Summary closers ("In summary...", "Em suma...")
      9. Hedging fluff ("It's important to note...")
    Verbatim chunk quotes are explicitly exempt; preserve as-is.
  - claude.ts callClaude() lazily loads _house-style.md once per process
    and PREPENDS it to every detective's system prompt:
        composedSystem = houseStyle + "---" + detective.systemPrompt
    This means all 7 detectives + future ones get the rules without any
    per-prompt change.

(W4.2) User: "Quando entra em uma página da investigação não tem como
voltar! UX terrível!"

  - New <BureauNav> sticky topbar with explicit "← home" + "🔎 bureau"
    buttons + clickable breadcrumb trail. Always visible at the top of
    every bureau page so the user can escape in one click.
  - Wired into /bureau, /h/[hypothesisId], /c/[slug], /jobs/[id]. Each
    page passes its sensible parent crumb (/bureau#hypotheses,
    /bureau#reports, /bureau#jobs).
  - Replaces the previous plain-text "disclosure.top / hypothesis /
    H-0004" line which had no visual affordance.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 13:27:58 -03:00
Luiz Gustavo
7826710051 W4: bilingual EN + PT-BR Investigation Bureau (CLAUDE.md §3 contract)
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 41s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Failing after 26s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
User flagged that the bureau was emitting English-only output, violating
the project's bilingual rule. Every narrative field now ships in both
languages: stored in sibling DB columns + rendered as adjacent markdown
sections per CLAUDE.md §3.

Migration 0007 (apply as supabase_admin):
  - public.hypotheses    +question_pt_br, +position_pt_br,
                         +argument_for_pt_br, +argument_against_pt_br
  - public.contradictions +topic_pt_br, +notes_pt_br
  - public.witnesses     +access_to_event_pt_br, +bias_notes_pt_br,
                         +verdict_pt_br
  - public.gaps          +description_pt_br, +suggested_next_move_pt_br
  - public.evidence: unchanged (verbatim_excerpt stays source-language)
  - JSONB siblings inside contradictions.chunks + gaps.scope handled at
    runtime (statement_pt_br, title_pt_br, dominant_model_pt_br,
    why_surprising_pt_br, what_it_implies_pt_br).

Detective prompts (all 7) rewritten with explicit bilingual JSON contract:
  - Output protocol section names every EN field + its _pt_br sibling
  - "Bilingual is mandatory" warning in the task instruction
  - Sentinel skip-states unchanged (NO_HYPOTHESES, NO_CONTRADICTIONS,
    INSUFFICIENT_TESTIMONY, INSUFFICIENT_HYPOTHESIS, NO_OUTLIERS,
    NO_NEW_EVIDENCE, INSUFFICIENT_ARTEFACTS)
  - Schneier: parallel arrays — hidden_assumptions[i] matches
    hidden_assumptions_pt_br[i], lengths must match
  - Case-Writer: interleaved §1 (EN) / §1 (PT-BR) per act in the body

Writer-side validation (all 7 tools):
  - Reject INSERT if PT-BR sibling missing when EN field is set
  - Persist both languages atomically in one INSERT (no half-updates)
  - Markdown renderers write adjacent EN+PT-BR sections in case files
    (## Argument for (EN) followed by ## Argumento a favor (PT-BR), etc.)

Detective parse layer (all 7 detectives):
  - Coerce both keys from JSON output
  - "incomplete_bilingual_*" skip reason when either side missing
  - Defensive: PT-BR fields trimmed + length-capped same as EN

Orchestrator propagates question_pt_br + topic_pt_br through job payload
to runHolmes / runCaseWriter, mirroring the chat-tool entry point.

Web (UI):
  - /api/jobs/[id] hydrates _pt_br siblings from pg
  - job-status-poller HypothesisCard: PT-BR primary, EN in <details>
    fallback when both exist
  - ContradictionCard: PT-BR statement primary + secondary EN quote
  - WitnessCard: PT-BR verdict primary + secondary EN quote, panels in PT
  - GapCard: PT-BR title/why/implies primary
  - /bureau hub: SELECTs both columns, renders PT-BR primary
  - /h/[id]: ArgumentPanel renders PT-BR primary with collapsible EN
    fallback when both exist
  - BureauSnapshot homepage: position_pt_br / topic_pt_br / verdict_pt_br
    primary
  - DocBureauPanel /d/[doc]: same primary-PT-BR pattern
  - New web/lib/i18n/pick.ts helper (unused yet by chat/agents — kept
    for future locale-driven switching when both languages are equally
    full; current rule is PT-BR-first since the user is brasileiro)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:02:59 -03:00
Luiz Gustavo
d4a2e4f51e W3.10: clickable detective tiles + quick-launch form + doc bureau panel
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 37s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 40s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Builds on top of W3.9 to turn the homepage Bureau from a read-only
dashboard into a working command center.

UI improvements (web/components/bureau-snapshot.tsx):
  - Detective tiles are now <Link>s — each navigates to its primary
    artefact section in /bureau (Holmes→#hypotheses, Locard→#evidence,
    Dupin→#contradictions, Schneier→#hypotheses, Poirot→#witnesses,
    Taleb→#outliers, Tetlock→#hypotheses, Case-Writer→#reports). Hover
    bg matches the detective's tone color.
  - <QuickLaunch /> form inserted right under the tiles.

New <QuickLaunch /> client component:
  - Detective dropdown (7 active kinds; evidence_chain not yet exposed
    here since it needs a doc_id better picked from the doc page).
  - Single input swaps placeholder + aria-label by kind: question for
    Holmes, topic for Dupin/Taleb/Case-Writer, hypothesis_id for
    Schneier/Tetlock, person_id for Poirot.
  - Submits to POST /api/bureau/launch and redirects to /jobs/[id]
    via the next.js router.
  - Loading state ("queueing…") + error display inline.

POST /api/bureau/launch (web/app/api/bureau/launch/route.ts):
  - Same 8-kind validator as the chat tool's request_investigation.
  - Auth required when Supabase is configured (triggered_by = user:email).
  - Returns { job_id, kind, detective, status_url, eta_seconds }.

DocBureauPanel on /d/[docId] (web/components/doc-bureau-panel.tsx):
  - Server component inserted between the doc header and
    AnomalyHighlights.
  - Surfaces every bureau artefact that touches the doc:
    · Evidence whose source_page_id starts with docId/p
    · Hypotheses citing any of those evidence_ids
    · Contradictions whose chunks[] has any item with this doc_id
    · Gaps/outliers with scope.doc_id == docId
    · Case reports whose markdown body references docId (filesystem scan)
  - Empty state shows "Investigation Bureau — untouched" with a CTA
    linking back to the homepage to launch the first investigation.
  - When non-empty, header counts total artefacts + links to /bureau
    for the full view.

Metadata (web/app/layout.tsx):
  - description rewritten from "Investigative wiki of the US Department
    of War UAP/UFO archive (war.gov/ufo)" to one that names the bureau
    + the 8 detectives. Affects SERP previews + social-card defaults.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 23:33:00 -03:00
Luiz Gustavo
67185ff518 W3.9: surface the Investigation Bureau on the homepage + /bureau hub
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 40s
CI / Scripts — Python smoke (push) Failing after 3s
CI / Web — npm audit (push) Failing after 31s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Closes a UX gap the user surfaced: W3.5-3.8 built 8 detectives, 4 new
URL endpoints (/jobs/[id], /h/[id], /c/[slug], /api/h/[id]/red-team)
and a chat tool, but the homepage was unchanged — the bureau was
invisible unless you knew the URL or asked the chat to invoke
request_investigation.

Homepage (web/app/page.tsx):
  - Title `▍ war.gov/ufo — Investigative Wiki` → `▍ The Disclosure Bureau`
  - Subtitle expanded from "Holmes · Poirot · Dupin · Locard" to all 8
    detectives (Holmes · Locard · Dupin · Schneier · Poirot · Taleb ·
    Tetlock · Case-Writer)
  - New `🔎 bureau` topbar link (gold, between graph/stats and batch)
  - BureauSnapshot inserted right after the header

BureauSnapshot (web/components/bureau-snapshot.tsx) — server component:
  - 8 detective tiles with role labels (each in its tone color)
  - 6 clickable counters (evidence / hypotheses / contradictions /
    witnesses / outliers / case reports) — anchor to /bureau#section
  - 6 "recent artefacts" columns surfacing the last 3-4 of each kind:
    hypotheses with prior→posterior + band + ↳reviewed_by marker,
    contradictions with topic + resolution_status, evidence with
    Grade badge + verbatim quote, outliers with title + scope.kind,
    witness analyses with canonical_name + credibility + verdict,
    case reports with slug + link to /c/<slug>
  - "Recent jobs" strip linking to /jobs/[id] color-coded by status
  - Reports read from /data/ufo/case/reports/ via fs.readdir + stat,
    sorted by mtime — no DB round-trip needed for that section

/bureau (web/app/bureau/page.tsx) — full hub:
  - Header with full counts
  - 7 sections (anchored to homepage counter links): Case reports,
    Hypotheses, Evidence, Contradictions, Outliers, Witnesses,
    Recent jobs table — each rendering up to 100 rows
  - Reports section parses frontmatter from each .md to surface topic
    + n_hypotheses + n_evidence on the card

Runtime fixes batched in:
  - Poirot: coerce entity_pk via Number() — node-postgres returns
    BIGINT as string by default; writer's Number.isFinite() rejected
    it as "person_entity_pk required" (j-edgar-hoover retry path)
  - Tetlock: write_calibration rationale cap 600 → 1200 chars. Prompt
    still asks ≤ 600 but a 2× slack beats failing the job on honest
    analysis. Observed live: Tetlock emitted ~620 chars on H-0003 and
    the writer rejected the entire calibration.
  - Case-Writer: Promise.all of 5 queries × max_parallel=2 jobs
    demanded up to 10 connections against the investigator role's
    rolconnlimit=4 → "too many connections for role investigator".
    Sequentialized — the LLM call is the hot path, not these queries.

Smoke results visible now on the homepage:
  - 3 hypotheses (H-0001/2/3) about green fireballs origin
  - 3 contradictions (R-0001/2/3) about color, geographic confinement,
    exclusive-green vs multicolored
  - 2 evidence cards (E-0002/3) Grade B
  - 3 outliers (G-0001/2/3) — including Taleb's deliberate
    meteor-shower-camouflage flag
  - 1 case report at /c/green-fireballs-sandia (Watson 13.4 KB,
    five-act narrative, fully cited)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 22:41:28 -03:00
Luiz Gustavo
dd75a67964 W3.8: Investigation Bureau complete — Poirot, Taleb, Tetlock, Case-Writer
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 45s
CI / Scripts — Python smoke (push) Failing after 5s
CI / Web — npm audit (push) Failing after 40s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 3s
Brings the bureau from 4 → 8 detectives. All eight run as Bun + claude-CLI
subprocesses against the same Supabase + investigation_jobs LISTEN/NOTIFY
queue, sharing search.ts hybridSearch and writer-side validators that
gate writes against schema + FK.

New detectives:

  Poirot (witness_analysis)
    - prompts/poirot.md — credibility / access / bias / corroboration /
      verdict; uses entity_mentions JOIN chunks to pull 12 chunks per
      person; resolves corroboration_refs chunk_ids defensively (accepts
      bare cNNNN even when the model emits pNNN/cNNNN).
    - INSERT into public.witnesses with W-NNNN naming.
    - Tone: purple (#9b5de5).

  Taleb (outlier_scan)
    - prompts/taleb.md — "surprise is relative to a model"; at most 3
      outliers; each requires explicit dominant_model + why_surprising +
      what_it_implies; fan-out into public.gaps with scope.kind="outlier".
    - Same unscoped-fallback as Dupin (Pass 1 with doc_id, Pass 2 widens
      to corpus if hits < 3).
    - Tone: yellow (#ffd23f).

  Tetlock (calibrate_hypothesis)
    - prompts/tetlock.md — honest Bayesian update; emits new_posterior +
      Δ + recommended_action ∈ {keep, downgrade, upgrade, supersede}.
    - write_calibration UPDATEs public.hypotheses + APPENDS a
      "## Calibration history" section to the H-NNNN.md case file
      (calibration is append-only — each datapoint matters). Posterior
      band auto-corrected to match Tetlock thresholds.
    - NO_NEW_EVIDENCE sentinel handled; pure 'keep' with |Δ|<0.005 only
      touches updated_at + reviewed_by.
    - Tone: teal (#26d4cc).

  Case-Writer (case_report)
    - prompts/case-writer.md — Dr. Watson assembles all artefacts
      (E-NNNN, H-NNNN, R-NNNN, W-NNNN, G-NNNN) into a five-act narrative.
      ILIKE filter on topic; doc_id optional scope.
    - Larger budget cap (≥ $0.50) + longer timeout for prose generation.
    - Writes case/reports/<slug>.md with frontmatter (topic + counts);
      no DB table for v0.
    - New page /c/[slug] renders the report via MarkdownBody + stat chips.
    - Tone: gold (#e0c080).

Hardening across the bureau:
  - Sentinel parsing now accepts backticked AND prose-trailing forms
    (Holmes NO_HYPOTHESES, Dupin NO_CONTRADICTIONS, Schneier
    INSUFFICIENT_HYPOTHESIS, Poirot INSUFFICIENT_TESTIMONY, Taleb
    NO_OUTLIERS, Tetlock NO_NEW_EVIDENCE, Case-Writer
    INSUFFICIENT_ARTEFACTS). Avoids the failure mode where the model
    refuses honestly but the runtime treated it as a parse error
    (observed live with Poirot+Hoover identifying the DIRECTOR
    false-positive disambiguation issue in entity_mentions).

Chat tool extensions (web/lib/chat/tools.ts):
  - request_investigation now accepts 7 kinds. Each routes to its
    detective with appropriate validation (hypothesis_id regex,
    person_id kebab-case, topic non-empty, doc_id for evidence_chain).
  - ETA per kind: Holmes/Dupin 60s, Poirot 45s, Schneier/Tetlock 30s,
    Taleb 50s, Case-Writer 180s (longer prose), Locard 30×n_chunks.

UI integration:
  - chat-bubble inline card paints each detective in its tone color.
  - /jobs/[id] page header swaps name/subtitle/tone per detective;
    question label adapts ("Topic" / "Hypothesis under attack" /
    "Witness under analysis" / "Topic to outlier-scan" / "Hypothesis
    under recalibration" / "Case to assemble").
  - job-status-poller renders: case-report link card (gold), outlier
    cards (yellow), witness cards (purple) — alongside existing
    hypothesis, evidence, contradiction cards.
  - /api/jobs/[id] hydrates witnesses (JOIN entities for canonical_name)
    + gaps (with scope JSONB).
  - /c/[slug] page reads /data/ufo/case/reports/<slug>.md and renders
    with MarkdownBody, frontmatter parsed for stat chips.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 22:11:39 -03:00
Luiz Gustavo
857dd771d2 W3.8: Schneier red-team detective + /h/[hypothesisId] dossier page
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 33s
CI / Scripts — Python smoke (push) Failing after 7s
CI / Web — npm audit (push) Failing after 38s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Adds the fourth AI detective in the Investigation Bureau runtime: Bruce
Schneier, who attacks an existing hypothesis as a red-team operator.

Runtime:
  - prompts/schneier.md — discipline (don't disprove, just attack;
    structured output with hidden_assumptions, failure_modes,
    alternative_explanations, recommended_tests, verdict_one_sentence;
    severity ∈ {low, medium, high}; emit INSUFFICIENT_HYPOTHESIS when
    the input is too thin)
  - src/detectives/schneier.ts — reads the hypothesis row + evidence
    chain (joined via evidence_refs FK), feeds Claude with the
    arguments + verbatim quotes, parses strict JSON object
  - src/tools/write_red_team_review.ts — UPDATEs hypotheses.reviewed_by
    + updated_at; APPENDS (or replaces if re-reviewed) a structured
    "## Red-team review (Schneier · X severity)" section to
    case/hypotheses/H-NNNN.md. Caps each list at 5 entries × 240 chars,
    validates verdict ≤ 280 chars.
  - orchestrator: new `red_team_review` kind dispatching to runSchneier

Chat + UI:
  - request_investigation gains kind=red_team_review + hypothesis_id arg
    (validated against H-NNNN regex); detective auto-resolves to schneier
  - chat-bubble inline card paints Schneier in red (#ff3344)
  - /jobs/[id] page swaps title/subtitle/tone per detective; the
    "Question" label becomes "Hypothesis under attack" for red_team_review

New /h/[hypothesisId] page (hypothesis dossier):
  - Server-rendered from public.hypotheses + public.evidence (joined
    via evidence_refs FK + chunk lookup)
  - Header: ID + creator + reviewer (highlighted when Schneier has
    visited), position as headline, question subtitle, Tetlock band
  - Prior + posterior bars with Δ-delta indicator
  - Argument grid: argument_for (green) vs argument_against (pink)
    side-by-side with [[wiki-link]] auto-linking to source chunks
  - Evidence chain: each E-NNNN with Grade A/B/C badge, verbatim
    blockquote, link to source page
  - Red-team review panel: parses the markdown section in the case
    file (severity badge, verdict, 4 bullet panels for
    hidden_assumptions / failure_modes / alternative_explanations /
    recommended_tests). Empty state when not yet reviewed.

RedTeamRequestButton client component + POST /api/h/[id]/red-team —
authenticated user can trigger Schneier in one click; UI swaps to
"acompanhar" link to /jobs/[id] once queued.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 21:48:12 -03:00
Luiz Gustavo
5ac53cb3e2 W3.7: Dupin contradiction-scan detective + UI integration
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 39s
CI / Scripts — Python smoke (push) Failing after 4s
CI / Web — npm audit (push) Failing after 37s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 4s
Adds the third AI detective in the Investigation Bureau runtime: C. Auguste
Dupin, who scans a corpus shortlist for pairs (or small groups) of chunks
that cannot both be true under any ordinary reading.

Runtime:
  - prompts/dupin.md — discipline (no contradiction without ≥2 distinct
    chunk_ids; reject same-vocabulary near-misses; FEW high-confidence
    over MANY weak ones; emit `NO_CONTRADICTIONS` when corpus is silent)
  - src/detectives/dupin.ts — hybridSearch with k=18 (more chunks than
    Holmes because contradictions emerge from comparing dispersed
    claims), strict JSON-array parsing, AT MOST 3 contradictions per call
  - src/tools/write_contradiction.ts — validates topic + ≥2 positions
    drawn from ≥2 distinct chunks, resolves chunk_pk via DB lookup
    (rejects positions citing unknown chunks), INSERTs into
    public.contradictions + writes case/contradictions/R-NNNN.md
  - orchestrator: new `contradiction_scan` kind dispatching to runDupin;
    payload { topic, doc_id?, lang?, context_chunks? }

Chat + UI:
  - request_investigation gains kind=contradiction_scan + topic arg;
    triggered detective auto-resolves to dupin
  - chat-bubble inline card renders dupin in orange (#ff8a4d) to
    distinguish from holmes (cyan) and locard (green)
  - /jobs/[id] page swaps title + subtitle + tone per detective;
    "Question" label becomes "Topic" for contradiction_scan
  - /api/jobs/[id] hydrates public.contradictions when outputs[] surfaces
    contradiction_ids
  - job-status-poller renders ContradictionCard: topic + N positions
    (verbatim statements quoted, stance label optional, link to source
    chunk) + optional notes panel, with resolution_status badge
    (open/resolved/irreconcilable)

R-NNNN shares the contradiction_id_seq slot with relation per
CLAUDE.md naming — same conceptual class (a connection between two
pieces of evidence in tension).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 21:34:04 -03:00
Luiz Gustavo
b76e81e4b3 W3.6: chat request_investigation tool + /jobs/[id] case-file viewer
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 44s
CI / Scripts — Python smoke (push) Failing after 3s
CI / Web — npm audit (push) Failing after 43s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 7s
Closes the loop between the chat UI and the Investigation Bureau runtime.

Chat tool (web/lib/chat/tools.ts):
  - request_investigation { kind, question, doc_id?, chunks?, claim? }
    INSERTs a row in public.investigation_jobs and returns
    { job_id, kind, status, eta_seconds, status_url, detective }.
  - kind=hypothesis_tournament → Holmes (1 question → 2-3 rival hypotheses)
  - kind=evidence_chain → Locard (1 doc → grade-A/B/C evidence with chain
    of custody, default top-5 anomaly chunks)
  - Plumbed user.email through ToolHandlerContext so triggered_by audits
    the requesting user.

Public job viewer:
  - GET /api/jobs/[id] joins investigation_jobs → public.evidence +
    public.hypotheses for the IDs surfaced in outputs[]. Returns one
    payload the page can render without n+1 round-trips. Strips
    triggered_by from the response (it carries the user's email).
  - app/jobs/[id]/page.tsx server-renders the case-file shell:
    detective lore header (Holmes blue or Locard green), question chip,
    scope chip with link back to the document.
  - components/job-status-poller.tsx client island that polls every 3 s
    while non-terminal, then once on terminal to hydrate evidence +
    hypotheses. Renders:
      · Phase tracker (queued → running → complete | failed)
      · Hypothesis cards w/ prior + posterior bars + Δ delta indicator
        + Tetlock band badge (high/medium/low/speculation)
      · Argument-for / argument-against with [[wiki-link]] auto-linking
        to /d/<doc>/p<NNN>#<cNNNN>
      · Evidence cards w/ Grade A/B/C badge + verbatim blockquote +
        bbox crop preview via /api/crop + custody-steps disclosure
      · Empty/in-flight panel ("os detetives estão lendo o corpus")
      · Failure panel surfacing error + partial outputs

Inline chat-bubble card (components/chat-bubble.tsx):
  - ToolTrace.richRender recognises request_investigation results and
    renders a detective banner with status + ETA + link to /jobs/[id]
    (target=_blank). Error case renders a red strip with the message.

UX flow now: user asks Sherlock a question → request_investigation
queues the job → chat card shows "🔎 Holmes · hypothesis_tournament ·
ETA ~60s" → user clicks → /jobs/<id> live-updates → 60 s later, 2-3
rival hypotheses + their arguments + chunk citations are rendered with
Bayesian update visible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 21:26:18 -03:00
Luiz Gustavo
eaf282c535 W2: rerank opt-in, analyze_image_region tool, RAG eval, graph cleanup, ADRs
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 40s
CI / Scripts — Python smoke (push) Failing after 3s
CI / Web — npm audit (push) Failing after 29s
CI / Retrieval — golden set (Recall@5 + MRR) (push) Failing after 3s
- TD#8 hybrid.ts: rerank_strategy {always|when_top_k_gt|never} + threshold
  (default skips rerank for top_k ≤ 15; chat tool uses threshold 10)
- O11 vision.ts + tools.ts: analyze_image_region tool — sharp-crops the
  bbox, claude CLI reads the temp PNG via Read tool, Sonnet vision answers
- TD#12 /graph: SigmaGraph replaces ForceGraphCanvas; react-force-graph-2d
  uninstalled (-37 transitive deps); force-graph-canvas.tsx deleted
- TD#27 messages/route.ts gatherContext slice sizes via CTX_* env vars
- TD#22 tests/rag/: golden.yaml (15 queries) + run.py (Recall@k + MRR +
  negative-pass rate) + baseline.json + CI job in .forgejo/workflows/ci.yml
- docs/adrs/: ADR-001..005 published from systems-atelier deliverables

Verified live on disclosure.top: top_k=5 path skips rerank (6.7s embed-only,
was 12-15s with rerank); rerank=always still available on demand.
First RAG baseline: Recall@5 = 0.2083, MRR = 0.25, Negative pass = 1.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 19:20:09 -03:00
Luiz Gustavo
55cac8a395 W0+W1+W1.2: security hardening, observability, autocomplete, glitchtip, forgejo CI
Some checks failed
CI / Web — typecheck + lint + build (push) Failing after 1m30s
CI / Scripts — Python smoke (push) Failing after 32s
CI / Web — npm audit (push) Failing after 37s
W0 — security hardening (5 fixes verified live on disclosure.top)
- middleware: gate /api/admin/* same as /admin/* (F1)
- imgproxy: tighten LOCAL_FILESYSTEM_ROOT from / to /var/lib/storage (F2)
- studio: real basic-auth label (bcrypt hash, middleware reference) (F3)
- relations: ENABLE ROW LEVEL SECURITY + public SELECT policy (F4)
- migration 0003: fold is_searchable + hybrid_search update into canonical (TD#2)

W1 — observability + resilience + autocomplete
- studio: HOSTNAME=0.0.0.0 so Next.js binds on loopback for healthcheck
- compose: PG_POOL_MAX=20, CLAUDE_CODE_OAUTH_TOKEN gated by separate env
- claude-code.ts: subprocess timeout configurable (CLAUDE_CODE_TIMEOUT_MS)
- openrouter.ts: retry with exponential backoff + Retry-After + in-memory
  circuit breaker (promotes FALLBACK after CB_THRESHOLD failures)
- lib/logger.ts: pino logger (NDJSON prod / pretty dev) + withRequest helper
- middleware: mints correlation_id, stamps x-correlation-id response header,
  emits structured http_request log per /api/* call
- messages/route.ts: switch to structured logger
- 60_meili_index.py: push documents + chunks into Meilisearch
- /api/search/autocomplete: parallel meili search (docs + chunks), 5-8ms p50
- search-autocomplete.tsx: debounced dropdown wired into search-panel

W1.2 — Glitchtip + Forgejo self-hosted
- compose: glitchtip-redis + glitchtip-web + glitchtip-worker (v4.2)
- compose: forgejo + forgejo-runner (server v9, runner v6) with group_add=988
- @sentry/nextjs SDK wired (instrumentation.ts + sentry.{client,server}.config.ts)
- /api/admin/throw smoke endpoint (gated by W0-F1 middleware)
- Synthetic event ingestion verified at glitchtip.disclosure.top
- forgejo.disclosure.top up, repo discadmin/disclosure-bureau created,
  runner registered (labels: ubuntu-latest, docker)
- .forgejo/workflows/ci.yml: typecheck + lint + build + npm audit + python
  syntax + compose validation

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 18:18:42 -03:00
Luiz Gustavo
e75ca5eda2 add clean LLM reading version of documents (the core goal)
Scanned docs are messy — duplicate transcriptions (typed + handwritten),
two classification variants of the same narrative, OCR noise, repeated
banners. The doc page showed raw chunks, so everything appeared twice.

40_reading_version.py generates ONE clean, deduplicated, well-structured
bilingual Markdown reading version per doc (Sonnet): merges duplicate versions
without losing unique lines, drops page furniture, formats transcripts as
dialogue. Faithful — invents nothing; redactions kept as markers.

/d/[docId] now defaults to a "📖 leitura" tab rendering this clean version,
with "🔍 trechos · scan original" preserving the faithful per-chunk + per-page
scan view. reading.md lives in raw/<doc>--subagent/ alongside the chunks.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 17:23:36 -03:00
Luiz Gustavo
5b62d0a3fe fix: UAP flag renders cleanly when type/rationale absent
~414 chunks have ufo_anomaly_detected=true but no type/rationale (extraction
left them null), so the flag rendered "UAP flag: anomaly —" with a dangling
em-dash. Build the label from the parts that exist: fall back to "anomalia"
for a missing type and omit the "—" when there's no rationale. The flag still
shows (the chunk genuinely contains UAP content), just without the noise.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 16:42:37 -03:00
Luiz Gustavo
b94f4869de fix: render image description when chunk has no bbox (no broken crop)
29 image chunks have an empty bbox {}, and `fm.bbox ?? default` doesn't catch
an empty object, so the crop URL got w=undefined → /api/crop 400 → broken-image
icon. Now validate bbox (w/h > 0); without it, render the image's textual
description instead of requesting an impossible crop.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 16:40:51 -03:00
Luiz Gustavo
a7e9dce6d2 rebuild entity layer from Sonnet-vision reextract pipeline
Add reextract pipeline (scripts/reextract/) that rebuilds doc-level entity
JSON from Sonnet-vision chunks via Opus, replacing the noisy per-page
extraction. Add synthesize scripts to regenerate wiki/entities from the 116
_reextract.json (30), aggregate missing page.md from chunks (31), and reprocess
805 pages the doc-rebuilder agent dropped on context overflow (32). Add
maintain scripts 43-56 for chunk-page sync, dedup, generic-entity marking, and
typed relation extraction.

Web: wire relations API + entity-relations component; entity/timeline/doc
pages consume the rebuilt layer.

Note: raw/, processing/, wiki/ remain gitignored (bulk data managed
separately); the 116 reextract JSONs and 7,798 rebuilt entity files live on
disk only. The 27 curated anchor events under wiki/entities/events/ are
preserved.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 12:20:24 -03:00
guto
7d13f93393 ship: synthesize 158 entities, AG-UI artifacts, chat persistence, auth flow
Fase 3 onda 2 — entity synthesis at scale:
- scripts/synthesize/20_entity_summary.py: queries DB for entities with
  total_mentions ≥ threshold + top-K verbatim chunk snippets via
  entity_mentions JOIN, prompts Sonnet (Holmes-Watson voice, bilingual),
  writes narrative_summary EN+PT-BR + summary_status=synthesized.
  Ran on 187 candidates (mentions ≥ 20) → 158 OK · 1 err · 29 skipped (no
  snippets). Combined with anchor curation: 20 curated + 158 synthesized
  = 178 entities with real narrative (vs 0 a day ago).

Fase 4 — chat with typed artifacts + persistence:
- lib/chat/agui.ts: AG-UI v1 typed Artifact union (citation, crop_image,
  entity_card, evidence_card, hypothesis_card, case_card, navigation_offer)
  alongside the existing event types.
- lib/chat/tools.ts + openrouter.ts: hybrid_search emits up to 6
  citation + crop_image artifacts per query. Provider collects them and
  returns in done.artifacts so the route can persist.
- api/sessions/[id]/messages: persist artifacts to messages.citations.
- components/chat-bubble.tsx: ArtifactCard renders inline cards (citation,
  crop_image, entity_card, navigation_offer) for streamed and persisted
  messages. activeId now persisted in localStorage so navigation between
  pages keeps the same conversation. New sessions are lazy (only when user
  has zero). loadMessages hydrates tools + artifacts from server. CRUD UI:
  rename (✎) + archive (🗑) buttons per session in the list.

Home search:
- doc-list-filters: input now fires hybrid_search (rerank=0 for speed)
  in parallel with the local title filter; chunk hits render above the doc
  grid with snippet + score + classification.
- api/search/hybrid: accept ?rerank=0 to skip the cross-encoder (1.3s vs 60s).

Auth flow:
- infra: SMTP_HOST=mail.spacemail.com:587 + DMARC published; mail now lands
  in inbox. GOTRUE_MAILER_AUTOCONFIRM=false (real email verification).
- kong.yml: proxy /auth/callback on api.disclosure.top → web:3000 so PKCE
  email links don't 404 at the gateway.
- web/app/auth/callback: handle both ?code= (OAuth) and ?token=&type=
  (PKCE); redirect to the public site host before verifyOtp so the session
  cookie lands on the right domain.

Audit deliverables:
- .nirvana/outputs/disclosure-bureau/.../systems-atelier/: 5 docs (code
  analysis, tech debt, discovery brief, system arch, 5 ADRs) authored by
  sa-principal that produced this roadmap. Kept in-tree for traceability.
2026-05-18 03:52:59 -03:00
guto
4459bd17e4 phase-0: kill stubs, ship 20 curated anchor events, configure SMTP
- scripts/03-dedup-entities.py: stop emitting placeholder narrative ("Stub. Will
  be enriched in Phase 7"); write summary_status=none + null fields instead.
- scripts/maintain/41_strip_stubs.py: idempotent migration that cleaned the
  22,096 entity .md files (now zero stub strings in wiki/).
- scripts/synthesize/01_anchor_events.py: curated 20 anchor UAP events
  (Roswell, Nimitz Tic-Tac, Phoenix Lights, Operação Prato, AATIP, etc.) with
  bilingual Holmes-Watson narrative via claude -p --model sonnet
  (CLAUDE_CODE_OAUTH_TOKEN). All summary_status=curated, confidence=high.
- web/api/timeline + timeline-view: filter narrative-less events by default,
  render "curado" badge for hand-vetted ones, drop the date display alone.
- CLAUDE-schema-full.md: document the summary_status enum and the four states.
- docker-compose.yml: SMTP_HOST=mail.spacemail.com configured;
  GOTRUE_MAILER_AUTOCONFIRM flipped to false (real email confirmation working).
- .nirvana/outputs/.../systems-atelier/: 5 deliverables of the architecture
  audit that produced this roadmap.
2026-05-18 00:44:17 -03:00
guto
19d0678e55 baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00