291748df63
The corpus had two parallel reverse-reference signals: the wiki/pages
entities_extracted blocks (Haiku page-level) and public.entity_mentions
(Sonnet chunk-level, ILIKE-matched). The entity page only consulted the
DB, so it showed "0 menções" for thousands of entities that were anchored
in pages or in cross-entity links the DB never indexed.
Resolved by collapsing all signals into the YAML frontmatter, which is
now the single runtime source for entity metadata.
scripts/maintain/42_sync_entity_stats.py walks every entity and writes:
mentioned_in: [...] # consolidated page refs
total_mentions: max(db, pages)
documents_count: max(db_docs, distinct page docs)
signal_sources:
db_chunks: int
page_refs: int
cross_refs: int
signal_strength: strong | weak | orphan | unverified
referenced_by: [[class/id]] # cross-entity backlinks
Outgoing wikilinks (e.g. OBJ.observed_in_event → EV) count toward the
entity's own cross_refs so anchored-but-not-mentioned entities don't
register as orphan.
OBJ canonical names like "7m long, 1.3m high, two rocket motors,
smooth flow, rotary drive null UAP (OBJ-EV1945-PEYERLSHOTDOWN-01)"
are rewritten to "Peyerl shot down UAP" derived from observed_in_event,
preserving the full description as an alias. --fix-obj-names did this
for every OBJ-* with >80 char canonical_name.
Default behaviour is conservative: --archive-only-junk archives only
single/double-char names and pure-numeric noise. Everything else stays
on disk with signal_strength marked, so the user can review later.
web/lib/retrieval/entity-pages.ts swapped from db-first to yaml-first.
The /e/[cls]/[id] page now reads counts straight from YAML and renders
a "força do sinal" badge with the per-source breakdown. Orphan entities
get a banner explaining they have no cross-references.
DB is still queried for ONE thing: the chunk text for preview cards on
the entity page, so we don't re-parse 21k markdown files on every render.
First-pass result: 9020 strong / 14520 weak / 10814 orphan; OBJ-EV1945-
PEYERLSHOTDOWN-01 now reads "Peyerl shot down UAP · fraca · 1 backlink"
in the live UI.
|
||
|---|---|---|
| .. | ||
| app | ||
| components | ||
| lib | ||
| .dockerignore | ||
| .env.local.example | ||
| Dockerfile | ||
| middleware.ts | ||
| next-env.d.ts | ||
| next.config.ts | ||
| package-lock.json | ||
| package.json | ||
| postcss.config.mjs | ||
| README.md | ||
| tailwind.config.ts | ||
| tsconfig.json | ||
web — Disclosure Bureau Next.js app
Next.js 15 + React 19 + Tailwind + Supabase + assistant-ui.
Quick start (local dev)
# 1. Install deps
npm install
# 2. (Optional) Start local Supabase
# Requires Docker. Skip if pointing at remote Supabase.
npx supabase init # first time only — creates supabase/ folder
npx supabase start # spins up Postgres/GoTrue/Storage on :54321
# 3. Configure env
cp .env.local.example .env.local
# Edit .env.local — paste local Supabase keys (printed by `supabase start`)
# 4. Apply migrations
psql postgresql://postgres:postgres@localhost:54322/postgres \
-f ../infra/supabase/migrations/0001_chat_schema.sql
# 5. Start dev
npm run dev
# http://localhost:3030
Without Supabase
The app degrades gracefully if Supabase env vars are unset:
- Wiki browsing works (read-only from filesystem)
- Auth bar shows "auth: disabled (dev)"
- Chat bubble shows "Auth not configured"
Useful for quick UI work without spinning up Docker.
Production (Coolify on VPS)
See ../infra/coolify/. Stack:
- Coolify orchestrates everything
- Supabase self-hosted:
db.disclosure.top,studio.disclosure.top - Next.js:
disclosure.top - Meilisearch (shared):
search.disclosure.top - Imgproxy (shared):
img.disclosure.top - Caddy: TLS + reverse proxy (built into Coolify)
Architecture
app/
├── page.tsx # home — 116 docs grouped by collection
├── auth/
│ ├── signin/page.tsx # magic-link form
│ ├── callback/route.ts # exchanges code for session
│ └── signout/route.ts
├── d/[docId]/
│ ├── page.tsx # doc detail
│ └── [page]/page.tsx # page reader (OCR + entity highlights + crops + sidebar PNG)
├── api/
│ ├── me/route.ts # GET current profile
│ ├── sessions/route.ts # GET list, POST new
│ ├── sessions/[id]/route.ts # GET detail, PATCH, DELETE
│ ├── sessions/[id]/messages/route.ts # POST send → assistant reply
│ ├── documents/, pages/, entities/, tables/ # read-only data
│ └── static/[...path]/route.ts # sandboxed file serve
components/
├── chat-bubble.tsx # floating Sherlock — auth-aware, session list
├── entity-modal.tsx # opens on entity click
├── reader-content.tsx # OCR + highlights + crops
└── auth-bar.tsx # sign in / out + budget tracker
lib/
├── wiki.ts # markdown reader (gray-matter)
├── entity-index.ts # match loader + text segmentation
└── supabase/{server,client}.ts # SSR helpers
middleware.ts # session refresh on every request
Tech notes
- No RAG: chat agent reads markdown directly. Wiki-link traversal substitutes for vector search.
- RLS-first: Supabase Row Level Security enforces "user sees only own sessions" at the DB layer.
- Magic-link auth: no passwords. GoTrue handles email delivery.
- Anti-abuse: per-user budget cap (default $5) + daily message quota (default 100) enforced via
check_budgetRPC before each Claude call.
Cost
Each chat turn costs ~$0.005-0.05 depending on context size (mostly Haiku $1/M input, $5/M output).