disclosure-bureau/scripts/synthesize
Luiz Gustavo a7e9dce6d2 rebuild entity layer from Sonnet-vision reextract pipeline
Add reextract pipeline (scripts/reextract/) that rebuilds doc-level entity
JSON from Sonnet-vision chunks via Opus, replacing the noisy per-page
extraction. Add synthesize scripts to regenerate wiki/entities from the 116
_reextract.json (30), aggregate missing page.md from chunks (31), and reprocess
805 pages the doc-rebuilder agent dropped on context overflow (32). Add
maintain scripts 43-56 for chunk-page sync, dedup, generic-entity marking, and
typed relation extraction.

Web: wire relations API + entity-relations component; entity/timeline/doc
pages consume the rebuilt layer.

Note: raw/, processing/, wiki/ remain gitignored (bulk data managed
separately); the 116 reextract JSONs and 7,798 rebuilt entity files live on
disk only. The 27 curated anchor events under wiki/entities/events/ are
preserved.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 12:20:24 -03:00
..
01_anchor_events.py phase-0: kill stubs, ship 20 curated anchor events, configure SMTP 2026-05-18 00:44:17 -03:00
20_entity_summary.py ship: synthesize 158 entities, AG-UI artifacts, chat persistence, auth flow 2026-05-18 03:52:59 -03:00
30_rebuild_wiki_from_reextract.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
31_aggregate_pages_from_chunks.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
32_reprocess_missing_pages.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00