disclosure-bureau/scripts/maintain
Luiz Gustavo fe19bb9c57 add page↔document navigation + DB repopulation tooling
Doc page (/d/[docId]/[page]) gains prev/next navigation bars (top + bottom):
within a doc it steps page-by-page; at the first/last page it jumps to the
previous/next document. Replaces the disabled-at-boundary links.

Indexer tooling for the VPS repopulation:
- 30-index-chunks-to-db.py: add --no-embed (fast BM25-only index; vectors
  backfilled separately) so the app is usable in minutes, not hours of CPU
  embedding.
- 57_load_relations_from_json.py: load typed relations into public.relations
  from reextract structured fields (deterministic ids, no fuzzy guessing).
- 58_backfill_embeddings.py: async pass to fill chunks.embedding (NULL rows)
  via the embed-service.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 14:28:14 -03:00
..
41_strip_stubs.py phase-0: kill stubs, ship 20 curated anchor events, configure SMTP 2026-05-18 00:44:17 -03:00
42_sync_entity_stats.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
43_fix_chunk_page_from_source_png.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
44_sync_chunk_page_to_db.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
45_resync_index_json.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
46_text_backfill_mentions.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
47_mark_unsearchable_chunks.sql rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
48_hybrid_search_filter_unsearchable.sql rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
49_dedup_aggressive.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
50_dedup_fuzzy_trigram.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
51_remap_entity_mentions.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
52_mark_generic_entities.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
53_add_is_generic_to_db.sql rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
54_sync_is_generic.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
55_relations_schema.sql rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
56_extract_relations.py rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00
57_load_relations_from_json.py add page↔document navigation + DB repopulation tooling 2026-05-21 14:28:14 -03:00
58_backfill_embeddings.py add page↔document navigation + DB repopulation tooling 2026-05-21 14:28:14 -03:00
run_full_dedup_pipeline.sh rebuild entity layer from Sonnet-vision reextract pipeline 2026-05-21 12:20:24 -03:00