discadmin/disclosure-bureau
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
disclosure-bureau/scripts
History
guto d5f6e6030a fix png-numbering: re-convert 34 zero-based docs + crop fallback 
34 of 116 docs were generated with 0-based PNG numbering (p-000.png …
p-008.png) but the Sonnet chunks reference 1-based page numbers in their
YAML frontmatter (page: 9 means the 9th sheet of paper). The /api/crop
handler built p-009.png and got a 500, the browser's Next/Image surfaced
400, and the chunk rendered as a black box on screen.

Fixes:
- web/app/api/crop/route.ts: try p-NNN.png first, fall back to
  p-(NNN-1).png if the 1-based file is missing. Cheap insurance for any
  doc that comes in with the old convention.
- scripts/01-convert-pdfs.sh: previously printf '%03d' "$num" with $num
  starting at 0 (e.g. "008") raised "invalid number" because Bash
  parsed it as octal. Wrap with $((10#$num)) to force decimal — this
  was silently corrupting page sequences and producing gaps like p-001
  ... p-008, p-011 (missing p-009/p-010).
- All 34 affected docs re-converted from PDFs with the patched script;
  every directory now has continuous 1-based PNGs.
- /processing/png/ rsync'd to VPS, web redeployed.

Smoke: /api/crop?doc=doc-341-…&page=9&… now returns 200 image/webp
instead of 500. Tested in browser: chunk c0026 (diagram, p9) renders
the real engineering diagram.
2026-05-18 11:45:40 -03:00
..
_archived-gemini ban gemini: quarantine 10 SDK-using scripts under scripts/_archived-gemini/ 2026-05-18 02:23:13 -03:00
maintain phase-0: kill stubs, ship 20 curated anchor events, configure SMTP 2026-05-18 00:44:17 -03:00
synthesize ship: synthesize 158 entities, AG-UI artifacts, chat persistence, auth flow 2026-05-18 03:52:59 -03:00
00-extract-war-gov.js baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
00b-coverage-report.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
00c-download-missing.js baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
01-convert-pdfs.sh fix png-numbering: re-convert 34 zero-based docs + crop fallback 2026-05-18 11:45:40 -03:00
02-vision-page.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
02b-enrich-with-web-metadata.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
03-dedup-entities.py phase-0: kill stubs, ship 20 curated anchor events, configure SMTP 2026-05-18 00:44:17 -03:00
04-lint.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
05-crop-bboxes.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
06-graph-export.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
07-test-agent.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
09-extract-uap-frames.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
12-incremental-orchestrator.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
14-build-document-md.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
15-consolidate-tables.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
16-extract-table-csv.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
17-enrich-entities.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
18-build-entity-index.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
19-detect-vision-mismatch.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
22-update-stub-messages.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
23-smart-dedup.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
24-document-synthesis.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
25-master-doc-test.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
26-chunk-harness.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
28-batch-rebuild-all.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
29-auto-resume-batch.sh baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
30-index-chunks-to-db.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
31-populate-entity-mentions.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
32-sync-mentioned-in-yaml.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
33-compact-progress-log.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
34-generate-doc-pitches.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
99-finalize-pipeline.sh baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
99-finalize-resume.sh baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
gen_all_chunks_doc65.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
gen_chunks_doc65_suba.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_d48_ocr_only.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_d49.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc65.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc65_assemble.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc65_page_rebuilder.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc65_s2_v2.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc65_s8.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc65_section2.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc65_section7.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc65_serial130_resume.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc65_suba_final.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc65_v2.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc255.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc_65.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc_d48.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
rebuild_doc_section3.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00
write_chunks_doc65.py baseline: Disclosure Bureau pipeline + Next.js UI + Supabase stack 2026-05-17 22:44:36 -03:00