# Task 15 — Lineage Consolidation, Build Manifest, Final Validation ## Goal Close the pipeline: consolidated lineage, reproducible build record, cross-format validation. ## Input - All `exports/*`, `lineage/`, all config/schema versions. ## Actions 1. Write `scripts/finalize.py`: - consolidate `lineage/` into `lineage/lineage.sqlite` (queryable: target row-range → source file + offsets + rule versions); - produce `BUILD_MANIFEST.md`: snapshot id, all config/mapping/schema versions, input inventory hash, per-artifact counts, run timestamps — the processing-flow documentation. 2. Cross-format validation: row counts and key aggregates (e.g., sum of cycles per test, point counts per measurement) identical across canonical, JSON, SQLite, PostgreSQL staging. 3. Lineage spot-check: resolve 20 random target rows back to source file + row offset; verify raw values match. ## Final Result (pipeline done when) - `BUILD_MANIFEST.md` complete; all validations pass; lineage resolvable. - **Final goal achieved:** cleansed canonical dataset conforming to the ontology, all anomalies quarantined and explained, four export formats generated from one snapshot, full source→target lineage, entire pipeline reproducible from source + versioned config.