b173ac82a9
- adapt meta-/code-/obs- rules to the local Python benchmark pipeline - replace db-sql-ddl, code-config-env-scope, test-e2e-pytest with pipeline equivalents - add code-python-style, data-determinism, data-naming-units, bench-methodology, build-pipeline-tasks - normalize specs and README typography to ASCII per code-data-formatting - rewrite root CLAUDE.md trigger table; add .cursorrules and .gitignore - specs: pipeline plan and task specs 00-11 - rules: 19 binding rule files adapted for this project - docs: CLAUDE.md rule-trigger table - config: .cursorrules commit convention, .gitignore excluding out/ and .venv/ - docs: rewrite README.md with pipeline diagrams, setup guide, result placeholders - config: requirements.txt for the closed dependency list - datagen: make_lab_config.py writes out/config/lab_config.yaml and .done marker
17 lines
920 B
Markdown
17 lines
920 B
Markdown
# Task 05 — Schema Grouping & Inconsistency Report
|
||
|
||
## Goal
|
||
Enumerate distinct record structures per measurement type; expose structural inconsistencies.
|
||
|
||
## Input
|
||
- `manifest/tokens.jsonl`, `manifest/schema_profiles.jsonl`.
|
||
|
||
## Actions
|
||
1. Write `scripts/group_schemas.py`: group files by `measurement_type × fingerprint`.
|
||
2. Emit `manifest/fingerprint_report.md` per measurement type: list of distinct fingerprints, file count each, canonical column set each, column-name diffs between fingerprints (added/removed/renamed candidates), dtype conflicts.
|
||
3. Emit `manifest/inconsistencies.csv`: one row per detected inconsistency `{measurement_type, kind(column_set|naming|dtype|placement), fingerprints involved, example files}`.
|
||
|
||
## Intermediate Result (done when)
|
||
- The number of distinct schemas per measurement type is known and documented.
|
||
- `inconsistencies.csv` complete — this is the work queue for Task 07.
|