b173ac82a9
- adapt meta-/code-/obs- rules to the local Python benchmark pipeline - replace db-sql-ddl, code-config-env-scope, test-e2e-pytest with pipeline equivalents - add code-python-style, data-determinism, data-naming-units, bench-methodology, build-pipeline-tasks - normalize specs and README typography to ASCII per code-data-formatting - rewrite root CLAUDE.md trigger table; add .cursorrules and .gitignore - specs: pipeline plan and task specs 00-11 - rules: 19 binding rule files adapted for this project - docs: CLAUDE.md rule-trigger table - config: .cursorrules commit convention, .gitignore excluding out/ and .venv/ - docs: rewrite README.md with pipeline diagrams, setup guide, result placeholders - config: requirements.txt for the closed dependency list - datagen: make_lab_config.py writes out/config/lab_config.yaml and .done marker
86 lines
3.9 KiB
Markdown
86 lines
3.9 KiB
Markdown
# Data / Determinism - one seed, byte-identical regeneration
|
|
|
|
**Purpose + scope.** The reproducibility contract for all generated data.
|
|
Binding for task 03 (CSV generation) and for every piece of code that
|
|
produces values which end up inside the corpus or its conversions.
|
|
|
|
The entire evaluation rests on this: benchmark results are only comparable
|
|
because every format holds EXACTLY the same data, and the study is only
|
|
reviewable because anyone can regenerate that data bit-for-bit.
|
|
|
|
---
|
|
|
|
## 1. The seed
|
|
|
|
- **The global seed is `20260711`.** It lives in `lab_config.yaml`
|
|
(written by task 01) and is READ from there - never hardcoded in a
|
|
generation script (see [code-config-yaml.md](code-config-yaml.md)).
|
|
- Changing the seed is a project-level decision, never a side effect. It
|
|
invalidates the whole `./out/` tree, every checksum, and every measured
|
|
result (see [meta-no-assumptions.md](meta-no-assumptions.md)).
|
|
|
|
## 2. Sanctioned randomness - seeded generators only
|
|
|
|
- The ONLY random sources are `random.Random(seed)` and
|
|
`numpy.random.default_rng(seed)`, constructed explicitly from the
|
|
config seed.
|
|
- Derive per-entity generators deterministically when parallel or
|
|
out-of-order generation needs them (e.g.
|
|
`default_rng([seed, batch_index, coupon_index])`) - never from time,
|
|
never from a global unseeded state.
|
|
- Forbidden anywhere in generation code: module-level `random.*` calls,
|
|
`numpy.random.*` legacy global functions, `os.urandom`, `uuid.uuid4`,
|
|
`hash()`-dependent ordering (Python string hashing is salted per
|
|
process).
|
|
|
|
## 3. No wall-clock in the data
|
|
|
|
- Every timestamp inside the corpus is SIMULATED - computed from the lab
|
|
schedule model - never `datetime.now()` / `time.time()`.
|
|
- Real wall-clock time may appear only in logs and in measurement records
|
|
(benchmark wall times in `./out/bench/` are measurements, not corpus
|
|
data).
|
|
|
|
## 4. Deterministic everything else
|
|
|
|
Randomness is not the only nondeterminism; all of these are binding:
|
|
|
|
- **Iteration order.** Iterate in a defined order (sorted paths, explicit
|
|
index loops). Never rely on `os.listdir` / `glob` / `set` / dict-of-set
|
|
ordering for anything that affects output. Sort directory listings
|
|
before use.
|
|
- **Float formatting.** Format floats with an explicit spec (e.g.
|
|
`f"{x:.6g}"`), chosen once per column family; repr drift between runs
|
|
or Python versions must not change file bytes.
|
|
- **Line endings and encoding.** Generated files are UTF-8 with `\n`
|
|
newlines, on every platform (`open(..., newline="")` for csv writers).
|
|
- **Parallelism.** If generation is parallelized, each worker's output is
|
|
deterministic (derived generators, fixed partitioning) and the merge
|
|
order is fixed.
|
|
|
|
## 5. The proof - MANIFEST sha256
|
|
|
|
- `./out/csv/MANIFEST.csv` records sha256 per generated file (spec 03).
|
|
**Regenerating with the same `lab_config.yaml` MUST reproduce every
|
|
sha256 exactly.** This is the acceptance test for this whole rule -
|
|
run generation twice when touching generator code and diff the
|
|
manifests.
|
|
- Downstream conversions must be equally deterministic in CONTENT (same
|
|
rows, same values); byte-identity of database files is not required
|
|
(SQLite/PostgreSQL internals may differ), which is why their gate is
|
|
row counts + query-result checksums instead
|
|
(see [test-pipeline-validation.md](test-pipeline-validation.md)).
|
|
|
|
## 6. Anti-patterns
|
|
|
|
- **Hardcoding `20260711`** in a script instead of reading the config.
|
|
- **An unseeded or globally-seeded random call** anywhere in generation.
|
|
- **`datetime.now()` in corpus values** - simulated schedule only.
|
|
- **Ordering that depends on the filesystem or hash salting** - sort
|
|
explicitly.
|
|
- **"Harmless" float repr changes** - a formatting change is a corpus
|
|
change; it must be deliberate and re-baselines every checksum.
|
|
- **Regenerating the corpus to "refresh" it** - identical input config
|
|
must yield identical output; if bytes changed without a config change,
|
|
that is a determinism bug to fix, not a new baseline to accept.
|