Files
LabDataStorageEvaluation/docs/rules/data-naming-units.md
administrator b173ac82a9 fix(docs): fix formatting issues in README.md
- adapt meta-/code-/obs- rules to the local Python benchmark pipeline
- replace db-sql-ddl, code-config-env-scope, test-e2e-pytest with pipeline equivalents
- add code-python-style, data-determinism, data-naming-units, bench-methodology, build-pipeline-tasks
- normalize specs and README typography to ASCII per code-data-formatting
- rewrite root CLAUDE.md trigger table; add .cursorrules and .gitignore
- specs: pipeline plan and task specs 00-11
- rules: 19 binding rule files adapted for this project
- docs: CLAUDE.md rule-trigger table
- config: .cursorrules commit convention, .gitignore excluding out/ and .venv/
- docs: rewrite README.md with pipeline diagrams, setup guide, result placeholders
- config: requirements.txt for the closed dependency list
- datagen: make_lab_config.py writes out/config/lab_config.yaml and .done marker
2026-07-11 13:39:13 -04:00

4.9 KiB

Data / Naming and units - IDs, timestamps, SI-explicit column names

Purpose + scope. The naming grammar for entities, timestamps, and measurement columns across ALL representations of the corpus: CSV, JSON-LD, SQLite, PostgreSQL, RDF. Binding for generation, conversion, queries, and reporting. This file also fixes two points the specs leave ambiguous (sections 3 and 4).


1. Entity identifiers

The hierarchy encodes containment left-to-right; each level appends one segment:

Entity Code grammar Example
batch B<nnn> B721
wafer <batch>-W<n> B721-W1
coupon <wafer>-C<nn> (01..49, the 7x7 grid) B721-W1-C07
track <coupon>-T<n> (1..3) B721-W1-C07-T2
run R<n> (1..4) R3
  • CSV stores the bare code (B721-W1-C07) in <entity>_code-style columns.
  • JSON-LD / RDF use the prefixed CURIE form as the node identity: batch:B721, wafer:B721-W1, coupon:B721-W1-C07, track:B721-W1-C07-T2, run:R1 (spec 00_PLAN global conventions).
  • SQL stores the bare code as the <singular>_code TEXT UNIQUE column next to the integer surrogate key (see db-sql-schema.md).
  • A code is never parsed to recover its parent where a proper reference exists; the parent relation is explicit in every format (directory nesting, partOf, FK).

2. Timestamps

  • ISO-8601, UTC, Z suffix, second precision: 2026-03-14T09:30:00Z.
  • All corpus timestamps are simulated (see data-determinism.md section 3).
  • In SQL: TIMESTAMPTZ in PostgreSQL; TEXT in ISO-8601 form in SQLite (sortable lexicographically by construction).

3. Measurement columns - unit suffix is part of the name

Every numeric measurement column ends in an explicit SI unit token. The canonical tokens (as they appear in CSV headers):

Token Unit Example columns
_um micrometre thickness_um, position_um
_nm nanometre ra_nm
_GPa gigapascal hardness_GPa, reduced_modulus_GPa
_mN millinewton max_load_mN, friction_force_mN
_wtpct weight percent au_wtpct
_um3 cubic micrometre wear_volume_um3
_m metre sliding_distance_m
_mm_s millimetre per second speed_mm_s
_V volt discharge_voltage_V
_W watt power_W
_C degree Celsius temperature_C
_pct percent (non-composition) rh_pct, cpu_util_pct
_s / _mb seconds / megabytes (benchmark outputs) wall_s, peak_rss_mb
  • Unit tokens are plain ASCII - never µ, °, or Unicode superscripts (see code-data-formatting.md).
  • Dimensionless quantities carry no token (cof, cycle, k_archard).
  • New measurement columns pick from this table or add a row to it in the same commit.

4. Canonical case mapping CSV -> SQL (ambiguity resolved)

The specs write CSV headers with SI capitalization (hardness_GPa, spec 03) but SQL DDL in lowercase (hardness_gpa, spec 05). Binding resolution:

  • CSV headers keep SI capitalization (_GPa, _mN, _V, _W, _C) - they mirror instrument output conventions.
  • SQL / JSON-LD / RDF property names are the plain lowercase of the CSV name (hardness_gpa, max_load_mn). One mechanical rule, no per-column table to maintain.
  • Converters map with csv_name.lower() - never hand-maintained rename dicts that can drift.

5. au_wtpct_mean (spec gap closed)

Specs 05/06 index coupons(au_wtpct_mean) but spec 03 never names the column. Binding definition:

  • au_wtpct_mean = the arithmetic mean of au_wtpct over the coupon's 400 XRF map points, computed by task 03.
  • It is written into coupon_info.csv by generation (not derived later by converters), so every format inherits the identical value, and Q2 ("coupons with mean Au = 10 +/- 0.5 wt%") means the same rows everywhere.

6. Benchmark artifact columns

Result files under ./out/bench/ follow the same discipline: results_raw.csv columns are format, query, run, wall_s, peak_rss_mb, cpu_s, cpu_util_pct, read_mb, rows, checksum_ok (spec 09); storage_sizes.csv columns are format, variant, bytes (see build-pipeline-tasks.md). Extending either file extends this list in the same commit.

7. Anti-patterns

  • A measurement column without a unit token (thickness, load) - unreadable and un-greppable across formats.
  • Unit in a comment instead of the name - comments do not survive format conversion; names do.
  • Parsing B721-W1-C07 to find the wafer where a real reference exists in the format at hand.
  • A hand-maintained CSV->SQL rename mapping - the mapping is lower(), nothing else.
  • Recomputing au_wtpct_mean in a converter - it is generated once into coupon_info.csv; converters copy it.
  • Unicode units in identifiers (µm, °C) - ASCII tokens only.