Overview

The three variants

• Variant A - composite natural PK. Row identity of a bulk row is its natural composite key (track_id, cycle), stored as a clustered WITHOUT ROWID table. FK columns are plain integers, validated by the loader against the corpus manifest (no engine constraint).
• Variant B - composite natural PK + enforced FK. Same physical layout as A, plus REFERENCES constraints with PRAGMA foreign_keys=ON, so the engine itself guarantees referential integrity.
• Variant C - surrogate key on every table. Classic style: each table gets a single-column auto-increment key named <singular>_id (friction_cycle_id, never a bare id). Correctness then requires an additional UNIQUE(track_id, cycle) index - a second B-tree.

Corpus context

The corpus models a Pt-Au coating study: 4 deposition batches → 12 wafers → 588 coupons (480 friction-tested) → 1,440 tracks → 1.44M cycle records. Generated size: 141.8 MiB of CSV; the production target for extrapolation is 600 GB - 6 TB, which is why per-row bytes on the bulk tables matter.

CREATE TABLE friction_cycles (
  track_id INTEGER NOT NULL,   -- reference to tracks(track_id), loader-validated
  cycle    INTEGER NOT NULL,
  cof      REAL    NOT NULL,
  PRIMARY KEY (track_id, cycle)
) WITHOUT ROWID;              -- clustered: the PK is the table's only B-tree

PRAGMA foreign_keys = ON;

CREATE TABLE friction_cycles (
  track_id INTEGER NOT NULL REFERENCES tracks(track_id),
  cycle    INTEGER NOT NULL,
  cof      REAL    NOT NULL,
  PRIMARY KEY (track_id, cycle)
) WITHOUT ROWID;

CREATE TABLE friction_cycles (
  friction_cycle_id INTEGER PRIMARY KEY,        -- rowid alias; unused by any FK
  track_id INTEGER NOT NULL REFERENCES tracks(track_id),
  cycle    INTEGER NOT NULL,
  cof      REAL    NOT NULL,
  UNIQUE (track_id, cycle)                      -- required; duplicates the table
);

• Data: the full generated corpus subset - tracks (1,440), friction_cycles (1,440,000), friction_loop_points (2,880,000) - read from the same CSV files in the same order for every variant.
• Load: SQLite, PRAGMA journal_mode=OFF, synchronous=OFF, cache_size=-262144; inserts via executemany in 50k-row batches; VACUUM before measuring the file size.
• Reads: warm OS page cache, best of 3 runs per pattern; fixed deterministic probe sets (39 tracks for range reads, 150 (track_id, cycle) pairs for point lookups); result-set sizes asserted, so every variant answers the identical question.
• Caveat: at 150 MB everything fits in RAM, which flatters variant C - at the 600 GB production target the working set exceeds RAM, and twice the pages means twice the cache pressure and disk reads. Warm-cache deltas are therefore a lower bound.

Database size after VACUUM (MiB, scale 0-200)

Load time, 4.32M rows (s, scale 0-10)

Relative read time, variant A = 100% (lower is better)

• +74% disk. In SQLite the surrogate id column is a rowid alias and costs nothing in the table itself - but the mandatory UNIQUE(track_id, cycle) index is a full second B-tree whose entries contain almost every column of the row. Two trees instead of one on a 3-column table is close to a doubling.
• +59% on grouped aggregation (the pipeline's forced-full-scan Q5). In A/B the table is clustered by track_id, so GROUP BY track_id streams the tree in group order. In C the engine walks the UNIQUE index for ordering and hops into the table B-tree for every cof value - the classic double access path.
• +5-13% on ranges and plain scans. More pages (190 vs 109 MiB) means more page traversals and cell decoding, even fully cached.
• ~0% on point lookups. Tree depth is the same either way; a single-row probe does not expose the second tree.
• PostgreSQL outlook. PG has no WITHOUT ROWID: the composite-PK table is a heap plus one PK index either way, but variant C still adds the id column to every heap tuple and the second index - expect +30-50% on the bulk tables, plus higher COPY-time FK validation cost.

Variant B  Composite natural PK on bulk tables + enforced foreign keys

• Referential integrity is guaranteed by the DBMS itself at a measured cost of +0.5 s of load time and nothing else.
• Disk footprint and read profile are identical to the leanest variant A: 109.1 MiB, single clustered B-tree per bulk table.
• Dimension tables (batches, wafers, coupons, runs, tracks, instruments) keep their classic single-column surrogate keys - <singular>_id, never a bare id - because other tables reference their rows.
• Variant C remains a valid design where fact rows must be individually addressable from outside (annotations, reprocessing links). No table in this schema references an individual cycle or loop point, so its cost (+74% disk, up to +59% on aggregation, growing at 600 GB scale) buys nothing here.