96ed7bc918
- tools: common/track_summary.py fixes the cross-engine steady-state algorithm - rules: db-sql-schema records the FK enforcement decision and track_summary definition - specs: 05 aligned with data-naming-units, simtra_profiles added - docs: FK key-schema study added under docs/research/, README updated - replace manual CSV reading with CorpusReader for better data handling - streamline argument parsing and dependency checks using common pipeline functions - enhance marker writing for task completion tracking - remove unused regex and validation error classes for cleaner code
33 lines
2.6 KiB
Markdown
33 lines
2.6 KiB
Markdown
# 05 - Convert to SQLite (optimized)
|
|
|
|
## Context / Goal
|
|
Build the single-file relational variant optimized for size and single-user query speed.
|
|
|
|
## Dependencies
|
|
03.
|
|
|
|
## Input
|
|
CSV tree.
|
|
|
|
## Processing instructions
|
|
- One database `./out/sqlite/tribo.db`. PRAGMAs during load: `journal_mode=OFF, synchronous=OFF, cache_size=-262144, foreign_keys=ON`; after load: `journal_mode=WAL`, `ANALYZE`, `VACUUM`.
|
|
- Foreign keys are enforced on every table, including bulk tables (variant B of the measured key-schema study; owner decision 2026-07-11, see db-sql-schema rule). Full study with ER diagrams and measurements: [../research/Tribology_FK_Architecture.html](../research/Tribology_FK_Architecture.html).
|
|
- Schema (16-table style, integer surrogate keys everywhere; no TEXT foreign keys in bulk tables; SQL column names are the lowercase of the CSV headers per data-naming-units):
|
|
- `batches, wafers, coupons, runs, instruments, tracks` - dimension tables, TEXT natural keys kept as unique columns.
|
|
- `simtra_profiles(batch_id INT, row_no INT, angle_deg REAL, energy_ev REAL, pt_flux REAL, au_flux REAL, PRIMARY KEY(batch_id, row_no)) WITHOUT ROWID`.
|
|
- `xrf_points(coupon_id INT, grid_x INT, grid_y INT, pt_wtpct REAL, au_wtpct REAL, PRIMARY KEY(coupon_id, grid_x, grid_y)) WITHOUT ROWID`.
|
|
- `profilometry_points(coupon_id INT, grid_x INT, grid_y INT, thickness_um REAL, PRIMARY KEY(coupon_id, grid_x, grid_y)) WITHOUT ROWID`.
|
|
- `nanoindentation(coupon_id INT, indent_id INT, x_um REAL, y_um REAL, hardness_gpa REAL, reduced_modulus_gpa REAL, max_load_mn REAL, PRIMARY KEY(coupon_id, indent_id)) WITHOUT ROWID`.
|
|
- `afm(coupon_id INT PRIMARY KEY, ra_nm REAL, rq_nm REAL, image_file TEXT)`.
|
|
- `friction_cycles(track_id INT, cycle INT, cof REAL, PRIMARY KEY(track_id, cycle)) WITHOUT ROWID`.
|
|
- `friction_loop_points(track_id INT, cycle INT, pt INT, position_um REAL, friction_force_mn REAL, PRIMARY KEY(track_id, cycle, pt)) WITHOUT ROWID`.
|
|
- `wear(track_id INT PRIMARY KEY, wear_volume_um3 REAL, k_archard REAL, sliding_distance_m REAL)`.
|
|
- `track_summary(track_id INT PRIMARY KEY, cof_ss_mean REAL, cof_ss_std REAL, run_in_cycles INT)` - precomputed during load (algorithm fixed in db-sql-schema rule, section 5).
|
|
- Indexes for Q1-Q7: `coupons(au_wtpct_mean)`, `tracks(run_id)`, `tracks(coupon_id)`, `runs(environment)`, composite `tracks(environment, load_mn)` (denormalize environment/load onto tracks).
|
|
- Batch inserts with `executemany`, 50k rows per transaction.
|
|
- Validate row counts against MANIFEST.
|
|
|
|
## Output
|
|
- `./out/sqlite/tribo.db`; size appended to `storage_sizes.csv`.
|
|
- Marker `./out/.done/05.ok`.
|