96ed7bc918
- tools: common/track_summary.py fixes the cross-engine steady-state algorithm - rules: db-sql-schema records the FK enforcement decision and track_summary definition - specs: 05 aligned with data-naming-units, simtra_profiles added - docs: FK key-schema study added under docs/research/, README updated - replace manual CSV reading with CorpusReader for better data handling - streamline argument parsing and dependency checks using common pipeline functions - enhance marker writing for task completion tracking - remove unused regex and validation error classes for cleaner code
2.6 KiB
2.6 KiB
05 - Convert to SQLite (optimized)
Context / Goal
Build the single-file relational variant optimized for size and single-user query speed.
Dependencies
Input
CSV tree.
Processing instructions
- One database
./out/sqlite/tribo.db. PRAGMAs during load:journal_mode=OFF, synchronous=OFF, cache_size=-262144, foreign_keys=ON; after load:journal_mode=WAL,ANALYZE,VACUUM. - Foreign keys are enforced on every table, including bulk tables (variant B of the measured key-schema study; owner decision 2026-07-11, see db-sql-schema rule). Full study with ER diagrams and measurements: ../research/Tribology_FK_Architecture.html.
- Schema (16-table style, integer surrogate keys everywhere; no TEXT foreign keys in bulk tables; SQL column names are the lowercase of the CSV headers per data-naming-units):
batches, wafers, coupons, runs, instruments, tracks- dimension tables, TEXT natural keys kept as unique columns.simtra_profiles(batch_id INT, row_no INT, angle_deg REAL, energy_ev REAL, pt_flux REAL, au_flux REAL, PRIMARY KEY(batch_id, row_no)) WITHOUT ROWID.xrf_points(coupon_id INT, grid_x INT, grid_y INT, pt_wtpct REAL, au_wtpct REAL, PRIMARY KEY(coupon_id, grid_x, grid_y)) WITHOUT ROWID.profilometry_points(coupon_id INT, grid_x INT, grid_y INT, thickness_um REAL, PRIMARY KEY(coupon_id, grid_x, grid_y)) WITHOUT ROWID.nanoindentation(coupon_id INT, indent_id INT, x_um REAL, y_um REAL, hardness_gpa REAL, reduced_modulus_gpa REAL, max_load_mn REAL, PRIMARY KEY(coupon_id, indent_id)) WITHOUT ROWID.afm(coupon_id INT PRIMARY KEY, ra_nm REAL, rq_nm REAL, image_file TEXT).friction_cycles(track_id INT, cycle INT, cof REAL, PRIMARY KEY(track_id, cycle)) WITHOUT ROWID.friction_loop_points(track_id INT, cycle INT, pt INT, position_um REAL, friction_force_mn REAL, PRIMARY KEY(track_id, cycle, pt)) WITHOUT ROWID.wear(track_id INT PRIMARY KEY, wear_volume_um3 REAL, k_archard REAL, sliding_distance_m REAL).track_summary(track_id INT PRIMARY KEY, cof_ss_mean REAL, cof_ss_std REAL, run_in_cycles INT)- precomputed during load (algorithm fixed in db-sql-schema rule, section 5).
- Indexes for Q1-Q7:
coupons(au_wtpct_mean),tracks(run_id),tracks(coupon_id),runs(environment), compositetracks(environment, load_mn)(denormalize environment/load onto tracks). - Batch inserts with
executemany, 50k rows per transaction. - Validate row counts against MANIFEST.
Output
./out/sqlite/tribo.db; size appended tostorage_sizes.csv.- Marker
./out/.done/05.ok.