b173ac82a9
- adapt meta-/code-/obs- rules to the local Python benchmark pipeline - replace db-sql-ddl, code-config-env-scope, test-e2e-pytest with pipeline equivalents - add code-python-style, data-determinism, data-naming-units, bench-methodology, build-pipeline-tasks - normalize specs and README typography to ASCII per code-data-formatting - rewrite root CLAUDE.md trigger table; add .cursorrules and .gitignore - specs: pipeline plan and task specs 00-11 - rules: 19 binding rule files adapted for this project - docs: CLAUDE.md rule-trigger table - config: .cursorrules commit convention, .gitignore excluding out/ and .venv/ - docs: rewrite README.md with pipeline diagrams, setup guide, result placeholders - config: requirements.txt for the closed dependency list - datagen: make_lab_config.py writes out/config/lab_config.yaml and .done marker
24 lines
1.3 KiB
Markdown
24 lines
1.3 KiB
Markdown
# 06 - Convert to PostgreSQL (indexed + partitioned)
|
||
|
||
## Context / Goal
|
||
Build the production-grade relational variant: the system-of-record candidate per the existing Storage ADR (PostgreSQL primary, RDF as deferred virtual layer).
|
||
|
||
## Dependencies
|
||
03.
|
||
|
||
## Input
|
||
CSV tree.
|
||
|
||
## Processing instructions
|
||
- Local PostgreSQL 16, database `tribo`. Same logical schema as task 05 with PostgreSQL types (`INT`, `REAL`, `TIMESTAMPTZ`, `TEXT`).
|
||
- `friction_cycles` and `friction_loop_points`: declarative RANGE partitioning by `track_id` (12 partitions of 120 tracks) - models the partitioning strategy needed at 600 GB scale.
|
||
- Load via `COPY FROM STDIN` (psycopg3 `copy`), not INSERT.
|
||
- Indexes after load: PK b-trees; `tracks(environment, load_mn)`, `coupons(au_wtpct_mean)`, `nanoindentation(coupon_id)`, BRIN on `friction_cycles(cycle)` per partition.
|
||
- `track_summary` as a materialized view (`cof_ss_mean` = avg of cycles > run_in threshold; `run_in_cycles` via first cycle where |cof - cof_ss| < 2σ).
|
||
- `VACUUM ANALYZE` after load. Record `pg_total_relation_size` per table and total into `storage_sizes.csv`.
|
||
- Also export a compressed dump `./out/pg/tribo.dump` (pg_dump -Fc) for archiving-format comparison.
|
||
|
||
## Output
|
||
- Live `tribo` database + `./out/pg/tribo.dump`.
|
||
- Marker `./out/.done/06.ok`.
|