Files
LabDataStorageEvaluation/docs/specs_old/13_export_postgresql.md
administrator b173ac82a9 fix(docs): fix formatting issues in README.md
- adapt meta-/code-/obs- rules to the local Python benchmark pipeline
- replace db-sql-ddl, code-config-env-scope, test-e2e-pytest with pipeline equivalents
- add code-python-style, data-determinism, data-naming-units, bench-methodology, build-pipeline-tasks
- normalize specs and README typography to ASCII per code-data-formatting
- rewrite root CLAUDE.md trigger table; add .cursorrules and .gitignore
- specs: pipeline plan and task specs 00-11
- rules: 19 binding rule files adapted for this project
- docs: CLAUDE.md rule-trigger table
- config: .cursorrules commit convention, .gitignore excluding out/ and .venv/
- docs: rewrite README.md with pipeline diagrams, setup guide, result placeholders
- config: requirements.txt for the closed dependency list
- datagen: make_lab_config.py writes out/config/lab_config.yaml and .done marker
2026-07-11 13:39:13 -04:00

1.0 KiB

Task 13 — Export: PostgreSQL

Goal

System-of-record load package: DDL + COPY-based bulk load (per ADR ingestion path).

Input

  • canonical/ snapshot, schema/ddl_postgresql.sql.

Actions

  1. Generate exports/postgresql/:
    • 01_schema.sql (DDL, versioned);
    • 02_lookups.sql (atmospheres, measurement_types, column_definitions, instruments);
    • per-table CSV staging files from canonical (streamed);
    • 03_copy_load.sql (\copy statements in dependency order);
    • 04_post_load.sql (indexes, constraints validation, ANALYZE);
    • load.sh / load.ps1 orchestration.
  2. Point tables loaded via COPY; headers via INSERT; partitioning DDL for friction_data_points per Phase-1 volume estimates.
  3. Include lineage table load (row-range granularity).

Intermediate Result (done when)

  • Package executes cleanly on an empty database (test locally if a PG instance is available; otherwise dry-validate SQL syntax).
  • Post-load counts script confirms parity with canonical. EXPORT_MANIFEST.json written.