# 00_PLAN - Tribology Lab Data Storage Evaluation Pipeline

> Provenance: this plan and the 11 task specifications were generated from
> [../Initial_Prompt.md](../Initial_Prompt.md). The prompt is a historical
> artifact, not a contract; the specs and [docs/rules/](../rules/) supersede
> it wherever they differ.

## Goal
Build a complete, reproducible pipeline that:
1. Defines a simulated tribology laboratory (Sandia Pt-Au LDRD context) with real instruments and a realistic test workflow.
2. Generates ~150 MB of physically plausible measurement data in CSV (the canonical raw format).
3. Converts the CSV corpus into 4 target storage formats: JSON-LD, SQLite (optimized), PostgreSQL (indexed + partitioned), RDF TripleStore.
4. Benchmarks 7 laboratory-standard retrieval scenarios (Q1-Q7) against all 5 formats, measuring wall time, peak RAM, CPU utilization, disk I/O, and disk footprint.
5. Extrapolates measured results to 600 GB and beyond (0.6 / 1.2 / 6 TB) using complexity-aware scaling models.
6. Produces summary tables and charts that clearly rank storage formats for laboratory use (analysis, reporting, search/filter, archiving, inter-lab exchange).

## Execution environment
- Claude Code agent inside Cursor, local filesystem access, on-premise only (no cloud services).
- Python 3.11+ (tabs for indentation), PostgreSQL 16 local instance, SQLite via stdlib `sqlite3`, `rdflib` for RDF, optional local triplestore (Jena Fuseki / GraphDB Free) - if unavailable, use `rdflib` + on-disk `oxigraph` (pip) as the queryable store.
- All artifacts written under `./out/` relative to repo root:
 - `./out/csv/` `./out/json/` `./out/sqlite/` `./out/pg/` (dumps) `./out/rdf/` `./out/bench/` `./out/report/`

## Task files and dependencies
```mermaid
flowchart TD
    T01[01_lab_configuration] --> T02[02_process_flow_diagrams]
    T01 --> T03[03_csv_data_generation]
    T03 --> T04[04_convert_json]
    T03 --> T05[05_convert_sqlite]
    T03 --> T06[06_convert_postgresql]
    T03 --> T07[07_convert_rdf]
    T04 --> T08[08_benchmark_queries]
    T05 --> T08
    T06 --> T08
    T07 --> T08
    T03 --> T08
    T08 --> T09[09_benchmark_execution]
    T09 --> T10[10_extrapolation_model]
    T09 --> T11[11_reporting]
    T10 --> T11
```

## Parallelism
- T04-T07 are independent; run in parallel after T03.
- T02 is independent of data generation; can run any time after T01.

## Execution order
01 → 02 → 03 → (04 | 05 | 06 | 07) → 08 → 09 → 10 → 11

## Global conventions
- IDs: `batch:B721`, `wafer:B721-W1`, `coupon:B721-W1-C07`, `track:B721-W1-C07-T2`, run IDs `run:R1..R4`.
- All timestamps ISO-8601 UTC. All units SI-explicit in column names (`_mN`, `_um`, `_GPa`, `_wtpct`, `_nm`).
- Random seed fixed at 20260711 for full reproducibility.
- Python indentation: tabs.
- Every task writes a completion marker `./out/.done/<task>.ok` containing row/byte counts for downstream validation.