Files

administrator b173ac82a9 fix(docs): fix formatting issues in README.md

- adapt meta-/code-/obs- rules to the local Python benchmark pipeline
- replace db-sql-ddl, code-config-env-scope, test-e2e-pytest with pipeline equivalents
- add code-python-style, data-determinism, data-naming-units, bench-methodology, build-pipeline-tasks
- normalize specs and README typography to ASCII per code-data-formatting
- rewrite root CLAUDE.md trigger table; add .cursorrules and .gitignore
- specs: pipeline plan and task specs 00-11
- rules: 19 binding rule files adapted for this project
- docs: CLAUDE.md rule-trigger table
- config: .cursorrules commit convention, .gitignore excluding out/ and .venv/
- docs: rewrite README.md with pipeline diagrams, setup guide, result placeholders
- config: requirements.txt for the closed dependency list
- datagen: make_lab_config.py writes out/config/lab_config.yaml and .done marker

2026-07-11 13:39:13 -04:00

1.5 KiB

Raw Blame History

09 - Benchmark Execution & Resource Measurement

Context / Goal

Execute all 35 (Q1-Q7 × 5) benchmark cells with rigorous resource metering; produce the raw measurement table that feeds extrapolation and reporting.

Dependencies

Processing instructions

Write bench_runner.py (tabs):

Each cell runs as a separate subprocess so RSS/CPU are attributable. For PostgreSQL, also sample the backend PID via pg_stat_activity + psutil.
Metrics per run: wall time (s), peak RSS (MB, psutil memory_info().rss sampled at 50 ms), CPU time user+sys (s), CPU utilization % (cpu_time / wall / cores), read bytes (psutil io_counters), result row count, checksum-match flag.
Protocol: 1 warm-up run (discarded) + 3 measured runs per cell; report median and min/max. Randomize cell order to spread cache effects; record run order.
Cold-cache variant: repeat the full matrix once after dropping caches if permitted (sync; echo 3 > /proc/sys/vm/drop_caches requires root - if not available, skip and note it).
Failures/timeouts: hard timeout 30 min per run; record TIMEOUT and continue (expected possibility: RDF Q5).
Storage footprint: consolidate storage_sizes.csv (all formats, incl. gzip-compressed archival sizes of each: tar.gz of CSV tree, gzip JSON, sqlite db gz, pg dump, nt.gz).

Output

./out/bench/results_raw.csv: format, query, run, wall_s, peak_rss_mb, cpu_s, cpu_util_pct, read_mb, rows, checksum_ok.
./out/bench/results_median.csv - aggregated.
./out/bench/storage_sizes.csv - final consolidated.
Marker ./out/.done/09.ok.

1.5 KiB Raw Blame History Unescape Escape