LabDataStorageEvaluation/docs/rules/code-python-style.md

Code / Python style - tabs, 3.11+, closed dependency list, streaming

Purpose + scope. Baseline style and dependency policy for every Python file in this repository (pipeline tasks 01-11, query implementations, helper tools). Binding for all contributors.

---

1. Language and formatting

• Python 3.11+. Use modern stdlib freely (pathlib, dataclasses, tomllib if needed); no compatibility shims for older versions.
• Indentation: TABS. Set by the project owner and repeated in the specs (00_PLAN, 03, 09). Every Python file in this repository indents with tab characters, never spaces. Configure the editor per file type; do not "fix" tab-indented files to spaces.
• from __future__ import annotations at the top of every module.
• Type hints on public function signatures; internal helpers at author's discretion.
• English for all identifiers, docstrings, and comments (see meta-language-style.md - Russian is the conversation language, not the code language).
• Comments follow code-comments-policy.md: default to none, keep only non-obvious WHY.

2. Dependencies - a closed list

The stdlib is the default. The ONLY permitted third-party packages:

Package	Used by	For
numpy	03	vectorized generation, seeded RNG
PyYAML	01, 03, 10	reading/writing the YAML configs
rdflib	04, 07	JSON-LD validation, RDF serialization
oxigraph	07, 09	embedded queryable triplestore
ijson	08, 09	streaming JSON parsing
psycopg (v3)	06, 08, 09	PostgreSQL COPY and queries
psutil	04-07, 09	RSS / CPU / IO measurement; converters record their own peak RSS / CPU time into their .done markers for the task 10 sizing model
matplotlib	11	report charts
pandas	08 (optional)	ONLY as the separately-measured second CSV query variant; never in the pipeline itself

Adding any other package requires updating this table first. In particular forbidden: ORMs, logging frameworks (obs-python-logging.md), CLI frameworks (argparse suffices), requests/HTTP clients (nothing to call).

Task 03 (data generation) is stricter by spec: stdlib + numpy only.

3. Script shape - every task is a self-contained CLI

• One entry script per pipeline task, runnable as python <script>.py [flags].
• argparse with --log-level (see obs-python-logging.md) and task-appropriate per-run flags (code-config-yaml.md section 4).
• A module docstring stating the task number, what it produces, and its spec file.
• main() -> int returning the exit code; sys.exit(main()) under if __name__ == "__main__":.
• Exit code 0 only on full success (outputs written, validation passed, marker written).

4. Streaming discipline - the corpus never fits in memory by design

The generated corpus is ~150 MB and its scaled-up target is 600 GB; code must behave as if the data never fits in RAM:

• Write generated rows as you produce them; never accumulate the full cycle/loop dataset in a list (spec 03: "stream rows; never hold the full cycle dataset in memory").
• Read CSV/JSON with iterators (csv.reader row-by-row, ijson events); never read() a bulk file whole or json.load the full corpus.
• Batch database work (executemany / COPY in chunks), keeping only the current batch in memory.
• Bounded aggregation state is fine (per-track summaries, counters); unbounded accumulation of raw rows is the thing being forbidden.

5. Anti-patterns

• Spaces for indentation in Python files - the project standard is tabs.
• An unlisted third-party import - extend the section 2 table first or use the stdlib.
• pd.read_csv in pipeline code - pandas exists only as the separately-measured Q-variant in spec 08.
• Loading a bulk artifact whole (json.load on the full corpus, readlines() on a cycles CSV) - stream it.
• A task importable only from a specific CWD - resolve paths from a repo-root anchor, not the current directory.
• Two tasks sharing mutable module state - tasks communicate through ./out/ artifacts only (see build-pipeline-tasks.md).