# Code / Python style - tabs, 3.11+, closed dependency list, streaming

**Purpose + scope.** Baseline style and dependency policy for every Python
file in this repository (pipeline tasks 01-11, query implementations,
helper tools). Binding for all contributors.

---

## 1. Language and formatting

- **Python 3.11+.** Use modern stdlib freely (`pathlib`, `dataclasses`,
  `tomllib` if needed); no compatibility shims for older versions.
- **Indentation: TABS.** Set by the project owner and repeated in the
  specs (00_PLAN, 03, 09). Every Python file in this repository indents
  with tab characters, never spaces. Configure the editor per file type;
  do not "fix" tab-indented files to spaces.
- `from __future__ import annotations` at the top of every module.
- Type hints on public function signatures; internal helpers at author's
  discretion.
- English for all identifiers, docstrings, and comments (see
  [meta-language-style.md](meta-language-style.md) - Russian is the
  conversation language, not the code language).
- Comments follow [code-comments-policy.md](code-comments-policy.md):
  default to none, keep only non-obvious WHY.

## 2. Dependencies - a closed list

The stdlib is the default. The ONLY permitted third-party packages:

| Package | Used by | For |
|---|---|---|
| `numpy` | 03 | vectorized generation, seeded RNG |
| `PyYAML` | 01, 03, 10, 11 | reading/writing the YAML configs |
| `rdflib` | 04, 07 | JSON-LD validation, RDF serialization |
| `oxigraph` | 07, 09 | embedded queryable triplestore |
| `ijson` | 08, 09 | streaming JSON parsing |
| `psycopg` (v3) | 06, 08, 09 | PostgreSQL COPY and queries |
| `psutil` | 04-07, 09 | RSS / CPU / IO measurement; converters record their own peak RSS / CPU time into their `.done` markers for the task 10 sizing model |
| `matplotlib` | 11 | report charts |
| `pandas` | 08 (optional) | ONLY as the separately-measured second CSV query variant; never in the pipeline itself |

Adding any other package requires updating this table first. In
particular forbidden: ORMs, logging frameworks
([obs-python-logging.md](obs-python-logging.md)), CLI frameworks
(argparse suffices), `requests`/HTTP clients (nothing to call).

Task 03 (data generation) is stricter by spec: **stdlib + numpy only.**

## 3. Script shape - every task is a self-contained CLI

- One entry script per pipeline task, runnable as
  `python <script>.py [flags]`.
- `argparse` with `--log-level` (see
  [obs-python-logging.md](obs-python-logging.md)) and task-appropriate
  per-run flags ([code-config-yaml.md](code-config-yaml.md) section 4).
- A module docstring stating the task number, what it produces, and its
  spec file.
- `main() -> int` returning the exit code; `sys.exit(main())` under
  `if __name__ == "__main__":`.
- Exit code 0 only on full success (outputs written, validation passed,
  marker written).

## 4. Streaming discipline - the corpus never fits in memory by design

The generated corpus is ~150 MB and its scaled-up target is 600 GB; code
must behave as if the data never fits in RAM:

- **Write generated rows as you produce them**; never accumulate the full
  cycle/loop dataset in a list (spec 03: "stream rows; never hold the
  full cycle dataset in memory").
- **Read CSV/JSON with iterators** (`csv.reader` row-by-row, `ijson`
  events); never `read()` a bulk file whole or `json.load` the full
  corpus.
- Batch database work (`executemany` / `COPY` in chunks), keeping only
  the current batch in memory.
- Bounded aggregation state is fine (per-track summaries, counters);
  unbounded accumulation of raw rows is the thing being forbidden.

## 5. Anti-patterns

- **Spaces for indentation** in Python files - the project standard is
  tabs.
- **An unlisted third-party import** - extend the section 2 table first
  or use the stdlib.
- **`pd.read_csv` in pipeline code** - pandas exists only as the
  separately-measured Q-variant in spec 08.
- **Loading a bulk artifact whole** (`json.load` on the full corpus,
  `readlines()` on a cycles CSV) - stream it.
- **A task importable only from a specific CWD** - resolve paths from a
  repo-root anchor, not the current directory.
- **Two tasks sharing mutable module state** - tasks communicate through
  `./out/` artifacts only (see
  [build-pipeline-tasks.md](build-pipeline-tasks.md)).