b173ac82a9
- adapt meta-/code-/obs- rules to the local Python benchmark pipeline - replace db-sql-ddl, code-config-env-scope, test-e2e-pytest with pipeline equivalents - add code-python-style, data-determinism, data-naming-units, bench-methodology, build-pipeline-tasks - normalize specs and README typography to ASCII per code-data-formatting - rewrite root CLAUDE.md trigger table; add .cursorrules and .gitignore - specs: pipeline plan and task specs 00-11 - rules: 19 binding rule files adapted for this project - docs: CLAUDE.md rule-trigger table - config: .cursorrules commit convention, .gitignore excluding out/ and .venv/ - docs: rewrite README.md with pipeline diagrams, setup guide, result placeholders - config: requirements.txt for the closed dependency list - datagen: make_lab_config.py writes out/config/lab_config.yaml and .done marker
133 lines
6.0 KiB
Markdown
133 lines
6.0 KiB
Markdown
# Code / Data / Formatting - ASCII-only typographic characters in source files
|
||
|
||
**Binding scope.** Strings that end up in any of the following MUST use ASCII
|
||
typographic characters; never their Unicode "smart" equivalents:
|
||
|
||
- Python source, SQL and SPARQL query files
|
||
- Generated data and artifacts written to disk (CSV, JSON-LD, N-Triples,
|
||
Turtle, MANIFEST and benchmark result CSVs, log messages)
|
||
- Configuration files (`.yaml`, `.json`, `.toml`)
|
||
- PowerShell / shell scripts (any file processed by Windows tooling)
|
||
- Markdown documentation (`docs/`, `README.md`, `CLAUDE.md`) - the specs
|
||
were normalized to ASCII typography on 2026-07-11 and stay that way
|
||
|
||
**Long dashes are an absolute exception, no carve-outs.** Em-dash (U+2014)
|
||
and en-dash (U+2013) MUST be replaced with ASCII hyphen-minus (`-`) in
|
||
**every** file in this repository, including markdown documentation, code
|
||
comments, and report prose. Reason: long dashes are visually
|
||
indistinguishable from a hyphen in many monospace fonts, break diffs / grep /
|
||
IDE search, and round-trip badly through Windows tooling chains (section 2).
|
||
|
||
---
|
||
|
||
## 1. The substitution table
|
||
|
||
| Unicode char | Codepoint | Name | ASCII replacement |
|
||
|---|---|---|---|
|
||
| em-dash | U+2014 | em-dash | `-` (single hyphen-minus) |
|
||
| en-dash | U+2013 | en-dash | `-` (single hyphen-minus) |
|
||
| horizontal bar | U+2015 | horizontal bar | `-` |
|
||
| minus sign | U+2212 | minus sign | `-` |
|
||
| ellipsis | U+2026 | horizontal ellipsis | `...` (three ASCII dots) |
|
||
| left double quote | U+201C | left double quotation mark | `"` |
|
||
| right double quote | U+201D | right double quotation mark | `"` |
|
||
| left single quote | U+2018 | left single quotation mark | `'` |
|
||
| right single quote | U+2019 | right single quotation mark | `'` |
|
||
| left guillemet | U+00AB | left guillemet | `"` |
|
||
| right guillemet | U+00BB | right guillemet | `"` |
|
||
| nbsp | U+00A0 | non-breaking space | regular space |
|
||
|
||
**Single hyphen, not double.** When replacing an em-dash that separates two
|
||
clauses, use a single `-` surrounded by spaces (`text - clause`), NOT `--`.
|
||
|
||
**Ranges use a single hyphen:** `Q1-Q7`, `120-200 MB`, `0.3-1.1 um` - never
|
||
an en-dash.
|
||
|
||
## 2. Why - documented failure modes
|
||
|
||
1. **PowerShell 5.1 reads no-BOM files as Windows-1252.** Non-ASCII UTF-8
|
||
bytes become mojibake and break string-literal parsing. This project is
|
||
developed on Windows; any script or config touched by PowerShell is
|
||
exposed.
|
||
|
||
2. **Windows -> git-bash -> tar / packaging round-trip.** At any step an
|
||
unset locale or wrong encoding assumption produces mojibake. A UTF-8 byte
|
||
sequence for U+2014 interpreted under a non-UTF-8 locale renders as a
|
||
`?` replacement character in the packaged output.
|
||
|
||
3. **Copy-paste from Word / Google Docs / Notion / web articles.** These
|
||
tools auto-substitute typography (`"` -> smart quotes, `'` -> smart
|
||
apostrophe, `--` -> em-dash, `...` -> ellipsis) silently. The
|
||
substitution survives copy into any editor and lands in source files
|
||
unnoticed. The original specs arrived with en/em dashes this way.
|
||
|
||
4. **Data files must diff and grep cleanly.** The CSV corpus, MANIFEST, and
|
||
benchmark results are compared by checksum and processed by streaming
|
||
parsers; a stray typographic character breaks byte-level reproducibility.
|
||
|
||
## 3. Where the rule does NOT apply
|
||
|
||
- **Scientific and unit notation is content, not typography.** Characters
|
||
such as `µ` (micro), `±`, `°C`, `×`, `Ø`, and Greek letters (`τ`, `σ`) in
|
||
formulas inside the specs are meaningful scientific notation and are kept.
|
||
Note the boundary: in **column names and code identifiers** the units are
|
||
always plain ASCII (`_um`, `_nm`, `_GPa` - see
|
||
[data-naming-units.md](data-naming-units.md)); the scientific characters
|
||
live only in explanatory prose.
|
||
- **International language strings** (Cyrillic, CJK, etc.) are meaningful
|
||
content, not typographic substitutions. UTF-8 throughout is the right
|
||
answer. (Repository artifacts are English; this matters only for quoted
|
||
material.)
|
||
- **Imported reference materials** in `docs/examples/` are kept verbatim as
|
||
received; they are not binding documents.
|
||
|
||
## 4. Detection / enforcement
|
||
|
||
There is no automated check for this rule yet - **not in CI yet**. The rule is
|
||
enforced by review and by the editing discipline in this file. The intent of a
|
||
future detector is documented here so it can be built consistently:
|
||
|
||
- **What it flags.** Any occurrence of the Unicode codepoints in the section 1
|
||
substitution table inside in-scope files. The section 3 exemptions
|
||
(scientific notation, international content, `docs/examples/`) are the only
|
||
carve-outs.
|
||
- **Exit behavior.** Non-zero exit when any flagged codepoint is found under
|
||
the scan roots, so it can later gate a commit hook or a build step.
|
||
- **Tooling.** When this detector is created it MUST be a PowerShell script
|
||
(this is a Windows project) and live under `tools/`.
|
||
|
||
## 5. Examples
|
||
|
||
### Good
|
||
|
||
```python
|
||
# safe through every Windows tooling step
|
||
print("Corpus size 152.3 MB - within the 120-200 MB gate.")
|
||
label = "steady-state COF, cycles 500-1000"
|
||
```
|
||
|
||
### Bad
|
||
|
||
```python
|
||
# em-dash literal (U+2014) - mojibake risk on PowerShell read/write
|
||
print("Corpus size 152.3 MB [U+2014] within the gate.")
|
||
|
||
# en-dash range (U+2013) - breaks grep for "Q1-Q7"
|
||
label = "queries Q1[U+2013]Q7"
|
||
```
|
||
|
||
## 6. Anti-patterns
|
||
|
||
- **Copy-pasting strings from a word processor or web page** into source /
|
||
specs / config without normalization.
|
||
- **Trusting tar / packaging to "preserve as-is".** They preserve bytes; the
|
||
*encoding interpretation* is what gets lost when locale or encoding
|
||
declaration is wrong on either end.
|
||
- **Assuming the source file encoding alone is enough.** The source file may
|
||
be fine - the failure is at downstream consumers (PowerShell read, build
|
||
write, OS locale).
|
||
- **Using `--` instead of `-` "for emphasis".** Single ASCII hyphen
|
||
surrounded by spaces is the project's chosen separator.
|
||
- **Putting `µ` or `°` into a column name or identifier** - units in names
|
||
are ASCII (`_um`, `_C`); see [data-naming-units.md](data-naming-units.md).
|