fix(docs): fix formatting issues in README.md
- adapt meta-/code-/obs- rules to the local Python benchmark pipeline - replace db-sql-ddl, code-config-env-scope, test-e2e-pytest with pipeline equivalents - add code-python-style, data-determinism, data-naming-units, bench-methodology, build-pipeline-tasks - normalize specs and README typography to ASCII per code-data-formatting - rewrite root CLAUDE.md trigger table; add .cursorrules and .gitignore - specs: pipeline plan and task specs 00-11 - rules: 19 binding rule files adapted for this project - docs: CLAUDE.md rule-trigger table - config: .cursorrules commit convention, .gitignore excluding out/ and .venv/ - docs: rewrite README.md with pipeline diagrams, setup guide, result placeholders - config: requirements.txt for the closed dependency list - datagen: make_lab_config.py writes out/config/lab_config.yaml and .done marker
This commit is contained in:
administrator
132
docs/rules/code-data-formatting.md
Normal file
132
docs/rules/code-data-formatting.md
Normal file
@@ -0,0 +1,132 @@
|
||||
# Code / Data / Formatting - ASCII-only typographic characters in source files
|
||||
|
||||
**Binding scope.** Strings that end up in any of the following MUST use ASCII
|
||||
typographic characters; never their Unicode "smart" equivalents:
|
||||
|
||||
- Python source, SQL and SPARQL query files
|
||||
- Generated data and artifacts written to disk (CSV, JSON-LD, N-Triples,
|
||||
Turtle, MANIFEST and benchmark result CSVs, log messages)
|
||||
- Configuration files (`.yaml`, `.json`, `.toml`)
|
||||
- PowerShell / shell scripts (any file processed by Windows tooling)
|
||||
- Markdown documentation (`docs/`, `README.md`, `CLAUDE.md`) - the specs
|
||||
were normalized to ASCII typography on 2026-07-11 and stay that way
|
||||
|
||||
**Long dashes are an absolute exception, no carve-outs.** Em-dash (U+2014)
|
||||
and en-dash (U+2013) MUST be replaced with ASCII hyphen-minus (`-`) in
|
||||
**every** file in this repository, including markdown documentation, code
|
||||
comments, and report prose. Reason: long dashes are visually
|
||||
indistinguishable from a hyphen in many monospace fonts, break diffs / grep /
|
||||
IDE search, and round-trip badly through Windows tooling chains (section 2).
|
||||
|
||||
---
|
||||
|
||||
## 1. The substitution table
|
||||
|
||||
| Unicode char | Codepoint | Name | ASCII replacement |
|
||||
|---|---|---|---|
|
||||
| em-dash | U+2014 | em-dash | `-` (single hyphen-minus) |
|
||||
| en-dash | U+2013 | en-dash | `-` (single hyphen-minus) |
|
||||
| horizontal bar | U+2015 | horizontal bar | `-` |
|
||||
| minus sign | U+2212 | minus sign | `-` |
|
||||
| ellipsis | U+2026 | horizontal ellipsis | `...` (three ASCII dots) |
|
||||
| left double quote | U+201C | left double quotation mark | `"` |
|
||||
| right double quote | U+201D | right double quotation mark | `"` |
|
||||
| left single quote | U+2018 | left single quotation mark | `'` |
|
||||
| right single quote | U+2019 | right single quotation mark | `'` |
|
||||
| left guillemet | U+00AB | left guillemet | `"` |
|
||||
| right guillemet | U+00BB | right guillemet | `"` |
|
||||
| nbsp | U+00A0 | non-breaking space | regular space |
|
||||
|
||||
**Single hyphen, not double.** When replacing an em-dash that separates two
|
||||
clauses, use a single `-` surrounded by spaces (`text - clause`), NOT `--`.
|
||||
|
||||
**Ranges use a single hyphen:** `Q1-Q7`, `120-200 MB`, `0.3-1.1 um` - never
|
||||
an en-dash.
|
||||
|
||||
## 2. Why - documented failure modes
|
||||
|
||||
1. **PowerShell 5.1 reads no-BOM files as Windows-1252.** Non-ASCII UTF-8
|
||||
bytes become mojibake and break string-literal parsing. This project is
|
||||
developed on Windows; any script or config touched by PowerShell is
|
||||
exposed.
|
||||
|
||||
2. **Windows -> git-bash -> tar / packaging round-trip.** At any step an
|
||||
unset locale or wrong encoding assumption produces mojibake. A UTF-8 byte
|
||||
sequence for U+2014 interpreted under a non-UTF-8 locale renders as a
|
||||
`?` replacement character in the packaged output.
|
||||
|
||||
3. **Copy-paste from Word / Google Docs / Notion / web articles.** These
|
||||
tools auto-substitute typography (`"` -> smart quotes, `'` -> smart
|
||||
apostrophe, `--` -> em-dash, `...` -> ellipsis) silently. The
|
||||
substitution survives copy into any editor and lands in source files
|
||||
unnoticed. The original specs arrived with en/em dashes this way.
|
||||
|
||||
4. **Data files must diff and grep cleanly.** The CSV corpus, MANIFEST, and
|
||||
benchmark results are compared by checksum and processed by streaming
|
||||
parsers; a stray typographic character breaks byte-level reproducibility.
|
||||
|
||||
## 3. Where the rule does NOT apply
|
||||
|
||||
- **Scientific and unit notation is content, not typography.** Characters
|
||||
such as `µ` (micro), `±`, `°C`, `×`, `Ø`, and Greek letters (`τ`, `σ`) in
|
||||
formulas inside the specs are meaningful scientific notation and are kept.
|
||||
Note the boundary: in **column names and code identifiers** the units are
|
||||
always plain ASCII (`_um`, `_nm`, `_GPa` - see
|
||||
[data-naming-units.md](data-naming-units.md)); the scientific characters
|
||||
live only in explanatory prose.
|
||||
- **International language strings** (Cyrillic, CJK, etc.) are meaningful
|
||||
content, not typographic substitutions. UTF-8 throughout is the right
|
||||
answer. (Repository artifacts are English; this matters only for quoted
|
||||
material.)
|
||||
- **Imported reference materials** in `docs/examples/` are kept verbatim as
|
||||
received; they are not binding documents.
|
||||
|
||||
## 4. Detection / enforcement
|
||||
|
||||
There is no automated check for this rule yet - **not in CI yet**. The rule is
|
||||
enforced by review and by the editing discipline in this file. The intent of a
|
||||
future detector is documented here so it can be built consistently:
|
||||
|
||||
- **What it flags.** Any occurrence of the Unicode codepoints in the section 1
|
||||
substitution table inside in-scope files. The section 3 exemptions
|
||||
(scientific notation, international content, `docs/examples/`) are the only
|
||||
carve-outs.
|
||||
- **Exit behavior.** Non-zero exit when any flagged codepoint is found under
|
||||
the scan roots, so it can later gate a commit hook or a build step.
|
||||
- **Tooling.** When this detector is created it MUST be a PowerShell script
|
||||
(this is a Windows project) and live under `tools/`.
|
||||
|
||||
## 5. Examples
|
||||
|
||||
### Good
|
||||
|
||||
```python
|
||||
# safe through every Windows tooling step
|
||||
print("Corpus size 152.3 MB - within the 120-200 MB gate.")
|
||||
label = "steady-state COF, cycles 500-1000"
|
||||
```
|
||||
|
||||
### Bad
|
||||
|
||||
```python
|
||||
# em-dash literal (U+2014) - mojibake risk on PowerShell read/write
|
||||
print("Corpus size 152.3 MB [U+2014] within the gate.")
|
||||
|
||||
# en-dash range (U+2013) - breaks grep for "Q1-Q7"
|
||||
label = "queries Q1[U+2013]Q7"
|
||||
```
|
||||
|
||||
## 6. Anti-patterns
|
||||
|
||||
- **Copy-pasting strings from a word processor or web page** into source /
|
||||
specs / config without normalization.
|
||||
- **Trusting tar / packaging to "preserve as-is".** They preserve bytes; the
|
||||
*encoding interpretation* is what gets lost when locale or encoding
|
||||
declaration is wrong on either end.
|
||||
- **Assuming the source file encoding alone is enough.** The source file may
|
||||
be fine - the failure is at downstream consumers (PowerShell read, build
|
||||
write, OS locale).
|
||||
- **Using `--` instead of `-` "for emphasis".** Single ASCII hyphen
|
||||
surrounded by spaces is the project's chosen separator.
|
||||
- **Putting `µ` or `°` into a column name or identifier** - units in names
|
||||
are ASCII (`_um`, `_C`); see [data-naming-units.md](data-naming-units.md).
|
||||
Reference in New Issue
Block a user