Repository navigation
Add packagename.Normalize as the PEP 503 entry point - #68
Conversation
| {"ΑΣ\u0301", "ας\u0301"}, | ||
| {"Α\u0301Σ", "α\u0301ς"}, | ||
| {"ΑΣ\u0301Β", "ασ\u0301β"}, | ||
| {"ΣΑΣ_ΣΑΣ", "σας-σας"}, |
There was a problem hiding this comment.
⚪ Low
Only ., :, ' and U+0301 exercise isCaseIgnorable; the other hand-listed Word_Break runes (U+00B7, U+0387, U+2019, U+FF0E, …) and the Other_Lowercase branch of isCased have no case, and FuzzNormalize's reference skips Σ. A typo in one of those runes would pass every test.
A few Python-checked cases would cover it, e.g. {"Α·Σ", "α·ς"}, {"ΑΣ·Β", "ασ·β"}, {"ΑΣ’Β", "ασ’β"}, {"ªΣ", "ªς"}, {"ʰΣ", "ʰσ"} (expected values derived by hand, so please confirm under CPython 3.13).
| ) | ||
|
|
||
| // TestNormalizeMatchesExtras checks that the PEP 503 and PEP 685 entry points | ||
| // agree, including on non-ASCII and invalid names. The transformation itself is covered by |
There was a problem hiding this comment.
⚪ Low
The comment says agreement is checked on invalid names, but inputs has no invalid UTF-8, İ or Σ (the only non-ASCII input is ÄBC_def). Either add "A\xff_B", "İstanbul" and "ΑΣ.Β", or drop "and invalid".
|
|
||
| import "github.com/posit-dev/go-python-packaging/internal/canonname" | ||
|
|
||
| // Normalize canonicalizes a Python project name per PEP 503: lowercase the |
There was a problem hiding this comment.
⚪ Low · tentative
"per PEP 503" isn't exact: PEP 503's formula is re.sub(r"[-_.]+", "-", name).lower() (collapse, then lowercase), while this follows canonicalize_name (lowercase, then collapse). They differ for a final sigma next to .: ΑΣ.Β gives ασ-β here and ας-β under the PEP 503 formula.
Consider naming canonicalize_name as the contract and noting the difference.
jonyoder
left a comment
There was a problem hiding this comment.
Approving. The inline notes are all low; the CHANGELOG wording is the one worth fixing before the next tag.
Summary
packagenamepackage withNormalize, the PEP 503 entry point for project-name normalization. Callers no longer have to useextras.Normalizefor project names, which looked like a bug at the call site. Closes Add a PEP 503-named entry point for project-name normalization #25.internal/canonname.extras.Normalizeandpackagename.Normalizeare thin wrappers, so they cannot drift apart.canonicalize_name, including non-ASCII. Lowercasing now follows Python'sstr.lower():Ä→ä,İ→i+ U+0307, andΣ→ςat the end of a word (checked on the original name, as upstream does, soΑΣ.Β→ασ-βbutΑΣ_Β→ας-β). Invalid UTF-8 bytes pass through unchanged instead of becoming U+FFFD. Already-normalized names return without allocating, and there is still no regex.Connected to: #25
Behavior change
extras.Normalizereturns different output for non-ASCII and invalid UTF-8 input. Valid PEP 508 names are ASCII, so their results do not change. This is listed under Fixed inCHANGELOG.md.Go 1.27.2 uses Unicode 17.0 and CPython 3.13 uses 15.1, so the final-sigma rule can differ for characters that were added or reclassified after 15.1.
Test Plan
go build ./...,go vet ./...,go test ./...gofmt -l .prints nothing;golangci-lint config verifyandgolangci-lint run ./...report 0 issuesgo list -deps ./...does not containpgregory.net/rapidinternal/canonname: base table, Python-derived Unicode and final-sigma tables, invalid UTF-8,isNormalized, zero allocations on the fast path, andFuzzNormalize(60 s run, no failures)packagename: pypatest_canonicalize_nameconformance cases, plus a test that it agrees withextras.Normalizecanonicalize_name(pinned4eb0753) under CPython 3.13:ʕU+0295 and U+1171E)