[mypyc] Add support for str.lower() and str.upper()

[VaggelisD]

Fixes mypyc/mypyc#1088

Follow up on #19375 with full Unicode support:

• ASCII fast path
• Shared CPyStr_ChangeCase helper, parameterized by function pointers
• _PyUnicode_ToLowerFull/_PyUnicode_ToUpperFull for Unicode which handle 1-to-N expansion (e.g., ß→SS); This was a sticky point with the previous PR which relied on Py_UNICODE_TOLOWER/TOUPPER.
• Temporary len * 3 UCS-4 buffer for the Unicode path; This is because each Unicode char may be expanded from 1 byte to 3

Benchmarks (x10 M calls each)

Input	Method	mypyc	CPython	Speedup
"" (empty)	lower	0.070s	0.182s	2.60x
"" (empty)	upper	0.069s	0.181s	2.62x
ASCII len 1 (convert)	lower	0.187s	0.234s	1.25x
ASCII len 1 (convert)	upper	0.161s	0.246s	1.53x
ASCII len 1 (no-op)	lower	0.157s	0.234s	1.49x
ASCII len 1 (no-op)	upper	0.151s	0.245s	1.62x
ASCII len 10	lower	0.189s	0.267s	1.41x
ASCII len 10	upper	0.189s	0.278s	1.47x
ASCII len 100	lower	0.434s	0.510s	1.18x
ASCII len 100	upper	0.431s	0.527s	1.22x
Unicode len 1 (Latin)	lower	0.152s	0.309s	2.03x
Unicode len 1 (Latin)	upper	0.156s	0.320s	2.05x
Unicode len 10 (Latin)	lower	0.438s	0.543s	1.24x
Unicode len 10 (Latin)	upper	0.441s	0.530s	1.20x
Unicode len 100 (Latin)	lower	2.560s	2.897s	1.13x
Unicode len 100 (Latin)	upper	2.564s	2.001s	0.78x
Unicode len 1 (Greek)	lower	0.216s	0.304s	1.41x
Unicode len 1 (Greek)	upper	0.218s	0.313s	1.44x
Unicode len 10 (Greek)	lower	0.526s	0.587s	1.12x
Unicode len 10 (Greek)	upper	0.479s	0.478s	1.00x
1-to-N expansion len 1	lower	0.332s	0.318s	0.96x
1-to-N expansion len 1	upper	0.258s	0.323s	1.25x
1-to-N expansion len 10	lower	0.614s	0.655s	1.07x
1-to-N expansion len 10	upper	0.526s	0.542s	1.03x

[VaggelisD]

Oops, looks like the APIs I based the PR on are not exported past 3.13+, will need to figure out a solution. The str.lower() and str.upper() primitives continue to claim victims even after a year

[VaggelisD]

That was a mistake afaict, CPY_3_13_FEATURES is defined by us so that'd always evaluate to true

[VaggelisD]

I was testing this branch on Python 3.11; For 3.13+ I had to fallback to CPython's generic dispatch for Unicode strings.

This introduces the following regression for the latter versions (tested on 3.14)

Input	Method	mypyc	CPython	Speedup
"" (empty)	lower	0.118s	0.144s	1.22x
"" (empty)	upper	0.104s	0.143s	1.38x
ASCII len 1 (convert)	lower	0.200s	0.234s	1.17x
ASCII len 1 (convert)	upper	0.196s	0.232s	1.18x
ASCII len 1 (no-op)	lower	0.194s	0.224s	1.15x
ASCII len 1 (no-op)	upper	0.199s	0.229s	1.15x
ASCII len 10	lower	0.222s	0.256s	1.15x
ASCII len 10	upper	0.226s	0.256s	1.13x
ASCII len 100	lower	0.476s	0.499s	1.05x
ASCII len 100	upper	0.490s	0.510s	1.04x
Unicode len 1 (Latin)	lower	0.404s	0.305s	0.75x
Unicode len 1 (Latin)	upper	0.420s	0.313s	0.74x
Unicode len 10 (Latin)	lower	0.652s	0.544s	0.83x
Unicode len 10 (Latin)	upper	0.566s	0.481s	0.85x
Unicode len 1 (Greek)	lower	0.406s	0.308s	0.76x
Unicode len 1 (Greek)	upper	0.404s	0.303s	0.75x
1-to-N len 1	lower	0.425s	0.326s	0.77x
1-to-N len 1 (ß→SS)	upper	0.425s	0.326s	0.77x

[JukkaL]

Thanks for the PR! Can you also compare performance against mypyc master, on Python 3.14?

[JukkaL]

Since lower and upper calls are very common, we might eventually want to include lower case / upper case tables for at least UCS-2 within librt (that are only included in the build if the primitives are actually used, i.e. as optional files). But the ascii case is very common, so this is already helpful.

[VaggelisD]

I applied the inlining and reran the benchmarks:

• For our branch vs Python 3.14, it looks like the regressions disappeared, not entirely sure why, maybe inlining did all that magic (the previous ones were on Python 3.11)

• For our branch vs master, the results are logical i.e regression kicks in for large Unicode strings but other than that we're on par or faster.

  Input	Method	mypyc (branch)	mypyc (master)	Python	vs master	vs CPython
  "" (empty)	lower	0.063s	0.159s	0.211s	2.52x	3.35x
  "" (empty)	upper	0.062s	0.163s	0.206s	2.63x	3.32x
  ASCII len 1 (convert)	lower	0.157s	0.230s	0.260s	1.46x	1.66x
  ASCII len 1 (convert)	upper	0.150s	0.272s	0.272s	1.81x	1.81x
  ASCII len 1 (no-op)	lower	0.156s	0.229s	0.269s	1.47x	1.72x
  ASCII len 1 (no-op)	upper	0.150s	0.232s	0.272s	1.55x	1.81x
  ASCII len 10	lower	0.189s	0.260s	0.310s	1.38x	1.64x
  ASCII len 10	upper	0.185s	0.266s	0.302s	1.44x	1.63x
  ASCII len 100	lower	0.429s	0.548s	0.554s	1.28x	1.29x
  ASCII len 100	upper	0.436s	0.517s	0.557s	1.19x	1.28x
  ASCII len 100 (no-op)	lower	0.440s	0.512s	0.551s	1.16x	1.25x
  ASCII len 100 (no-op)	upper	0.442s	0.515s	0.563s	1.17x	1.27x
  Unicode len 1 (Latin)	lower	0.353s	0.354s	0.392s	1.00x	1.11x
  Unicode len 1 (Latin)	upper	0.356s	0.420s	0.380s	1.18x	1.07x
  Unicode len 10 (Latin)	lower	0.517s	0.579s	0.571s	1.12x	1.10x
  Unicode len 10 (Latin)	upper	0.523s	0.544s	0.586s	1.04x	1.12x
  Unicode len 100 (Latin)	lower	2.407s	2.476s	2.431s	1.03x	1.01x
  Unicode len 100 (Latin)	upper	2.587s	2.645s	2.637s	1.02x	1.02x
  Unicode len 1 (Greek)	lower	0.357s	0.353s	0.396s	0.99x	1.11x
  Unicode len 1 (Greek)	upper	0.362s	0.347s	0.392s	0.96x	1.08x
  Unicode len 10 (Greek)	lower	0.582s	0.564s	0.643s	0.97x	1.10x
  Unicode len 10 (Greek)	upper	0.577s	0.572s	0.672s	0.99x	1.16x
  1-to-N len 1 (İ)	lower	0.375s	0.364s	0.415s	0.97x	1.11x
  1-to-N len 10 (İ)	lower	0.641s	0.632s	0.676s	0.99x	1.05x
  1-to-N len 1 (ß→SS)	upper	0.330s	0.324s	0.370s	0.98x	1.12x
  1-to-N len 10 (ß)	upper	0.613s	0.609s	0.648s	0.99x	1.06x
  1-to-N len 1 (ffi→FFI)	upper	0.350s	0.341s	0.385s	0.97x	1.10x