sceCesRefersUcsProfileCp858 threw. It now returns a single-byte profile
accepted by sceCesSbcToUtf8, sceCesUtf8ToSbc and the MBCS context
conversions. Code page 858 is code page 850 with byte 0xD5 mapped to the
euro sign (U+20AC) instead of U+0131; that is the whole high half of
ICU's ibm-858_P100-1997.ucm. Bytes 0x00-0x7F stay unchanged, as in the
other single-byte profiles.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FzhWRMauD3ouc8kSxT2NnT
Brings in main with #9204. The measured CP932 and CP51932 encoding rules (JIS form aliases, U+FEFF, the C1 range and U+000E) apply to those profiles only; the strict JIS X 0208 profiles keep #9204's tables.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Keep #9204's strict JIS X 0208 profiles with their SS2 and SS3 choices inside the new helpers: decodeEucJp sends the measured CP51932 profile to decodeEucJpScalar and the strict profiles to #9204's rules, and only the X0208Ss2Ss3 profile encodes JIS X 0212. The EUC-KR, UHC and Big5 UTF-8 forms decode with decodeScalarUtf8.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Implement sceCesUcsProfileInitGb18030_2000 and sceCesUcsProfileInitGb18030_2005.
Two-byte cells are the CP936 table plus the cells that differ, and four-byte
sequences are runs over the BMP and the supplementary planes, generated from
ICU's gb-18030-2000.ucm (before ICU-8427) and gb-18030-2005.ucm with ICU's
GB 18030 ranges. The GB, MBC and MBCS string conversions accept both profiles
and produce four-byte UTF-8 and surrogate pairs for supplementary characters.
Implement sceCesUcsProfileInitEucJpX0208, sceCesUcsProfileInitEucJpX0208Ss2,
sceCesUcsProfileInitEucJpX0208Ss2Ss3 and sceCesUcsProfileInitSJis1997X0208.
The JIS X 0208 table is JIS0208.TXT, derived from the CP932 table without
rows 13 and 89-92 and with six cells changed. The EUC-JP variants add SS2
half-width katakana and SS3 JIS X 0212 as their names say. Cell 0x2140 and
the Shift-JIS single bytes 0x5C and 0x7E, where the sources disagree, throw.
The UTF-16 and UTF-32 string forms and the UTF-8 to Shift-JIS and EUC-JP
calls now share the measured string loop and UTF-8 decoder, and the
Japanese tests expect JIS X 0212 to be neither decoded nor encoded.
sceCesMbcsUcsContextInit rejects a null profile.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
sceCesRefersUcsProfile threw for every code page except 1250, 1251 and
1252. Twenty-seven more now return a profile that sceCesSbcToUtf8,
sceCesUtf8ToSbc and the MBCS context conversions accept: ISO 8859-1,
-2, -4, -5, -9, -10, -13, -14, -15 and -16, code pages 437, 737, 775,
850, 852, 855, 860, 861, 862, 863, 865 and 866, KOI8-R, KOI8-U and
Windows 1254, 1256 and 1258.
SingleByte.hpp holds the high half of each table, taken from the
public mapping files at unicode.org: ISO8859, Microsoft's PC tables,
VENDORS/MISC for KOI8 and Microsoft's best-fit tables for 1254, 1256
and 1258, the same source and derivation as the existing Cp1250 table,
whose high half equals bestfit1250's byte table. Bytes 0x00-0x7F are
unchanged in all of them, and every table maps its 256 bytes to 256
distinct BMP characters. The three existing profiles keep their
tables; one lookup now serves them all.
Profiles whose public table leaves bytes undefined (ISO 8859-3, -6, -7,
-8, -11, code pages 857, 864, 869), maps bytes to private use (874,
1253, 1255, 1257) or has no public table here still throw.
guest_ces checks that each new profile is a distinct, stable pointer,
four bytes per table against the source, a round trip of every byte
through sceCesSbcToUtf8 and sceCesUtf8ToSbc, UNASSIGNED_CODE for a
character outside the table, and a KOI8-R string through an MBCS
context in both directions.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Add the GBK (CP936) and EUC-CN profiles, their scalar conversions to
and from UTF-8, UTF-16, UTF-32 and UCS-2, and accept both profiles in
the MBCS string conversions.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Implement sceCesSbcTo{Ucs2,Utf16,Utf32}* and their inverses, and the
UTF-16 and UTF-32 MBCS string conversions and length queries, on the
existing single-byte, CP932 and CP51932 profiles.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Implement the single-character conversions between the CP932 and
CP51932 profiles and UTF-8, UTF-16, UTF-32 and UCS-2 in both
directions, and the Shift-JIS to JIS X 0208 row/cell arithmetic.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Dean