Commit Graph
52 Commits
Author SHA1 Message Date
Dean 2a16d79fb9 Merge pull request #10722 from rooklz/fix/cescs-error-length
fix(libSceCesCs): report the length of a rejected MBCS character as the console does
2026-10-26 12:10:41 +13:00
Rookie 8e08731a36 fix(libSceCesCs): report the length of a rejected MBCS character as the console does 2026-10-25 13:10:08 +00:00
Rookie 22e5cd7e1a fix(libSceCesCs): decode KS X 1001 cell 0xA2E8 as U+327E as the console does 2026-10-25 12:46:16 +00:00
Rookie 12e3082520 feat(libSceCesCs): encode UCS-2, UTF-8, UTF-16 and UTF-32 characters into any MBCS profile 2026-10-25 01:57:27 +00:00
orezraeyandClaude Opus 5.5 76e43cade8 Merge fix/libscecescs-mbcs-strings-measured-ps5 into fix/libscecescs-jis-encode-measured-ps5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 19:13:28 -03:00
orezraeyandClaude Opus 5.5 67ef849d7c Merge fix/libscecescs-scalar-measured-ps5 into fix/libscecescs-mbcs-strings-measured-ps5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 19:08:57 -03:00
orezraeyandClaude Opus 5.5 a349c9dcd2 Merge main into fix/libscecescs-scalar-measured-ps5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 19:03:24 -03:00
orezraeyandClaude Opus 5.5 26090c534a Merge fix/libscecescs-mbcs-strings-measured-ps5 into fix/libscecescs-jis-encode-measured-ps5
Brings in main with #9204. The measured CP932 and CP51932 encoding rules (JIS form aliases, U+FEFF, the C1 range and U+000E) apply to those profiles only; the strict JIS X 0208 profiles keep #9204's tables.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 17:41:53 -03:00
orezraeyandClaude Opus 5.5 cce3beb67a Merge main into fix/libscecescs-mbcs-strings-measured-ps5
Keep #9204's strict JIS X 0208 profiles with their SS2 and SS3 choices inside the new helpers: decodeEucJp sends the measured CP51932 profile to decodeEucJpScalar and the strict profiles to #9204's rules, and only the X0208Ss2Ss3 profile encodes JIS X 0212. The EUC-KR, UHC and Big5 UTF-8 forms decode with decodeScalarUtf8.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 17:39:22 -03:00
Rookie b62b28831d feat(libSceCesCs): add the GB18030-2000 and GB18030-2005 profiles
Implement sceCesUcsProfileInitGb18030_2000 and sceCesUcsProfileInitGb18030_2005.
Two-byte cells are the CP936 table plus the cells that differ, and four-byte
sequences are runs over the BMP and the supplementary planes, generated from
ICU's gb-18030-2000.ucm (before ICU-8427) and gb-18030-2005.ucm with ICU's
GB 18030 ranges. The GB, MBC and MBCS string conversions accept both profiles
and produce four-byte UTF-8 and surrogate pairs for supplementary characters.
2026-10-23 21:16:05 +01:00
Rookie c345ed42d2 feat(libSceCesCs): add the strict JIS X 0208 EUC-JP and Shift-JIS profiles
Implement sceCesUcsProfileInitEucJpX0208, sceCesUcsProfileInitEucJpX0208Ss2,
sceCesUcsProfileInitEucJpX0208Ss2Ss3 and sceCesUcsProfileInitSJis1997X0208.
The JIS X 0208 table is JIS0208.TXT, derived from the CP932 table without
rows 13 and 89-92 and with six cells changed. The EUC-JP variants add SS2
half-width katakana and SS3 JIS X 0212 as their names say. Cell 0x2140 and
the Shift-JIS single bytes 0x5C and 0x7E, where the sources disagree, throw.
2026-10-23 19:33:21 +01:00
orezraeyandClaude Opus 5.5 556b195d59 Merge main into fix/libscecescs-jis-encode-measured-ps5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 09:56:08 -03:00
orezraeyandClaude Opus 5.5 e4ed848821 Merge main into fix/libscecescs-mbcs-strings-measured-ps5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 09:55:06 -03:00
orezraeyandClaude Opus 5.5 8e10ddbb5c Merge main into fix/libscecescs-scalar-measured-ps5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 09:54:01 -03:00
jacob6769 509243a7e1 Merge remote-tracking branch 'origin/main' into feat/ces-korean-big5-scalars
# Conflicts:
#	docs/dev/debt/libSceCesCs.md
2026-10-23 21:52:29 +09:00
orezraey 567b164839 Merge branch 'fix/libscecescs-mbcs-strings-measured-ps5' into fix/libscecescs-jis-encode-measured-ps5 2026-10-23 09:09:26 -03:00
orezraey ab5ab9a3c5 Merge branch 'fix/libscecescs-scalar-measured-ps5' into fix/libscecescs-mbcs-strings-measured-ps5
# Conflicts:
#	core/libs/prx/libSceCesCs/Export.cpp
2026-10-23 09:08:30 -03:00
orezraey b41fdbce97 Merge remote-tracking branch 'origin/main' into fix/libscecescs-scalar-measured-ps5
# Conflicts:
#	docs/dev/debt/libSceCesCs.md
2026-10-23 09:05:18 -03:00
jacob6769 0f54186f5a Merge remote-tracking branch 'origin/main' into feat/ces-korean-big5-scalars
# Conflicts:
#	core/libs/prx/libSceCesCs/Export.cpp
#	core/libs/prx/libSceCesCs/PendingExports.cpp
#	docs/dev/debt/libSceCesCs.md
2026-10-23 20:34:04 +09:00
orezraeyandClaude Opus 5.5 d3f7b11e85 docs(libSceCesCs): record the measured Shift-JIS decoding sweep
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 08:27:22 -03:00
orezraeyandClaude Opus 5.5 4ec8b2d289 fix(libSceCesCs): encode EUC-JP and Shift-JIS as measured on a PS5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 08:27:22 -03:00
orezraeyandClaude Opus 5.5 23411115af refactor(libSceCesCs): let every MBCS string form check its output through the shared loop
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 08:27:15 -03:00
orezraeyandClaude Opus 5.5 5b014ba000 fix(libSceCesCs): terminate and count MBCS string conversions as measured on a PS5
The UTF-16 and UTF-32 string forms and the UTF-8 to Shift-JIS and EUC-JP
calls now share the measured string loop and UTF-8 decoder, and the
Japanese tests expect JIS X 0212 to be neither decoded nor encoded.
sceCesMbcsUcsContextInit rejects a null profile.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 08:17:51 -03:00
orezraey 0a33d53861 Merge branch 'fix/libscecescs-eucjp-measured-ps5' into fix/libscecescs-mbcs-strings-measured-ps5
# Conflicts:
#	core/libs/tests/GuestCes.cpp
2026-10-23 08:06:41 -03:00
orezraey 7735835ba8 Merge remote-tracking branch 'origin/main' into fix/libscecescs-eucjp-measured-ps5
# Conflicts:
#	core/libs/tests/GuestCes.cpp
#	docs/dev/debt/libSceCesCs.md
2026-10-23 08:02:15 -03:00
orezraey 803744fe85 Merge remote-tracking branch 'origin/main' into fix/libscecescs-scalar-measured-ps5
# Conflicts:
#	core/libs/prx/libSceCesCs/Export.cpp
#	core/libs/tests/GuestCes.cpp
#	docs/dev/debt/libSceCesCs.md
2026-10-23 07:55:22 -03:00
jacob6769 1d5d71aa21 Merge remote-tracking branch 'origin/main' into feat/ces-korean-big5-scalars
# Conflicts:
#	core/libs/prx/libSceCesCs/Export.cpp
#	core/libs/prx/libSceCesCs/PendingExports.cpp
#	docs/dev/debt/libSceCesCs.md
2026-10-23 19:17:52 +09:00
Simone 79abbd62db Merge #8976 into feat/libSceCesCs-mbc-sjis-code 2026-10-23 11:08:22 +02:00
Simone cd4bfc33e0 Merge #8972 into feat/libSceCesCs-gbk-euc-cn 2026-10-23 11:08:22 +02:00
Dean Galvin 4c33e22394 Merge main into feat/libSceCesCs-sbc-mbcs-utf16-utf32 2026-10-23 19:33:14 +13:00
Dean Galvin 88b0b798ed Merge main into feat/libSceCesCs-japanese-scalar 2026-10-23 15:51:39 +13:00
orezraeyandClaude Opus 5.5 56561e0268 docs(libSceCesCs): the Cp1251 profile also works in MBCS contexts
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-22 22:35:11 -03:00
orezraeyandClaude Opus 5.5 9ae9b105bc fix(libSceCesCs): convert single EUC-JP characters as measured on a PS5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-22 22:08:28 -03:00
orezraeyandClaude Opus 5.5 106283e3ed fix(libSceCesCs): convert single-byte characters as measured on a PS5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-22 21:58:44 -03:00
NikoandClaude Opus 5.5 27c3f6ba2d feat(libSceCesCs): add 27 single-byte code page profiles
sceCesRefersUcsProfile threw for every code page except 1250, 1251 and
1252. Twenty-seven more now return a profile that sceCesSbcToUtf8,
sceCesUtf8ToSbc and the MBCS context conversions accept: ISO 8859-1,
-2, -4, -5, -9, -10, -13, -14, -15 and -16, code pages 437, 737, 775,
850, 852, 855, 860, 861, 862, 863, 865 and 866, KOI8-R, KOI8-U and
Windows 1254, 1256 and 1258.

SingleByte.hpp holds the high half of each table, taken from the
public mapping files at unicode.org: ISO8859, Microsoft's PC tables,
VENDORS/MISC for KOI8 and Microsoft's best-fit tables for 1254, 1256
and 1258, the same source and derivation as the existing Cp1250 table,
whose high half equals bestfit1250's byte table. Bytes 0x00-0x7F are
unchanged in all of them, and every table maps its 256 bytes to 256
distinct BMP characters. The three existing profiles keep their
tables; one lookup now serves them all.

Profiles whose public table leaves bytes undefined (ISO 8859-3, -6, -7,
-8, -11, code pages 857, 864, 869), maps bytes to private use (874,
1253, 1255, 1257) or has no public table here still throw.

guest_ces checks that each new profile is a distinct, stable pointer,
four bytes per table against the source, a round trip of every byte
through sceCesSbcToUtf8 and sceCesUtf8ToSbc, UNASSIGNED_CODE for a
character outside the table, and a KOI8-R string through an MBCS
context in both directions.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-21 16:36:05 +02:00
SimoneandClaude Opus 5.5 c020ce8934 feat(libSceCesCs): implement generic MBCS character, Shift-JIS code and JIS X 0213 helpers
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-21 15:24:37 +02:00
SimoneandClaude Opus 5.5 f974e91f08 feat(libSceCesCs): convert GBK and EUC-CN through the CP936 table
Add the GBK (CP936) and EUC-CN profiles, their scalar conversions to
and from UTF-8, UTF-16, UTF-32 and UCS-2, and accept both profiles in
the MBCS string conversions.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-21 15:24:24 +02:00
SimoneandClaude Opus 5.5 79fe0b903c feat(libSceCesCs): add UTF-16 and UTF-32 forms of the SBC and MBCS string conversions
Implement sceCesSbcTo{Ucs2,Utf16,Utf32}* and their inverses, and the
UTF-16 and UTF-32 MBCS string conversions and length queries, on the
existing single-byte, CP932 and CP51932 profiles.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-21 15:23:56 +02:00
simonbronkandClaude Opus 5.5 3990593a19 feat(libSceCesCs): implement Shift-JIS and EUC-JP scalar conversions
Implement the single-character conversions between the CP932 and
CP51932 profiles and UTF-8, UTF-16, UTF-32 and UCS-2 in both
directions, and the Shift-JIS to JIS X 0208 row/cell arithmetic.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-21 15:23:34 +02:00
jacob6769andClaude Opus 5.5 b50e5c7a4d feat(libSceCesCs): implement the EUC-KR, UHC and Big5 scalar conversions
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-21 21:29:50 +09:00
jacob6769andClaude Opus 5.5 dc7be1b072 feat(libSceCesCs): convert EUC-KR, UHC and Big5 through the CP949 and CP950 tables
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-21 12:42:02 +02:00
adhimiw 1dcbe42379 feat(libSceCesCs): declare the missing firmware character set exports 2026-10-20 11:14:38 +05:30
Adrià 784f82b4b0 Merge pull request #7848 from SibteProf/feat/ces-unicode-scalar-conversions
feat(libSceCesCs): implement Unicode scalar conversions
2026-10-20 04:36:33 +02:00
SibteProf f9d6f3e932 feat(libSceCesCs): implement Unicode scalar conversions 2026-10-19 15:46:11 +05:00
Bl4ckk01 f4b090fbfd feat(libSceCesCs): implement Shift-JIS CP932 encoding conversions 2026-10-18 21:41:40 +02:00
primesoul-web 3f03bc0e24 feat(libs): finish the CesCs conversions and the AudioOut, AudioIn and GameUpdate exports
libSceCesCs goes to 17 of 17: three-byte EUC-JP JIS X 0212 (0x8F)
characters convert through a table read from the Unicode consortium's
JIS0212.TXT, both directions, so sceCesEucJpToUcs2, the MbcsStrTo*
and *Get*Len forms and the context conversions accept them, and a
character with cells in both JIS X 0208 and JIS X 0212 still encodes
to the two-byte form. sceCesRefersUcsProfileCp1250 returns a profile
with the Windows-1250 table of Microsoft's bestfit1250.txt, accepted
by the SBC and context conversions.

The AudioOut HDMI bitstream Sys family follows the Ex ports: SysOpen
opens a port only for mode 5, the TrueHD MAT route EVO-PLAYER-PS5
reads from the firmware 12.70 mode tables, paced at 192 kHz with no
host device; SysConfigureOutput accepts the observed HDMI MAT switch;
SysClose closes a Sys port. sceAudioOut2MasteringGetState fills the
states by states id with the descriptors KytyPS5 declares, SetParam
answers a null parameter with the mastering invalid-API-parameter
error, Set3DLatency follows KytyPS5's output validation (0x80268001
above 2, any user), SetSystemDebugState records the debug state id,
and GetSpeakerArrayCoefficients returns the front pair KytyPS5 and
prosper both return.

sceAudioInOpen accepts param 0x10 as float mono like KytyPS5; HqOpen
answers a longer grain or a lower rate with the invalid-size and
invalid-frequency errors; AsyncOpen also accepts the 384-sample grain
SharpProspero documents. sceGameUpdateGetAddcontLatestVersion answers
a null entitlement label with 0x80412803, as prosper does.
2026-10-18 18:52:36 +02:00
ps54allandClaude Sonnet 5.5 db69769f42 feat(libSceCesCs): convert two-byte EUC-JP characters through the CP932 table
The EUC-JP profile threw on every two-byte character. It now maps the rows
0xA1-0xFE through a 94x94 table taken from Microsoft code page 932 (JIS X 0208
plus NEC row 13 and the NEC-selected IBM rows), and encodes a code point to its
lowest cell. Unassigned cells answer UNASSIGNED_CODE, a bad trail byte
INVALID_ENCODE, and a cut pair SRC_BUFFER_END. Three-byte JIS X 0212 characters
still throw.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-14 16:10:41 +02:00
ps54allandClaude Opus 5.5 61e10efbd6 feat(libSceCesCs): convert EUC-JP (CP51932) strings through a profile context
sceCesUcsProfileInitEucJpCp51932 returns a profile for EUC-JP, and
sceCesMbcsUcsContextInit now keeps the profile in the context. The
multibyte string conversions (UTF-8, UCS-2, their length queries) and
sceCesEucJpToUcs2 convert ASCII and the JIS X 0201 half-width katakana
for that profile, and every character for the existing CP1252/CP1251
profiles. Two- and three-byte JIS X 0208/0212 characters throw, since
their mapping table is not present.

Metal Gear Solid 4 Master Collection (PPSA18194) imports the set in its
game executable, which fails to load without it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-14 14:32:42 +02:00
Alexandros Pournaras 5a0d74d1f8 style(libSceCesCs): reorder functions 2026-10-14 12:20:58 +03:00
Alexandros PournarasandClaude Sonnet 5.5 4941c586cd feat(libSceCesCs): declare sceCesRefersUcsProfileCp1250
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-14 12:04:43 +03:00