external/cde
1
0
Fork 0
mirror of git://git.code.sf.net/p/cdesktopenv/code synced 2025-03-09 15:50:02 +00:00
Code Issues Releases Wiki Activity

Make 'read' compatible with Shift-JIS

Browse source 
This commit fixes a bug in the 'read' built-in: it did not properly
skip over multibyte characters. The bug never affects UTF-8 locales
because all UTF-8 bytes have the high-order bit set. But Shift-JIS
characters may include a byte corresponding to the ASCII backslash
character, which cauased buggy behaviour when using 'read' without
the '-r' option that disables backslash escape processing.

It also makes the regression tests compatible with Shift-JIS
locales. They failed with syntax errors.

src/cmd/ksh93/bltins/read.c:
- Use the multibyte macros when skipping over word characters.
  Based on a patch from the old ast-developers mailing list:
  https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01848.html

src/cmd/ksh93/include/defs.h:
- Be a bit smarter about causing the compiler to optimise out
  multibyte code when SHOPT_MULTIBYTE is disabled. See the updated
  comment for details.

src/cmd/ksh93/tests/locale.sh:
- Put all the supported locales in an array for future tests.
- Add test for the 'read' bug. Include it in a loop that tests
  64 SHIFT-JIS character combinations. Only one fails on old ksh:
  the one where the final byte corresponds to the ASCII backslash.
  It doesn't hurt to test all the others anyway.

src/cmd/ksh93/tests/basic.sh,
src/cmd/ksh93/tests/builtins.sh,
src/cmd/ksh93/tests/quoting2.sh:
- Fix syntax errors that occurred in SHIFT-JIS locales as the
  parser was processing literal UTF-8 characters. Not executing
  that code is not enough; we need to make sure it never gets
  parsed as well. This is done by wrapping the commands containing
  literal UTF-8 strings in an 'eval' command as a single-quoted
  operand.

.github/workflows/ci.yml:
- Run the tests in the ja_JP.SJIS locale instead of ja_JP.UTF-8.
  UTF-8 is already covered by the nl_NL.UTF-8 test run; that should
  be good enough.
This commit is contained in:
Martijn Dekker 2021-02-18 14:46:10 +00:00
parent 8c2d8e5f46
commit c2cb0eae19
9 changed files with 65 additions and 31 deletions
Download patch file Download diff file Expand all files Collapse all files

2
src/cmd/ksh93/include/version.h
View file

@ -20,7 +20,7 @@
#define SH_RELEASE_FORK "93u+m" /* only change if you develop a new ksh93 fork */
#define SH_RELEASE_SVER "1.0.0-alpha" /* semantic version number: https://semver.org */
#define SH_RELEASE_DATE "2021-02-17" /* must be in this format for $((.sh.version)) */
#define SH_RELEASE_DATE "2021-02-18" /* must be in this format for $((.sh.version)) */
#define SH_RELEASE_CPYR "(c) 2020-2021 Contributors to ksh " SH_RELEASE_FORK
/* Scripts sometimes field-split ${.sh.version}, so don't change amount of whitespace. */