mirror of
git://git.code.sf.net/p/cdesktopenv/code
synced 2025-03-09 15:50:02 +00:00
c2cb0eae19
This commit fixes a bug in the 'read' built-in: it did not properly skip over multibyte characters. The bug never affects UTF-8 locales because all UTF-8 bytes have the high-order bit set. But Shift-JIS characters may include a byte corresponding to the ASCII backslash character, which cauased buggy behaviour when using 'read' without the '-r' option that disables backslash escape processing. It also makes the regression tests compatible with Shift-JIS locales. They failed with syntax errors. src/cmd/ksh93/bltins/read.c: - Use the multibyte macros when skipping over word characters. Based on a patch from the old ast-developers mailing list: https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01848.html src/cmd/ksh93/include/defs.h: - Be a bit smarter about causing the compiler to optimise out multibyte code when SHOPT_MULTIBYTE is disabled. See the updated comment for details. src/cmd/ksh93/tests/locale.sh: - Put all the supported locales in an array for future tests. - Add test for the 'read' bug. Include it in a loop that tests 64 SHIFT-JIS character combinations. Only one fails on old ksh: the one where the final byte corresponds to the ASCII backslash. It doesn't hurt to test all the others anyway. src/cmd/ksh93/tests/basic.sh, src/cmd/ksh93/tests/builtins.sh, src/cmd/ksh93/tests/quoting2.sh: - Fix syntax errors that occurred in SHIFT-JIS locales as the parser was processing literal UTF-8 characters. Not executing that code is not enough; we need to make sure it never gets parsed as well. This is done by wrapping the commands containing literal UTF-8 strings in an 'eval' command as a single-quoted operand. .github/workflows/ci.yml: - Run the tests in the ja_JP.SJIS locale instead of ja_JP.UTF-8. UTF-8 is already covered by the nl_NL.UTF-8 test run; that should be good enough.
46 lines
2.6 KiB
C
46 lines
2.6 KiB
C
/***********************************************************************
|
|
* *
|
|
* This software is part of the ast package *
|
|
* Copyright (c) 1982-2012 AT&T Intellectual Property *
|
|
* and is licensed under the *
|
|
* Eclipse Public License, Version 1.0 *
|
|
* by AT&T Intellectual Property *
|
|
* *
|
|
* A copy of the License is available at *
|
|
* http://www.eclipse.org/org/documents/epl-v10.html *
|
|
* (with md5 checksum b35adb5213ca9657e911e9befb180842) *
|
|
* *
|
|
* Information and Software Systems Research *
|
|
* AT&T Research *
|
|
* Florham Park NJ *
|
|
* *
|
|
* David Korn <dgk@research.att.com> *
|
|
* *
|
|
***********************************************************************/
|
|
|
|
#define SH_RELEASE_FORK "93u+m" /* only change if you develop a new ksh93 fork */
|
|
#define SH_RELEASE_SVER "1.0.0-alpha" /* semantic version number: https://semver.org */
|
|
#define SH_RELEASE_DATE "2021-02-18" /* must be in this format for $((.sh.version)) */
|
|
#define SH_RELEASE_CPYR "(c) 2020-2021 Contributors to ksh " SH_RELEASE_FORK
|
|
|
|
/* Scripts sometimes field-split ${.sh.version}, so don't change amount of whitespace. */
|
|
/* Arithmetic $((.sh.version)) uses the last 10 chars, so the date must be at the end. */
|
|
#if _AST_ksh_release
|
|
# define SH_RELEASE SH_RELEASE_FORK "/" SH_RELEASE_SVER " " SH_RELEASE_DATE
|
|
#else
|
|
# ifdef _AST_git_commit
|
|
# define SH_RELEASE SH_RELEASE_FORK "/" SH_RELEASE_SVER "+" _AST_git_commit " " SH_RELEASE_DATE
|
|
# else
|
|
# define SH_RELEASE SH_RELEASE_FORK "/" SH_RELEASE_SVER "+dev " SH_RELEASE_DATE
|
|
# endif
|
|
#endif
|
|
|
|
/*
|
|
* For shcomp: the version number (0-255) for the binary bytecode header.
|
|
* Only increase very rarely, i.e.: if incompatible changes are made that
|
|
* cause bytecode from newer versions to fail on older versions of ksh.
|
|
*
|
|
* The version number was last increased in 2021 for ksh 93u+m because
|
|
* most of the predefined aliases were converted to builtin commands.
|
|
*/
|
|
#define SHCOMP_HDR_VERSION 4
|