mirror of
git://git.code.sf.net/p/cdesktopenv/code
synced 2025-03-09 15:50:02 +00:00
c2cb0eae19
This commit fixes a bug in the 'read' built-in: it did not properly skip over multibyte characters. The bug never affects UTF-8 locales because all UTF-8 bytes have the high-order bit set. But Shift-JIS characters may include a byte corresponding to the ASCII backslash character, which cauased buggy behaviour when using 'read' without the '-r' option that disables backslash escape processing. It also makes the regression tests compatible with Shift-JIS locales. They failed with syntax errors. src/cmd/ksh93/bltins/read.c: - Use the multibyte macros when skipping over word characters. Based on a patch from the old ast-developers mailing list: https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01848.html src/cmd/ksh93/include/defs.h: - Be a bit smarter about causing the compiler to optimise out multibyte code when SHOPT_MULTIBYTE is disabled. See the updated comment for details. src/cmd/ksh93/tests/locale.sh: - Put all the supported locales in an array for future tests. - Add test for the 'read' bug. Include it in a loop that tests 64 SHIFT-JIS character combinations. Only one fails on old ksh: the one where the final byte corresponds to the ASCII backslash. It doesn't hurt to test all the others anyway. src/cmd/ksh93/tests/basic.sh, src/cmd/ksh93/tests/builtins.sh, src/cmd/ksh93/tests/quoting2.sh: - Fix syntax errors that occurred in SHIFT-JIS locales as the parser was processing literal UTF-8 characters. Not executing that code is not enough; we need to make sure it never gets parsed as well. This is done by wrapping the commands containing literal UTF-8 strings in an 'eval' command as a single-quoted operand. .github/workflows/ci.yml: - Run the tests in the ja_JP.SJIS locale instead of ja_JP.UTF-8. UTF-8 is already covered by the nl_NL.UTF-8 test run; that should be good enough.
32 lines
1 KiB
YAML
32 lines
1 KiB
YAML
name: CI
|
|
|
|
on: [push]
|
|
|
|
jobs:
|
|
|
|
Linux:
|
|
name: Linux
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Checkout sources
|
|
uses: actions/checkout@v2
|
|
- name: Build
|
|
run: bin/package make
|
|
- name: Regression tests
|
|
run: |
|
|
PS4="$PS4[ci.yml] "
|
|
set -o xtrace
|
|
export TZ=UTC
|
|
ulimit -n 1024
|
|
: default regression tests &&
|
|
script -q -e -c "bin/shtests" &&
|
|
: regression tests with OS-provided multibyte locales &&
|
|
LANG=nl_NL.UTF-8 script -q -e -c "bin/shtests --locale --nocompile" &&
|
|
LANG=ja_JP.SJIS script -q -e -c "bin/shtests --locale --nocompile" &&
|
|
: disable most SHOPTs, rebuild ksh &&
|
|
sed --regexp-extended --in-place=.orig \
|
|
'/^SHOPT (2DMATCH|AUDIT|BGX|BRACEPAT|DYNAMIC|EDPREDICT|ESH|FIXEDARRAY|HISTEXPAND|MULTIBYTE|NAMESPACE|OPTIMIZE|SUID_EXEC|STATS|VSH)=/ s/=1/=0/' \
|
|
src/cmd/ksh93/SHOPT.sh &&
|
|
bin/package make &&
|
|
: default regression tests with SHOPTs disabled &&
|
|
script -q -e -c "bin/shtests"
|