1
0
Fork 0
mirror of git://git.code.sf.net/p/cdesktopenv/code synced 2025-02-15 04:32:24 +00:00
cde/.github/workflows/ci.yml

33 lines
1 KiB
YAML
Raw Normal View History

name: CI
on: [push]
jobs:
Linux:
name: Linux
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v2
- name: Build
run: bin/package make
- name: Regression tests
run: |
PS4="$PS4[ci.yml] "
set -o xtrace
export TZ=UTC
ulimit -n 1024
: default regression tests &&
script -q -e -c "bin/shtests" &&
Make 'read' compatible with Shift-JIS This commit fixes a bug in the 'read' built-in: it did not properly skip over multibyte characters. The bug never affects UTF-8 locales because all UTF-8 bytes have the high-order bit set. But Shift-JIS characters may include a byte corresponding to the ASCII backslash character, which cauased buggy behaviour when using 'read' without the '-r' option that disables backslash escape processing. It also makes the regression tests compatible with Shift-JIS locales. They failed with syntax errors. src/cmd/ksh93/bltins/read.c: - Use the multibyte macros when skipping over word characters. Based on a patch from the old ast-developers mailing list: https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01848.html src/cmd/ksh93/include/defs.h: - Be a bit smarter about causing the compiler to optimise out multibyte code when SHOPT_MULTIBYTE is disabled. See the updated comment for details. src/cmd/ksh93/tests/locale.sh: - Put all the supported locales in an array for future tests. - Add test for the 'read' bug. Include it in a loop that tests 64 SHIFT-JIS character combinations. Only one fails on old ksh: the one where the final byte corresponds to the ASCII backslash. It doesn't hurt to test all the others anyway. src/cmd/ksh93/tests/basic.sh, src/cmd/ksh93/tests/builtins.sh, src/cmd/ksh93/tests/quoting2.sh: - Fix syntax errors that occurred in SHIFT-JIS locales as the parser was processing literal UTF-8 characters. Not executing that code is not enough; we need to make sure it never gets parsed as well. This is done by wrapping the commands containing literal UTF-8 strings in an 'eval' command as a single-quoted operand. .github/workflows/ci.yml: - Run the tests in the ja_JP.SJIS locale instead of ja_JP.UTF-8. UTF-8 is already covered by the nl_NL.UTF-8 test run; that should be good enough.
2021-02-18 14:46:10 +00:00
: regression tests with OS-provided multibyte locales &&
LANG=nl_NL.UTF-8 script -q -e -c "bin/shtests --locale --nocompile" &&
Make 'read' compatible with Shift-JIS This commit fixes a bug in the 'read' built-in: it did not properly skip over multibyte characters. The bug never affects UTF-8 locales because all UTF-8 bytes have the high-order bit set. But Shift-JIS characters may include a byte corresponding to the ASCII backslash character, which cauased buggy behaviour when using 'read' without the '-r' option that disables backslash escape processing. It also makes the regression tests compatible with Shift-JIS locales. They failed with syntax errors. src/cmd/ksh93/bltins/read.c: - Use the multibyte macros when skipping over word characters. Based on a patch from the old ast-developers mailing list: https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01848.html src/cmd/ksh93/include/defs.h: - Be a bit smarter about causing the compiler to optimise out multibyte code when SHOPT_MULTIBYTE is disabled. See the updated comment for details. src/cmd/ksh93/tests/locale.sh: - Put all the supported locales in an array for future tests. - Add test for the 'read' bug. Include it in a loop that tests 64 SHIFT-JIS character combinations. Only one fails on old ksh: the one where the final byte corresponds to the ASCII backslash. It doesn't hurt to test all the others anyway. src/cmd/ksh93/tests/basic.sh, src/cmd/ksh93/tests/builtins.sh, src/cmd/ksh93/tests/quoting2.sh: - Fix syntax errors that occurred in SHIFT-JIS locales as the parser was processing literal UTF-8 characters. Not executing that code is not enough; we need to make sure it never gets parsed as well. This is done by wrapping the commands containing literal UTF-8 strings in an 'eval' command as a single-quoted operand. .github/workflows/ci.yml: - Run the tests in the ja_JP.SJIS locale instead of ja_JP.UTF-8. UTF-8 is already covered by the nl_NL.UTF-8 test run; that should be good enough.
2021-02-18 14:46:10 +00:00
LANG=ja_JP.SJIS script -q -e -c "bin/shtests --locale --nocompile" &&
: disable most SHOPTs, rebuild ksh &&
Fix compile/regress fails on compiling without SHOPT_* options Many compile-time options were broken so that they could not be turned off without causing compile errors and/or regression test failures. This commit now allows the following to be disabled: SHOPT_2DMATCH # two dimensional ${.sh.match} for ${var//pat/str} SHOPT_BGX # one SIGCHLD trap per completed job SHOPT_BRACEPAT # C-shell {...,...} expansions (, required) SHOPT_ESH # emacs/gmacs edit mode SHOPT_HISTEXPAND # csh-style history file expansions SHOPT_MULTIBYTE # multibyte character handling SHOPT_NAMESPACE # allow namespaces SHOPT_STATS # add .sh.stats variable SHOPT_VSH # vi edit mode The following still break ksh when disabled: SHOPT_FIXEDARRAY # fixed dimension indexed array SHOPT_RAWONLY # make viraw the only vi mode SHOPT_TYPEDEF # enable typeset type definitions Compiling without SHOPT_RAWONLY just gives four regression test failures in pty.sh, but turning off SHOPT_FIXEDARRAY and SHOPT_TYPEDEF causes compilation to fail. I've managed to tweak the code to make it compile without those two options, but then dozens of regression test failures occur, often in things nothing directly to do with those options. It looks like the separation between the code for these options and the rest was never properly maintained. Making it possible to disable SHOPT_FIXEDARRAY and SHOPT_TYPEDEF may involve major refactoring and testing and may not be worth it. This commit has far too many tweaks to list. Notables fixes are: src/cmd/ksh93/data/builtins.c, src/cmd/ksh93/data/options.c: - Do not compile in the shell options and documentation for disabled features (braceexpand, emacs/gmacs, vi/viraw), so the shell is not left with no-op options and inaccurate self-doc. src/cmd/ksh93/data/lexstates.c: - Comment the state tables to associte them with their IDs. - In the ST_MACRO table (sh_lexstate9[]), do not make the S_BRACE state for position 123 (ASCII for '{') conditional upon SHOPT_BRACEPAT (brace expansion), otherwise disabling this causes glob patterns of the form {3}(x) (matching 3 x'es) to stop working as well -- and that is ksh globbing, not brace expansion. src/cmd/ksh93/edit/edit.c: ed_read(): - Fixed a bug: SIGWINCH was not handled by the gmacs edit mode. src/cmd/ksh93/sh/name.c: nv_putval(): - The -L/-R left/right adjustment options to typeset do not count zero-width characters. This is the behaviour with SHOPT_MULTIBYTE enabled, regardless of locale. Of course, what a zero-width character is depends on the locale, but control characters are always considered zero-width. So, to avoid a regression, add some fallback code for non-SHOPT_MULTIBYTE builds that skips ASCII control characters (as per iscntrl(3)) so they are still considered to have zero width. src/cmd/ksh93/tests/shtests: - Export the SHOPT_* macros from SHOPT.sh to the tests as environment variables, so the tests can check for them and decide whether or how to run tests based on the compile-time options that the tested binary was presumably compiled with. - Do not run the C.UTF-8 tests if SHOPT_MULTIBYTE is not enabled. src/cmd/ksh93/tests/*.sh: - Add a bunch of checks for SHOPT_* env vars. Since most should have a value 0 (off) or 1 (on), the form ((SHOPT_FOO)) is a convenient way to use them as arithmetic booleans. .github/workflows/ci.yml: - Make GitHub do more testing: run two locale tests (Dutch and Japanese UTF-8 locales), then disable all the SHOPTs that we can currently disable, recompile ksh, and run the tests again.
2021-02-08 17:16:43 +00:00
sed --regexp-extended --in-place=.orig \
Fix multiple bugs in .sh.match (#455) This commit backports all of the relevant .sh.match bugfixes from ksh93v-. Most of the .sh.match rewrite is from versions 2012-08-24 and 2012-10-04, with patches from later releases of 93v- and ksh2020 also applied. Note that there are still some remaining bugs in .sh.match, although now the total count of .sh.match bugs should be less that before. These are the relevant changes in the ksh93v- changelog that were backported: 12-08-07 .sh.match no longer gets set for patterns in PS4 during set -x. 12-08-10 Rewrote .sh.match expansions fixing several bugs and improving performance. 12-08-22 .sh.match now handles subpatterns that had no matches with ${var//pattern} correctly. 12-08-21 A bug in setting .sh.match after ${var//pattern/string} when string is empty has been fixed. 12-08-21 A bug in setting .sh.match after [[ string == pattern ]] has been fixed. 12-08-31 A bug that could cause a core dump after typeset -m var=.sh.match has been fixed. 12-09-10 Fixed a bug in typeset -m the .sh.match is being renamed. 12-09-07 Fixed a bug in .sh.match code that coud cause the shell to quitely 13-02-21 The 12-01-16 bug fix prevented .sh.match from being used in the replacement string. The previous code was restored and a different fix which prevented .sh.match from being computed for nested replacement has been used instead. 13-05-28 Fixed two bug for typeset -c and typeset -m for variable .sh.match. Changes: - The SHOPT_2DMATCH option has been removed. This was already the default behavior previously, and now it's documented in the man page. - init.c: Backported the sh_setmatch() rewrite from 93v- 2012-08-24 and 2012-10-04. - Backported the libast 93v- strngrpmatch() function, as the .sh.match rewrite requires this API. - Backported the sh_match regression tests from ksh93v-, with many other sh_match tests backported from ksh2020. Much of the sh_match script is based on code from Roland Mainz: https://marc.info/?l=ast-developers&m=134606574109162&w=2 https://marc.info/?l=ast-developers&m=134490505607093 - tests/{substring,treemove}.sh: Backported other relevant .sh.match fixes, with tests added to the substring and treemove test scripts. - tests/types.sh: One of the (now reverted) memory leak bugfixes introduced a CI test failure in this script, so for that test the error message has been improved. - string/strmatch.c: The original ksh93v- code for the strngrpmatch() changes introduced a crash that could occur because strlen would be used on a null pointer. This has been fixed by avoiding strlen if the string is null. One nice side effect of these changes is a considerable performance improvement in the shbench[1] gsub benchmark (results from 20 iterations with CCFLAGS=-Os): -------------------------------------------------- name /tmp/ksh-current /tmp/ksh-matchfixes -------------------------------------------------- gsub.ksh 0.883 [0.822-0.959] 0.457 [0.442-0.505] -------------------------------------------------- Despite all of the many fixes and improvements in the backported 93v- .sh.match code, there are a few remaining bugs: - .sh.match is printed with a default [0] subscript (see also https://github.com/ksh93/ksh/issues/308#issuecomment-1025016088): $ arch/*/bin/ksh -c 'echo ${!.sh.match}' .sh.match[0] This bug appears to have been introduced by the changes from ksh93v- 2012-08-24. - The wrong variable name is given for 'parameter not set' errors (from https://marc.info/?l=ast-developers&m=134489094602596): $ arch/*/bin/ksh -u $ x=1234 $ true "${x//~(X)([012])|([345])/}" $ compound co $ typeset -m co.array=.sh.match $ printf "%q\n" "${co.array[2][0]}" arch/linux.i386-64/bin/ksh: co.array[2][(null)]: parameter not set - .sh.match leaks out of subshells. Further information and a reproducer can be found here: https://marc.info/?l=ast-developers&m=136292897330187 [1]: https://github.com/ksh-community/shbench
2022-02-09 00:01:40 +00:00
'/^SHOPT (AUDIT|BGX|BRACEPAT|DEVFD|DYNAMIC|EDPREDICT|ESH|FIXEDARRAY|HISTEXPAND|MULTIBYTE|NAMESPACE|OPTIMIZE|SPAWN|STATS|SUID_EXEC|VSH)=/ s/=1?/=0/' \
Fix compile/regress fails on compiling without SHOPT_* options Many compile-time options were broken so that they could not be turned off without causing compile errors and/or regression test failures. This commit now allows the following to be disabled: SHOPT_2DMATCH # two dimensional ${.sh.match} for ${var//pat/str} SHOPT_BGX # one SIGCHLD trap per completed job SHOPT_BRACEPAT # C-shell {...,...} expansions (, required) SHOPT_ESH # emacs/gmacs edit mode SHOPT_HISTEXPAND # csh-style history file expansions SHOPT_MULTIBYTE # multibyte character handling SHOPT_NAMESPACE # allow namespaces SHOPT_STATS # add .sh.stats variable SHOPT_VSH # vi edit mode The following still break ksh when disabled: SHOPT_FIXEDARRAY # fixed dimension indexed array SHOPT_RAWONLY # make viraw the only vi mode SHOPT_TYPEDEF # enable typeset type definitions Compiling without SHOPT_RAWONLY just gives four regression test failures in pty.sh, but turning off SHOPT_FIXEDARRAY and SHOPT_TYPEDEF causes compilation to fail. I've managed to tweak the code to make it compile without those two options, but then dozens of regression test failures occur, often in things nothing directly to do with those options. It looks like the separation between the code for these options and the rest was never properly maintained. Making it possible to disable SHOPT_FIXEDARRAY and SHOPT_TYPEDEF may involve major refactoring and testing and may not be worth it. This commit has far too many tweaks to list. Notables fixes are: src/cmd/ksh93/data/builtins.c, src/cmd/ksh93/data/options.c: - Do not compile in the shell options and documentation for disabled features (braceexpand, emacs/gmacs, vi/viraw), so the shell is not left with no-op options and inaccurate self-doc. src/cmd/ksh93/data/lexstates.c: - Comment the state tables to associte them with their IDs. - In the ST_MACRO table (sh_lexstate9[]), do not make the S_BRACE state for position 123 (ASCII for '{') conditional upon SHOPT_BRACEPAT (brace expansion), otherwise disabling this causes glob patterns of the form {3}(x) (matching 3 x'es) to stop working as well -- and that is ksh globbing, not brace expansion. src/cmd/ksh93/edit/edit.c: ed_read(): - Fixed a bug: SIGWINCH was not handled by the gmacs edit mode. src/cmd/ksh93/sh/name.c: nv_putval(): - The -L/-R left/right adjustment options to typeset do not count zero-width characters. This is the behaviour with SHOPT_MULTIBYTE enabled, regardless of locale. Of course, what a zero-width character is depends on the locale, but control characters are always considered zero-width. So, to avoid a regression, add some fallback code for non-SHOPT_MULTIBYTE builds that skips ASCII control characters (as per iscntrl(3)) so they are still considered to have zero width. src/cmd/ksh93/tests/shtests: - Export the SHOPT_* macros from SHOPT.sh to the tests as environment variables, so the tests can check for them and decide whether or how to run tests based on the compile-time options that the tested binary was presumably compiled with. - Do not run the C.UTF-8 tests if SHOPT_MULTIBYTE is not enabled. src/cmd/ksh93/tests/*.sh: - Add a bunch of checks for SHOPT_* env vars. Since most should have a value 0 (off) or 1 (on), the form ((SHOPT_FOO)) is a convenient way to use them as arithmetic booleans. .github/workflows/ci.yml: - Make GitHub do more testing: run two locale tests (Dutch and Japanese UTF-8 locales), then disable all the SHOPTs that we can currently disable, recompile ksh, and run the tests again.
2021-02-08 17:16:43 +00:00
src/cmd/ksh93/SHOPT.sh &&
bin/package make &&
: default regression tests with SHOPTs disabled &&
script -q -e -c "bin/shtests"