Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
This documents significant changes in the 93u+m branch of AT&T ksh93.
|
|
|
|
For full details, see the git log at:
|
|
|
|
https://github.com/modernish/ksh
|
|
|
|
|
|
|
|
Any uppercase BUG_* names are modernish shell bug IDs.
|
|
|
|
|
Fix bugs in testing if a parameter is set
This fixes three related bugs:
1. Expansions like ${var+set} remained static when used within a
'for', 'while' or 'until' loop; the expansions din't change
along with the state of the variable, so they could not be used
to check whether a variable is set within a loop if the state of
that variable changed in the course of the loop. (BUG_ISSETLOOP)
2. ${IFS+s} always yielded 's', and [[ -v IFS ]] always yielded
true, even if IFS is unset. (BUG_IFSISSET)
3. IFS was incorrectly exempt from '-u' ('-o nounset') checks.
src/cmd/ksh93/sh/macro.c: varsub():
- When getting a node pointer (np) to the parameter to test,
special-case IFS by checking if it has a value and not setting
the pointer if not. The node to IFS always exists, even after
'unset -v IFS', so before this fix it always followed the code
path for a parameter that is set. This fixes BUG_IFSISSET for
${IFS+s} and also fixes set -u (-o nounset) with IFS.
- Before using the 'nv_isnull' macro to check if a regular variable
is set, call nv_optimize() if needed. This fixes BUG_ISSETLOOP.
Idea from Kurtis Rader: https://github.com/att/ast/issues/1090
Of course this only works if SHOPT_OPTIMIZE==1 (the default),
but if not, then this bug is not triggered in the first place.
- Add some comments for future reference.
src/cmd/ksh93/bltins/test.c: test_unop():
- Fix BUG_IFSISSET for [[ -v IFS ]]. The nv_optimize() method
doesn't seem to have any effect here, so the only way that I can
figure out is to special-case IFS, nv_getval()'ing it to check if
IFS has a value in the current scope.
src/cmd/ksh93/tests/variables.sh:
- Add regression tests for checking if a varariable is set within a
loop, within and outside a function with that variable made local
(to check if the scope is honoured). Repeat these tests for a
regular variable and for IFS, for ${foo+set} and [[ -v foo ]].
(cherry picked from commit a2cf79cb98fa3e47eca85d9049d1d831636c9b16)
2020-05-20 14:50:43 +00:00
|
|
|
2020-05-20:
|
|
|
|
|
|
|
|
- Fix BUG_ISSETLOOP. Expansions like ${var+set} remained static when used
|
|
|
|
within a 'for', 'while' or 'until' loop; the expansions din't change along
|
|
|
|
with the state of the variable, so they could not be used to check whether a
|
|
|
|
variable is set within a loop if the state of that variable changed in the
|
|
|
|
course of the loop.
|
|
|
|
|
|
|
|
- Fix BUG_IFSISSET. ${IFS+s} always yielded 's', and [[ -v IFS ]] always
|
|
|
|
yielded true, even if IFS is unset. This applied to IFS only.
|
|
|
|
|
2020-05-19 14:49:56 +00:00
|
|
|
2020-05-19:
|
|
|
|
|
|
|
|
- Fix 'command -p'. The -p option causes the operating system's standard
|
|
|
|
utilities path (as output by 'getconf PATH') to be searched instead of $PATH.
|
|
|
|
Before this fix, this was broken on non-interactive shells as the internal
|
|
|
|
variable holding the default PATH value was not correctly initialised.
|
|
|
|
|
Fix 'test -t 1' in $(command substitutions)
Standard output (FD 1) tested as being on a terminal within a
command substitution, which makes no sense as the command
substitution is supposed to be catching standard output.
ksh -c 'v=$(echo begincomsub
[ -t 1 ] && echo oops
echo endcomsub)
echo "$v"'
This should not output "oops".
This is one of the many bugs with ksh93 virtual (non-forked)
subshells. On the abandoned Vashist/Rader ksh2020 branch, this bug
was fixed by changing quite a lot of code, which introduced and/or
exposed another bug:
https://github.com/att/ast/issues/1079
https://github.com/att/ast/commit/8e1e405e
https://github.com/att/ast/issues/1088
That issue was unresolved when the ksh2020 branch was abandoned.
The safer and more conservative fix is simply forcing the subshell
to fork if we're in a non-forked command substitution and testing
'-t 1'. It is hard to imagine a situation where this would cause a
noticable performance hit.
Note that this fix does not affect ksh93-specific "shared"
non-subshell ${ command substitutions; } which are executed in the
main shell environment, so that variables survive, etcetera.
'test -t 1' continues to wrongly return true there, but command
substitutions of that form cannot be forked because that would
defeat their purpose.
src/cmd/ksh93/bltins/test.c:
- Fix 'test -t 1', '[ -t 1 ]' and '[[ -t 1 ]]' by forking the
current subshell if it is a virtual/non-forked subshell
(shp->subshell), and a command substitution (shp->comsub), but
NOT a "shared" ${ command substitution; } (!shp->subshare).
src/cmd/ksh93/tests/bracket.sh:
- Add two regression tests for this issue, which were adapted from
the Vashist/Rader ksh2020 branch.
NEWS, src/cmd/ksh93/include/version.h:
- Update.
(cherry picked from commit b8ef05e457ead65b83417699b8dd8632f855e2fa)
2020-05-16 19:06:49 +00:00
|
|
|
2020-05-16:
|
|
|
|
|
|
|
|
- Fix 'test -t 1', '[ -t 1 ]', '[[ -t 1 ]]' in command substitutions.
|
|
|
|
Standard output (file descriptor 1) tested as being on a terminal within a
|
|
|
|
command substitution, which makes no sense as the command substitution is
|
|
|
|
supposed to be catching standard output.
|
|
|
|
v=$(echo begincomsub
|
|
|
|
[ -t 1 ] && echo oops
|
|
|
|
echo endcomsub)
|
|
|
|
echo "$v"
|
|
|
|
This now does not output "oops".
|
|
|
|
|
2020-05-14 10:36:16 +00:00
|
|
|
2020-05-14:
|
|
|
|
|
|
|
|
- Fix syncing history when print -s -f is used. For example, the
|
|
|
|
following now correctly adds a 'cd' command to the history:
|
|
|
|
print -s -f 'cd -- %q\n' "$PWD"
|
|
|
|
Ref.: https://github.com/att/ast/issues/425
|
|
|
|
https://github.com/att/ast/pull/442
|
|
|
|
|
2020-05-16 14:04:35 +00:00
|
|
|
- Fix BUG_PUTIOERR: Output builtins now correctly detect and report
|
|
|
|
input/output errors. This allows scripts to check for a nonzero exit
|
|
|
|
status on the 'print', 'printf' and 'echo' builtins and prevent possible
|
|
|
|
infinite loops if SIGPIPE is ignored.
|
|
|
|
|
2020-05-14 16:41:26 +00:00
|
|
|
- Add a convenient bin/run_ksh_tests script to the source tree that
|
|
|
|
sets up the necessary environment and runs the ksh regression tests.
|
|
|
|
|
Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
2020-05-13:
|
|
|
|
|
|
|
|
- Fix BUG_CASELIT: an undocumented 'case' pattern matching misbehaviour that
|
|
|
|
goes back to the original Bourne shell, but wasn't discovered until 2018.
|
|
|
|
If a pattern doesn't match as a pattern, it was tried again as a literal
|
|
|
|
string. This broke common validation use cases, e.g.:
|
|
|
|
n='[0-9]'
|
|
|
|
case $n in
|
|
|
|
( [0-9] ) echo "$n is a number" ;;
|
|
|
|
esac
|
|
|
|
would output "[0-9] is a number" as the literal string fallback matches the
|
|
|
|
pattern. As this misbehaviour was never documented anywhere (not for Bourne,
|
|
|
|
ksh88, or ksh93), and it was never replicated in other shells (not even in
|
|
|
|
ksh88 clones pdksh and mksh), it is unlikely any scripts rely on it.
|
|
|
|
Of course, a literal string fallback, should it be needed, is trivial to
|
|
|
|
implement correctly without this breakage:
|
|
|
|
case $n in
|
|
|
|
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
|
|
|
|
esac
|
2020-05-13 14:00:33 +00:00
|
|
|
Ref.: https://github.com/att/ast/issues/476
|
|
|
|
|
|
|
|
- Fix BUG_REDIRIO: ksh used to redirect standard output by default when no
|
|
|
|
file descriptor was specified with the rarely used '<>' reading/writing
|
|
|
|
redirection operator. It now redirects standard input by default, as POSIX
|
|
|
|
specifies and as all other POSIX shells do. To redirect standard output
|
|
|
|
for reading and writing, you now need '1<>'.
|
|
|
|
Ref.: https://github.com/att/ast/issues/75
|
|
|
|
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_07
|