Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
This documents significant changes in the 93u+m branch of AT&T ksh93.
|
2020-06-14 04:28:38 +00:00
|
|
|
For full details, see the git log at: https://github.com/ksh93/ksh
|
Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
|
|
|
|
|
Any uppercase BUG_* names are modernish shell bug IDs.
|
2020-07-01 17:14:10 +00:00
|
|
|
|
Fix octal number arguments in printf integer arithmetic
Bug 1: POSIX requires numbers used as arguments for all the %d,
%u... in printf to be interpreted as in the C language, so
printf '%d\n' 010
should output 8 when the posix option is on. However, it outputs 10.
This bug was introduced as a side effect of a change introduced in
the 2012-02-07 version of ksh 93u+m, which caused the recognition
of leading-zero numbers as octal in arithmetic expressions to be
disabled outside ((...)) and $((...)). However, POSIX requires
leading-zero octal numbers to be recognised for printf, too.
The change in question introduced a sh.arith flag that is set while
we're processing a POSIX arithmetic expression, i.e., one that
recognises leading-zero octal numbers.
Bug 2: Said flag is not reset in a command substitution used within
an arithmetic expression. A command substitution should be a
completely new context, so the following should both output 10:
$ ksh -c 'integer x; x=010; echo $x'
10 # ok; it's outside ((…)) so octals are not recognised
$ ksh -c 'echo $(( $(integer x; x=010; echo $x) ))'
8 # bad; $(comsub) should create new non-((…)) context
src/cmd/ksh93/bltins/print.c: extend():
- For the u, d, i, o, x, and X conversion modifiers, set the POSIX
arithmetic context flag before calling sh_strnum() to convert the
argument. This fixes bug 1.
src/cmd/ksh93/sh/subshell.c: sh_subshell():
- When invoking a command substitution, save and unset the POSIX
arithmetic context flag. Restore it at the end. This fixes bug 2.
Reported-by: @stephane-chazelas
Resolves: https://github.com/ksh93/ksh/issues/326
2021-09-13 02:23:50 +00:00
|
|
|
2021-09-13:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug introduced in 93u+ 2012-02-07 that caused the 'printf' builtin
|
|
|
|
|
(and its 'print -f' equivalent) to fail to recognise integer arguments with a
|
|
|
|
|
leading zero as octal numbers. For example, 'printf "%d\n" 010' now once
|
|
|
|
|
again outputs '8' instead of '10'.
|
|
|
|
|
|
|
|
|
|
- Disable the POSIX arithmetic context while running a command substitution
|
|
|
|
|
invoked from within an arithmetic expression. This fixes a bug that caused
|
|
|
|
|
integer arguments with a leading zero to be incorrectly interpreted as octal
|
|
|
|
|
numbers in non-POSIX arithmetic contexts within such command substitutions.
|
|
|
|
|
|
Fix 'ps' output for hashbangless scripts on Linux/macOS
When invoking a script without an interpreter (#!hashbang) path,
ksh forks, but there is no exec syscall in the child. The existing
command line is overwritten in fixargs() with the name of the new
script and associated arguments. In the generic/fallback version of
fixargs() which is used on Linux and macOS, if the new command line
is longer than the existing one, it is truncated. This works well
when calling a script with a shorter name.
However, it generates a misleading name in the common scenario
where a script is invoked from an interactive shell, which
typically has a short command line. For instance, if "/tmp/script"
is invoked, "ksh" gets replaced with "/tm" in "ps" output.
A solution is found in the fact that, on these systems, the
environment is stored immediately after the command line arguments.
This space can be made available for use by a longer command line
by moving the environment strings out of the way.
src/cmd/ksh93/sh/main.c: fixargs():
- Refactor BSD setproctitle(3) version to be more self-contained.
- In the generic (Linux/macOS) version, on init (i.e. mode==0), if
the command line is smaller than 128 bytes and the environment
strings have not yet been moved (i.e. if they still immediately
follow the command line arguments in memory), then strdup the
environment strings, pointing the *environment[] members to the
new strings and adding the length of the strings to the maximum
command line buffer size.
Reported-by: @gkamat
Resolves: https://github.com/ksh93/ksh/pull/300
2021-09-12 03:16:51 +00:00
|
|
|
2021-09-12:
|
|
|
|
|
|
|
|
|
|
- When invoking a script without an interpreter/hashbang path on Linux and
|
|
|
|
|
macOS, ksh can now update 'ps' output to show longer command lines.
|
|
|
|
|
|
Fix `backtick` comsubs by making them act like $(modern) ones
ksh93 currently has three command substitution mechanisms:
- type 1: old-style backtick comsubs that use a pipe;
- type 3: $(modern) comsubs that use a temp file, currently with
fallback to a pipe if a temp file cannot be created;
- type 2: ${ shared-state; } comsubs; same as type 3, but shares
state with parent environment.
Type 1 is buggy. There are at least two reproducers that make it
hang. The Red Hat patch applied in 4ce486a7 fixed a hang in
backtick comsubs but reintroduced another hang that was fixed in
ksh 93v-. So far, no one has succeeded in making pipe-based comsubs
work properly.
But, modern (type 3) comsubs use temp files. How does it make any
sense to have two different command substitution mechanisms at the
execution level? The specified functionality between backtick and
modern command substitutions is exactly the same; the difference
*should* be purely syntactic.
So this commit removes the type 1 comsub code at the execution
level, treating them all like type 3 (or 2). As a result, the
related bugs vanish while the regression tests all pass.
The only side effect that I can find is that the behaviour of bug
https://github.com/ksh93/ksh/issues/124 changes for backtick
comsubs. But it's broken either way, so that's neutral.
So this commit can now be added to my growing list of ksh93 issues
fixed by simply removing code.
src/cmd/ksh93/sh/xec.c:
- Remove special code for type 1 comsubs from iousepipe(),
sh_iounpipe(), sh_exec() and _sh_fork().
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/subshell.c:
- Remove pipe support from sh_subtmpfile(). This also removes the
use of a pipe as a fallback for $(modern) comsubs. Instead, panic
and error out if temp file creation fails. If the shell cannot
create a temporary file, there are fatal system problems anyway
and a script should not continue.
- No longer pass comsub type to sh_subtmpfile().
All other changes:
- Update sh_subtmpfile() calls.
src/cmd/ksh93/tests/subshell.sh:
- Add two regression tests based on reproducers from bug reports.
Resolves: https://github.com/ksh93/ksh/issues/305
Resolves: https://github.com/ksh93/ksh/issues/316
2021-08-13 06:01:46 +00:00
|
|
|
2021-08-13:
|
|
|
|
|
|
|
|
|
|
- An issue was fixed that could cause old-style `backtick` command
|
|
|
|
|
substitutions to hang in certain cases.
|
|
|
|
|
|
[[ ... ]]: fix '!' to negate another '!'
Bug: [[ ! ! 1 -eq 1 ]] returns false, but should return true.
This bug was reported for bash, but ksh has it too:
https://lists.gnu.org/archive/html/bug-bash/2021-06/msg00006.html
Op 24-05-21 om 17:47 schreef Chet Ramey:
> On 5/22/21 2:45 PM, Vincent Menegaux wrote:
>> Previously, these commands:
>>
>> [[ ! 1 -eq 1 ]]; echo $?
>> [[ ! ! 1 -eq 1 ]]; echo $?
>>
>> would both result in `1', since parsing `!' set CMD_INVERT_RETURN
>> instead of toggling it.
>
> Interestingly, ksh93 produces the same result as bash. I agree
> that it's more intuitive to toggle it.
Also interesting is that '!' as an argument to the simple
'test'/'[' command does work as expected (on both bash and ksh93):
'test ! ! 1 -eq 1' and '[ ! ! 1 -eq 1 ]' return 0/true.
Even the man page for [[ is identical for bash and ksh93:
| ! expression
| True if expression is false.
This suggests it's supposed to be a logical negation operator, i.e.
'!' is implicitly documented to negate another '!'. Bolsky & Korn's
1995 ksh book, p. 167, is slightly more explicit about it:
"! test-expression. Logical negation of test-expression."
I also note that multiple '!' negators in '[[' work as expected on
mksh, yash and zsh.
src/cmd/ksh93/sh/parse.c: test_primary():
- Fix bitwise logic for '!': xor the TNEGATE bit into tretyp
instead of or'ing it, which has the effect of toggling it.
2021-06-03 13:47:25 +00:00
|
|
|
2021-06-03:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug in the [[ compound command: the '!' logical negation operator
|
|
|
|
|
now correctly negates another '!', e.g., [[ ! ! 1 -eq 1 ]] now returns
|
|
|
|
|
0/true. Note that this has always been the case for 'test'/'['.
|
|
|
|
|
|
2021-05-18 22:02:49 +00:00
|
|
|
2021-05-18:
|
|
|
|
|
|
|
|
|
|
- Fixed SHLVL so that replacing ksh by itself (exec ksh) will not increase it.
|
|
|
|
|
|
2021-05-19 04:53:56 +00:00
|
|
|
- Fixed a regression introduced on 2020-08-05 that caused a non-interactive
|
|
|
|
|
shell to exit if an I/O redirection of a function call encountered an error.
|
|
|
|
|
|
2021-05-13 07:30:28 +00:00
|
|
|
2021-05-13:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug with 'test -t 1' that was introduced on 2021-04-26:
|
|
|
|
|
v=$(test -t 1 >/dev/tty && echo ok) did not assign 'ok' to v.
|
|
|
|
|
|
2021-05-10 16:40:44 +00:00
|
|
|
2021-05-10:
|
|
|
|
|
|
|
|
|
|
- Release 1.0.0-beta.1.
|
|
|
|
|
|
2021-05-08 03:43:37 +00:00
|
|
|
2021-05-07:
|
|
|
|
|
|
|
|
|
|
- Backported three ksh 93v- math.tab changes, allowing for an exp10()
|
|
|
|
|
arithmetic function if one exists in the C library, a new float()
|
|
|
|
|
function, and lastly an updated int() function that rounds to zero
|
|
|
|
|
instead of being an alias to floor().
|
|
|
|
|
|
Fix two more 'command' bugs
BUG 1: Though 'command' is specified/documented as a regular
builtin, preceding assignments survive the invocation (as with
special or declaration builtins) if 'command' has no command
arguments in these cases:
$ foo=wrong1 command; echo $foo
wrong1
$ foo=wrong2 command -p; echo $foo
wrong2
$ foo=wrong3 command -x; echo $foo
wrong3
Analysis: sh_exec(), case TCOM (simple command), contains the
following loop that skips over 'command' prefixes, preparsing any
options and remembering the offset in the 'command' variable:
src/cmd/ksh93/sh/xec.c
1059 while(np==SYSCOMMAND || !np && com0
&& nv_search(com0,shp->fun_tree,0)==SYSCOMMAND)
1060 {
1061 register int n = b_command(0,com,&shp->bltindata);
1062 if(n==0)
1063 break;
1064 command += n;
1065 np = 0;
1066 if(!(com0= *(com+=n)))
1067 break;
1068 np = nv_bfsearch(com0, shp->bltin_tree, &nq, &cp);
1069 }
This skipping is not done if the preliminary b_command() call on
line 1061 (with argc==0) returns zero. This is currently the case
for command -v/-V, so that 'command' is treated as a plain and
regular builtin for those options.
The cause of the bug is that this skipping is even done if
'command' has no arguments. So something like 'foo=bar command' is
treated as simply 'foo=bar', which of course survives.
So the fix is for b_command() to return zero if there are no
arguments. Then b_command() itself needs changing to not error out
on the second/main b_command() call if there are no arguments.
src/cmd/ksh93/bltins/whence.c: b_command():
- When called with argc==0, return a zero offset not just for -v
(X_FLAG) or -V (V_FLAG), but also if there are no arguments left
(!*argv) after parsing options.
- When called with argc>0, do not issue a usage error if there are
no arguments, but instead return status 0 (or, if -v/-V was given,
status 2 which was the status of the previous usage message).
This way, 'command -v $emptyvar' now also works as you'd expect.
BUG 2: 'command -p' sometimes failed after executing certain loops.
src/cmd/ksh93/sh/path.c: defpath_init():
- astconf() returns a pointer to memory that may be overwritten
later, so duplicate the string returned. Backported from ksh2020.
(re: f485fe0f, aa4669ad, <https://github.com/att/ast/issues/959>)
src/cmd/ksh93/tests/builtins.sh:
- Update the test for BUG_CMDSPASGN to check every variant of
'command' (all options and none; invoking/querying all kinds of
command and none) with a preceding assignment. (re: fae8862c)
This also covers bug 2 as 'command -p' was failing on macOS prior
to the fix due to a loop executed earlier in another test.
2021-05-05 01:42:45 +00:00
|
|
|
2021-05-05:
|
|
|
|
|
|
|
|
|
|
- Fixed: a preceding variable assignment like foo=bar in 'foo=bar command'
|
|
|
|
|
(with no command arguments after 'command') incorrectly survived the
|
|
|
|
|
'command' regular built-in command invocation.
|
|
|
|
|
|
|
|
|
|
- Fixed: 'command -p some_utility' intermittently failed to find the utility
|
|
|
|
|
under certain conditions due to a memory corruption issue.
|
|
|
|
|
|
2021-05-03 01:59:09 +00:00
|
|
|
2021-05-03:
|
|
|
|
|
|
Fix $RANDOM to act consistently in subshells (#294)
This fixes the following:
1. Using $RANDOM in a virtual/non-forked subshell no longer
influences the reproducible $RANDOM sequence in the parent
environment.
2. When invoking a subshell $RANDOM is now re-seeded (as mksh and
bash do) so that invocations in repeated subshells (including
forked subshells) longer produce identical sequences by default.
3. Program flow corruption that occurred in scripts on executing
( ( simple_command & ) ).
src/cmd/ksh93/include/variables.h:
- Move 'struct rand' here as it will be needed in subshell.c. Add
rand_seed member to save the pseudorandom generator seed. Remove
the pointer to the shell state as it's redundant.
src/cmd/ksh93/sh/init.c:
- put_rand(): Store given seed in rand_seed while calling srand().
No longer pointlessly limit the number of possible seeds with the
RANDMASK bitmask (that mask is to limit the values to 0-32767,
it should not limit the number of possible sequences to 32768).
- nget_rand(): Instead of using rand(), use rand_r() to update the
random_seed value. This makes it possible to save/restore the
current seed of the pseudorandom generator.
- Add sh_reseed_rand() function that reseeds the pseudorandom
generator by calling srand() with a bitwise-xor combination of
the current PID, the current time with a granularity of 1/10000
seconds, and a sequence number that is increased on each
invocation.
- nv_init(): Set the initial seed using sh_reseed_rand() here
instead of in sh_main(), as this is where the other struct rand
members are initialised.
src/cmd/ksh93/sh/main.c: sh_main():
- Remove the srand() call that was replaced by the sh_reseed_rand()
call in init.c.
src/cmd/ksh93/sh/subshell.c: sh_subshell():
- Upon entering a virtual subshell, save the current $RANDOM seed
and state, then reseed $RANDOM for the subshell.
- Upon exiting a virtual subshell, restore $RANDOM seed and state
and reseed the generator using srand() with the restored seed.
src/cmd/ksh93/sh/xec.c: sh_exec():
- When optimizing out a subshell that is the last command, still
act like a subshell: reseed $RANDOM and increase ${.sh.subshell}.
- Fix a separate bug discovered while implementing this. Do not
optimize '( simple_command & )' when in a virtual subshell; doing
this causes program flow corruption.
- When optimizing '( simple_command & )', also reseed $RANDOM and
increment ${.sh.subshell}.
src/cmd/ksh93/tests/subshell.sh,
src/cmd/ksh93/tests/variables.sh:
- Add various tests for all of the above.
Co-authored-by: Johnothan King <johnothanking@protonmail.com>
Resolves: https://github.com/ksh93/ksh/issues/285
2021-05-03 03:03:46 +00:00
|
|
|
- Subshells (even if non-forked) now keep a properly separated state of the
|
|
|
|
|
pseudorandom generator used for $RANDOM, so that using $RANDOM in a
|
|
|
|
|
non-forked subshell no longer influences a reproducible $RANDOM sequence in
|
|
|
|
|
the parent environment. In addition, upon invoking a subshell, $RANDOM is now
|
|
|
|
|
reseeded (as mksh and bash do).
|
|
|
|
|
|
|
|
|
|
- Fixed program flow corruption that occurred in scripts on executing a
|
|
|
|
|
background job in a nested subshell, as in ( ( simple_command & ) ).
|
|
|
|
|
|
2021-05-03 01:59:09 +00:00
|
|
|
- Completed the 2021-04-30 fix for ${var<OP>'{}'} where <OP> is '-', '+',
|
|
|
|
|
':-' or ':+' by fixing a bug that caused an extra '}' to be output.
|
|
|
|
|
|
2021-05-03 23:34:06 +00:00
|
|
|
- Following the resolution of Austin Group bug 1393[*] that is set to be
|
|
|
|
|
included in the next version of the POSIX standard, the 'command' prefix
|
|
|
|
|
in POSIX mode (set -o posix) no longer disables the declaration properties
|
|
|
|
|
of declaration built-ins. This reverts a change introduced on 2020-09-11.
|
|
|
|
|
[*] https://austingroupbugs.net/view.php?id=1393
|
|
|
|
|
|
2021-05-04 02:13:14 +00:00
|
|
|
- Fixed arithmetic assignment operations for multidimensional indexed arrays
|
|
|
|
|
|
2021-04-30 04:10:04 +00:00
|
|
|
2021-04-30:
|
|
|
|
|
|
2021-04-30 18:45:50 +00:00
|
|
|
- The emacs 'ESC .' (M-.) and vi '_' commands now take shell quoting into
|
|
|
|
|
account when repeating a word from the previous command line. For example, if
|
|
|
|
|
the previous command is 'ls Stairway\ To\ Heaven.mp3', then they now insert
|
|
|
|
|
'Stairway\ To\ Heaven.mp3' instead of 'Heaven.mp3'. Thanks to Govind Kamat.
|
|
|
|
|
|
2021-04-30 04:10:04 +00:00
|
|
|
- Fixed a bug introduced on 2020-09-05 that caused "echo ${var:+'{}'}"
|
|
|
|
|
to be misparsed.
|
|
|
|
|
|
2021-04-30 23:47:39 +00:00
|
|
|
- Fixed: the effects of 'builtin', 'exec' and 'ulimit' leaked out of a parent
|
|
|
|
|
virtual subshell if run from a ${ shared-state; } command substitution.
|
|
|
|
|
|
2021-04-26 17:09:36 +00:00
|
|
|
2021-04-26:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug introduced on 2021-02-20 in which a shared-state command
|
|
|
|
|
substitution stopped sharing its state with the calling shell environment
|
|
|
|
|
if it executed a command that locally redirected standard outpuut.
|
|
|
|
|
|
2021-04-22 02:25:24 +00:00
|
|
|
2021-04-22:
|
|
|
|
|
|
|
|
|
|
- shcomp (the shell bytecode compiler) was fixed to correctly compile process
|
|
|
|
|
substitutions used as the file name to a redirection, as in 'cmd < <(cmd)'.
|
|
|
|
|
|
Fix LINENO after unsetting it a virtual subshell (#283)
There is a TODO note in variables.sh that notes the value of LINENO
is wrong after a virtual subshell. The following script should
print '6', but the bug causes it to print '1' instead:
$ cat /tmp/lineno
#!/bin/ksh
(
unset LINENO
:
)
echo $LINENO
This bug started to occur after the bugfix applied in 7b994b6a.
However, that commit is not where the cause of bug was (when that
bugfix is applied to ksh versions 2008-07-25 through 2012-01-01,
$LINENO works fine). Rather, the cause of this bug was introduced
in 93u+ 2012-02-29. In that version, the mp->nvfun pointer was only
copied from np->nvfun if the variable can be freed from memory.
This is what caused 7b994b6a to break $LINENO in subshells, so to
fix this bug the mp->nvfun and np->nvfun must point to the same
object, even when the variable isn't freed from memory.
src/cmd/ksh93/sh/subshell.c: nv_restore():
- Always copy the np->nvfun pointer to mp->nvfun. To prevent
crashes, the value of np->nvfun->nofree is set to the value given
by the nofree variable, which is set before _nv_unset. See also
commit 7e7f1372, which fixed a crash that happened because
_nv_unset discards the NV_NOFREE flag.
src/cmd/ksh93/tests/variables.sh:
- Remove the workaround for LINENO after a virtual subshell.
- Add a regression test for the value of LINENO when unset in a
virtual subshell, then used after the subshell. Note that before
commit 997ad43b LINENO's value was corrupted after being unset in
a subshell, so the test checks for corruption of the LINENO
variable (in prior commits LINENO was set to '49' because of the
previous bug).
2021-04-22 18:16:25 +00:00
|
|
|
- Fixed a bug introduced on 2020-07-13 that set LINENO to the wrong line
|
|
|
|
|
number after leaving a virtual subshell in which LINENO had been unset.
|
|
|
|
|
|
2021-04-21 18:28:46 +00:00
|
|
|
2021-04-21:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug introduced on 2020-09-28 that caused an interactive ksh to exit
|
|
|
|
|
if a profile script (such as ~/.kshrc) contains a syntax error.
|
|
|
|
|
|
2021-04-21 02:34:54 +00:00
|
|
|
2021-04-20:
|
|
|
|
|
|
|
|
|
|
- Fixed three problems with the /opt/ast/bin/getconf built-in command:
|
|
|
|
|
1. The -l/--lowercase option did not change all variable names to lower case.
|
|
|
|
|
2. The -q/--quote option now quotes all string values. Previously, it only
|
|
|
|
|
quoted string values that had a space or other non-shellsafe character.
|
|
|
|
|
3. The -c/--call, -n/--name and -s/--standard options matched all variable
|
|
|
|
|
names provided by 'getconf -a', even if none were actual matches.
|
|
|
|
|
|
2021-04-21 02:56:03 +00:00
|
|
|
- The readonly attribute of ksh variables is no longer imported from
|
|
|
|
|
or exported to other ksh shell instances through the environment.
|
|
|
|
|
|
2021-04-16 13:45:01 +00:00
|
|
|
2021-04-16:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug in emacs mode: after using tab completion to complete the name
|
|
|
|
|
of a directory, it was not possible to type numbers after the slash.
|
|
|
|
|
|
Fix <>; redirection for final command exec optimization (#277)
The <>; operator doesn't work correctly if it's used as the last
command of a -c script. Reproducer:
$ echo test > a; ksh -c 'echo x 1<>; a'; cat a
x
st
This bug is caused by ksh running the last command of -c scripts
with execve(2) instead of posix_spawn(3) or fork(2). The <>;
operator is noted by the man page as being incompatible with the
exec builtin (see also the ksh93u+ man page), so it's not
surprising this bug occurs when ksh runs a command using execve:
> <>;word cannot be used with the exec and redirect built-ins.
The ksh2020 fix simply removed the code required for ksh to use
this optimization at all. It's not a performance friendly fix and
only papers over the bug, so this commit provides a better fix.
This bug was first reported at:
https://github.com/att/ast/issues/9
In addition, this commit re-enables the execve(2) optimization for
the last command for scripts loaded from a file. It was enabled in
in older ksh versions, and was only disabled in interactive shells:
https://github.com/ksh93/ast-open-history/blob/2011-06-30/src/cmd/ksh93/sh/main.c#L593-L599
It was changed on 2011-12-24 to only be used for -c scripts:
https://github.com/ksh93/ast-open-history/blob/2011-12-24/src/cmd/ksh93/sh/main.c#L593-L599
We think there is no good reason why scripts loaded from a file
should be optimised less than scripts loaded from a -c argument.
They're both scripts; there's no essential difference between them.
So this commit reverts that change. If there is a bug left in the
optimization after this fix, this revert increases the chance of
exposing it so that it can be fixed.
src/cmd/ksh93/sh/xec.c:
- The IOREWRITE flag is set when handling the <>; operator, so to
fix this bug, avoid exec'ing the last command if it uses <>;. See
also commit 17ebfbf6, which fixed another issue related to the
execve optimization.
src/cmd/ksh93/tests/io.sh:
- Enable a regression test that was failing because of this bug.
- Add the reproducer from https://github.com/att/ast/issues/9 as a
regression test.
src/cmd/ksh93/sh/main.c:
- Only avoid the non-forking optimization in interactive shells.
src/cmd/ksh93/tests/signal.sh:
- Add an extra comment to avoid the non-forking optimization in the
regression test for rhbz#1469624.
- If the regression test for rhbz#1469624 fails, show the incorrect
exit status in the error message.
src/cmd/ksh93/tests/builtins.sh,
src/cmd/ksh93/tests/options.sh:
- This bugfix was causing the options regression test to segfault
when run under shcomp. The cause is the same as
<https://github.com/ksh93/ksh/issues/165>, so as a workaround,
avoid parsing process substitutions with shcomp until that is
fixed. This workaround should also avoid the other problem
detailed in <https://github.com/ksh93/ksh/issues/274>.
Resolves: https://github.com/ksh93/ksh/issues/274
2021-04-15 17:29:50 +00:00
|
|
|
- Fixed an optimization bug that caused the <>; redirection operator to fail
|
2021-04-17 20:31:38 +00:00
|
|
|
when used with the last command in a -c script.
|
Fix <>; redirection for final command exec optimization (#277)
The <>; operator doesn't work correctly if it's used as the last
command of a -c script. Reproducer:
$ echo test > a; ksh -c 'echo x 1<>; a'; cat a
x
st
This bug is caused by ksh running the last command of -c scripts
with execve(2) instead of posix_spawn(3) or fork(2). The <>;
operator is noted by the man page as being incompatible with the
exec builtin (see also the ksh93u+ man page), so it's not
surprising this bug occurs when ksh runs a command using execve:
> <>;word cannot be used with the exec and redirect built-ins.
The ksh2020 fix simply removed the code required for ksh to use
this optimization at all. It's not a performance friendly fix and
only papers over the bug, so this commit provides a better fix.
This bug was first reported at:
https://github.com/att/ast/issues/9
In addition, this commit re-enables the execve(2) optimization for
the last command for scripts loaded from a file. It was enabled in
in older ksh versions, and was only disabled in interactive shells:
https://github.com/ksh93/ast-open-history/blob/2011-06-30/src/cmd/ksh93/sh/main.c#L593-L599
It was changed on 2011-12-24 to only be used for -c scripts:
https://github.com/ksh93/ast-open-history/blob/2011-12-24/src/cmd/ksh93/sh/main.c#L593-L599
We think there is no good reason why scripts loaded from a file
should be optimised less than scripts loaded from a -c argument.
They're both scripts; there's no essential difference between them.
So this commit reverts that change. If there is a bug left in the
optimization after this fix, this revert increases the chance of
exposing it so that it can be fixed.
src/cmd/ksh93/sh/xec.c:
- The IOREWRITE flag is set when handling the <>; operator, so to
fix this bug, avoid exec'ing the last command if it uses <>;. See
also commit 17ebfbf6, which fixed another issue related to the
execve optimization.
src/cmd/ksh93/tests/io.sh:
- Enable a regression test that was failing because of this bug.
- Add the reproducer from https://github.com/att/ast/issues/9 as a
regression test.
src/cmd/ksh93/sh/main.c:
- Only avoid the non-forking optimization in interactive shells.
src/cmd/ksh93/tests/signal.sh:
- Add an extra comment to avoid the non-forking optimization in the
regression test for rhbz#1469624.
- If the regression test for rhbz#1469624 fails, show the incorrect
exit status in the error message.
src/cmd/ksh93/tests/builtins.sh,
src/cmd/ksh93/tests/options.sh:
- This bugfix was causing the options regression test to segfault
when run under shcomp. The cause is the same as
<https://github.com/ksh93/ksh/issues/165>, so as a workaround,
avoid parsing process substitutions with shcomp until that is
fixed. This workaround should also avoid the other problem
detailed in <https://github.com/ksh93/ksh/issues/274>.
Resolves: https://github.com/ksh93/ksh/issues/274
2021-04-15 17:29:50 +00:00
|
|
|
|
Allow invoking path-bound built-in commands by direct path or preceding `PATH` assignment (#275)
Path-bound builtins on ksh (such as /opt/ast/bin/cat) break some
basic assumptions about paths in the shell that should hold true,
e.g., that a path output by whence -p or command -v should actually
point to an executable command. This commit should fix the
following:
1. Path-bound built-ins (such as /opt/ast/bin/cat) can now be
executed by invoking the canonical path (independently of the
value of $PATH), so the following will now work as expected:
$ /opt/ast/bin/cat --version
version cat (AT&T Research) 2012-05-31
$ (PATH=/opt/ast/bin:$PATH; "$(whence -p cat)" --version)
version cat (AT&T Research) 2012-05-31
In the event an external command by that path exists, the
path-bound builtin will now override it when invoked using the
canonical path. To invoke a possible external command at that
path, you can still use a non-canonical path, e.g.:
/opt//ast/bin/cat or /opt/ast/./bin/cat
2. Path-bound built-ins will now also be found on a PATH set
locally using an assignment preceding the command, so something
like the following will now work as expected:
$ PATH=/opt/ast/bin cat --version
version cat (AT&T Research) 2012-05-31
The builtin is not found by sh_exec() because the search for
builtins happens long before invocation-local preceding
assignments are processsed. This only happens in sh_ntfork(),
before forking, or in sh_fork(), after forking. Both sh_ntfork()
and sh_fork() call path_spawn() to do the actual path search, so
a check there will cover both cases.
This does mean the builtin will be run in the forked child if
sh_fork() is used (which is the case on interactive shells with
job.jobcontrol set, or always after compiling with SHOPT_SPAWN
disabled). Searching for it before forking would mean
fundamentally redesigning that function to be basically like
sh_ntfork(), so this is hard to avoid.
src/cmd/ksh93/sh/path.c: path_spawn():
- Before doing anything else, check if the passed path appears in
the builtins tree as a pathbound builtin. If so, run it. Since a
builtin will only be found if a preceding PATH assignment
temporarily changed the PATH, and that assignment is currently in
effect, we can just sh_run() the builtin so a nested sh_exec()
invocation will find and run it.
- If 'spawn' is not set (i.e. we must return), set errno to 0 and
return -2. See the change to sh_ntfork() below.
src/cmd/ksh93/sh/xec.c:
- sh_exec(): When searching for built-ins and the restricted option
isn't active, also search bltin_tree for names beginning with a
slash.
- sh_ntfork(): Only throw an error if the PID value returned is
exactly -1. This allows path_spawn() to return -2 after running a
built-in to tell sh_ntfork() to do the right things to restore
state.
src/cmd/ksh93/sh/parse.c: simple():
- When searching for built-ins at parse time, only exclude names
containing a slash if the restricted option is active. This
allows finding pointers to built-ins invoked by literal path like
/opt/ast/bin/cat, as long as that does not result from an
expansion. This is not actually necessary as sh_exec() will also
cover this case, but it is an optimisation.
src/lib/libcmd/getconf.c:
- Replace convoluted deferral to external command by a simple
invocation of the path to the native getconf command determined
at compile time (by src/lib/libast/comp/conf.sh). Based on:
https://github.com/ksh93/ksh/issues/138#issuecomment-816384871
If there is ever a system that has /opt/ast/bin/getconf as its
default native external 'getconf', then there would still be an
infinite recursion crash, but this seems extremely unlikely.
Resolves: https://github.com/ksh93/ksh/issues/138
2021-04-15 03:08:12 +00:00
|
|
|
2021-04-14:
|
|
|
|
|
|
|
|
|
|
- Path-bound built-ins (such as /opt/ast/bin/cat) can now be executed by
|
|
|
|
|
invoking the canonical path, so the following will now work as expected:
|
|
|
|
|
$ /opt/ast/bin/cat --version
|
|
|
|
|
version cat (AT&T Research) 2012-05-31
|
|
|
|
|
$ (PATH=/opt/ast/bin:$PATH; "$(whence -p cat)" --version)
|
|
|
|
|
version cat (AT&T Research) 2012-05-31
|
|
|
|
|
Non-canonical paths such as /opt/ast/./bin/cat will not find the built-ins.
|
|
|
|
|
|
|
|
|
|
- Path-bound built-ins will now also be found on a PATH set locally using an
|
|
|
|
|
assignment preceding the command, so the following will now work as expected:
|
|
|
|
|
$ PATH=/opt/ast/bin cat --version
|
|
|
|
|
version cat (AT&T Research) 2012-05-31
|
|
|
|
|
|
Fix the exit status returned when a command isn't executable (#273)
Previous discussion: https://github.com/att/ast/issues/485
If ksh attempts to execute a non-executable command found in the
PATH, in some instances the error message and return status are
incorrect. In the example below, ksh returns with exit status 126
when using the -c execve(2) optimization or when using fork(2) in
an interactive shell. However, using posix_spawn(3) causes the exit
status to change:
$ echo 'print cannot execute' > /tmp/x
# Runs command with spawnveg (i.e., posix_spawn or vfork)
$ ksh -c 'PATH=/tmp; x; echo $?'
ksh: x: not found
127
# Runs command with execve
$ ksh -c 'PATH=/tmp; x'; echo $?
ksh: x: cannot execute [Permission denied]
126
# Runs command with fork
$ ksh -ic 'PATH=/tmp; x; echo $?'
ksh: x: cannot execute [Permission denied]
126
Since 'x' is in the PATH but can't be executed, the correct exit
status is 126, not 127. It's worth noting this bug doesn't cause
the regression tests to fail with ksh93u+m, but it does cause one
test to fail when run under dtksh:
path.sh[706]: Long nonexistent command name: got status 126, ''
This commit backports various fixes for this bug from ksh2020, with
additional fixes applied (since there were still some additional
issues the ksh2020 patch didn't fix). The lacking regression test
for exit status 126 in path.sh has been rewritten to test for more
scenarios where ksh failed to return the correct error message
and/or exit status. I can also confirm with this patch applied the
path.sh regression tests now pass when run under dtksh.
src/cmd/ksh93/sh/path.c:
- Add a comment to path_absolute() describing 'oldpp' is the
current pointer in the while loop and 'pp' is the next pointer.
Backported from:
https://github.com/att/ast/commit/a6cad450
- The patch from ksh2020 didn't fix this bug in the SHOPT_SPAWN
code (because ksh2020 prefers fork(2)), so issues with the exit
status could still occur when using spawnveg. To fix this, always
set 'noexec' to the value of errno if can_execute fails. Before
this fix, errno was discarded if 'pp' was a null pointer and
can_execute failed.
- If a command couldn't be executed and the error wasn't ENOENT,
save errno in a 'not_executable' variable. If an executable
command couldn't be found in the PATH, exit with status 126 and
set errno to the saved value. This was based on a ksh2020 bugfix,
but it has been reworked a little bit to fix a bug that caused a
mismatch between the error message shown and errno. Example with
a non-executable file in PATH:
$ nonexec
ksh2020: nonexec: cannot execute [No such file or directory]
The ksh2020 patch: <https://github.com/att/ast/pull/493>
- Backport a ksh2020 bugfix for directories in the PATH when
running one of the added regression tests on OpenBSD:
https://github.com/att/ast/pull/767
src/cmd/ksh93/data/msg.c,
src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/sh/{path,xec}.c:
- If a command name is too long (ENAMETOOLONG), then it wasn't
found in the PATH. For that case return exit status 127, like
for ENOENT.
src/cmd/ksh93/tests/path.sh:
- Replace the old test with a new set of more extensive tests.
These tests check the error message and exit status when ksh
attempts to run a command using any of the following:
- execve(2), used with the last command run with -c (*A tests).
- posix_spawn(3)/vfork(2), used in noninteractive scripts (*B tests).
- fork(2), used in interactive shells with job control (*C tests).
- command -x (*D tests).
- exec(1) (*E tests).
- Add a regression test from ksh2020 for attempting to execute a
directory:
https://github.com/att/ast/pull/758
src/lib/libast/include/ast.h,
src/lib/libast/include/wait.h:
- Avoid bitshifts in macros for static error codes. The return
values of command not found and exec related errors are static
values and should not require any macro magic for calculation.
Backported from: https://github.com/att/ast/commit/c073b102
- Simplify EXIT_* and W* macros to use 8 bits.
2021-04-15 02:37:57 +00:00
|
|
|
2021-04-13:
|
|
|
|
|
|
|
|
|
|
- Fixed a few bugs that could cause ksh to show the wrong error message and/or
|
|
|
|
|
return the wrong exit status if a command couldn't be executed. In
|
|
|
|
|
scenarios where the command was found in the PATH but it was not executable,
|
|
|
|
|
ksh now returns with exit status 126. Otherwise, ksh will return with exit
|
|
|
|
|
status 127 (such as if the command isn't found or if the command name is
|
|
|
|
|
too long).
|
|
|
|
|
|
2021-04-13 02:15:34 +00:00
|
|
|
2021-04-12:
|
|
|
|
|
|
|
|
|
|
- Corrected a memory fault when an attempt was made to unset the default
|
|
|
|
|
nameref KSH_VERSION from the shell environment prior to any other name
|
2021-04-23 21:02:30 +00:00
|
|
|
reference variable creation or modification.
|
2021-04-13 02:15:34 +00:00
|
|
|
|
2021-04-12 00:24:33 +00:00
|
|
|
2021-04-11:
|
|
|
|
|
|
|
|
|
|
- Fixed two related regressions introduced on 2020-06-16:
|
|
|
|
|
1. The += assignment failed to append the value of variables when used
|
|
|
|
|
in an invocation-local scope. The following should print '5', but
|
|
|
|
|
the regression resulted in '3' being printed instead:
|
|
|
|
|
$ integer foo=2; foo+=3 command eval 'echo $foo'
|
|
|
|
|
3
|
|
|
|
|
2. Any += assignment used in an invocation-local scope could modify
|
|
|
|
|
readonly variables.
|
|
|
|
|
|
Reset arithmetic recursion level on all errors (re: 264ba48b)
The recursion level for arithmetic expressions is kept track of in
a static 'level' variable in streval.c. It is reset when arithmetic
expressions throw an error.
But an error for an arithmetic expression may also occur elsewhere
-- at least in one case: when an arithmetic expression attempts to
change a read-only variable. In that case, the recursion level is
never reset because that code does not have access to the static
'level' variable.
If many such conditions occur (as in the new readonly.sh regression
tests), an arithmetic command like 'i++' may eventually fail with a
'recursion too deep' error.
To mitigate the problem, MAXLEVEL in streval.c was changed from 9
to 1024 in 264ba48b (as in the ksh 93v- beta). This commit leaves
that increase, but adds a proper fix.
src/cmd/ksh93/include/defs.h:
- Add global sh.arithrecursion (a.k.a. shp->arithrecursion)
variable to keep track of the arithmetic recursion level,
replacing the static 'level' variable in streval.c.
src/cmd/ksh93/sh/xec.c: sh_exec():
- Reset sh.arithrecursion before starting a new simple command
(TCOM), a new subshell with parentheses (TPAR), a new pipe
(TFIL), or a new [[ ... ]] command (TTST). These are the same
places where 'echeck' is set to 1 for --errexit and ERR trap
checks, so it should cover everything.
src/cmd/ksh93/sh/streval.c:
- Change all uses of 'level' to sh.arithrecursion.
- _seterror, aritherror(): No longer bother to reset the level
to zero here; xec.c should have this covered for all cases now.
src/cmd/ksh93/tests/arith.sh:
- Add tests for main shell and subshell.
2021-04-10 23:08:26 +00:00
|
|
|
2021-04-10:
|
|
|
|
|
|
|
|
|
|
- Fixed: the internal count of the recursion level for arithmetic expressions
|
|
|
|
|
was not reset when certain errors occurred in a virtual subshell. This could
|
|
|
|
|
cause an erroneous "recursion to deep" error when a loop executed many
|
|
|
|
|
subshells containing arithmetic expressions with errors, e.g. for testing.
|
|
|
|
|
|
2021-04-09 22:26:07 +00:00
|
|
|
2021-04-09:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused ksh to enable -c during the shell's initialization
|
|
|
|
|
if the only argument passed was --posix.
|
|
|
|
|
|
|
|
|
|
- Fixed a related bug that caused 'set --posix' to leave the braceexpand and
|
|
|
|
|
letoctal shell options unchanged.
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused 'set --default' to unset the restricted option
|
|
|
|
|
in restricted shells.
|
|
|
|
|
|
2021-04-08 13:39:30 +00:00
|
|
|
2021-04-08:
|
|
|
|
|
|
|
|
|
|
- Path-bound builtins will now be used by restricted shells if /opt/ast/bin
|
|
|
|
|
is in the $PATH upon invoking the shell or before setting it to restricted.
|
|
|
|
|
|
2021-04-09 02:49:48 +00:00
|
|
|
- Fixed a bug that caused "printf '%T\n' now" to ignore $LC_ALL and $LC_TIME
|
|
|
|
|
if the current locale was previously set, unset then set again.
|
|
|
|
|
|
2021-04-08 12:06:22 +00:00
|
|
|
2021-04-07:
|
|
|
|
|
|
|
|
|
|
- The $LC_TIME variable is now recognized by ksh and if set to an invalid
|
|
|
|
|
locale will show an error.
|
|
|
|
|
|
2021-04-08 12:24:17 +00:00
|
|
|
- Fixed BUG_CSUBSTDO: If standard output is closed before running a command
|
|
|
|
|
substitution, redirecting any other file descriptor no longer closes standard
|
|
|
|
|
output inside of the command substitution.
|
|
|
|
|
|
Fix unsetting array element after expanding array subscript range
Simple reproducer:
set -A arr a b c d; : ${arr[1..2]}; unset arr[1]; echo ${arr[@]}
Output:
a
Expected output:
a c d
The ${arr[1..2]} expansion broke the subsequent 'unset' command
so that it unsets element 1 and on, instead of only 1.
This regression was introduced in nv_endsubscript() on 2009-07-31:
https://github.com/ksh93/ast-open-history/commit/c47896b4/src/cmd/ksh93/sh/array.c
That change checks for the ARRAY_SCAN attribute which enables
processing ranges of array elements instead of single array
elements, and restores it after. That restore is evidently not
correct as it causes the subsequent unset command to malfunction.
If we revert that change, the bug disappears and the regression
tests show no failures. However, I don't know what this was meant
to accomplish and what other bug we might introduce by reverting
this. However, no corresponding regression test was added along
with the 2009-07-31 change, nor is there any corresponding message
in the changelog. So this looks to be one of those mystery changes
that we'll never know the reason for.
Since we currently have proof that this change causes breakage and
no evidence that it fixes anything, I'll go ahead and revert it
(and add a regression test, of course). If that causes another
regression, hopefully someone will find it at some point.
src/cmd/ksh93/sh/array.c: nv_endsubscript():
- Revert the 2009-07-31 change that saves/restores the ARRAY_SCAN
attribute.
- Keep the 'ap' pointer as it is now used by newer code. Move the
declaration up to the beginning of the block, as is customary.
src/cmd/ksh93/sh/init.c:
- Cosmetic change: remove an unused array_scan() macro that I found
when grepping the code for ARRAY_SCAN. The macro was introduced
in version 2001-06-01 but the code that used it was replaced in
version 2001-07-04, without removing the macro itself.
Resolves: https://github.com/ksh93/ksh/issues/254
2021-04-05 21:11:23 +00:00
|
|
|
2021-04-05:
|
|
|
|
|
|
|
|
|
|
- Fixed a regression, introduced in ksh 93t+ 2009-07-31, that caused a command
|
|
|
|
|
like 'unset arr[3]' to unset not just element 3 of the array but all elements
|
|
|
|
|
starting from 3, if a range expansion like ${arr[5..10]} was previously used.
|
|
|
|
|
|
Backport bugfixes for arrays of 'enum' types from ksh 93v- beta
These fixes are applied rather blindly as no one has yet managed to
understand the almost entirely uncommented arrays and variables
handling code (arrays.c, name.c, nvdisc.c, nvtree.c, nvtype.c).
Hopefully we'll figure all that out at some point. In the meantime
these backported fixes appear to work fine, and these bugs impact
the usability of 'enum', so I'm just going to have to violate my
own policy and backport these fixes without understanding them.
Thanks to @JohnoKing for putting in a lot of work tracing these.
Further discussion at: https://github.com/ksh93/ksh/issues/87
src/cmd/ksh93/sh/array.c:
- nv_arraysettype():
* Further simplify the function. After my initial simplification
of it (re: 5491fe97), I don't believe there's actually a need
to save a duplicate copy of the value. Use the pointer returned
by nv_getval() directly to restore the value.
* Cope with a null value (nv_getval() returning a NULL pointer).
This is needed for compatibility with the backported fix in
nvtype.c (below).
- array_putval(): If the array's value pointer (up->cp) is a
pointer to the empty string, it is set to NULL before calling
nv_putv() to prevent an empty string from being deleted. Backport
a fix from 93v- that restores the pointer to the empty string if
the NV_NOFREE attribute is set. Removing it somehow causes these
regressions:
enum.sh[86]: ${array[@]} doesn't yield all values for
associative enum arrays (expected 'green blue blue red
yellow green red orange'; got 'green blue blue yellow
green orange')
enum.sh[94]: unsetting associative enum array does not work
(got 'Color_t -A Colors=([foo]=red [rood]=red)')
enum.sh[116]: assigning first enum element to indexed array
failed (expected 'red red'; got 'BUG BUG')
- nv_associative(): Do not increase the 'nelem' (number of
elements) value of the array's 'header' struct if the array is
associative and of an enum type. The original 93v- fix only
checked for the NV_INTEGER attribute, but backporting that caused
several regressions. Using a debug output command I've determined
that the exact value of 'type' is somehow consistently set to
0x26 if the array is associative and of an enum type, which is
NV_INTEGER | NV_LTOU | NV_RJUST as defined in include/nval.h. I
cannot find where/how that value is determined. In any case this
fix, based on but more specific than the 93v- one, appears to
work fine. Removing it somehow causes this regression:
enum.sh[94]: unsetting associative enum array does not work
(got 'Color_t -A Colors=()')
src/cmd/ksh93/sh/nvtype.c: nv_settype():
- Another fix backported from 93v-. If the variable is an array,
also set the type of element 0 of that array using a call to
nv_arraysettype(). The value may be null. Removing this somehow
causes this regression:
enum.sh[94]: unsetting associative enum array does not work
(got 'Color_t -A Colors=()')
src/cmd/ksh93/tests/enum.sh:
- Add tests for all the bugs fixed here, plus some hypothetical
bugs (e.g., do the same tests for indexed enum type arrays as for
associative enum type arrays, even though indexed enum type
arrays didn't have all the same problems).
Co-authored-by: Johnothan King <johnothanking@protonmail.com>
Resolves: https://github.com/ksh93/ksh/issues/87
2021-04-06 04:45:46 +00:00
|
|
|
- Several fixes for arrays of a type created by 'enum' were backported from ksh
|
|
|
|
|
93v-, further to the two enum array fixes already applied on 2021-02-01:
|
|
|
|
|
1. The array[@]} expansion was fixed for associative arrays of an enum type.
|
|
|
|
|
2. Assignments now work correctly for all enum values for both indexed and
|
|
|
|
|
associative arrays.
|
|
|
|
|
3. 'unset' will now completely unset an associative array of an enum type.
|
|
|
|
|
|
2021-04-05 05:43:19 +00:00
|
|
|
2021-04-04:
|
|
|
|
|
|
Fix bell character handling when redrawing command line (#250)
To set a window title in bash and zsh, the $PS1 prompt can be set
with the title placed between $'\E]0;' and $'\a':
set -o emacs # Or vi mode
typeset -A fmt=(
[start_title]=$'\E]0;'
[end_title]=$'\a'
)
PS1="${fmt[start_title]}$(hostname): $(uname)${fmt[end_title]}\$ "
This also works in ksh unless the shell receives SIGWINCH. With a
$PS1 that sets a window title, the prompt breaks until two
interrupts are received. This is caused by ed_setup() skipping
$'\a' (the bell character) when setting up the e_prompt buffer
which is an edited version of the final line of the PS1 prompt for
use when redrawing the command line.
One fix would be to avoid cutting out the bell character. But if
the prompt contains a bell, we only want the terminal to beep when
a new prompt is printed, and not upon refreshing the command line,
e.g. when receiving SIGWINCH or pressing Ctrl+L.
To avoid the problem, this commit adds code that cuts out sequences
of the form ESC ] <number> ; <text> BELL from the prompt redraw
buffer altogether. They are not needed there because these
sequences will already have taken effect when the full prompt was
printed by io_prompt().
This commit also adds a tweak that should improve the recognition
of other escape sequences to count their length.
src/cmd/ksh93/edit/edit.c: ed_setup():
- When preparing the e_prompt buffer, cut out dtterm/xterm
Operating System Commands that set window/icon title, etc.
See: https://invisible-island.net/xterm/ctlseqs/ctlseqs.html
- When counting the length of escape sequences in that part of PS1,
try to recognize some more types of sequences. These changes are
part of a ksh2020 patch: https://github.com/att/ast/issues/399
src/cmd/ksh93/sh.1:
- Document that any '!' in escape sequences in the PS1 prompt needs
to be changed to '!!'. To avoid breaking compatibility, this
requirement is documented instead of backporting the changes to
io_prompt() from https://github.com/att/ast/issues/399 which try
to remove that requirement for specific escape sequences.
Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-04-05 07:06:53 +00:00
|
|
|
- A bug was fixed that caused a broken prompt display upon redrawing the
|
|
|
|
|
command line if the last line of the prompt includes an xterm escape
|
|
|
|
|
sequence that is terminated by $'\a' (the bell character).
|
|
|
|
|
|
2021-04-05 05:43:19 +00:00
|
|
|
- Harden readonly variables. Readonly variables or arrays no longer allow
|
|
|
|
|
attribute changes which would otherwise allow their value to be altered.
|
|
|
|
|
Expanded support for readonly variables within multidimensional arrays.
|
|
|
|
|
|
2021-04-04 21:18:43 +00:00
|
|
|
2021-04-03:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused the uname builtin's -d option to change the output
|
|
|
|
|
of the -o option.
|
|
|
|
|
|
|
|
|
|
- Fixed a possible crash that could occur when showing the domain name
|
|
|
|
|
with the uname builtin's -d option.
|
|
|
|
|
|
2021-04-02 00:19:19 +00:00
|
|
|
2021-03-31:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused 'cd -' to ignore the current value of $OLDPWD
|
|
|
|
|
when it's set to a different directory in a new scope.
|
|
|
|
|
|
|
|
|
|
- Fixed a related bug that caused ksh to use the wrong value for $PWD
|
|
|
|
|
when in a new scope.
|
|
|
|
|
|
2021-03-30 10:25:20 +00:00
|
|
|
2021-03-29:
|
|
|
|
|
|
|
|
|
|
- Fixed an intermittent crash that could occur in vi mode when using the 'b'
|
|
|
|
|
or 'B' commands to go back one word.
|
|
|
|
|
|
2021-03-27 21:51:16 +00:00
|
|
|
2021-03-27:
|
|
|
|
|
|
|
|
|
|
- The 'test' builtin will now show an error message when given the invalid ']]'
|
|
|
|
|
or '=~' operators; it also properly returns with exit status 2 now (instead
|
|
|
|
|
of exit status 1). If the invalid operator is supported by [[ ... ]] (such
|
|
|
|
|
as '=~'), test will now suggest the usage of [[ ... ]] instead.
|
|
|
|
|
|
Add --globcasedetect shell option for globbing and completion
One of the best-kept secrets of libast/ksh93 is that the code
includes support for case-insensitive file name generation (a.k.a.
pathname expansion, a.k.a. globbing) as well as case-insensitive
file name completion on interactive shells, depending on whether
the file system is case-insensitive or not. This is transparently
determined for each directory, so a path pattern that spans
multiple file systems can be part case-sensitive and part case-
insensitive. In more precise terms, each slash-separated path name
component pattern P is treated as ~(i:P) if its parent directory
exists on a case-insensitive file system. I recently discovered
this while dealing with <https://github.com/ksh93/ksh/issues/223>.
However, that support is dead code on almost all current systems.
It depends on pathconf(2) having a _PC_PATH_ATTRIBUTES selector.
The 'c' attribute is supposedly returned if the given directory is
on a case insensitive file system. There are other attributes as
well (at least 'l', see src/lib/libcmd/rm.c). However, I have been
unable to find any system, current or otherwise, that has
_PC_PATH_ATTRIBUTES. Google and mailing list searches yield no
relevant results at all. If anyone knows of such a system, please
add a comment to this commit on GitHub, or email me.
An exception is Cygwin/Windows, on which the "c" attribute was
simply hardcoded, so globbing/completion is always case-
insensitive. As of Windows 10, that is wrong, as it added the
possibility to mount case-sensitive file systems.
On the other hand, this was never activated on the Mac, even
though macOS has always used a case-insensitive file like Windows.
But, being UNIX, it can also mount case-sensitive file systems.
Finally, Linux added the possibility to create individual case-
insensitive ext4 directories fairly recently, in version 5.2.
https://www.collabora.com/news-and-blog/blog/2020/08/27/using-the-linux-kernel-case-insensitive-feature-in-ext4/
So, since this functionality latently exists in the code base, and
three popular OSs now have relevant file system support, we might
as well make it usable on those systems. It's a nice idea, as it
intuitively makes sense for globbing and completion behaviour to
auto-adapt to file system case insensitivity on a per-directory
basis. No other shell does this, so it's a nice selling point, too.
However, the way it is coded, this is activated unconditionally on
supported systems. That is not a good idea. It will surprise users.
Since globbing is used with commands like 'rm', we do not want
surprises. So this commit makes it conditional upon a new shell
option called 'globcasedetect'. This option is only compiled into
ksh on systems where we can actually detect FS case insensitivity.
To implement this, libast needs some public API additions first.
*** libast changes ***
src/lib/libast/features/lib:
- Add probes for the linux/fs.h and sys/ioctl.h headers.
Linux needs these to use ioctl(2) in pathicase(3) (see below).
src/lib/libast/path/pathicase.c,
src/lib/libast/include/ast.h,
src/lib/libast/man/path.3,
src/lib/libast/Mamfile:
- Add new pathicase(3) public API function. This uses whatever
OS-specific method it can detect at compile time to determine if
a particular path is on a case-insensitive file system. If no
method is available, it only sets errno to ENOSYS and returns -1.
Currently known to work on: macOS, Cygwin, Linux 5.2+, QNX 7.0+.
- On systems (if any) that have the mysterious _PC_PATH_ATTRIBUTES
selector for pathconf(2), call astconf(3) and check for the 'c'
attribute to determine case insensitivity. This should preserve
compatibility with any such system.
src/lib/libast/port/astconf.c:
- dynamic[]: As case-insensitive globbing is now optional on all
systems, do not set the 'c' attribute by default on _WINIX
(Cygwin/Windows) systems.
- format(): On systems that do not have _PC_PATH_ATTRIBUTES, call
pathicase(3) to determine the value for the "c" (case
insensitive) attribute only. This is for compatibility as it is
more efficient to call pathicase(3) directly.
src/lib/libast/misc/glob.c,
src/lib/libast/include/glob.h:
- Add new GLOB_DCASE public API flag to glob(3). This is like
GLOB_ICASE (case-insensitive matching) except it only makes the
match case-insensitive if the file system for the current
pathname component is determined to be case-insensitive.
- gl_attr(): For efficiency, call pathicase(3) directly instead of
via astconf(3).
- glob_dir(): Only call gl_attr() to determine file system case
insensitivity if the GLOB_DCASE flag was passed. This makes case
insensitive globbing optional on all systems.
- glob(): The options bitmask needs to be widened to fit the new
GLOB_DCASE option. Define this centrally in a new GLOB_FLAGMASK
macro so it is easy to change it along with GLOB_MAGIC (which
uses the remaining bits for a sanity check bit pattern).
src/lib/libast/path/pathexists.c:
- For efficiency, call pathicase(3) directly instead of via
astconf(3).
*** ksh changes ***
src/cmd/ksh93/features/options,
src/cmd/ksh93/SHOPT.sh:
- Add new SHOPT_GLOBCASEDET compile-time option. Set it to probe
(empty) by default so that the shell option is compiled in on
supported systems only, which is determined by new iffe feature
test that checks if pathicase(3) returns an ENOSYS error.
src/cmd/ksh93/data/options.c,
src/cmd/ksh93/include/shell.h:
- Add -o globcasedetect shell option if compiling with
SHOPT_GLOBCASEDET.
src/cmd/ksh93/sh/expand.c: path_expand():
- Pass the new GLOB_DCASE flag to glob(3) if the
globcasedetect/SH_GLOBCASEDET shell option is set.
src/cmd/ksh93/edit/completion.c:
- While file listing/completion is based on globbing and
automatically becomes case-insensitive when globbing does, it
needs some additional handling to make a string comparison
case-insensitive in corresponding cases. Otherwise, partial
completions may be deleted from the command line upon pressing
tab. This code was already in ksh 93u+ and just needs to be
made conditional upon SHOPT_GLOBCASEDET and globcasedetect.
- For efficiency, call pathicase(3) directly instead of via
astconf(3).
src/cmd/ksh93/sh.1:
- Document the new globcasedetect shell option.
2021-03-22 15:01:03 +00:00
|
|
|
2021-03-22:
|
|
|
|
|
|
|
|
|
|
- A new --globcasedetect shell option is added to ksh on OSs where we can check
|
|
|
|
|
for a case-insensitive file system (currently macOS, Windows/Cygwin, Linux
|
Fix more compiler warnings, typos and other minor issues (#260)
Many of these changes are minor typo fixes. The other changes
(which are mostly compiler warning fixes) are:
NEWS:
- The --globcasedetect shell option works on older Linux kernels
when used with FAT32/VFAT file systems, so remove the note about
it only working with 5.2+ kernels.
src/cmd/ksh93/COMPATIBILITY:
- Update the documentation on function scoping with an addition
from ksh93v- (this does apply to ksh93u+).
src/cmd/ksh93/edit/emacs.c:
- Check for '_AST_ksh_release', not 'AST_ksh_release'.
src/cmd/INIT/mamake.c,
src/cmd/INIT/ratz.c,
src/cmd/INIT/release.c,
src/cmd/builtin/pty.c:
- Add more uses of UNREACHABLE() and noreturn, this time for the
build system and pty.
src/cmd/builtin/pty.c,
src/cmd/builtin/array.c,
src/cmd/ksh93/sh/name.c,
src/cmd/ksh93/sh/nvtype.c,
src/cmd/ksh93/sh/suid_exec.c:
- Fix six -Wunused-variable warnings (the name.c nv_arrayptr()
fixes are also in ksh93v-).
- Remove the unused 'tableval' function to fix a -Wunused-function
warning.
src/cmd/ksh93/sh/lex.c:
- Remove unused 'SHOPT_DOS' code, which isn't enabled anywhere.
https://github.com/att/ast/issues/272#issuecomment-354363112
src/cmd/ksh93/bltins/misc.c,
src/cmd/ksh93/bltins/trap.c,
src/cmd/ksh93/bltins/typeset.c:
- Add dictionary generator function declarations for former
aliases that are now builtins (re: 1fbbeaa1, ef1621c1, 3ba4900e).
- For consistency with the rest of the codebase, use '(void)'
instead of '()' for print_cpu_times.
src/cmd/ksh93/sh/init.c,
src/lib/libast/path/pathshell.c:
- Move the otherwise unused EXE macro to pathshell() and only
search for 'sh.exe' on Windows.
src/cmd/ksh93/sh/xec.c,
src/lib/libast/include/ast.h:
- Add an empty definition for inline when compiling with C89.
This allows the timeval_to_double() function to be inlined.
src/cmd/ksh93/include/shlex.h:
- Remove the unused 'PIPESYM2' macro.
src/cmd/ksh93/tests/pty.sh:
- Add '# err_exit #' to count the regression test added in
commit 113a9392.
src/lib/libast/disc/sfdcdio.c:
- Move diordwr, dioread, diowrite and dioexcept behind
'#ifdef F_DIOINFO' to fix one -Wunused-variable warning and
multiple -Wunused-function warnings (sfdcdio() only uses these
functions when F_DIOINFO is defined).
src/lib/libast/string/fmtdev.c:
- Fix two -Wimplicit-function-declaration warnings on Linux by
including sys/sysmacros.h in fmtdev().
2021-04-08 18:58:07 +00:00
|
|
|
and QNX 7.0+). When this option is turned on, file name generation
|
Add --globcasedetect shell option for globbing and completion
One of the best-kept secrets of libast/ksh93 is that the code
includes support for case-insensitive file name generation (a.k.a.
pathname expansion, a.k.a. globbing) as well as case-insensitive
file name completion on interactive shells, depending on whether
the file system is case-insensitive or not. This is transparently
determined for each directory, so a path pattern that spans
multiple file systems can be part case-sensitive and part case-
insensitive. In more precise terms, each slash-separated path name
component pattern P is treated as ~(i:P) if its parent directory
exists on a case-insensitive file system. I recently discovered
this while dealing with <https://github.com/ksh93/ksh/issues/223>.
However, that support is dead code on almost all current systems.
It depends on pathconf(2) having a _PC_PATH_ATTRIBUTES selector.
The 'c' attribute is supposedly returned if the given directory is
on a case insensitive file system. There are other attributes as
well (at least 'l', see src/lib/libcmd/rm.c). However, I have been
unable to find any system, current or otherwise, that has
_PC_PATH_ATTRIBUTES. Google and mailing list searches yield no
relevant results at all. If anyone knows of such a system, please
add a comment to this commit on GitHub, or email me.
An exception is Cygwin/Windows, on which the "c" attribute was
simply hardcoded, so globbing/completion is always case-
insensitive. As of Windows 10, that is wrong, as it added the
possibility to mount case-sensitive file systems.
On the other hand, this was never activated on the Mac, even
though macOS has always used a case-insensitive file like Windows.
But, being UNIX, it can also mount case-sensitive file systems.
Finally, Linux added the possibility to create individual case-
insensitive ext4 directories fairly recently, in version 5.2.
https://www.collabora.com/news-and-blog/blog/2020/08/27/using-the-linux-kernel-case-insensitive-feature-in-ext4/
So, since this functionality latently exists in the code base, and
three popular OSs now have relevant file system support, we might
as well make it usable on those systems. It's a nice idea, as it
intuitively makes sense for globbing and completion behaviour to
auto-adapt to file system case insensitivity on a per-directory
basis. No other shell does this, so it's a nice selling point, too.
However, the way it is coded, this is activated unconditionally on
supported systems. That is not a good idea. It will surprise users.
Since globbing is used with commands like 'rm', we do not want
surprises. So this commit makes it conditional upon a new shell
option called 'globcasedetect'. This option is only compiled into
ksh on systems where we can actually detect FS case insensitivity.
To implement this, libast needs some public API additions first.
*** libast changes ***
src/lib/libast/features/lib:
- Add probes for the linux/fs.h and sys/ioctl.h headers.
Linux needs these to use ioctl(2) in pathicase(3) (see below).
src/lib/libast/path/pathicase.c,
src/lib/libast/include/ast.h,
src/lib/libast/man/path.3,
src/lib/libast/Mamfile:
- Add new pathicase(3) public API function. This uses whatever
OS-specific method it can detect at compile time to determine if
a particular path is on a case-insensitive file system. If no
method is available, it only sets errno to ENOSYS and returns -1.
Currently known to work on: macOS, Cygwin, Linux 5.2+, QNX 7.0+.
- On systems (if any) that have the mysterious _PC_PATH_ATTRIBUTES
selector for pathconf(2), call astconf(3) and check for the 'c'
attribute to determine case insensitivity. This should preserve
compatibility with any such system.
src/lib/libast/port/astconf.c:
- dynamic[]: As case-insensitive globbing is now optional on all
systems, do not set the 'c' attribute by default on _WINIX
(Cygwin/Windows) systems.
- format(): On systems that do not have _PC_PATH_ATTRIBUTES, call
pathicase(3) to determine the value for the "c" (case
insensitive) attribute only. This is for compatibility as it is
more efficient to call pathicase(3) directly.
src/lib/libast/misc/glob.c,
src/lib/libast/include/glob.h:
- Add new GLOB_DCASE public API flag to glob(3). This is like
GLOB_ICASE (case-insensitive matching) except it only makes the
match case-insensitive if the file system for the current
pathname component is determined to be case-insensitive.
- gl_attr(): For efficiency, call pathicase(3) directly instead of
via astconf(3).
- glob_dir(): Only call gl_attr() to determine file system case
insensitivity if the GLOB_DCASE flag was passed. This makes case
insensitive globbing optional on all systems.
- glob(): The options bitmask needs to be widened to fit the new
GLOB_DCASE option. Define this centrally in a new GLOB_FLAGMASK
macro so it is easy to change it along with GLOB_MAGIC (which
uses the remaining bits for a sanity check bit pattern).
src/lib/libast/path/pathexists.c:
- For efficiency, call pathicase(3) directly instead of via
astconf(3).
*** ksh changes ***
src/cmd/ksh93/features/options,
src/cmd/ksh93/SHOPT.sh:
- Add new SHOPT_GLOBCASEDET compile-time option. Set it to probe
(empty) by default so that the shell option is compiled in on
supported systems only, which is determined by new iffe feature
test that checks if pathicase(3) returns an ENOSYS error.
src/cmd/ksh93/data/options.c,
src/cmd/ksh93/include/shell.h:
- Add -o globcasedetect shell option if compiling with
SHOPT_GLOBCASEDET.
src/cmd/ksh93/sh/expand.c: path_expand():
- Pass the new GLOB_DCASE flag to glob(3) if the
globcasedetect/SH_GLOBCASEDET shell option is set.
src/cmd/ksh93/edit/completion.c:
- While file listing/completion is based on globbing and
automatically becomes case-insensitive when globbing does, it
needs some additional handling to make a string comparison
case-insensitive in corresponding cases. Otherwise, partial
completions may be deleted from the command line upon pressing
tab. This code was already in ksh 93u+ and just needs to be
made conditional upon SHOPT_GLOBCASEDET and globcasedetect.
- For efficiency, call pathicase(3) directly instead of via
astconf(3).
src/cmd/ksh93/sh.1:
- Document the new globcasedetect shell option.
2021-03-22 15:01:03 +00:00
|
|
|
(globbing), as well as file name tab completion on interactive shells,
|
|
|
|
|
automatically become case-insensitive on file systems where the difference
|
|
|
|
|
between upper- and lowercase is ignored for file names. This is transparently
|
|
|
|
|
determined for each directory, so a path pattern that spans multiple file
|
|
|
|
|
systems can be part case-sensitive and part case-insensitive.
|
|
|
|
|
The option is not compiled into ksh on systems where we do not know of a
|
|
|
|
|
method to check for file system case insensitivity. The shell option can be
|
|
|
|
|
force-compiled by setting SHOPT_GLOBCASEDET to 1 in src/cmd/ksh93/SHOPT.sh,
|
|
|
|
|
but it won't have any effect on non-supported systems, so this is not
|
|
|
|
|
recommended. It can be removed from ksh by setting SHOPT_GLOBCASEDET to 0.
|
|
|
|
|
|
2021-03-17 22:34:45 +00:00
|
|
|
2021-03-17:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug with file name completion on the interactive shell in multibyte
|
|
|
|
|
locales. Upon encountering two filenames with multibyte characters starting
|
|
|
|
|
with the same byte, a partial multibyte character was autocompleted.
|
|
|
|
|
|
2021-03-16 16:13:13 +00:00
|
|
|
2021-03-16:
|
|
|
|
|
|
Allow proper tilde expansion overrides (#225)
Until now, when performing any tilde expansion like ~/foo or
~user/foo, ksh added a placeholder built-in command called
'.sh.tilde', ostensibly with the intention to allow users to
override it with a shell function or custom builtin. The multishell
ksh93 repo <https://github.com/multishell/ksh93/> shows this was
added sometime between 2002-06-28 and 2004-02-29. However, it has
never worked and crashed the shell.
This commit replaces that with something that works. Specific tilde
expansions can now be overridden using .set or .get discipline
functions associated with the .sh.tilde variable (see manual,
Discipline Functions).
For example, you can use either of:
.sh.tilde.set()
{
case ${.sh.value} in
'~tmp') .sh.value=${XDG_RUNTIME_DIR:-${TMPDIR:-/tmp}} ;;
'~doc') .sh.value=~/Documents ;;
'~ksh') .sh.value=/usr/local/src/ksh93/ksh ;;
esac
}
.sh.tilde.get()
{
case ${.sh.tilde} in
'~tmp') .sh.value=${XDG_RUNTIME_DIR:-${TMPDIR:-/tmp}} ;;
'~doc') .sh.value=~/Documents ;;
'~ksh') .sh.value=/usr/local/src/ksh93/ksh ;;
esac
}
src/cmd/ksh93/include/variables.h,
src/cmd/ksh93/data/variables.c:
- Add SH_TILDENOD for a new ${.sh.tilde} predefined variable.
It is initially unset.
src/cmd/ksh93/sh/macro.c:
- sh_btilde(): Removed.
- tilde_expand2(): Rewritten. I started out with the tiny version
of this function from the 2002-06-28 version of ksh. It uses the
stack instead of sfio, which is more efficient. A bugfix for
$HOME == '/' was retrofitted so that ~/foo does not become
//foo instead of /foo. The rest is entirely new code.
To implement the override functionality, it now checks if
${.sh.tilde} has any discipline function associated with it.
If it does, it assigns the tilde expression to ${.sh.tilde} using
nv_putval(), triggering the .set discipline, and then reads it
back using nv_getval(), triggering the .get discipline. The
resulting value is used if it is nonempty and does not still
start with a tilde.
src/cmd/ksh93/bltins/typeset.c,
src/cmd/ksh93/tests/builtins.sh:
- Since ksh no longer adds a dummy '.sh.tilde' builtin, remove the
ad-hoc hack that suppressed it from the output of 'builtin'.
src/cmd/ksh93/tests/tilde.sh:
- Add tests verifying everything I can think of, as well as tests
for bugs found and fixed during this rewrite.
src/cmd/ksh93/tests/pty.sh:
- Add test verifying that the .sh.tilde.set() discipline does not
modify the exit status value ($?) when performing tilde expansion
as part of tab completion.
src/cmd/ksh93/sh.1:
- Instead of "tilde substitution", call the basic mechanism "tilde
expansion", which is the term used everywhere else (including the
1995 Bolsky/Korn ksh book).
- Document the new override feature.
Resolves: https://github.com/ksh93/ksh/issues/217
2021-03-17 21:07:14 +00:00
|
|
|
- Tilde expansion can now be extended or modified by defining a .sh.tilde.get
|
|
|
|
|
or .sh.tilde.set discipline function. This replaces a 2004 undocumented
|
|
|
|
|
attempt to add this functionality via a .sh.tilde built-in, which never
|
|
|
|
|
worked and crashed the shell. See the manual for details on the new method.
|
|
|
|
|
|
2021-03-16 16:13:13 +00:00
|
|
|
- Fixed a bug in interactive shells: if a variable used by the shell called
|
|
|
|
|
a discipline function (such as PS1.get() or COLUMNS.set()), the value of $?
|
|
|
|
|
was set to the exit status of the discipline function instead of the last
|
|
|
|
|
command run.
|
|
|
|
|
|
2021-03-15 21:49:02 +00:00
|
|
|
2021-03-15:
|
|
|
|
|
|
|
|
|
|
- If the HOME variable is unset, the bare tilde ~ now expands to the current
|
|
|
|
|
user's system-configured home directory instead of merely the username.
|
|
|
|
|
|
2021-03-16 10:19:00 +00:00
|
|
|
- Tighten up potential invalid typeset attribute combos when more than
|
|
|
|
|
one numeric type has been requested. In particular, -F and -i are no
|
|
|
|
|
longer masked over by previously given float types.
|
|
|
|
|
|
Re-enable SHOPT_DEVFD, fixing process substitution fd leaks (#218)
This commit fixes a long-standing bug (present since at least
ksh93r) that caused a file descriptor leak when passing a process
substitution to a function, or (if compiled with SHOPT_SPAWN) to a
nonexistent command.
The leaks only occurred when ksh was compiled with SHOPT_DEVFD; the
FIFO method was unaffected.
src/cmd/ksh93/sh/xec.c: sh_exec():
- When a process substitution is passed to a built-in, the
remaining file descriptor is closed with sh_iorestore. Do the
same thing when passing a process substitution to a function.
This is done by delaying the sh_iorestore() call to 'setexit:'
where both built-ins and functions terminate and set the exit
status ($?).
This means that call now will not be executed if a longjmp is
done, e.g. due to an error in a special built-in. However, there
is already another sh_iorestore() call in main.c, exfile(), line
418, that handles that scenario.
- sh_ntfork() can fail, so rather than assume it will succeed,
handle a failure by closing extra file descriptors with
sh_iorestore(). This fixes the leak on command not found with
SHOPT_SPAWN.
src/cmd/ksh93/include/defs.h:
- Since the file descriptor leaks are now fixed, remove the
workaround that forced ksh to use the FIFO method.
src/cmd/ksh93/SHOPT.sh:
- Add SHOPT_DEVFD as a configurable option (default: probe).
src/cmd/ksh93/tests/io.sh:
- Add a regression test for the 'not found' file descriptor leak.
- Add a test to ensure it keeps working with 'command'.
Fixes: https://github.com/ksh93/ksh/issues/67
2021-03-13 13:46:42 +00:00
|
|
|
2021-03-13:
|
|
|
|
|
|
|
|
|
|
- Fixed a file descriptor leak that occurred when ksh used /dev/fd for
|
|
|
|
|
process substitutions passed to functions.
|
|
|
|
|
|
|
|
|
|
- Fixed a separate file descriptor leak that happened when a process
|
|
|
|
|
substitution was passed to a nonexistent command.
|
|
|
|
|
|
Fix unused process substitutions hanging (#214)
On systems where ksh needs to use the older and less secure FIFO
method for process substitutions (which is currently all of them as
the more modern and solid /dev/fd method is still broken, see #67),
process substitutions could leave background processes hanging in
these two scenarios:
1. If the parent process exits without opening a pipe to the child
process forked by the process substitution. The fifo_check()
function in xec.c, which is periodically called to check if the
parent process still exists while waiting for it to open the
FIFO, verified the parent process's existence by checking if the
PPID had reverted to 1, the traditional PID of init. However,
POSIX specifies that the PPID can revert to any implementation-
defined system process in that case. So this breaks on certain
systems, causing unused process substitutions to hang around
forever as they never detect that the parent disappeared.
The fix is to save the current PID before forking and having the
child check if the PPID has changed from that saved PID.
2. If command invoked from the main shell is passed a process
substitution, but terminates without opening the pipe to the
process substitution. In that case, the parent process never
disappears in the first place, because the parent process is the
main shell. So the same infinite wait occurs in unused process
substitutions, even after correcting problem 1.
The fix is to remember all FIFOs created for any number of
process substitutions passed to a single command, and unlink any
remaining FIFOs as they represent unused command substitutions.
Unlinking them FIFOs causes sh_open() in the child to fail with
ENOENT on the next periodic check, which can easily be handled.
Fixing these problems causes the FIFO method to act identically to
the /dev/fd method, which is good for compatibility. Even when #67
is fixed this will still be important, as ksh also runs on systems
that do not have /dev/fd (such as AIX, HP-UX, and QNX), so will
fall back to using FIFOs.
--- Fix problem 1 ---
src/cmd/ksh93/sh/xec.c:
- Add new static fifo_save_ppid variable.
- sh_exec(): If a FIFO is defined, save the current PID in
fifo_save_ppid for the forked child to use.
- fifo_check(): Compare PPID against the saved value instead of 1.
--- Fix problem 2 ---
To keep things simple I'm abusing the name-value pair routines used
for variables for this purpose. The overhead is negligible. A more
elegant solution is possible but would involve adding more code.
src/cmd/ksh93/include/defs.h: _SH_PRIVATE:
- Define new sh.fifo_tree pointer to a new FIFO cleanup tree.
src/cmd/ksh93/sh/args.c: sh_argprocsubs():
- After launching a process substitution in the background,
add the FIFO to the cleanup list before freeing it.
src/cmd/ksh93/sh/xec.c:
- Add fifo_cleanup() that unlinks all FIFOs in the cleanup list and
clears/closes the list. They should only still exist if the
command never used them, however, just run 'unlink' and don't
check for existence first as that would only add overhead.
- sh_exec():
* Call fifo_cleanup() on finishing all simple commands (when
setting $?) or when a special builtin fails.
* When forking, clear/close the cleanup list; we do not want
children doing duplicate cleanup, particularly as this can
interfere when using multiple process substitutions in one
command.
* Process substitution handling:
> Change FIFO check frequency from 500ms to 50ms.
Note that each check sends a signal that interrupts open(2),
causing sh_open() to reinvoke it. This causes sh_open() to
fail with ENOENT on the next check when the FIFO no longer
exists, so we do not need to add an additional check for
existence to fifo_check(). Unused process substitutions now
linger for a maximum of 50ms.
> Do not issue an error message if errno == ENOENT.
- sh_funct(): Process substitutions can be passed to functions as
well, and we do not want commands within the function to clean up
the FIFOs for the process substitutions passed to it from the
outside. The problem is solved by simply saving fifo_tree in a
local variable, setting it to null before running the function,
and cleaning it up before restoring the parent one at the end.
Since sh_funct() is called recursively for multiple-level
function calls, this correctly gives each function a locally
scoped fifo_tree.
--- Tests ---
src/cmd/ksh93/tests/io.sh:
- Add tests covering the failing scenarios.
Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-03-12 11:43:23 +00:00
|
|
|
2021-03-11:
|
|
|
|
|
|
|
|
|
|
- Fixed an intermittent bug that caused process substitutions to infinitely
|
|
|
|
|
loop in Linux virtual machines that use systemd.
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused process substitutions to leave lingering processes
|
|
|
|
|
if the command invoking them never reads from them.
|
|
|
|
|
|
2021-03-09 04:35:45 +00:00
|
|
|
2021-03-09:
|
|
|
|
|
|
|
|
|
|
- The ${!foo@} and ${!foo*} expansions yield variable names beginning with foo,
|
|
|
|
|
but excluded 'foo' itself. The fix for this is now backported from 93v- beta.
|
|
|
|
|
|
Fix test -v for numeric types & set/unset state for short int
This commit fixes two interrelated problems.
1. The -v unary test/[/[[ operator is documented to test if a
variable is set. However, it always returns true for variable
names with a numeric attribute, even if the variable has not
been given a value. Reproducer:
$ ksh -o nounset -c 'typeset -i n; [[ -v n ]] && echo $n'
ksh: n: parameter not set
That is clearly wrong; 'echo $n' should never be reached and the
error should not occur, and does not occur on mksh or bash.
2. Fixing the previous problem revealed serious breakage in short
integer type variables that was being masked. After applying
that fix and then executing 'typeset -si var=0':
- The conditional assignment expansions ${var=123} and
${var:=123} assigned 123 to var, even though it was set to 0.
- The expansions ${var+s} and ${var:+n} incorrectly acted as if
the variable was unset and empty, respectively.
- '[[ -v var ]]' and 'test -v var' incorrectly returned false.
The problems were caused by a different storage method for short
ints. Their values were stored directly in the 'union Value'
member of the Namval_t struct, instead of allocated on the stack
and referred to by a pointer, as regular integers and all other
types do. This inherently broke nv_isnull() as this leaves no
way to distinguish between a zero value and no value at all.
(I'm also pretty sure it's undefined behaviour in C to check for
a null pointer at the address where a short int is stored.)
The fix is to store short ints like other variables and refer
to them by pointers. The NV_INT16P combined bit mask already
existed for this, but nv_putval() did not yet support it.
src/cmd/ksh93/bltins/test.c: test_unop():
- Fix problem 1. For -v, only check nv_isnull() and do not check
for the NV_INTEGER attribute (which, by the way, is also used
for float variables by combining it with other bits).
See also 5aba0c72 where we recently fixed nv_isnull() to
work properly for all variable types including short ints.
src/cmd/ksh93/sh/name.c: nv_putval():
- Fix problem 2, part 1. Add support for NV_INT16P. The code is
simply copied and adapted from the code for regular integers, a
few lines further on. The regular NV_SHORT code is kept as this
is still used for some special variables like ${.sh.level}.
src/cmd/ksh93/bltins/typeset.c: b_typeset():
- Fix problem 2, part 2. Use NV_INT16P instead of NV_SHORT.
src/cmd/ksh93/tests/attributes.sh:
- Add set/unset/empty/nonempty tests for all numeric types.
src/cmd/ksh93/tests/bracket.sh,
src/cmd/ksh93/tests/comvar.sh:
- Update a couple of existing tests.
- Add test for [[ -v var ]] and [[ -n ${var+s} ]] on unset
and empty variables with many attributes.
src/cmd/ksh93/COMPATIBILITY:
- Add a note detailing the change to test -v.
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/sh.1:
- Correct 'typeset -C' documentation. Variables declared as
compound are *not* initially unset, but initially have the empty
compound value. 'typeset' outputs them as:
typeset -C foo=()
and not:
typeset -C foo
and nv_isnull() is never true for them. This may or may not
technically be a bug. I don't think it's worth changing, but
it should at least be documented correctly.
2021-03-10 00:38:00 +00:00
|
|
|
- test -v var, [ -v var ], and [[ -v var ]] did not correctly test if a
|
|
|
|
|
variable is set or unset after it has been given a numeric attribute with
|
|
|
|
|
'typeset' but not yet assigned a value. This has been fixed so that
|
|
|
|
|
[[ -v var ]] is now equivalent to [[ -n ${var+set} ]] as documented.
|
|
|
|
|
|
Fixes for -G/--globstar (re: 5312a59d)
The fix for '.' and '..' in regular globbing broke '.' and '..' in
globstar. No globstar pattern that contains '.' or '..' as any
pathname component still matched. This commit fixes that.
This commit also makes symlink/** mostly work, which it never has
done in any ksh93 version. It is correct and expected that symlinks
found by patterns are not resolved, but symlinks were not resolved
even when specified as explicit non-pattern pathname components.
For example, /tmp/** breaks if /tmp is a symlink (e.g. on macOS),
which looks like a bug.
src/lib/libast/include/glob.h,
src/lib/libast/misc/glob.c: glob_dir():
- Make symlink/** work. we can check if the string pointed to by
pat is exactly equal to *. If so, we are doing regular globbing
for that particular pathname element, and it's okay to resolve
symlinks. If not (if it's **), we're doing globstar and we should
not be matching symlinks.
- Let's also introduce proper identification of symlinks (GLOB_SYM)
and not lump them in with other special files (GLOB_DEV).
- Fix the bug with literal '.' and '..' components in globstar
patterns. In preceding code, the matchdir pointer gets set to the
complete glob pattern if we're doing globstar for the current
pathname element, null if not. The pat pointer gets set to the
elements of the pattern that are still left to be processed;
already-done elements are trimmed from it by increasing the
pointer. So, to do the right thing, we need to make sure that '.'
or '..' is skipped if, and only if, it is the final element in
the pattern (i.e., if pat does not contain a slash) and is not
specified literally as '.' or '..', i.e., only if '.' or '..' was
actually resolved from a glob pattern. After this change,
'**/.*', '**/../.*', etc. do the right thing, showing all your
hidden files and directories without undesirable '.' and '..'
results; '.' and '..' are skipped as final elements, unless you
literally specify '**/.', '**/..', '**/foo/bar/..', etc.
src/cmd/ksh93/COMPATIBILITY:
- Note the symlink/** globstar change.
src/cmd/ksh93/sh.1:
- Try to document the current globstar behaviour more exhausively.
src/cmd/ksh93/tests/glob.sh:
- Add tests. Try to cover all the corner cases.
src/cmd/ksh93/tests/shtests:
- Since tests in glob.sh do not use err_exit, they were not
counted. Special-case glob.sh for counting the tests: count the
lines starting with a test_* function call.
Resolves: https://github.com/ksh93/ksh/issues/146
2021-03-06 18:45:42 +00:00
|
|
|
2021-03-07:
|
|
|
|
|
|
Fix set/unset state for short integer (typeset -si) (#211)
This commit fixes at least three bugs:
1. When issuing 'typeset -p' for unset variables typeset as short
integer, a value of 0 was incorrectly diplayed.
2. ${x=y} and ${x:=y} were still broken for short integer types
(re: 9f2389ed). ${x+set} and ${x:+nonempty} were also broken.
3. A memory fault could occur if typeset -l followed a -s option
with integers. Additonally, now the last -s/-l wins out as the
option to utilize instead of it always being short.
src/cmd/ksh93/include/name.h:
- Fix the nv_isnull() macro by removing the direct exclusion of
short integers from this set/unset test. This breaks few things
(only ${.sh.subshell} and ${.sh.level}, as far as we can tell)
while potentially correcting many aspects of short integer use
(at least bugs 1 and 2 above), as this macro is widely used.
- union Value: add new pid_t *pidp pointer member for PID values
(see further below).
src/cmd/ksh93/bltins/typeset.c: b_typeset():
- To fix bug 3 above, unset the 'shortint' flag and NV_SHORT
attribute bit upon encountering the -l optiobn.
*** To fix ${.sh.subshell} to work with the new nv_isnull():
src/cmd/ksh93/sh/defs.h:
- Add new 'realsubshell' member to the shgd (aka shp->gd) struct
which will be the integer value for ${.sh.subshell}.
src/cmd/ksh93/sh/init.c,
src/cmd/ksh93/data/variables.c:
- Initialize SH_SUBSHELLNOD as a pointer to shgd->realsubshell
instead of using a short value (.s) directly. Using a pointer
allows nv_isnull() to return a positive for ${.sh.subshell} as
a non-null pointer is what it checks for.
- While we're at it, initialize PPIDNOD ($PPID) and SH_PIDNOD
(${.sh.pid}) using the new pdip union member, which is more
correct as they are values of type pid_t.
src/cmd/ksh93/sh/subshell.c,
src/cmd/ksh93/sh/xec.c:
- Update the ${.sh.subshell} increases/decreases to refer to
shgd->realsubshell (a.k.a. shp->gd->realsubshell).
*** To fix ${.sh.level} after changing nv_isnull():
src/cmd/ksh93/sh/macro.c: varsub():
- Add a specific exception for SH_LEVLNOD to the nv_isnull() test,
so that ${.sh.level} is always considered to be set. Its handling
throughout the code is too complex/special for a simple fix, so
we have to special-case it, at least for now.
*** Regression test additions:
src/cmd/ksh93/tests/attributes.sh:
- Add in missing short integer tests and correct the one that
existed. The -si test now yields 'typeset -x -r -s -i foo'
instead of 'typeset -x -r -s -i foo=0' which brings it in line
with all the others.
- Add in some other -l attribute tests for floats. Note, -lX test
was not added as the size of long double is platform dependent.
src/cmd/ksh93/tests/variables.sh:
- Add tests for ${x=y} and ${x:=y} used on short int variables.
Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-03-08 04:19:36 +00:00
|
|
|
- Fixed the typeset -p display of short integers without an assigned value.
|
|
|
|
|
Also, the last -s or -l attribute option supplied for an integer is used.
|
|
|
|
|
|
Fixes for -G/--globstar (re: 5312a59d)
The fix for '.' and '..' in regular globbing broke '.' and '..' in
globstar. No globstar pattern that contains '.' or '..' as any
pathname component still matched. This commit fixes that.
This commit also makes symlink/** mostly work, which it never has
done in any ksh93 version. It is correct and expected that symlinks
found by patterns are not resolved, but symlinks were not resolved
even when specified as explicit non-pattern pathname components.
For example, /tmp/** breaks if /tmp is a symlink (e.g. on macOS),
which looks like a bug.
src/lib/libast/include/glob.h,
src/lib/libast/misc/glob.c: glob_dir():
- Make symlink/** work. we can check if the string pointed to by
pat is exactly equal to *. If so, we are doing regular globbing
for that particular pathname element, and it's okay to resolve
symlinks. If not (if it's **), we're doing globstar and we should
not be matching symlinks.
- Let's also introduce proper identification of symlinks (GLOB_SYM)
and not lump them in with other special files (GLOB_DEV).
- Fix the bug with literal '.' and '..' components in globstar
patterns. In preceding code, the matchdir pointer gets set to the
complete glob pattern if we're doing globstar for the current
pathname element, null if not. The pat pointer gets set to the
elements of the pattern that are still left to be processed;
already-done elements are trimmed from it by increasing the
pointer. So, to do the right thing, we need to make sure that '.'
or '..' is skipped if, and only if, it is the final element in
the pattern (i.e., if pat does not contain a slash) and is not
specified literally as '.' or '..', i.e., only if '.' or '..' was
actually resolved from a glob pattern. After this change,
'**/.*', '**/../.*', etc. do the right thing, showing all your
hidden files and directories without undesirable '.' and '..'
results; '.' and '..' are skipped as final elements, unless you
literally specify '**/.', '**/..', '**/foo/bar/..', etc.
src/cmd/ksh93/COMPATIBILITY:
- Note the symlink/** globstar change.
src/cmd/ksh93/sh.1:
- Try to document the current globstar behaviour more exhausively.
src/cmd/ksh93/tests/glob.sh:
- Add tests. Try to cover all the corner cases.
src/cmd/ksh93/tests/shtests:
- Since tests in glob.sh do not use err_exit, they were not
counted. Special-case glob.sh for counting the tests: count the
lines starting with a test_* function call.
Resolves: https://github.com/ksh93/ksh/issues/146
2021-03-06 18:45:42 +00:00
|
|
|
- Fixed a bug with -G/--globstar introduced on 2020-08-09: patterns did not
|
|
|
|
|
match anything if any pathname component was '.' or '..', e.g. '**/./glob.c'
|
|
|
|
|
never matched. The 2020-08-09 fix does still apply to patterns like '.*'.
|
|
|
|
|
|
|
|
|
|
- Enhancement to -G/--globstar: symbolic links to directories are now followed
|
|
|
|
|
if they match a normal (non-**) glob pattern. For example, if '/lnk' is a
|
|
|
|
|
symlink to a directory, '/lnk/**' and '/l?k/**' now work as you would expect.
|
|
|
|
|
|
2021-03-07 17:01:17 +00:00
|
|
|
- Fixed a bug introduced on 2021-02-11 that caused job control on interactive
|
|
|
|
|
ksh sessions to misbehave if the login shell was replaced by ksh using 'exec'.
|
|
|
|
|
|
Fix ${x=y} and ${x:=y} for numeric types of x
These POSIX expansions first assign y to x if x is unset or empty,
respectively, and then they yield the value of x. This was not
working on any ksh93 version if x was typeset as numeric (integer
or float) but still unset, as in not assigned a value.
$ unset a; typeset -i a; printf '%q\n' "${a:=42}" "$a"
0
''
Expected output:
42
42
src/cmd/ksh93/sh/macro.c:
- Fix the test for set/unset variable. It was broken because it
only checked for the existence of the node, which exists after
'typeset', but did not check if a value had been assigned. This
additional check needs to be done with the nv_isnull() macro, but
only for expansions of the regular M_BRACE type. Special
expansions cannot have an unset state.
- As of commit 95294419, we know that an nv_optimize() call may be
needed before using nv_isnull() if the shell is compiled with
SHOPT_OPTIMIZE. Move the nv_optimize() call from that commit
forward to before the new check that calls nv_isnull(), and only
bother with it if the type is M_BRACE.
src/cmd/ksh93/tests/variables.sh:
- Add tests for this bug. Test float and integer, and also check
that ${a=b} and ${a:=b} correctly treat the value of 'b' as an
arithmetic expression of which the result is assigned to 'a' if
'a' was typeset as numeric.
src/cmd/ksh93/tests/attributes.sh,
src/cmd/ksh93/tests/comvar.sh,
src/cmd/ksh93/tests/nameref.sh,
src/cmd/ksh93/tests/types.sh:
- Fix a number of tests to report failures correctly.
Resolves: https://github.com/ksh93/ksh/issues/157
2021-03-06 03:56:52 +00:00
|
|
|
2021-03-06:
|
|
|
|
|
|
|
|
|
|
- Fixed an old expansion bug: expansions of type ${var=value} and ${var:=value}
|
|
|
|
|
did not perform an assignment and yielded the value 0 if 'var' was typeset as
|
|
|
|
|
numeric (integer or float) but had not yet been assigned a value.
|
|
|
|
|
|
2021-03-06 06:43:38 +00:00
|
|
|
- Fixed a bug introduced on 2020-08-19: Ctrl+D would break after an
|
|
|
|
|
interactive shell received SIGWINCH.
|
|
|
|
|
|
Fix command history corruption on syntax error (re: e999f6b1)
Analysis: When a syntax error occurs, the shell performs a
longjmp(3) back to exfile() in main.c on line 417:
415| if(jmpval)
416| {
417| Sfio_t *top;
418| sh_iorestore((void*)shp,0,jmpval);
419| hist_flush(shp->gd->hist_ptr);
420| sfsync(shp->outpool);
The first thing it does is restore the file descriptor state
(sh_iorestore), then it flushes the history file (hist_flush), then
it synchronises sfio's logical stream state with the physical
stream state using (sfsync).
However, the fix applied in e999f6b1 caused sh_iorestore() to sync
all sfio streams unconditionally. So this was done before
hist_flush(), which caused unpredictable behaviour, including
temporary and/or permanent history corruption, as this also synched
shp->outpool before hist_flush() had a chance to do its thing.
The fix is to only call sfsync() in sh_iorestore() if we're
actually about to call ftruncate(2), and not otherwise.
Moral of the story: bug fixes should be as specific as possible to
minimise the risk of side effects.
src/cmd/ksh93/sh/io.c: sh_iorestore():
- Only call sfsync() if we're about to truncate a file.
src/cmd/ksh93/tests/pty.sh:
- Add test.
Thanks to Marc Wilson for reporting the bug and to Johnothan King
for finding the commit that introduced it.
Resolves: https://github.com/ksh93/ksh/issues/209
Relevant: https://github.com/att/ast/issues/61
2021-03-07 00:27:33 +00:00
|
|
|
- Fixed a bug introduced on 2020-05-21: on an interactive shell, command lines
|
|
|
|
|
containing a syntax error were not added to the command history file and
|
|
|
|
|
sometimes corrupted the command history.
|
|
|
|
|
|
Remove obsolete quote balancing hack
The old Bourne shell failed to check for closing quotes and command
substitution backticks when encountering end-of-file in a parser
context (such as a script). ksh93 implemented a hack for partial
compatibility with this bug, tolerating unbalanced quotes and
backticks in backtick command subsitutions, 'eval', and command
line invocation '-c' scripts only.
This hack became broken for backtick command substitutions in
fe20311f/350b52ea as a memory leak was fixed by adding a newline to
the stack at the end of the command substitution. That extra
newline becomes part of any string whose quotes are not properly
terminated, causing problems such as the one detailed here:
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01889.html
$ touch abc
$ echo `ls "abc`
ls: abc
: not found
No other fix for the memory leak is known that doesn't cause other
problems. (The alternative fix detailed in the referenced mailing
list post causes a different corner-case regression.)
Besides, the hack has always caused other corner case bugs as well:
$ ksh -c '((i++'
Actual: ksh: i++(: not found
(If an external command 'i++(' existed, it would be run)
Expect: ksh: syntax error at line 1: `(' unmatched
$ ksh -c 'i=0; echo $((++i'
Actual: (empty line; the arithmetic expansion is ignored)
Expect: ksh: syntax error at line 1: `(' unmatched
$ ksh -c 'echo $(echo "hi)'
Actual: ksh: syntax error at line 1: `(' unmatched
Expect: ksh: syntax error at line 1: `"' unmatched
So, it's time to get rid of this hack. The old Bourne shell is
dead and buried. No other shell tries to support this breakage.
Tolerating syntax errors is just asking for strange side effects,
inconsistent states, and corner case bugs. We should not want to do
that. Old scripts that rely on this will just need to be fixed.
src/cmd/ksh93/sh/lex.c:
- struct lexdata: Remove 'char balance' member for remembering an
unbalanced quote or backtick.
- sh_lex(): Remove the back to remember and compensate for
unbalanced quotes/backticks that was executed only if we were
executing a script from a string, as opposed to a file.
src/cmd/ksh93/COMPATIBILITY:
- Note the change.
Resolves: https://github.com/ksh93/ksh/issues/199
2021-03-05 21:55:25 +00:00
|
|
|
2021-03-05:
|
|
|
|
|
|
|
|
|
|
- Unbalanced quotes and backticks now correctly produce a syntax error
|
|
|
|
|
in -c scripts, 'eval', and backtick-style command substitutions.
|
|
|
|
|
|
Fix arbitrary command execution vuln in array subscripts in arith
This commit fixes an arbitrary command execution vulnerability in
array subscripts used within the arithmetic subsystem.
One of the possible reproducers is:
var='1$(echo INJECTION >&2)' ksh -c \
'typeset -A a; ((a[$var]++)); typeset -p a'
Output before this commit:
INJECTION
typeset -A a=([1]=1)
The 'echo' command has been surreptitiously executed from an
external environment variable.
Output after this commit:
typeset -A a=(['1$(echo INJECTION >&2)']=1)
The value is correctly used as an array subscript and nothing in it
is parsed or executed. This is as it should be, as ksh93 supports
arbitrary subscripts for associative arrays.
If we think about it logically, the C-style arithmetic subsystem
simply has no business messing around with shell expansions or
quoting at all, because those don't belong to it. Shell expansions
and quotes are properly resolved by the main shell language before
the arithmetic subsystem is even invoked. It is particularly
important to maintain that separation because the shell expansion
mechanism also executes command substitutions.
Yet, the arithmetic subsystem subjected array subscripts that
contain `$` (and only array subscripts -- how oddly specific) to
an additional level of expansion and quote resolution. For some
unfathomable reason, there are two lines of code doing specifically
this. The vulnerability is fixed by simply removing those.
Incredibly, variants of this vulnerability are shared by bash, mksh
and zsh. Instead of fixing it, it got listed in Bash Pitfalls!
http://mywiki.wooledge.org/BashPitfalls#y.3D.24.28.28_array.5B.24x.5D_.29.29
src/cmd/ksh93/sh/arith.c:
- scope(): Remove these two lines that implement the vulnerability.
if(strchr(sub,'$'))
sub = sh_mactrim(shp,sub,0);
- scope(), arith(): Remove the NV_SUBQUOTE flag from two
nv_endsubscript() calls. That flag causes the array subscript to
retain the current level of shell quoting. The shell quotes
everything as in "double quotes" before invoking the arithmetic
subsystem, and the bad sh_mactrim() call removed one level of
quoting. Since we're no longer doing that, this flag should no
longer be passed, or subscripts may get extra backslash escapes.
src/cmd/ksh93/include/name.h,
src/cmd/ksh93/sh/array.c:
- nv_endsubscript(): The NV_SUBQUOTE flag was only passed from
arith.c. Since it is now unused, remove it.
src/cmd/ksh93/tests/arith.sh:
- Tweak some tests: fix typos, report wrong values.
- Add 21 tests. Most are based on reproducers contributed by
@stephane-chazelas and @hyenias. They verify that this
vulnerability is gone and that no quoting bugs were introduced.
Resolves: https://github.com/ksh93/ksh/issues/152
2021-03-04 13:37:13 +00:00
|
|
|
2021-03-04:
|
|
|
|
|
|
|
|
|
|
- Fixed an arbitrary command execution vulnerability that occurred when
|
|
|
|
|
parsing the subscripts of arrays within arithmetic commands and expansion.
|
|
|
|
|
|
2021-03-03 03:26:39 +00:00
|
|
|
2021-03-01:
|
|
|
|
|
|
|
|
|
|
- Fixed the retention of size attributes when 'readonly' or 'typeset -r'
|
|
|
|
|
was applied to an existing variable.
|
|
|
|
|
|
emacs: Fix three tab completion bugs
1. The editor accepted literal tabs without escaping in certain
cases, causing buggy and inconsistent completion behaviour.
https://github.com/ksh93/ksh/issues/71#issuecomment-656970959
https://github.com/ksh93/ksh/issues/71#issuecomment-657216472
2. After completing a filename by choosing from a file completion
menu, the terminal cursor was placed one position too far to the
right, corrupting command line display. This happened with
multiline active.
https://github.com/ksh93/ksh/issues/71#issue-655093805
3. A completion menu was displayed if the file name to be completed
was at the point where the rest of it started with a number,
even if that part uniquely identified it so the menu had 1 item.
https://www.mail-archive.com/ast-users@lists.research.att.com/msg00436.html
src/cmd/ksh93/edit/emacs.c:
- Cosmetic consistency: change two instances of cntl('[') to ESC.
- ed_emacsread(): Fix number 1 by refusing to continue into default
processing if a tab character was not used for tab completion.
Instead, beep and continue to the next read loop iteration. This
behaviour is consistent with most other shells, so I doubt there
will be objections. To enter a literal tab it's simple enough to
escape it with ^V (the 'stty lnext' character) or \.
- draw(): Fix number 2 by correcting an off-by-one error in the
ed_setcursor() call that updates the terminal's cursor display
in multiline mode. The 'old' and 'new' parameters need to have
identical values in this particular call to avoid the cursor
position being off by one to the right. This change makes it
match the corresponding ed_setcursor() call in vi.c. See below*
for details. Thanks to Lev Kujawski for the help in analysing.
src/cmd/ksh93/edit/completion.c: ed_expand():
- Fix number 3 by changing from '=' mode (menu-based completion) to
'\' mode (ordinary filename completion) if the menu would only
show one option, which was pointless and annoying. This never
happened in vi mode, so possibly the ed_expand() call in emacs.c
could have been improved instead. But I'm comfortable with fixing
it here and not in emacs.c, because this fixes it at a more
fundamental level, plus it's straightforward and obvious here.
Resolves: https://github.com/ksh93/ksh/issues/71
____
* Further details on bug number 2:
At https://github.com/ksh93/ksh/issues/71#issuecomment-786391565
Martijn Dekker wrote:
> I'm back to my original hypothesis that there is somehow an
> off-by-one error related to the ed_setcursor() call that gets
> executed when in multiline mode. I cannot confirm whether that
> off-by-one error is actually in the call itself, or occurs
> sometime earlier on one of the many possible occasions where
> ep->cursor is changed. But everything else appears to work
> correctly, so it's not unlikely that the problem is in the call
> itself.
>
> For reference, this is the original version of that call in
> emacs.c:
>
> ksh/src/cmd/ksh93/edit/emacs.c
> Lines 1556 to 1557 in df2b9bf
> if(ep->ed->e_multiline && option == REFRESH)
> ed_setcursor(ep->ed, ep->screen, ep->cursor-ep->screen, ep->ed->e_peol, -1);
>
> There is a corresponding call in the vi.c refresh() function
> (which does the same thing as draw() in emacs.c), where the third
> (old) and fourth (new) arguments are actually identical:
>
> ksh/src/cmd/ksh93/edit/vi.c
>
> Lines 2086 to 2087 in df2b9bf
> if(vp->ed->e_multiline && vp->ofirst_wind==INVALID)
> ed_setcursor(vp->ed, physical, last_phys+1, last_phys+1, -1);
>
> The expectation for this particular call is in fact that they
> should be identical, so that a delta of zero is calculated in
> that function. Delta not being zero is what causes the cursor to
> be positioned wrong.
>
> In vi.c, last_phys is a macro that is defined as editb.e_peol,
> and editb is a macro that is defined as (*vp->ed). Which means
> last_phys means vp->ed->e_peol, which is the same as
> ep->ed->e_peol in emacs.c. (These editors were originally
> separate programs by different authors, and I suppose this is how
> it shows. Korn didn't want to change all the variable names to
> integrate them, so made macros instead.)
>
> That leaves the question of why vi.c adds 1 to both last_phys
> a.k.a. e_peol arguments, and emacs.c uses e_peol for new without
> adding anything. Analysing the ed_setcursor() code could answer
> that question.
>
> So, this patch makes emacs.c do it the same way vi.c does. Let's
> make the third argument identical to the fourth. My brief testing
> shows the bug is fixed, and the regression tests yield no
> failures. This fix is also the most specific change possible, so
> there are few opportunities for side effects (I hope).
At https://github.com/ksh93/ksh/issues/71#issuecomment-786466652
Lev Kujawski wrote:
> I did a bit of research on this, and I think the fix to have the
> Emacs editing mode do the same as Vi is correct.
>
> From RELEASE:
> 08-05-01 In multiline edit mode, the refresh operation will now clear
> the remaining portion of the last line.
>
> Here's a fragment from the completion.c of the venerable but
> dated CDE DtKsh:
>
> else
> while (*com)
> {
> *out++ = ' ';
> out = strcopy(out,*com++);
> }
> *cur = (out-outbuff);
> /* restore rest of buffer */
> out = strcopy(out,stakptr(0));
> *eol = (out-outbuff);
>
> Noticeably missing is the code to add a space after file name
> completions. So, it seems plausible that if multiline editing
> mode was added beforehand,the ep->ed->p_eol !=
> ep->cursor-ep->screen case might never have occurred during
> testing.
>
> Setting the 'first' parameter to -1 seems to be a pretty explicit
> indicator that the author(s) intended the line clearing code to
> run, hence the entry in RELASE.
>
> The real issue is that if we update the cursor by calling
> ed_setcursor on line 1554 with old != new, the later call to
> setcursor on line 1583, here:
>
> I = (ncursor-nscreen) - ep->offset;
> setcursor(ep,i,0);
>
> will use outdated screen information to call setcursor, which,
> coincidentally, calls ed_setcursor.
2021-02-26 11:20:58 +00:00
|
|
|
2021-02-26:
|
|
|
|
|
|
|
|
|
|
- Fixed three long-standing bugs with tab completion in the emacs editor:
|
|
|
|
|
|
|
|
|
|
1. The editor accepted literal tabs without escaping in certain cases,
|
|
|
|
|
causing buggy and inconsistent completion behaviour. Details:
|
|
|
|
|
https://github.com/ksh93/ksh/issues/71#issuecomment-656970959
|
|
|
|
|
https://github.com/ksh93/ksh/issues/71#issuecomment-657216472
|
|
|
|
|
To enter a literal tab in emacs, you need to escape it with ^V or \.
|
|
|
|
|
|
|
|
|
|
2. After completing a filename by choosing from a file completion menu,
|
|
|
|
|
the terminal cursor was placed one position too far to the right,
|
|
|
|
|
corrupting command line display. This happened with multiline active.
|
|
|
|
|
Details: https://github.com/ksh93/ksh/issues/71#issue-655093805
|
|
|
|
|
|
|
|
|
|
3. A completion menu was displayed if the file name to be completed was
|
|
|
|
|
at the point where the rest of it started with a number, even if that
|
|
|
|
|
part uniquely identified it so the menu only showed one item. Details:
|
|
|
|
|
https://www.mail-archive.com/ast-users@lists.research.att.com/msg00436.html
|
|
|
|
|
|
Fix ${.sh.fun} leaking out of DEBUG trap
The value of the ${.sh.fun} variable, which is supposed to contain
the name of the function currently being executed, leaks out of the
DEBUG trap if it executes a function. Reproducer:
$ fn() { echo "executing the function"; }
$ trap fn DEBUG
$ trap - DEBUG
executing the function
$ echo ${.sh.fun}
fn
${.sh.fun} should be empty outside the function.
Annalysis:
The sh_debug() function in xec.c, which executes the DEBUG trap
action, contains these lines, which are part of restoring the state
after running the trap action with sh_trap():
nv_putval(SH_PATHNAMENOD,shp->st.filename,NV_NOFREE);
nv_putval(SH_FUNNAMENOD,shp->st.funname,NV_NOFREE);
shp->st = savst;
First the SH_PATHNAMENOD (${.sh.file}) and SH_FUNNAMENOD
(${.sh.fun}) variables get restored from the values in the shell's
scoped information struct (shp->st), but that is done *before*
restoring the parent scope with 'shp->st = savst;'. It should be
done after. Fixing the order is sufficient to fix the bug.
However, I am not convinced that these nv_putval() calls are good
for anything at all. Setting, unsetting, restoring, etc. the
${.sh.fun} and ${.sh.file} variables is already being handled
perfectly well elsewhere in the code for executing functions and
sourcing dot scripts. The DEBUG trap is neither here nor there.
There's no reason for it to get involved with these variables.
I was unable to break anything after simply removing those two
lines. So I strongly suspect this is another case, out of many now,
where a bug in ksh93 is properly fixed by removing some code.
I couldn't get ${.sh.file} to leak similarly -- I think this is
because SH_PATHNAMENOD (and not SH_FUNNOD) is set explicitly in
exfile() in main.c, masking this incorrect restore. It is the only
place where SH_PATHNAMENOD and SH_FUNNOD are not both set.
src/cmd/ksh93/sh/xec.c:
- Remove these two spurious nv_putval() calls.
src/cmd/ksh93/tests/variables.sh:
- Add regression test for leaking ${.sh.fun}.
2021-02-27 01:01:30 +00:00
|
|
|
- A bug with ${.sh.fun} in combination with the DEBUG trap has been fixed.
|
|
|
|
|
The ${.sh.fun} variable wrongly continued to contain the name of the last
|
|
|
|
|
function executed by the DEBUG trap after the trap action completed.
|
|
|
|
|
|
editors: fix broken SIGWINCH handling
In the emacs editor:
1. press ESC
2. change the size of your terminal window
and your screen is mysteriously cleared. (Until recent fixes, the
shell probably also crashed somewhere in the job control code.)
The cause is the way SIGWINCH is handled by ed_read() in edit.c.
For the emacs editor, it sends a Ctrl+L character to the input
buffer. The Ctrl+L command refreshes the command line. And it so
happens that ESC plus Ctrl+L is a command to clear the screen in
the emacs editor.
With the exeption of vi insert/command mode for which it uses a
shared flag, edit.c does not know the state of the editor, because
their data are internal to emacs.c and vi.c. So it doesn't know
whether you're in some mode that treats keyboard input specially.
Which means this way of dealing with SIGWINCH is fundamentally
misdesigned and is not worth fixing.
It gets sillier: in addition to sending keyboard commands, edit.c
was also communicating directly with emacs.c and vi.c via a flag,
e_nocrnl, which means 'please don't make Ctrl+L emit a linefeed'
(it normally refreshes on a new line but that is undesirable for
SIGWINCH). So there is already a hack that breaks the barrier
between edit.c and emacs.c/vi.c. Let's do that properly instead.
As of this commit, ed_read() does not send any fake keystrokes.
Instead, two extern functions, emacs_redraw() and vi_redraw(), are
defined for redrawing the command line. These are put in emacs.c
and vi.c so they have access to relevant static data and functions.
Then, instead of sending keyboard commands to the editor and
returning, ed_read() simply calls the redraw function for the
active editor, then continues and waits for input. Much cleaner.
src/cmd/ksh93/include/edit.h:
- Remove e_nocrnl flag from Edit_t struct.
- Define externs emacs_redraw() and vi_redraw(). Since Emacs_t and
Vi_t types are not known here, we have to declare void* pointers
and the functions will have to use typecasts.
src/cmd/ksh93/edit/edit.c:
- ed_read(): Call emacs_redraw() or vi_redraw() as per above.
- ed_getchar(): Remove comment about a nonexistent while loop.
src/cmd/ksh93/edit/emacs.c:
- Updates corresponding to removal of e_nocrnl flag.
- Add emacs_redraw(). This one is pretty simple. Refresh the
command line, then ed_flush() to update the cursor display.
src/cmd/ksh93/edit/vi.c:
- Updates corresponding to removal of e_nocrnl flag. Also remove a
similar internal 'nonewline' flag which is now equally redundant.
- Move the Ctrl+L handling code (minus writing the newline) into
the vi_redraw() function.
- Change two cases where vi set nonewline and sent Ctrl+L to itself
into simple vi_redraw() calls.
- Add vi_redraw(). This is more complicated as it incorporates the
previous Ctrl+L code. It needs an added refresh() call with a
check whether we're currently in command or insert mode, as those
use different refresh methods. Luckily edit.c already maintains
an *e_vi_insert flag in ed_getchar() that we can use. Since vi's
refresh() already calls ed_flush(), we don't need to add that.
2021-02-21 23:07:30 +00:00
|
|
|
2021-02-21:
|
|
|
|
|
|
|
|
|
|
- Fixed: The way that SIGWINCH was handled (i.e. the signal emitted when the
|
|
|
|
|
terminal window size changes) could cause strange emacs/vi editor behaviour.
|
|
|
|
|
|
DEBUG trap: restore status 2 trigger to skip command (re: d00b4b39)
So now we know what that faulty check for shp->indebug in sh_trap()
was meant to do: it was meant to pass down the trap handler's exit
status, via sh_debug(), down to sh_exec() (xec.c) so that it could
then skip the execution of the next command if the trap's exit
status is 2, as documented in the manual page. As of d00b4b39, exit
status 2 was not passed down, so this stopped working.
This commit reinstates that functionality, but without the exit
status bug in command substitutions caused by the old way.
src/cmd/ksh93/sh/fault.c: sh_trap():
- Save the trap's exit status before restoring the parent
envionment's exit status. Make this saved exit status the return
value of the function. (This does not break anything, AFAICT; the
majority of sh_trap() calls ignore the return value, and the few
that don't ignore it seem to expect it to return exactly this.)
src/cmd/ksh93/sh/xec.c: sh_exec():
- The sh_trap() fix has one side effect: whereas the exit status of
a skipped command was always 2 (as per the trap handler), now it
is always 0, because it gets reset in sh_exec() but no command is
executed. That is probably not a desirable change in behaviour,
so let's fix that here instead: set sh.exitval to 2 when skipping
commands.
src/cmd/ksh93/sh.1:
- Document that ${.sh.command} shell-quotes its arguments for use
by 'eval' and such. This fact was not documented anywhere, AFAIK.
src/cmd/ksh93/shell.3:
- Document that $? (exit status) is made local to trap handlers.
- Document that sh_trap() returns the trap handler's exit status.
src/cmd/ksh93/tests/basic.sh:
- Add test for this bug.
- Add a missing test for the exit status 255 functionality (if a
DEBUG trap handler yields this exit status and we're executing a
function or dot script, a return is triggered).
Fixes: https://github.com/ksh93/ksh/issues/187
2021-02-20 05:13:51 +00:00
|
|
|
2021-02-20:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug introduced on 2021-01-20: if a DEBUG trap action yielded exit
|
|
|
|
|
status 2, the execution of the next command was not skipped as documented.
|
|
|
|
|
|
Fix multiple buffer overflows with justified strings (-L/-R/-Z)
ksh crashed in various different and operating system-dependent
ways when attempting to create or apply justification strings
using typeset -L/-R/-Z, especially if large sizes are used.
The crashes had two immediate causes:
- In nv_newattr(), when applying justification attributes, a buffer
was allocated for the justified string that was exactly 8 bytes
longer than the original string. Any larger justification string
caused a buffer overflow (!!!).
- In nv_putval(), when applying existing attributes to a new value,
the corresponding memmove() either did not zero-terminate the
justified string (if the original string was longer than the
justified string) or could read memory past the original string
(if the original string was shorter than the justified string).
Both scenarios can cause a crash.
This commit fixes other minor issues as well, such as a mysterious
8 extra bytes allocated by several malloc/realloc calls. This may
have been some naive attempt to paper over the above bugs. It seems
no one can make any other kind of sense of it.
A readjustment bug with zero-filling was also fixed.
src/cmd/ksh93/sh/name.c:
- nv_putval():
. Get rid of the magical +8 bytes for malloc and realloc. Just
allocate one extra byte for the terminating zero.
. Fix the memmove operation to use strncpy instead, so that
buffer overflows are avoided in both scenarios described above.
Also make it conditional upon a size adjustment actually
happening (i.e. if 'dot' is nonzero).
. Mild refactoring: combine two 'if(sp)' blocks into one;
declare variables only used there locally for legibility.
- nv_newattr():
* Replace the fatally broken "let's allocate string length + 8
bytes no matter the size of the adjustment" routine with a new
one based on work by @hyenias (see comments in #142). It is
efficient with memory, taking into account numeric types,
growing strings, and shrinking strings.
* Fix zero-filling in readjustment after changing the initial
size of a -Z attribute. If the number was zero, all zeros were
still skipped, leaving an empty string.
Thanks to @hyenias for originally identifying this breakage and
laying the groundwork for fixing nv_newattr(), and to @lijog for
the crash analysis that revealed the key to the nv_putval() fix.
Resolves: https://github.com/ksh93/ksh/issues/142
Resolves: https://github.com/ksh93/ksh/issues/181
2021-02-20 12:27:53 +00:00
|
|
|
- Fixed multiple buffer overflows causing crashes in typeset -L/-R-/-Z.
|
|
|
|
|
|
|
|
|
|
- Fixed typeset -Z zero-filling: if the number was zero, all zeros
|
|
|
|
|
were skipped when changing the initial size value of the -Z attribute,
|
|
|
|
|
leaving an empty string.
|
|
|
|
|
|
2021-02-18 14:46:10 +00:00
|
|
|
2021-02-18:
|
|
|
|
|
|
|
|
|
|
- A bug was fixed in the 'read' builtin that caused it to fail to process
|
|
|
|
|
multibyte characters properly in Shift-JIS locales.
|
|
|
|
|
|
2021-02-17 14:29:12 +00:00
|
|
|
2021-02-17:
|
|
|
|
|
|
|
|
|
|
- Emacs mode fixes:
|
|
|
|
|
1. Erasing a backslash while doing a reverse search (^R) no longer deletes
|
|
|
|
|
extra characters.
|
|
|
|
|
2. The backslash now escapes a subsequent interrupt (^C) as documented.
|
|
|
|
|
|
Fix subshell scoping of changes in shared command substitution
A ${ shared-state command substitution; } (internally called
subshare) is documented to share its state with the parent shell
environment, so all changes made within the command substitution
survive outside of it. However, when it is run within a
virtual/non-forked subshell, variables that are not already local
to that subshell will leak out of it into the grandparent state.
Reproducer:
$ ksh -c '( v=${ bug=BAD; } ); echo "$bug"'
BAD
If the variable pre-exists in the subshell, the bug does not occur:
$ ksh -c '( bug=BAD1; v=${ bug=BAD2; } ); echo "$bug"'
(empty line, as expected)
The problem is that the sh_assignok() function, which is
responsible for variable scoping in virtual subshells, does not
ever bother to create a virtual subshell scope for a subshare.
That is an error if a subshare's parent (or higher-up ancestor)
environment is a virtual subshell, because a scope needs to be
created in that parent environment if none exists.
To make this bugfix possible, first we need to get something out of
the way. nv_restore() temporarily sets the subshell's pointer to
the preesnt working directory, shpwd, to null. This causes
sh_assignok() to assume that the subshell is a subshare (because
subshares don't store their own PWD) and refuse to create a scope.
However, nv_restore() sets it to null for a different purpose: to
temporarily disable scoping for *all* virtual subshells, making
restoring possible. This is a good illustration of why it's often
not a good idea to use the same variable for unrelated purposes.
src/cmd/ksh93/sh/subshell.c:
- Add a global static subshell_noscope flag variable to replace the
misuse of sh.shpwd described above.
- sh_assignok():
. Check subshell_noscope instead of shpwd to see if scope
creation is disabled. This makes it possible to distinguish
between restoring scope and handling subshares.
. If the current environment is a subshare that is in a virtual
subshell, create a scope in the parent subshell. This is done
by temporarily making the parent virtual subshell the current
subshell (by setting the global subshell_data pointer to it)
and calling sh_assignok() again, recursively.
- nv_restore(): To disable subshell scope creation while restoring,
set subshell_noscope instead of saving and unsetting sh.shpwd.
src/cmd/ksh93/tests/subshell.sh:
- Add tests. I like tests. Tests are good.
Fixes: https://github.com/ksh93/ksh/issues/143
2021-02-17 15:33:48 +00:00
|
|
|
- Fixed a longstanding bug with shared-state command substitutions of the form
|
|
|
|
|
${ command; }. If these were executed in a subshell, changes made within
|
|
|
|
|
could survive not only the command substitution but also the parent subshell.
|
|
|
|
|
|
Add new 'nobackslashctrl' shell option; other minor editor tweaks
The following emacs editor 'feature' kept making me want to go back
to bash. I forget a backslash escape in a command somewhere. So I
go back to insert it. I type the \, then want to go forward. My
right arrow key, instead of moving the cursor, then replaces my
backslash with garbage. Why? The backslash escapes the following
control character at the editor level and inserts it literally.
The vi editor has a variant of this which is much less harmful. It
only works in insert mode and the backslash only escapes the next
kill or erase character.
In both editors, this feature is completely redundant with the
'stty lnext' character which is ^V by default -- and works better
as well because it also escapes ^C, ^J (linefeed) and ^M (Return).
[In fact, you could even issue 'stty lnext \\' and get a much more
consistent version of this feature on any shell. You have to type
two backslashes to enter one, but it won't kill your cursor keys.]
If it were up to me alone, I'd simply remove this misfeature from
both editors. However, it is long-standing documented behaviour.
It's in the 1995 book. Plus, POSIX specifies the vi variant of it.
So, this adds a shell option instead. It was quite trivial to do.
Now I can 'set --nobackslashctrl' in my ~/.kshrc. What a relief!
Note: To keep .kshrc compatibile with older ksh versions, use:
command set --nobackslashctrl 2>/dev/null
src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/data/options.c:
- Add new SH_NOBACKSLCTRL/"nobackslashctrl" long-form option. The
"no" prefix shows it to the user as "backslashctrl" which is on
by default. This avoids unexpectedly changing historic behaviour.
src/cmd/ksh93/edit/emacs.c: ed_emacsread(),
src/cmd/ksh93/edit/vi.c: getline():
- Only set the flag for special backslash handling if
SH_NOBACKSLCTRL is off.
src/cmd/ksh93/sh.1,
src/cmd/ksh93/data/builtins.c:
- Document the new option (as "backslashctrl", on by default).
Other minor tweaks:
src/cmd/ksh93/edit/edit.c:
- ed_setup(): Add fallback #error if no tput method is set. This
should never be triggered; it's to catch future editing mistakes.
- escape(): cntl('\t') is nonsense as '\t' is already a control
character, so change this to just '\t'.
- xcommands(): Let's enable the ^X^D command for debugging
information on non-release builds.
src/cmd/ksh93/features/cmds:
- The tput feature tests assumed a functioning terminal in $TERM.
However, for all we know we might be compiling with no tty and
TERM=dumb. The tput commands won't work then. So set TERM=ansi
to use a standard default.
2021-02-16 00:07:03 +00:00
|
|
|
2021-02-15:
|
|
|
|
|
|
2021-02-16 06:50:12 +00:00
|
|
|
- Fixed a regression introduced by ksh93 (was not in ksh88): an empty 'case'
|
|
|
|
|
list on a single line ('case x in esac') was a syntax error.
|
|
|
|
|
|
2021-02-16 01:47:15 +00:00
|
|
|
- Fixed a bug in the emacs built-in editor, introduced on 2020-09-17, that
|
|
|
|
|
made the Meta-D and Meta-H keys delete single characters instead of words.
|
|
|
|
|
|
Add new 'nobackslashctrl' shell option; other minor editor tweaks
The following emacs editor 'feature' kept making me want to go back
to bash. I forget a backslash escape in a command somewhere. So I
go back to insert it. I type the \, then want to go forward. My
right arrow key, instead of moving the cursor, then replaces my
backslash with garbage. Why? The backslash escapes the following
control character at the editor level and inserts it literally.
The vi editor has a variant of this which is much less harmful. It
only works in insert mode and the backslash only escapes the next
kill or erase character.
In both editors, this feature is completely redundant with the
'stty lnext' character which is ^V by default -- and works better
as well because it also escapes ^C, ^J (linefeed) and ^M (Return).
[In fact, you could even issue 'stty lnext \\' and get a much more
consistent version of this feature on any shell. You have to type
two backslashes to enter one, but it won't kill your cursor keys.]
If it were up to me alone, I'd simply remove this misfeature from
both editors. However, it is long-standing documented behaviour.
It's in the 1995 book. Plus, POSIX specifies the vi variant of it.
So, this adds a shell option instead. It was quite trivial to do.
Now I can 'set --nobackslashctrl' in my ~/.kshrc. What a relief!
Note: To keep .kshrc compatibile with older ksh versions, use:
command set --nobackslashctrl 2>/dev/null
src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/data/options.c:
- Add new SH_NOBACKSLCTRL/"nobackslashctrl" long-form option. The
"no" prefix shows it to the user as "backslashctrl" which is on
by default. This avoids unexpectedly changing historic behaviour.
src/cmd/ksh93/edit/emacs.c: ed_emacsread(),
src/cmd/ksh93/edit/vi.c: getline():
- Only set the flag for special backslash handling if
SH_NOBACKSLCTRL is off.
src/cmd/ksh93/sh.1,
src/cmd/ksh93/data/builtins.c:
- Document the new option (as "backslashctrl", on by default).
Other minor tweaks:
src/cmd/ksh93/edit/edit.c:
- ed_setup(): Add fallback #error if no tput method is set. This
should never be triggered; it's to catch future editing mistakes.
- escape(): cntl('\t') is nonsense as '\t' is already a control
character, so change this to just '\t'.
- xcommands(): Let's enable the ^X^D command for debugging
information on non-release builds.
src/cmd/ksh93/features/cmds:
- The tput feature tests assumed a functioning terminal in $TERM.
However, for all we know we might be compiling with no tty and
TERM=dumb. The tput commands won't work then. So set TERM=ansi
to use a standard default.
2021-02-16 00:07:03 +00:00
|
|
|
- A new 'backslashctrl' shell option has been added. It is on by default.
|
|
|
|
|
Turning it off (set +o backslashctrl or set --nobackslashctrl) disables the
|
|
|
|
|
special escaping behaviour of the backslash character in the emacs and vi
|
|
|
|
|
built-in editors. Particularly in the emacs editor, this makes it much easier
|
|
|
|
|
to go back, insert a forgotten backslash into a command, and then continue
|
|
|
|
|
editing without having your next cursor key replace your backslash with
|
|
|
|
|
garbage. Note that Ctrl+V (or whatever other character was set using
|
|
|
|
|
'stty lnext') always escapes all control characters in either editing mode.
|
|
|
|
|
|
2021-02-14 19:39:01 +00:00
|
|
|
2021-02-14:
|
|
|
|
|
|
|
|
|
|
- Due to a deficiency in some UNIX veriants, the 'sleep' built-in command could
|
|
|
|
|
occasionally sleep for slightly less than the time specified. It now performs
|
|
|
|
|
an additional check against the system clock to make sure it sleeps at least
|
|
|
|
|
the given amount of time. Thanks to Lev Kujawski for adding this feature.
|
|
|
|
|
|
Fix bugs related to --posix shell option (re: 921bbcae, f45a0f16)
This fixes the following:
1. 'set --posix' now works as an equivalent of 'set -o posix'.
2. The posix option turns off braceexpand and turns on letoctal.
Any attempt to override that in a single command such as 'set -o
posix +o letoctal' was quietly ignored. This now works as long
as the overriding option follows the posix option in the command.
3. The --default option to 'set' now stops the 'posix' option, if
set or unset in the same 'set' command, from changing other
options. This allows the command output by 'set +o' to correctly
restore the current options.
src/cmd/ksh93/data/builtins.c:
- To make 'set --posix' work, we must explicitly list it in
sh_set[] as a supported option so that AST optget(3) recognises
it and won't override it with its own default --posix option,
which converts the optget(3) string to at POSIX getopt(3) string.
This means it will appear as a separate entry in --man output,
whether we want it to or not. So we might as well use it as an
example to document how --optionname == -o optionname, replacing
the original documentation that was part of the '-o' description.
src/cmd/ksh93/sh/args.c: sh_argopts():
- Add handling for explitit --posix option in data/builtins.c.
- Move SH_POSIX syncing SH_BRACEEXPAND and SH_LETOCTAL from
sh_applyopts() into the option parsing loop here. This fixes
the bug that letoctal was ignored in 'set -o posix +o letoctal'.
- Remember if --default was used in a flag, and do not sync options
with SH_POSIX if the flag is set. This makes 'set +o' work.
src/cmd/ksh93/include/argnod.h,
src/cmd/ksh93/data/msg.c,
src/cmd/ksh93/sh/args.c: sh_printopts():
- Do not potentially translate the 'on' and 'off' labels in 'set
-o' output. No other shell does, and some scripts parse these.
src/cmd/ksh93/sh/init.c: sh_init():
- Turn on SH_LETOCTAL early along with SH_POSIX if the shell was
invoked as sh; this makes 'sh -o' and 'sh +o' show expected
options (not that anyone does this, but correctness is good).
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/include/shell.h:
- The state flags were in defs.h and most (but not all) of the
shell options were in shell.h. Gather all the shell state and
option flag definitions into one place in shell.h for clarity.
- Remove unused SH_NOPROFILE and SH_XARGS option flags.
src/cmd/ksh93/tests/options.sh:
- Add tests for these bugs.
src/lib/libast/misc/optget.c: styles[]:
- Edit default optget(3) option self-documentation for clarity.
Several changed files:
- Some SHOPT_PFSH fixes to avoid compiling dead code.
2021-02-14 23:51:19 +00:00
|
|
|
- A few bugs were fixed that 93u+m introduced along with the new '-o posix'
|
|
|
|
|
shell option on 2020-09-01:
|
|
|
|
|
1. 'set --posix' now works as the expected equivalent of 'set -o posix'.
|
|
|
|
|
2. As of 2020-09-18, the posix option turns off braceexpand and turns on
|
|
|
|
|
letoctal. Any attempt to override that in a single command such as
|
|
|
|
|
'set -o posix +o letoctal' was quietly ignored. This now works as long
|
|
|
|
|
as the overriding option follows the posix option on the command line.
|
|
|
|
|
3. The --default option to 'set' now stops the 'posix' option, if set or
|
|
|
|
|
unset in the same 'set' command, from changing other options. This allows
|
|
|
|
|
the command output by 'set +o' to correctly restore the current options.
|
|
|
|
|
|
2021-02-11 13:41:10 +00:00
|
|
|
2021-02-11:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused ksh to lose track of all running background jobs if
|
|
|
|
|
a shared-state command substitution of the form v=${ cmd; } was used twice.
|
|
|
|
|
|
Fix most of job control (-m/-o monitor) in scripts
If I haven't missed anything, this should make the non-interactive
aspects of job control in scripts work as expected, except for the
"<command unknown>" issue in the output of 'bg', 'fg' and 'jobs'
(which is not such a high priority as those commands are really
designed for interactive use).
Plus, I believe I now finally understand what these three are for:
* The job.jobcontrol variable is set to nonzero by job_init() in
jobs.c if, and only if, the shell is interactive *and* managed to
get control of the terminal. Therefore, any changing of terminal
settings (tcsetpgrp(3), tty_set()) should only be done if
job.jobcontrol is nonzero. This commit changes several checks for
sh_isoption(SH_INTERACTIVE) to checks for job.jobcontrol for
better consistency with this.
* The state flag, sh_isstate(SH_MONITOR), determines whether the
bits of job control that are relevant for both scripts and
interactive shells are active, which is mostly making sure that a
background job gets its own process group (setpgid(3)).
* The shell option, sh_isoption(SH_MONITOR), is just that. When the
user turns it on or off, the state flag is synched with it. It
should usually not be directly checked for, as the state may be
temporarily turned off without turning off the option.
Prior discussion:
https://www.mail-archive.com/austin-group-l@opengroup.org/msg06456.html
src/cmd/ksh93/bltins/typeset.c, src/cmd/ksh93/sh/args.c:
- Move synching the SH_MONITOR state flag with the SH_MONITOR
shell option from b_set() (the 'set' builtin) to sh_applyopts()
which is indirectly called from b_set() and is also used when
parsing the shell invocation command line. This ensures -m is
properly enabled in both scenarios.
src/cmd/ksh93/sh/jobs.c:
- job_init(): Do not refuse to initialise job control on
non-interactive shells. Instead, skip everything that should only
be done on interactive shells (i.e., everything to do with the
terminal). This function is now even more of a mess than it was
before, so refactoring may be desirabe at some point.
- job_close(), job_set(), job_reset(), job_wait(): Do not reset the
terminal process group (tcsetpgrp()) if job.jobcontrol isn't on.
src/cmd/ksh93/sh/xec.c:
- sh_exec(): TFORK: For SIGINT handling, check the SH_MONITOR
state flag, not the shell option.
- sh_exec(): TFORK: Do not turn off the SH_MONITOR state flag in
forked children. The non-interactive part of job control should
stay active. Instead, turn off the SH_INTERACTIVE state flag so
we don't get interactive shell behaviour (i.e. job control noise
on the terminal) in forked subshells.
- _sh_fork(), sh_ntfork(): Do not reset the terminal process group
(tcsetpgrp()) if job.jobcontrol isn't on. Do not turn off the
SH_MONITOR state flag in forked children.
src/cmd/ksh93/sh/subshell.c: sh_subfork():
- Do not turn off the monitor option and state in forked subshells.
The non-interactive part of job control should stay active.
src/cmd/ksh93/bltins/misc.c: b_bg():
- Check isstate(SH_MONITOR) instead of sh_isoption(SH_MONITOR) &&
job.jobcontrol before throwing a 'no job control' error.
This fixes a minor bug: fg, bg and disown could quietly fail.
src/cmd/ksh93/tests/jobs.sh:
- Add tests for 'fg' with job control IDs (%%, %1) in scripts.
- Add test checking that a background job launched from a subsell
with job control enabled correctly becomes the leader of its own
process group.
Makes progress on: https://github.com/ksh93/ksh/issues/119
2021-02-11 22:05:00 +00:00
|
|
|
- Job control (the -m/-o monitor option) has been fixed for scripts. Background
|
|
|
|
|
jobs are now correctly assigned their own process group when run from
|
|
|
|
|
subshells (except command substitutions). The 'fg' command now also works for
|
|
|
|
|
scripts as it does on other shells, though 'wait' should be preferred.
|
|
|
|
|
|
2021-02-05 03:42:10 +00:00
|
|
|
2021-02-05:
|
|
|
|
|
|
|
|
|
|
- Fixed a longstanding bug that caused redirections that store a file
|
|
|
|
|
descriptor > 10 in a variable, such as {var}>file, to stop working if
|
|
|
|
|
brace expansion (the -B or -o braceexpand option) was turned off. (Note
|
|
|
|
|
that '{var}' is not a brace expansion as it does not contain ',' or '..'.)
|
|
|
|
|
|
2021-02-04 17:31:22 +00:00
|
|
|
2021-02-04:
|
|
|
|
|
|
|
|
|
|
- Fixed ksh crashing if an autoloaded function tried to autoload itself.
|
|
|
|
|
ksh now errors out gracefully with an "autoload loop" error message.
|
|
|
|
|
|
|
|
|
|
- Fixed crash on trying a very long nonexistent command.
|
|
|
|
|
|
2021-02-01 16:19:04 +00:00
|
|
|
2021-02-01:
|
|
|
|
|
|
2021-02-01 23:35:18 +00:00
|
|
|
- Fixed a bug in 'typeset': the '-s' modifier option for short integer will
|
|
|
|
|
now only be applied if the integer option '-i' is also present, avoiding
|
|
|
|
|
inconsistent results and a crash.
|
|
|
|
|
|
2021-02-01 16:19:04 +00:00
|
|
|
- Fixed: scalar arrays (-a) and associative arrays (-A) of a type created by
|
|
|
|
|
'enum' allowed values not specified by the enum type, corrupting results.
|
|
|
|
|
|
|
|
|
|
- Fixed: the "${array[@]}" expansion for associative arrays of a type created
|
|
|
|
|
by 'enum' expanded to random numbers instead of the array's values.
|
|
|
|
|
|
command -x: fix efficiency; always run external cmd (re: acf84e96)
This commit fixes 'command -x' to adapt to OS limitations with
regards to data alignment in the arguments list. A feature test is
added that detects if the OS aligns the argument on 32-bit or
64-bit boundaries or not at all, allowing 'command -x' to avoid
E2BIG errors while maximising efficiency.
Also, as of now, 'command -x' is a way to bypass built-ins and
run/query an external command. Built-ins do not limit the length of
their argument list, so '-x' never made sense to use for them. And
because '-x' hangs on Linux and macOS on every ksh93 release
version to date (see acf84e96), few use it, so there is little
reason not to make this change.
Finally, this fixes a longstanding bug that caused the minimum exit
status of 'command -x' to be 1 if a command with many arguments was
divided into several command invocations. This is done by replacing
broken flaggery with a new SH_XARG state flag bit.
src/cmd/ksh93/features/externs:
- Add new C feature test detecting byte alignment in args list.
The test writes a #define ARG_ALIGN_BYTES with the amount of
bytes the OS aligns arguments to, or zero for no alignment.
src/cmd/ksh93/include/defs.h:
- Add new SH_XARG state bit indicating 'command -x' is active.
src/cmd/ksh93/sh/path.c: path_xargs():
- Leave extra 2k in the args buffer instead of 1k, just to be sure;
some commands add large environment variables these days.
- Fix a bug in subtracting the length of existing arguments and
environment variables. 'size -= strlen(cp)-1;' subtracts one less
than the size of cp, which makes no sense; what is necessary is
to substract the length plus one to account for the terminating
zero byte, i.e.: 'size -= strlen(cp)+1'.
- Use the ARG_ALIGN_BYTES feature test result to match the OS's
data alignment requirements.
- path_spawn(): E2BIG: Change to checking SH_XARG state bit.
src/cmd/ksh93/bltins/whence.c: b_command():
- Allow combining -x with -p, -v and -V with the expected results
by setting P_FLAG to act like 'whence -p'. E.g., as of now,
command -xv printf
is equivalent to
whence -p printf
but note that 'whence' has no equivalent of 'command -pvx printf'
which searches $(getconf PATH) for a command.
- When -x will run a command, now set the new SH_XARG state flag.
src/cmd/ksh93/sh/xec.c: sh_exec():
- Change to using the new SH_XARG state bit.
- Skip the check for built-ins if SH_XARG is active, so that
'command -x' now always runs an external command.
src/lib/libcmd/date.c, src/lib/libcmd/uname.c:
- These path-bound builtins sometimes need to run the external
system command by the same name, but they did that by hardcoding
an unportable direct path. Now that 'command -x' runs an external
command, change this to using 'command -px' to guarantee using
the known-good external system utility in the default PATH.
- In date.c, fix the format string passed to 'command -px date'
when setting the date; it was only compatible with BSD systems.
Use the POSIX variant on non-BSD systems.
2021-01-30 05:51:22 +00:00
|
|
|
2021-01-30:
|
|
|
|
|
|
|
|
|
|
- The -x option to the 'command' built-in now causes it to bypass built-ins
|
|
|
|
|
so that it always runs/queries an external command. See 'command --man'.
|
|
|
|
|
|
|
|
|
|
- Fixed a bug in 'command -x' that caused the minimum exit status to be 1 if
|
|
|
|
|
a command with many arguments was divided into several command invocations.
|
|
|
|
|
|
|
|
|
|
- The 2020-08-16 fix is improved with a compile-time feature test that
|
2021-02-01 16:19:04 +00:00
|
|
|
detects if the OS requires extra bytes per argument in the arguments list,
|
command -x: fix efficiency; always run external cmd (re: acf84e96)
This commit fixes 'command -x' to adapt to OS limitations with
regards to data alignment in the arguments list. A feature test is
added that detects if the OS aligns the argument on 32-bit or
64-bit boundaries or not at all, allowing 'command -x' to avoid
E2BIG errors while maximising efficiency.
Also, as of now, 'command -x' is a way to bypass built-ins and
run/query an external command. Built-ins do not limit the length of
their argument list, so '-x' never made sense to use for them. And
because '-x' hangs on Linux and macOS on every ksh93 release
version to date (see acf84e96), few use it, so there is little
reason not to make this change.
Finally, this fixes a longstanding bug that caused the minimum exit
status of 'command -x' to be 1 if a command with many arguments was
divided into several command invocations. This is done by replacing
broken flaggery with a new SH_XARG state flag bit.
src/cmd/ksh93/features/externs:
- Add new C feature test detecting byte alignment in args list.
The test writes a #define ARG_ALIGN_BYTES with the amount of
bytes the OS aligns arguments to, or zero for no alignment.
src/cmd/ksh93/include/defs.h:
- Add new SH_XARG state bit indicating 'command -x' is active.
src/cmd/ksh93/sh/path.c: path_xargs():
- Leave extra 2k in the args buffer instead of 1k, just to be sure;
some commands add large environment variables these days.
- Fix a bug in subtracting the length of existing arguments and
environment variables. 'size -= strlen(cp)-1;' subtracts one less
than the size of cp, which makes no sense; what is necessary is
to substract the length plus one to account for the terminating
zero byte, i.e.: 'size -= strlen(cp)+1'.
- Use the ARG_ALIGN_BYTES feature test result to match the OS's
data alignment requirements.
- path_spawn(): E2BIG: Change to checking SH_XARG state bit.
src/cmd/ksh93/bltins/whence.c: b_command():
- Allow combining -x with -p, -v and -V with the expected results
by setting P_FLAG to act like 'whence -p'. E.g., as of now,
command -xv printf
is equivalent to
whence -p printf
but note that 'whence' has no equivalent of 'command -pvx printf'
which searches $(getconf PATH) for a command.
- When -x will run a command, now set the new SH_XARG state flag.
src/cmd/ksh93/sh/xec.c: sh_exec():
- Change to using the new SH_XARG state bit.
- Skip the check for built-ins if SH_XARG is active, so that
'command -x' now always runs an external command.
src/lib/libcmd/date.c, src/lib/libcmd/uname.c:
- These path-bound builtins sometimes need to run the external
system command by the same name, but they did that by hardcoding
an unportable direct path. Now that 'command -x' runs an external
command, change this to using 'command -px' to guarantee using
the known-good external system utility in the default PATH.
- In date.c, fix the format string passed to 'command -px date'
when setting the date; it was only compatible with BSD systems.
Use the POSIX variant on non-BSD systems.
2021-01-30 05:51:22 +00:00
|
|
|
maximising the efficiency of 'command -x' for the system it runs on.
|
|
|
|
|
|
Fix field splitting bug triggered by DEBUG trap
An unquoted variable expansion evaluated in a DEBUG trap action
caused IFS field splitting to be deactivated in code executed after
the trap action. Thanks to Koichi Nakashima for the reproducer:
| v=''
| trap ': $v' DEBUG
| A="a b c"
| set -- $A
| printf '%s\n' "$@"
|
| Expected
|
| a
| b
| c
|
| Actual
|
| a b c
src/cmd/ksh93/sh/fault.c: sh_trap():
- Remove incorrect save/restore of sh.ifstable, the internal state
table for field splitting. This reverts three lines added in ksh
93t+ 2009-11-30. Analysis: As an expansion is split into fields
(macro.c, lines 2367-2471), sh.ifstable is modified. If that
happens within a DEBUG trap, any modifications in ifstable are
undone by the restoring memccpy, leaving an inconsistent state.
src/cmd/ksh93/COMPATIBILITY:
- Document the DEBUG trap fixes, particularly the incorrect
inheritance by subshells and functions that some scripts may now
rely on because this bug is so longstanding. (re: 2a835a2d)
src/cmd/ksh93/tests/basic.sh:
- Add relevant tests.
Resolves: https://github.com/ksh93/ksh/issues/155
TODO: add a -T (-o functrace) option as in bash, which should allow
subshells and ksh-style functions to inherit DEBUG traps.
P.S.: The very handy multishell repo allows us to use 'git blame'
to trace the origin of the recently fixed DEBUG trap bugs.
The off-by-one error causing various bugs, reverted in 2a835a2d,
was introduced in ksh 93t 2008-07-25:
https://github.com/multishell/ksh93/commit/8e947ccf
(fault.c, line 321)
The incorrect check causing the exit status bug, reverted in
d00b4b39, was introduced in ksh 93t 2008-11-04:
https://github.com/multishell/ksh93/commit/b1ade268
(fault.c, line 459)
The ifstable save/restore causing the field splitting bug, reverted
in this commit, was introduced in ksh 93t+ 2009-11-30:
https://github.com/multishell/ksh93/commit/53d9f009
(fault.c, lines 440, 444, 482)
So all the bugs reported in #155 were fixed by simply reverting
these specific changes. I think that they are some experiments that
the developers simply forgot to remove. I've suspected such a thing
multiple times before. ksh93 was developed by researchers who were
genius innovators, but incredibly sloppy maintainers.
2021-01-24 15:46:46 +00:00
|
|
|
2021-01-24:
|
|
|
|
|
|
2021-01-24 22:45:08 +00:00
|
|
|
- Fixed a bug in 'typeset': combining the -u option with -F or -E caused the
|
|
|
|
|
variable to become a hexadecimal floating point in error.
|
|
|
|
|
|
Fix field splitting bug triggered by DEBUG trap
An unquoted variable expansion evaluated in a DEBUG trap action
caused IFS field splitting to be deactivated in code executed after
the trap action. Thanks to Koichi Nakashima for the reproducer:
| v=''
| trap ': $v' DEBUG
| A="a b c"
| set -- $A
| printf '%s\n' "$@"
|
| Expected
|
| a
| b
| c
|
| Actual
|
| a b c
src/cmd/ksh93/sh/fault.c: sh_trap():
- Remove incorrect save/restore of sh.ifstable, the internal state
table for field splitting. This reverts three lines added in ksh
93t+ 2009-11-30. Analysis: As an expansion is split into fields
(macro.c, lines 2367-2471), sh.ifstable is modified. If that
happens within a DEBUG trap, any modifications in ifstable are
undone by the restoring memccpy, leaving an inconsistent state.
src/cmd/ksh93/COMPATIBILITY:
- Document the DEBUG trap fixes, particularly the incorrect
inheritance by subshells and functions that some scripts may now
rely on because this bug is so longstanding. (re: 2a835a2d)
src/cmd/ksh93/tests/basic.sh:
- Add relevant tests.
Resolves: https://github.com/ksh93/ksh/issues/155
TODO: add a -T (-o functrace) option as in bash, which should allow
subshells and ksh-style functions to inherit DEBUG traps.
P.S.: The very handy multishell repo allows us to use 'git blame'
to trace the origin of the recently fixed DEBUG trap bugs.
The off-by-one error causing various bugs, reverted in 2a835a2d,
was introduced in ksh 93t 2008-07-25:
https://github.com/multishell/ksh93/commit/8e947ccf
(fault.c, line 321)
The incorrect check causing the exit status bug, reverted in
d00b4b39, was introduced in ksh 93t 2008-11-04:
https://github.com/multishell/ksh93/commit/b1ade268
(fault.c, line 459)
The ifstable save/restore causing the field splitting bug, reverted
in this commit, was introduced in ksh 93t+ 2009-11-30:
https://github.com/multishell/ksh93/commit/53d9f009
(fault.c, lines 440, 444, 482)
So all the bugs reported in #155 were fixed by simply reverting
these specific changes. I think that they are some experiments that
the developers simply forgot to remove. I've suspected such a thing
multiple times before. ksh93 was developed by researchers who were
genius innovators, but incredibly sloppy maintainers.
2021-01-24 15:46:46 +00:00
|
|
|
- Fixed: an unquoted variable expansion evaluated in a DEBUG trap action caused
|
|
|
|
|
IFS field splitting to be deactivated in code executed after the trap action.
|
|
|
|
|
This bug was introduced in ksh 93t+ 2009-11-30.
|
|
|
|
|
|
2021-01-24 00:49:36 +00:00
|
|
|
2021-01-23:
|
|
|
|
|
|
|
|
|
|
- Fixed: when the DEBUG trap was redefined in a subshell, the DEBUG trap in
|
|
|
|
|
the parent environment was corrupted or the shell crashed.
|
2021-01-24 03:51:00 +00:00
|
|
|
When a redirection was used in a DEBUG trap action, the trap was disabled.
|
Fix field splitting bug triggered by DEBUG trap
An unquoted variable expansion evaluated in a DEBUG trap action
caused IFS field splitting to be deactivated in code executed after
the trap action. Thanks to Koichi Nakashima for the reproducer:
| v=''
| trap ': $v' DEBUG
| A="a b c"
| set -- $A
| printf '%s\n' "$@"
|
| Expected
|
| a
| b
| c
|
| Actual
|
| a b c
src/cmd/ksh93/sh/fault.c: sh_trap():
- Remove incorrect save/restore of sh.ifstable, the internal state
table for field splitting. This reverts three lines added in ksh
93t+ 2009-11-30. Analysis: As an expansion is split into fields
(macro.c, lines 2367-2471), sh.ifstable is modified. If that
happens within a DEBUG trap, any modifications in ifstable are
undone by the restoring memccpy, leaving an inconsistent state.
src/cmd/ksh93/COMPATIBILITY:
- Document the DEBUG trap fixes, particularly the incorrect
inheritance by subshells and functions that some scripts may now
rely on because this bug is so longstanding. (re: 2a835a2d)
src/cmd/ksh93/tests/basic.sh:
- Add relevant tests.
Resolves: https://github.com/ksh93/ksh/issues/155
TODO: add a -T (-o functrace) option as in bash, which should allow
subshells and ksh-style functions to inherit DEBUG traps.
P.S.: The very handy multishell repo allows us to use 'git blame'
to trace the origin of the recently fixed DEBUG trap bugs.
The off-by-one error causing various bugs, reverted in 2a835a2d,
was introduced in ksh 93t 2008-07-25:
https://github.com/multishell/ksh93/commit/8e947ccf
(fault.c, line 321)
The incorrect check causing the exit status bug, reverted in
d00b4b39, was introduced in ksh 93t 2008-11-04:
https://github.com/multishell/ksh93/commit/b1ade268
(fault.c, line 459)
The ifstable save/restore causing the field splitting bug, reverted
in this commit, was introduced in ksh 93t+ 2009-11-30:
https://github.com/multishell/ksh93/commit/53d9f009
(fault.c, lines 440, 444, 482)
So all the bugs reported in #155 were fixed by simply reverting
these specific changes. I think that they are some experiments that
the developers simply forgot to remove. I've suspected such a thing
multiple times before. ksh93 was developed by researchers who were
genius innovators, but incredibly sloppy maintainers.
2021-01-24 15:46:46 +00:00
|
|
|
DEBUG traps were also incorrectly inherited by subshells and ksh functions.
|
|
|
|
|
All this was caused by a bug introduced in ksh 93t 2008-07-25.
|
2021-01-24 00:49:36 +00:00
|
|
|
|
Build system: make SHOPT_* editable again; allow indenting Mamfiles
The build system is adapted to make SHOPT_* compile-time options
editable without nmake. We can now easily change ksh's compile-time
options by editing src/cmd/ksh93/SHOPT.sh. The bin/package script
is adapted to turn these into compile flags. This resolves the most
important drawback of not using nmake.
Also, mamake now has support for indented Mam (Make Abstract
Machine) code. Only one type of block (make...done) is supported in
Mamfiles, so they are easy to indent automatically. A script to
(re)do this is included.
Since nmake is not going to be restored (it has too many problems
that no one is interested in fixing), this at least makes mamake
significantly easier to work with.
The Makefiles are deleted. They may still be handy for reference to
understand the Mamfiles, but they haven't actually matched the
Mamfiles for a while -- and you can still look in the git history.
Deleting them requires some adaptations to bin/package and mamake.c
because, even though they do not use those files, they still looked
for them to decide whether to build code in a directory.
Finally, this commit incorporates some #pragmas for clang to
suppress annoying warnings about the coding style used in this
historic code base. (gcc does not complain so much.)
src/cmd/ksh93/SHOPT.sh:
- Added.
bin/package, src/cmd/INIT/package.sh:
- cd into our own directory in case we were run from another dir.
- $makefiles: only look for Mamfiles.
- Add ksh compile-options via KSH_SHOPTFLAGS. Include SHOPT.sh.
- make_recurse(): Do not write a missing Makefile.
- finalize environment: Look for Mamfiles instead of Makefiles.
src/cmd/INIT/mamake.c:
- Tell clang to suppress annoying warnings about coding style.
- Update version string and self-documentation.
- input(): Add support for indented Mam code by skipping initial
whitespace on each input line.
- files[]: Instead of looking for various of Makefiles to decide
where to build, only look for Mamfiles.
src/Makefile, src/cmd/INIT/Makefile, src/cmd/Makefile,
src/cmd/builtin/Makefile, src/cmd/ksh93/Makefile, src/lib/Makefile,
src/lib/libast/Makefile, src/lib/libcmd/Makefile,
src/lib/libdll/Makefile, src/lib/libsum/Makefile:
- Removed.
src/Mamfile, src/cmd/INIT/Mamfile, src/cmd/Mamfile,
src/cmd/builtin/Mamfile, src/cmd/ksh93/Mamfile, src/lib/Mamfile,
src/lib/libast/Mamfile, src/lib/libcmd/Mamfile,
src/lib/libdll/Mamfile, src/lib/libsum/Mamfile:
- Indent the code with tabs.
- In ksh93/Mamfile, add ${KSH_SHOPT_FLAGS} to every $CC command.
- In ksh93/Mamfile, add "prev SHOPT.sh" for every *.o file
so they are rebuilt whenever SHOPT.sh changes.
bin/Mamfile_indent:
- Added, in case someone wants to re-indent a Mamfile.
src/cmd/INIT/proto.c, src/cmd/INIT/ratz.c, src/cmd/INIT/release.c,
src/lib/libast/features/common, src/lib/libast/include/ast.h:
- Tell clang to suppress annoying warnings about coding style that
it disapproves of (mainly concerning the use of parentheses).
src/cmd/INIT/cc.darwin, src/cmd/INIT/cc.freebsd,
src/cmd/INIT/cc.openbsd:
- Remove now-redundant clang warning suppression flags.
Resolves: https://github.com/ksh93/ksh/issues/60
2021-01-22 23:23:14 +00:00
|
|
|
2021-01-22:
|
|
|
|
|
|
|
|
|
|
- Compile-time shell options can now be edited in src/cmd/ksh93/SHOPT.sh
|
|
|
|
|
before building.
|
|
|
|
|
|
2021-01-20 17:40:09 +00:00
|
|
|
2021-01-20:
|
|
|
|
|
|
|
|
|
|
- Fixed: executing a DEBUG trap in a command substitution had side effects
|
|
|
|
|
on the exit status ($?) of non-trap commands.
|
Fix field splitting bug triggered by DEBUG trap
An unquoted variable expansion evaluated in a DEBUG trap action
caused IFS field splitting to be deactivated in code executed after
the trap action. Thanks to Koichi Nakashima for the reproducer:
| v=''
| trap ': $v' DEBUG
| A="a b c"
| set -- $A
| printf '%s\n' "$@"
|
| Expected
|
| a
| b
| c
|
| Actual
|
| a b c
src/cmd/ksh93/sh/fault.c: sh_trap():
- Remove incorrect save/restore of sh.ifstable, the internal state
table for field splitting. This reverts three lines added in ksh
93t+ 2009-11-30. Analysis: As an expansion is split into fields
(macro.c, lines 2367-2471), sh.ifstable is modified. If that
happens within a DEBUG trap, any modifications in ifstable are
undone by the restoring memccpy, leaving an inconsistent state.
src/cmd/ksh93/COMPATIBILITY:
- Document the DEBUG trap fixes, particularly the incorrect
inheritance by subshells and functions that some scripts may now
rely on because this bug is so longstanding. (re: 2a835a2d)
src/cmd/ksh93/tests/basic.sh:
- Add relevant tests.
Resolves: https://github.com/ksh93/ksh/issues/155
TODO: add a -T (-o functrace) option as in bash, which should allow
subshells and ksh-style functions to inherit DEBUG traps.
P.S.: The very handy multishell repo allows us to use 'git blame'
to trace the origin of the recently fixed DEBUG trap bugs.
The off-by-one error causing various bugs, reverted in 2a835a2d,
was introduced in ksh 93t 2008-07-25:
https://github.com/multishell/ksh93/commit/8e947ccf
(fault.c, line 321)
The incorrect check causing the exit status bug, reverted in
d00b4b39, was introduced in ksh 93t 2008-11-04:
https://github.com/multishell/ksh93/commit/b1ade268
(fault.c, line 459)
The ifstable save/restore causing the field splitting bug, reverted
in this commit, was introduced in ksh 93t+ 2009-11-30:
https://github.com/multishell/ksh93/commit/53d9f009
(fault.c, lines 440, 444, 482)
So all the bugs reported in #155 were fixed by simply reverting
these specific changes. I think that they are some experiments that
the developers simply forgot to remove. I've suspected such a thing
multiple times before. ksh93 was developed by researchers who were
genius innovators, but incredibly sloppy maintainers.
2021-01-24 15:46:46 +00:00
|
|
|
This bug was introduced in ksh 93t 2008-11-04.
|
2021-01-20 17:40:09 +00:00
|
|
|
|
typeset: add error msgs for incompatible options; improve usage msg
This adds informative error messages if incompatible options are
given. It also documents the exclusive -m, -n and -T options on
separate usage lines, as was already done with -f. The usage
message for incompatible options now looks something like this:
| $ ksh -c 'typeset -L10 -F -f -i foo'
| ksh: typeset: -i/-F/-E/-X cannot be used with -L/-R/-Z
| ksh: typeset: -f cannot be used with other options
| Usage: typeset [-bflmnprstuxACHS] [-a[type]] [-i[base]] [-E[n]]
| [-F[n]] [-L[n]] [-M[mapping]] [-R[n]] [-X[n]]
| [-h string] [-T[tname]] [-Z[n]] [name[=value]...]
| Or: typeset -f [name...]
| Or: typeset -m [name=name...]
| Or: typeset -n [name=name...]
| Or: typeset -T [tname[=(type definition)]...]
| Help: typeset [ --help | --man ] 2>&1
(see also the previous commit, e21a053e)
Unfortunately the first "Usage" line has some redundancies with the
"Or:" lines showing separate usages. It doesn't seem to be possible
to avoid this; it's a flaw in how libast generates everything
(usage, help, manual) from one huge getopt(3) string. I still think
the three added "Or:" lines are an improvement as it wasn't
previously shown that these options need to be used on their own.
src/cmd/ksh93/bltins/typeset.c: b_typeset():
- Instead of only showing a generic usage message, add an
informative error message if incompatible options were given.
- Conflicting options detection was failing because NV_LJUST and
NV_EXPNOTE have the same bitmask value. Use a new 'isadjust'
flag for -L/-R/-Z to remember if one of these was set.
- Detect conflict between -L/-R/-Z and a float option, not just -i.
src/cmd/ksh93/include/name.h, src/cmd/ksh93/data/msg.c:
- Add the two new error messages for incompatible options.
src/cmd/ksh93/data/builtins.c: sh_opttypeset[]:
- Add a space after 'float' in in "[+float?\btypeset -lE\b]" as
this makes 'float' appear on its own line, improving formatting.
- Show -m, -n, -T on separate usage lines like -f, as none of these
can be combined with other options.
- Remove "cannot be combined with other options" from -m and -n
descriptions, as that should now be clear from the separate usage
lines -- and even if not, the error message is now informative.
src/cmd/ksh93/sh.1, src/cmd/ksh93/COMPATIBILITY:
- Update.
src/cmd/ksh93/tests/types.sh:
- Remove obsolete test: 'typeset -RF' is no longer accepted.
(It crashed in 93u+, so this is not an incompatibility...)
Resolves: https://github.com/ksh93/ksh/issues/48
2021-01-21 00:24:13 +00:00
|
|
|
- The typeset builtin command now gives an informative error message if an
|
|
|
|
|
incompatible combination of options is given.
|
|
|
|
|
|
2021-01-19 18:47:41 +00:00
|
|
|
2021-01-19:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash when using 'cd' in a virtual/non-forking subshell in a
|
|
|
|
|
situation where the current working directory cannot be determined.
|
|
|
|
|
|
2021-01-08 22:15:48 +00:00
|
|
|
2021-01-08:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash on exceeding the maximum size of the $PS1 prompt.
|
|
|
|
|
The maximum size is also increased from 160 to 256 bytes.
|
|
|
|
|
|
2021-01-07 17:34:47 +00:00
|
|
|
2021-01-07:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash that could occur while ksh updated ${.sh.match}.
|
|
|
|
|
|
Implement hash tables for virtual subshells (re: 102868f8, 9d428f8f)
The forking fix implemented in 102868f8 and 9d428f8f, which stops
the main shell's hash table from being cleared if PATH is changed
in a subshell, can cause a significant performance penalty for
certain scripts that do something like
( PATH=... command foo )
in a subshell, especially if done repeatedly. This is because the
hash table is cleared (and hence a subshell forks) even for
temporary PATH assignments preceding commands.
It also just plain doesn't work. For instance:
$ hash -r; (ls) >/dev/null; hash
ls=/bin/ls
Simply running an external command in a subshell caches the path in
the hash table that is shared with a main shell. To remedy this, we
would have to fork the subshell before forking any external
command. And that would be an unacceptable performance regression.
Virtual subshells do not need to fork when changing PATH if they
get their own hash tables. This commit adds these. The code for
alias subshell trees (which was removed in ec888867 because they
were broken and unneeded) provided the beginning of a template for
their implementation.
src/cmd/ksh93/sh/subshell.c:
- struct subshell: Add strack pointer to subshell hash table.
- Add sh_subtracktree(): return pointer to subshell hash table.
- sh_subfuntree(): Refactor a bit for legibility.
- sh_subshell(): Add code for cleaning up subshell hash table.
src/cmd/ksh93/sh/name.c:
- nv_putval(): Remove code to fork a subshell upon resetting PATH.
- nv_rehash(): When in a subshell, invalidate a hash table entry
for a subshell by creating the subshell scope if needed, then
giving that entry the NV_NOALIAS attribute to invalidate it.
src/cmd/ksh93/sh/path.c: path_search():
- To set a tracked alias/hash table entry, use sh_subtracktree()
and pass the HASH_NOSCOPE flag to nv_search() so that any new
entries are added to the current subshell table (if any) and do
not influence any parent scopes.
src/cmd/ksh93/bltins/typeset.c: b_alias():
- b_alias(): For hash table entries, use sh_subtracktree() instead
of forking a subshell. Keep forking for normal aliases.
- setall(): To set a tracked alias/hash table entry, pass the
HASH_NOSCOPE flag to nv_search() so that any new entries are
added to the current subshell table (if any) and do not influence
any parent scopes.
src/cmd/ksh93/sh/init.c: put_restricted():
- Update code for clearing the hash table (when changing $PATH) to
use sh_subtracktree().
src/cmd/ksh93/bltins/cd_pwd.c:
- When invalidating path name bindings to relative paths, use the
subshell hash tree if applicable by calling sh_subtracktree().
- rehash(): Call nv_rehash() instead of _nv_unset()ting the hash
table entry; this is needed to work correctly in subshells.
src/cmd/ksh93/tests/leaks.sh:
- Add leak tests for various PATH-related operations in the main
shell and in a virtual subshell.
- Several pre-existing memory leaks are exposed by the new tests
(I've confirmed these in 93u+). The tests are disabled and marked
TODO for now, as these bugs have not yet been fixed.
src/cmd/ksh93/tests/subshell.sh:
- Update.
Resolves: https://github.com/ksh93/ksh/issues/66
2021-01-07 21:18:33 +00:00
|
|
|
- Any changes to the hash table (a.k.a. "tracked aliases", i.e. cached $PATH
|
|
|
|
|
searches) in a subshell now no longer affect the parent shell's hash table.
|
|
|
|
|
|
2021-01-05 04:55:07 +00:00
|
|
|
2021-01-05:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug in 'cd' that caused 'cd ./foo' to search for 'foo' in $CDPATH.
|
|
|
|
|
|
2021-01-03 23:54:36 +00:00
|
|
|
2021-01-03:
|
|
|
|
|
|
|
|
|
|
- The invocation
|
|
|
|
|
$ ksh +s
|
|
|
|
|
caused an infinite loop and corrupted ~/.sh_history. This is now fixed so
|
|
|
|
|
that the '-s' option is automatically turned on if there are no non-option
|
|
|
|
|
command arguments, as documented in Bolsky & Korn (1995), p. 261.
|
|
|
|
|
|
2020-11-26 13:50:30 +00:00
|
|
|
2020-10-22:
|
|
|
|
|
|
|
|
|
|
- Fixed: 'typeset -F0', 'typeset -E0', and 'typeset -X0' floating point
|
|
|
|
|
numerics having a precision of 0 with variable assignment.
|
|
|
|
|
'typeset -F0 x; x=4.56' worked but not 'typeset -F0 x=4.56'.
|
|
|
|
|
|
2020-11-26 13:30:24 +00:00
|
|
|
2020-10-21:
|
|
|
|
|
|
|
|
|
|
- Fixed: More concisely correct the exporting of uppercase and lowercase
|
|
|
|
|
variables when only the export and change case attributes were applied.
|
|
|
|
|
This fix improves upon the previous 2020-09-30 modifications.
|
|
|
|
|
|
Mitigate PWD race condition in non-forking subshells
Virtual/non-forking subshells that change the present working
directory (PWD) with 'cd' suffer from a serious race condition. The
PWD is changed within the same process. This means it may not be
possible to change back to the original PWD when exiting the
subshell, as some other process may destroy the PWD or modify its
permissions in the meantime. ksh did not handle this error
condition at all, so, after exiting a subshell that invoked 'cd',
it could silently end up running the script's following command(s)
in the wrong directory. Which might be 'rm -rf *'. So, ouch.
The proper and obvious fix is never to allow a virtual subshell to
change the PWD, as it can never be guaranteed you can return to a
previous directory. If the PWD is changed in a child process, there
is no need to restore it in the parent process, and this whole
problem is avoided. So subshells really should always fork on
encountering a 'cd' command.
But forking is slow. It is not uncommon for scripts to 'cd' in a
subshell that is run repeatedly in a loop.
There is also the issue of custom builtins that can be added to ksh
via shared libraries. In the standard shell language, 'cd' is the
only command that changes the PWD, so we could just make that
command fork the subshell it is run from. But there's no telling
what a custom builtin might do.
So this commit implements a compromise that will not affect
performance unless there is the pathological condition of a PWD
that has been rendered inaccessible in some way:
1. When entering a virtual subshell, if the parent shell's PWD
proves inaccessible upon saving it, the subshell will now fork into
a separate process, avoiding the unrestorable PWD problem.
2. If some attack renders the parent shell's PWD unrestorable
*after* ksh enters a virtual subshell, ksh will now error out when
exiting it. There is nothing else left to do then. Continuing would
mean running arbitrary commands in the wrong PWD.
src/cmd/ksh93/sh/subshell.c:
- Put all the code/variables only needed for fchdir() behind '#if
_lib_fchdir'. This makes it clearer what's what.
(I don't know if there is still any system out there without
fchdir(3); I haven't found any. The chdir(3) fallback version may
be removed later as there is no way to make it remotely secure.)
- Fix the attempt to use the O_PATH mode for open(2) as a fallback
for nonexistent O_SEARCH on Linux. Define _GNU_SOURCE on Linux,
or <fcntl.h> (which is included indirectly) won't define O_PATH.
- Fix use of O_SEARCH. The code was simply wrong, repeating an
open(".",O_RDONLY) instead. Since a nonexistent O_SEARCH is now
redefined as either O_PATH or O_RDONLY, we can simply
open(".",O_SEARCH) and be done with it.
- Fix fatal error handling. Introduce fatal error condition for
failure to fchdir(3) back to the parent's PWD; rename 'duped' to
'fatalerror' and use it for error numbers; save and restore errno
on fatal error so the message will report the cause. (We must
call errormsg() near the end of sh_subshell() to avoid crashes.)
- If open(".",O_SEARCH) was not able get a file descriptor to our
PWD on entry, then call sh_subfork() immediately before running
the subshell commands. (Forking earlier causes a crash.)
- When restoring the PWD, if fchdir(3) fails, do *not* fall back to
chdir(3). We already know the PWD is inaccessible, so if chdir(3)
"succeeds" then, it's very likely to be a substitute injected by
an attacker.
src/cmd/ksh93/bltins/cd_pwd.c:
- If we don't have fchdir(3), then sh_subshell() must fall back to
chdir(2) to restore the PWD. That is highly vulnerable, as a
well-timed rename would allow an attacker to usurp the PWD. We
can't do anything about that if some custom builtin changes the
PWD, but we can at least make 'cd' always fork a subshell, which
slows down ksh but removes the need for the parent shell ever to
restore the PWD. (There is certainly no popular system where this
is relevant and there might not be any such current system.)
This commit adds no regression test because a portable regression
test is not really doable. Different kernels, external /bin/pwd
utilities, etc. all have quite different behaviour under the
pathological condition of an inaccessible PWD, so both the
before-fix and the after-fix behaviour differs. See link below.
Resolves: https://github.com/ksh93/ksh/issues/141
Thanks to Stéphane Chazelas for the bug report.
2020-10-06 21:13:41 +00:00
|
|
|
2020-10-06:
|
|
|
|
|
|
|
|
|
|
- The security of virtual/non-forking subshells that locally change the present
|
|
|
|
|
working directory (PWD) using 'cd' has been improved in two ways.
|
|
|
|
|
1. On entering a subshell, if the parent shell's PWD proves inaccessible upon
|
|
|
|
|
saving it, the subshell will now fork into a separate process so the
|
|
|
|
|
parent process never changes its PWD, avoiding the need to restore it.
|
|
|
|
|
2. If some attack renders the parent shell's PWD unrestorable *after* ksh
|
|
|
|
|
enters a virtual subshell, ksh will now error out on exiting it, as
|
|
|
|
|
continuing would mean running arbitrary commands in the wrong PWD.
|
|
|
|
|
Hopefully this is an acceptable compromise between performance and security.
|
|
|
|
|
The proper fix would be to always fork a subshell when changing the working
|
|
|
|
|
directory within it, but the resulting slowdown would likely be unpopular.
|
|
|
|
|
|
2020-09-29 22:34:02 +00:00
|
|
|
2020-09-30:
|
|
|
|
|
|
|
|
|
|
- Fixed: 'typeset -xu' and 'typeset -xl' (export + change case) failed to
|
|
|
|
|
change the case of a variable's value in certain conditions.
|
|
|
|
|
|
Fix pipefail with (errexit or ERR trap) regression
ksh 93u+ introduced a regression in the combination of the
'set -o pipefail' and 'set -e'/'set -o errexit' options:
$ ksh93 -o errexit -o pipefail -c \
'(exit 3) | true; echo "still here despite $? status"'
still here despite 3 status
The bug is in how the the huge sh_exec() function in xec.c handles
the 'echeck' flag. Near the end of sh_exec(), this flag triggers a
sh_chktrap() call to check whether to trigger any traps, including
the ERR trap -- and that same function also handles the errexit
option, which is basically the same as 'trap "exit" ERR'.
We can learn more easily how sh_exec() works by inserting debug
warnings in all its 'switch(type&COMMSK)' cases, like:
case TCOM:
errormsg(SH_DICT,ERROR_warn(0),"[DEBUG] TCOM");
... and same for all the others. With that done, the output
of a very simple dummy pipeline looks as follows:
$ arch/*/bin/ksh -c 'true | true | true'
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TFIL
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TFORK
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TFORK
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TSETIO
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TCOM
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TCOM
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TCOM
So, it looks like sh_exec() handles this pipeline as follows:
TFIL
|_____TFORK
| |_____TCOM
|_____TFORK
| |_____TCOM
|_____TSETIO
|_____TCOM
Each time a pipeline like command1 | command2 | ... is executed,
sh_exec() is invoked with type TFIL; this then recursively invokes
sh_exec() to handle the individual elements. The last element of
the pipe triggers a sh_exec() run with type TSETIO; since it is run
in the current shell environment, it is effectively treated as a
command with an input redirection. All the previous elements are of
type TFORK instead, because they are executed asynchronously in
separate, forked subshell processes. Finally, the TFORK or TSETIO
code then recursively calls sh_exec() again with type TCOM to
actually execute the commands.
When reading the code, we find that the 'echeck' flag is set as
part of the TSETIO code. This makes sense of why only an error in
the last element of the pipe triggers the errexit/ERR trap action.
So that's the bug: the flag is set in the wrong place.
This can be fixed by setting that flag in the TFIL handling code
instead, as this is what calls everything else and collects all the
exit statuses. So the sh_chktrap() call is now executed after
handling the entire pipeline, at the TFIL recursion level.
This also allows getting rid of the special-casing in the buggy
TSETIO version. The SH_ERREXIT state is restored at the end of each
sh_exec() call, so since we're now doing this at a lower recursion
level, it will already have been restored.
src/cmd/ksh93/sh/xec.c: sh_exec():
- Fix the bug as per the above.
src/cmd/ksh93/tests/options.sh:
- Add tests for errexit and ERR trap combined with pipefail.
src/cmd/ksh93/tests/basic.sh:
- Tweak a couple of tests that reported a trap wasn't triggered
even if it was actually triggered more than once.
Fixes: https://github.com/ksh93/ksh/issues/121
Thanks to Stéphane Chazelas for the bug report.
2020-09-30 14:13:34 +00:00
|
|
|
- A ksh 93u+ regression was fixed in the combination of ERR trap handling and
|
|
|
|
|
the 'pipefail' option. A pipeline now triggers the ERR trap correctly again
|
|
|
|
|
if the 'pipefail' option is active and any of the pipeline elements return a
|
|
|
|
|
nonzero exit status. Similarly, if both the 'errexit' and 'pipefail' options
|
|
|
|
|
are active, ksh now correctly exits if any pipeline element returns nonzero.
|
|
|
|
|
|
2020-10-01 04:13:00 +00:00
|
|
|
- Autoloading a function no longer causes the calling script's $LINENO to be
|
|
|
|
|
off by the number of lines in the function definition file that was loaded.
|
|
|
|
|
This also corrects line numbers in warnings and error messages.
|
|
|
|
|
|
Fix signal/trap behaviour in ksh functions (rhbz#1454804)
Prior discussion:
https://bugzilla.redhat.com/1454804
On 2017-05-23 13:33:25 UTC, Paulo Andrade wrote:
> In previous ksh versions, when exiting the scope of a ksh
> (not posix) function, it would restore the trap table of
> the "calling context" and if the reason the function exited
> was a signal, it would call sh_fault() passing as argument
> the signal value.
> Newer ksh checks it, but calls kill(getpid(), signal_number)
> after restoring the trap table, but only calls for SIGINT and
> SIGQUIT.
[...]
> The old way appears to have been more appropriate, but there
> must be a reason to only pass SIGINT and SIGQUIT as it is an
> explicit patch.
The last paragraph is where I differ. This would not be the first
example of outright breakage that appeared to be added deliberately
and that 93u+m has fixed or removed, see e.g. 8477d2ce ('printf %H'
had code that deleted all multibyte characters), cefe087d, or
781f0a39. Sometimes it seems the developers added a little
experiment and then forgot all about it, so it became a misfeature.
In this instance, the correct pre-2012 ksh behaviour is still
explicitly documented in (k)sh.1: "A trap condition that is not
caught or ignored by the function causes the function to terminate
and the condition to be passed on to the caller". Meaning, if there
is no function-local trap, the signal defaults to the parent scope.
There is no language that limits this to SIGINT and SIGQUIT only.
It also makes no sense at all to do so -- signals such as SIGPIPE,
SIGTERM, or SIGSEGV need to be caught by default and to do
otherwise results in misbehaviour by default.
src/cmd/ksh93/sh/xec.c: sh_funscope():
- When resending a signal after restoring the global traps state,
remove the spurious check that limits this to SIGINT and SIGQUIT.
- Replace it with a check for nsig!=0, as that means there were
parent trap states to restore. Otherwise 'kill' may be called
with an invalid signal argument, causing a crash on macOS.
src/cmd/ksh93/tests/signal.sh:
- Update a test to check that a function-local SIGTERM trap is
triggered correctly when signalled from another process.
- Complete the tests for 3aee10d7; this bug needed fixing before
we could test that previous fix in a ksh function scope.
- Add a test for triggering global traps from ksh functions,
testing multiple POSIX-standard signals.
2020-09-29 00:28:08 +00:00
|
|
|
2020-09-28:
|
|
|
|
|
|
|
|
|
|
- While executing a ksh-style function, ksh 93u+ ignored all signals for which
|
|
|
|
|
the function had not set a local trap, except for SIGINT and SIGQUIT. This
|
|
|
|
|
was contrary to the manual, which states that a "trap condition that is not
|
|
|
|
|
caught or ignored by the function causes the function to terminate and the
|
|
|
|
|
condition to be passed on to the caller". This has now been fixed in 93u+m to
|
|
|
|
|
match the documentation, so that e.g. global traps work as expected again.
|
|
|
|
|
|
2020-09-27 19:26:09 +00:00
|
|
|
2020-09-27:
|
|
|
|
|
|
2020-09-28 02:47:53 +00:00
|
|
|
- The shell's lexical analysis of a 'case' statement within a do...done block
|
2020-09-27 19:26:09 +00:00
|
|
|
within a command substitution of the form $(...) has been fixed so that code
|
|
|
|
|
like the following no longer throws a spurious syntax error:
|
|
|
|
|
x=$(for i in 1; do case $i in word) true;; esac; done)
|
|
|
|
|
Previously, this required a leading parenthesis before 'word', although the
|
|
|
|
|
syntax error claimed that the ';;' was unexpected.
|
|
|
|
|
|
2020-09-26 18:26:18 +00:00
|
|
|
2020-09-26:
|
|
|
|
|
|
|
|
|
|
- 'whence -f' now completely ignores the existence of functions, as documented.
|
|
|
|
|
|
2020-09-26 18:28:55 +00:00
|
|
|
- ksh now does not import environment variables whose names are not valid in
|
|
|
|
|
the shell language, as it would be impossible to change or unset them.
|
|
|
|
|
However, they stay in the environment to be passed to child processes.
|
|
|
|
|
|
Fix argv rewrite on invoking hashbangless script (rhbz#1047506)
The fixargs() function is invoked when ksh needs to run a script
without a #!/hashbang/path. Instead of letting the kernel invoke a
shell, ksh exfile()s the script itself from sh_main(). In the
forked child, it calls fixargs() to set the argument list in the
environment to the args of the new script, so that 'ps' and
/proc/PID/cmdline show the expected output.
But fixargs() is broken because, on systems other than HP-UX (on
which ksh uses pstat(2)), ksh simply inserts a terminating zero.
The arguments list is not a zero-terminated C string. Unix systems
expect the entire arguments buffer to be zeroed out, otherwise 'ps'
and /proc/*/cmdline will have fragments of previous command lines
in the output.
The Red Hat patch for this bug is:
https://src.fedoraproject.org/rpms/ksh/blob/642af4d6/f/ksh-20120801-argvfix.patch
However, that fix is incomplete because 'command_len' was also
hardcoded to be limited to 64 characters (!), which still gave
invalid 'ps' output if the erased command line was longer.
src/cmd/ksh93/sh/main.c: fixargs():
- Remove CMD_LENGTH macro which was defined as 64.
- Remove code that limited the erasure of the arguments buffer to
CMD_LENGTH characters. That code also had quite a dodgy strdup()
call -- it copies arguments to the heap, but they are never freed
(or even used), so it's a memory leak. Also, none of this is
ever done if the length is calculated using pstat(2) on HP-UX,
which is a clear indication that it's unnecessary.
(I think this code block must have been some experiment they
forgot to remove. One reason why I think so is that a 64 byte
arguments limit never made sense, even in the 1980s when they
wrote ksh on 80-column CRT displays. Another indication of this
is that fixing it didn't require adding anything; the code to do
the right thing was already there, it was just being overridden.)
- Zero out the full arguments length as in the Red Hat patch.
src/cmd/ksh93/tests/basic.sh:
- Add test. It's sort of involved because 'ps' is one of the least
portable commands in practice, in spite of standardisation.
2020-09-25 06:19:43 +00:00
|
|
|
2020-09-25:
|
|
|
|
|
|
whence -v/-a: report path to autoloadable functions
Since at least 1999, whence -v on pdksh (and its successor mksh)
reports the path where an autoloadable function may be found:
$ mkdir ~/fun; FPATH=~/fun
$ echo 'myfn() { echo hi; }' >~/fun/myfn
$ whence -v myfn
myfn is a undefined (autoload from /home/user/fun/myfn) function
Whereas ksh93 only reports, rather uselessly:
myfn is an undefined function
As of this commit, whence -v/-a on ksh 93u+m does the same as
pdksh, but with correct grammar:
myfn is an undefined function (autoload from /home/user/fun/myfn)
This may be a small violation of my own "no new features" policy
for 93u+m, but I couldn't resist. This omission has been annoying
me, and it's just embarrassing to lack a pdksh feature :)
src/cmd/ksh93/include/path.h,
src/cmd/ksh93/data/msg.c:
- Add e_autoloadfrom[] = " (autoload from %s)" message.
src/cmd/ksh93/bltins/whence.c: whence():
- Report the path (if any) when reporting an undefined function.
This needs to be done in two places:
1. When a function has been explicitly marked undefined with
'autoload', we need to do a quick path_search() loop to find
the path. (These undefined functions take precedence over
regular commands, so are reported first.)
2. When a function is not explicitly autoloaded but merely
available in $FPATH, that path search was already done, so all
we need to do is report it. (These are reported last.)
Note that the output remains as on 93u+ if no function definition
file is found on $FPATH. This is also like pdksh/mksh.
src/cmd/ksh93/data/builtins.c:
- Bump 'whence' version date. The inline docs never detailed very
exactly what 'whence -v' reports, so no need for further edits.
src/cmd/ksh93/tests/path.sh:
- Regress-test the new whence behaviour plus actual autoloading,
including the command override behaviour of autoloaded functions.
2020-09-25 13:39:08 +00:00
|
|
|
- whence -v/-a now reports the path to the file that an "undefined" (i.e.
|
|
|
|
|
autoloadable) function will be loaded from when invoked, if found in $FPATH.
|
|
|
|
|
|
Fix argv rewrite on invoking hashbangless script (rhbz#1047506)
The fixargs() function is invoked when ksh needs to run a script
without a #!/hashbang/path. Instead of letting the kernel invoke a
shell, ksh exfile()s the script itself from sh_main(). In the
forked child, it calls fixargs() to set the argument list in the
environment to the args of the new script, so that 'ps' and
/proc/PID/cmdline show the expected output.
But fixargs() is broken because, on systems other than HP-UX (on
which ksh uses pstat(2)), ksh simply inserts a terminating zero.
The arguments list is not a zero-terminated C string. Unix systems
expect the entire arguments buffer to be zeroed out, otherwise 'ps'
and /proc/*/cmdline will have fragments of previous command lines
in the output.
The Red Hat patch for this bug is:
https://src.fedoraproject.org/rpms/ksh/blob/642af4d6/f/ksh-20120801-argvfix.patch
However, that fix is incomplete because 'command_len' was also
hardcoded to be limited to 64 characters (!), which still gave
invalid 'ps' output if the erased command line was longer.
src/cmd/ksh93/sh/main.c: fixargs():
- Remove CMD_LENGTH macro which was defined as 64.
- Remove code that limited the erasure of the arguments buffer to
CMD_LENGTH characters. That code also had quite a dodgy strdup()
call -- it copies arguments to the heap, but they are never freed
(or even used), so it's a memory leak. Also, none of this is
ever done if the length is calculated using pstat(2) on HP-UX,
which is a clear indication that it's unnecessary.
(I think this code block must have been some experiment they
forgot to remove. One reason why I think so is that a 64 byte
arguments limit never made sense, even in the 1980s when they
wrote ksh on 80-column CRT displays. Another indication of this
is that fixing it didn't require adding anything; the code to do
the right thing was already there, it was just being overridden.)
- Zero out the full arguments length as in the Red Hat patch.
src/cmd/ksh93/tests/basic.sh:
- Add test. It's sort of involved because 'ps' is one of the least
portable commands in practice, in spite of standardisation.
2020-09-25 06:19:43 +00:00
|
|
|
- When ksh invoked a shell script that does not have a leading
|
|
|
|
|
#!/hashbang/path, 'ps' and /proc/<PID>/cmdline showed corrupted output if
|
|
|
|
|
the new script's command line was shorter than that of the invoking script.
|
|
|
|
|
This has been fixed by wiping the arguments buffer correctly.
|
|
|
|
|
|
2020-09-24 05:36:44 +00:00
|
|
|
2020-09-24:
|
|
|
|
|
|
|
|
|
|
- An omission made it impossible to turn off brace expansion within command
|
|
|
|
|
substitutions (`...`, $(...) or ${ ...; }) as the code for parsing these
|
|
|
|
|
did not check the -B/braceexpand option. This check has now been added.
|
|
|
|
|
|
2020-09-22 23:56:09 +00:00
|
|
|
2020-09-23:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash that could occur when running a pipeline containing
|
|
|
|
|
backtick-style command substitutions with job control enabled.
|
|
|
|
|
|
Fix typeset -l/-u crash on special vars (rhbz#1083713)
When using typeset -l or -u on a variable that cannot be changed
when the shell is in restricted mode, ksh crashed.
This fixed is inspired by this Red Hat fix, which is incomplete:
https://src.fedoraproject.org/rpms/ksh/blob/642af4d6/f/ksh-20120801-tpstl.patch
The crash was caused by the nv_shell() function. It walks though a
discipline function tree to get the pointer to the interpreter
associated with it. Evidently, the problem is that some pointer in
that walk is not set correctly for all special variables.
Thing is, ksh only has one shell language interpreter, and only one
global data structure (called 'sh') to keep its main state[*]. Yet,
the code is full of 'shp' pointers to that structure. Most (not
all) functions pass that pointer around to each other, accessing
that struct indirectly, ostensibly to account for the non-existent
possibility that there might be more than one interpreter state.
The "why" of that is an interesting cause for speculation that I
may get to sometime. For now, it is enough to know that, in the
code as it is, it matters not one iota what pointer to the shell
interpreter state is used; they all point to the same thing (unless
it's broken, as in this bug).
So, rather than fixing nv_shell() and/or associated pointer
assignments, this commit simply removes it, and replaces it with
calls to sh_getinterp(), which always returns a pointer to sh (see
init.c, where that function is defined as literally 'return &sh').
[*] Defined in shell.h, with the _SH_PRIVATE part in defs.h
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/name.c:
- Remove nv_shell().
src/cmd/ksh93/sh/init.c:
- In all the discipline functions for special variables, initialise
shp using sh_getinterp() instead of nv_shell().
src/cmd/ksh93/tests/variables.sh:
- Add regression test for typeset -l/-u on all special variables.
2020-09-23 17:15:45 +00:00
|
|
|
- Fixed a crash that occurred when using 'typeset -u' or 'typeset -l' on a
|
|
|
|
|
special variable such as PATH, ENV or SHELL.
|
|
|
|
|
|
2020-09-21 23:42:15 +00:00
|
|
|
2020-09-21:
|
|
|
|
|
|
|
|
|
|
- A bug was fixed that caused command substitutions embedded in here-documents
|
|
|
|
|
to lose the output of the commands they ran. This bug occurred when ksh was
|
|
|
|
|
compiled with the SHOPT_SPAWN compile-time option.
|
|
|
|
|
|
2020-09-22 00:45:59 +00:00
|
|
|
- Bugfix: var=$(< file) now reads the file even if the standard inout, standard
|
|
|
|
|
output and/or standard error file descriptors are closed.
|
|
|
|
|
|
Multiple 'whence' and path search fixes
Hopefully this doesn't introduce new bugs, but it does fix at
least the following:
1. When whence -v/-a found an "undefined" (i.e. autoloadable)
function in $FPATH, it actually loaded the function as a side
effect of reporting on its existence (!). Now it only reports.
2. 'whence' will now canonicalise paths properly. Examples:
$ whence ///usr/lib/../bin//./env
/usr/bin/env
$ (cd /; whence -v dev/../usr/bin//./env)
dev/../usr/bin//./env is /usr/bin/env
3. 'whence' no longer prefixes a spurious double slash when doing
something like 'cd / && whence bin/echo'. On Cygwin, an initial
double slash denotes a network server, so this was not just a
cosmetic problem.
4. 'whence -a' now reports a "tracked alias" (a.k.a. hash table
entry, i.e. cached $PATH search) even if an actual alias by the
same name exists. This needed fixing because in fact the hash
table entry continues to be used when bypassing the alias.
Aliases and "tracked aliases" are not remotely the same thing;
confusing nomenclature is not a reason to report wrong results.
5. When using 'hash' or 'alias -t' on a command that is also a
builtin to force caching a $PATH search for the external
command, 'whence -a' double-reported the path:
$ hash printf; whence -a printf
printf is a shell builtin
printf is /usr/bin/printf
printf is a tracked alias for /usr/bin/printf
This is now fixed so that the second output line is gone.
Plus, if there were multiple versions of the command on $PATH,
the tracked alias was reported at the end, which is the wrong
order. This is also fixed.
src/cmd/ksh93/bltins/whence.c: whence():
- Refactor the do...while loop that handles whence -v/-a for path
searches in such a way that the code actually makes sense and
stops looking like higher esotericism. Just doing this fixed #2,
#4 and #5 above (the latter two before I even noticed them). For
instance, the path_fullname() call to canonicalise paths was
already there; it was just never used.
- Remove broken 'notrack' flaggery for deciding whether to report a
hash table entry a.k.a. "tracked alias"; instead, check the hash
table (shp->track_tree).
src/cmd/ksh93/sh/path.c:
- path_search(): Re #3: When prefixing the PWD, first check if
we're in '/' and if so, don't prefix it; otherwise, adding the
next slash causes an initial double slash. (Since '/' is the only
valid single-character absolute path, all we need to do is check
if the second character pwd[1] is non-null.)
- path_search(): Re #1: Stop autoloading when called by 'whence':
* The 'flag==2' check to avoid autoloading a function was
broken. The flag value is 2 on the first whence() loop
iteration, but 3 on subsequent ones. Change to 'flag >= 2'.
* However, this only fixes it if the function file does not have
the x permission bit, as executable files are handled by
path_absolute() which unconditionally autoloads functions!
So, pass on our flag parameter when callling path_absolute().
- path_absolute(): Re #1: Add flag parameter. Do not autoload
functions if flag >= 2.
src/cmd/ksh93/include/path.h,
src/cmd/ksh93/bltins/typeset.c,
src/cmd/ksh93/sh/main.c,
src/cmd/ksh93/sh/xec.c:
- Re #1: Update path_absolute() calls, adding a 0 flag parameter.
src/cmd/ksh93/include/name.h:
- Remove now-unused pathcomp member from union Value. It was
introduced in 99065353 to allow examining the value of a tracked
alias. This commit uses nv_getval() instead.
src/cmd/ksh93/tests/builtins.sh,
src/cmd/ksh93/tests/path.sh:
- Add and tweak various related tests.
Fixes: https://github.com/ksh93/ksh/issues/84
2020-09-20 05:56:09 +00:00
|
|
|
2020-09-20:
|
|
|
|
|
|
|
|
|
|
- Bugfix: when whence -v/-a found an "undefined" (i.e. autoloadable) function
|
|
|
|
|
in $FPATH, it actually loaded the function as a side effect of reporting on
|
|
|
|
|
its existence. Now it only reports, as documented.
|
|
|
|
|
|
|
|
|
|
- 'whence' will now canonicalise paths properly, resolving '.' and '..'
|
|
|
|
|
elements in paths given to it. It also no longer prefixes a spurious
|
|
|
|
|
double slash when doing something like 'cd / && whence bin/echo'.
|
|
|
|
|
|
2020-09-18 18:32:34 +00:00
|
|
|
2020-09-18:
|
|
|
|
|
|
|
|
|
|
- Setting the 'posix' option now turns off the 'braceexpand' option, as brace
|
|
|
|
|
expansion is not specified by POSIX and potentially incompatible with sh
|
|
|
|
|
scripts. In addition, 'set -o posix' now turns on the 'letoctal' option
|
|
|
|
|
instead of controlling that behaviour directly. 'set +o posix' does the
|
|
|
|
|
reverse of these.
|
|
|
|
|
|
emacs, vi: Support repeat parameters to VT220 keys (re: f2a3f4e3)
In the vi and emacs line editors, repeat count parameters can now
also be used for the arrow keys and the forward-delete key. E.g.,
in emacs mode, <ESC> 7 <left-arrow> will now move the cursor seven
positions to the left. In vi control mode, this would be entered
as: 7 <left-arrow>.
src/cmd/ksh93/edit/emacs.c:
- ed_emacsread(): Upon getting ^[ (ESC), save current repeat count
in a new variable; restore and reset it upon the next character.
- escape(): Minor bugfix: when processing a ^[[x sequence where 'x'
is a character other than '~' (which would be DEL), also reinsert
the final character into the buffer so scripts can detect them.
src/cmd/ksh93/edit/vi.c:
- cntlmode(): Do not reset the repeat count if the command is '[',
the character following ESC in VT220 escape sequences.
- mvcursor():
* Do not use getcount() to get the character following '[', as
that was parsing repetition parameters in the wrong place.
There wouldn't be any, so this would reset the repeat count.
* After that, no more need for the special-casing of ^[[3~ (DEL)
introduced in f2a3f4e3. Move it to within the 'switch' block.
* When handling left and right arrows and Home and End keys, do
not modify cursor directly but ed_ungetchar() the corresponding
traditional command keys as with the rest. Otherwise a repeat
count parameter would now wrongly survive those keys.
src/cmd/ksh93/sh.1:
- Document control character notation used for vi mode docs.
- Since vi control mode beeps and aborts on ESC except if a
subsequent [ is already in the input buffer upon receiving ESC,
document that VT220 escape sequences only preserve repeat counts
when entered into the input buffer all at once.
- Don't skip the initial ESC in the documentation of the VT220
escape sequences. In control mode, skipping the initial ESC still
works as before, but that is now undocumented, as it's really
nothing more than an artefact of VT220 escape processing.
- Move the two long paragraphs on '-o viraw' and canonical (i.e.
line-based) input processing from the vi editor introduction to
the options section under 'viraw'. It is much too arcane for the
intro, and besides, ksh 93u+ (and hence also 93u+m) has
SHOPT_VIRAW enabled by default, so the shell is compiled to force
this option on at all times, making it even less relevant for
most users.
2020-09-17 15:28:03 +00:00
|
|
|
2020-09-17:
|
|
|
|
|
|
|
|
|
|
- In the vi and emacs line editors, repeat count parameters can now also be
|
|
|
|
|
used for the arrow keys and the forward-delete key. E.g., in emacs mode,
|
|
|
|
|
<ESC> 7 <left-arrow> will now move the cursor seven positions to the left.
|
|
|
|
|
In vi control mode, this would be entered as: 7 <left-arrow>.
|
|
|
|
|
|
2020-09-18 01:03:39 +00:00
|
|
|
- When a background job on an interactive shell received SIGINT or SIGPIPE, the
|
|
|
|
|
job termination message was empty. It now shows "Interrupt" or "Broken Pipe".
|
|
|
|
|
|
A few job control (-m, -o monitor) fixes (rhbz#960034)
This patch from Red Hat fixes the following:
1. ksh was ignoring the -m (-o monitor) option when specified on
the invocation command line.
2. Scripts did not properly terminate their background processes
on Ctrl+C if the -m option was turned off. Reproducer:
xterm &
read junk
When run as a script without turning on -m, pressing Ctrl+C
should terminate the xterm, and now does.
3. Scripts no longer attempt to set the terminal foreground process
group ID, as only interactive shells should be doing that.
This makes some progress on https://github.com/ksh93/ksh/issues/119
but we're a long way from fixing all of that.
src/cmd/ksh93/sh/main.c: exfile():
- On non-interactive shells, do not turn off the monitor option.
Instead, if it was turned on, turn on the SH_MONITOR state flag.
src/cmd/ksh93/edit/edit.c: ed_getchar():
- On Ctrl+C, issue SIGINT to the current process group using
killpg(2) instead of going via sh_fault(), which handles a
signal only for the current shell process.
src/cmd/ksh93/sh/jobs.c: job_reap(), job_reset(),
src/cmd/ksh93/sh/xec.c: sh_exec():
- Only attempt to set the terminal foreground process group ID
using tcsetpgrp(3) if the shell is interactive.
Original patch: https://src.fedoraproject.org/rpms/ksh/blob/642af4d6/f/ksh-20120801-kshmfix.patch
This was applied to Red Hat's ksh 93u+ on 8 July 2013.
2020-09-18 02:42:27 +00:00
|
|
|
- The -m (-o monitor) option is no longer ignored when specified on the shell
|
|
|
|
|
invocation command line.
|
|
|
|
|
|
|
|
|
|
- A script that is interrupted with Ctrl+C now terminates its background jobs
|
|
|
|
|
as expected, unless the -m (-o monitor) option was turned on.
|
|
|
|
|
|
2020-09-14 10:15:05 +00:00
|
|
|
2020-09-14:
|
|
|
|
|
|
|
|
|
|
- Corrected rounding of floating point values by ksh's printf %f formatting
|
|
|
|
|
operator. Fix contributed by @hyenias.
|
|
|
|
|
|
Handle forward-delete key in emacs and vi editors
On every modern system, the forward-delete key on PC/Mac keyboards
generates the VT220 sequence ESC [ 3 ~. Every other shell with an
editor handles this now, ksh93 seems to be the last not to.
src/cmd/ksh93/edit/emacs.c: escape():
- Handle the ^[[3 as part of normal escape processing, then read an
extra character to check for the final '~'. If detected, insert
an ERASECHAR key event.
src/cmd/ksh93/edit/vi.c: mvcursor():
- Replace the ^[[3~ sequence by an 'x' command. We have to
special-case its processing, because vi mode parses numbers as
repetition operators. The escape sequence contains a number,
making it incompatible with normal command handling. This means
number repetitions don't work with the forward-delete key. If
that annoys anyone enough to fix it, a patch would be welcome.
For now, it will do to make the forward-delete key stop
exhibiting bizarre behaviour (beep + change case + move forward).
src/cmd/ksh93/sh.1
- Copy-edit emacs documentation for VT220-style sequences; map them
to their actual key, otherwise it's meaningless to the reader.
- Document the new forward-delete key behaviour for emacs mode.
- Leave the forward-delete key for vi mode undocumented for now, as
repetitions don't work, so it doesn't really match the vi canon.
(OTOH, it doesn't work in vim, either...)
2020-09-14 22:49:05 +00:00
|
|
|
- The forward-delete key now works as expected in emacs and vi editing modes.
|
|
|
|
|
|
Fix 'command' expansion bug and POSIX compliance
The 'command' name can now result from an expansion, e.g.:
c=command; "$c" ls
set -- command ls; "$@"
both work now. This fixes BUG_CMDEXPAN.
If -o posix is on, 'command' now disables not only the "special"
but also the "declaration" properties of builtin commands that it
invokes. This is because POSIX specifies 'command' as a simple
regular builtin, and any command name following 'command' is just
an argument to the 'command' command, so there is nothing that
allows any further arguments (such as assignment-arguments) to be
treated specially by the parser. So, if and only if -o posix is on:
a. Arguments that start with a variable name followed by '=' are
always treated as regular words subject to normal shell syntax.
b. Since assignment-arguments are not processed as assignments
before the command itself, 'command' can now stop the shell from
exiting (as required by the standard) if a command that it
invokes (such as 'export') tries to modify a readonly variable.
This fixes BUG_CMDSPEXIT.
Most of 'command' is integrated in the parser and parse tree
executer, so that is where it needed fixing.
src/cmd/ksh93/sh/parse.c: simple():
- If the posix option is on, do not skip past SYSCOMMAND so that
any declaration builtin commands that are arguments to 'command'
are not detected and thus not treated specially at parsetime.
src/cmd/ksh93/sh/xec.c: sh_exec():
- When detecting SYSCOMMAND in order to skip past it, not only
compare the Namval_t pointer 'np' to SYSCOMMAND, but also handle
the case where that pointer is NULL, as when the command name
results from an expansion. In that case, search the function tree
shp->fun_tree for the name and see if that yields the SYSCOMMAND
pointer. fun_tree is initialised with a dtview to bltin_tree, so
searching fun_tree instead allows for overriding 'command' with a
shell function (which the POSIX standard requires us to allow).
src/cmd/ksh93/sh.1,
src/cmd/ksh93/data/builtins.c:
- Update documentation to match these changes.
- Various related edits and improvements.
src/cmd/ksh93/tests/builtins.sh:
- Check that 'command' works if resulting from an expansion.
- Check that 'command' can be overridden by a shell function.
2020-09-11 07:33:29 +00:00
|
|
|
2020-09-11:
|
|
|
|
|
|
|
|
|
|
- The 'command' regular builtin utility (which runs a simple command, removing
|
|
|
|
|
special properties) has been made fully POSIX compliant.
|
|
|
|
|
1. The 'command' name can now result from an expansion (fixing BUG_CMDEXPAN),
|
|
|
|
|
e.g. 'c=command; "$c" ls' and 'set -- command ls; "$@"' now work.
|
|
|
|
|
2. If and only if the POSIX mode (the new -o posix shell option) is active,
|
|
|
|
|
then the 'command' utility now disables not only "special" but also
|
|
|
|
|
"declaration" properties of builtin commands that it invokes, meaning:
|
|
|
|
|
a. arguments that start with a variable name followed by '=' are
|
|
|
|
|
always treated as regular words subject to normal shell syntax;
|
|
|
|
|
b. 'command' can now stop the shell from exiting if a command that it
|
|
|
|
|
invokes tries to modify a readonly variable (fixing BUG_CMDSPEXIT).
|
|
|
|
|
|
Reinstate 'r' and 'history' as preset aliases for interactive ksh
Following a community discussion, it became clear that 'r' is
particularly problematic as a regular builtin, as the name can and
does conflict with at least one legit external command by that
name. There was a consensus against removing it altogether and
letting users set the alias in their login scripts. However,
aliases are easier to bypass, remove or rename than builtins are.
My compromise is to reinstate 'r' as a preset alias on interactive
shells only, along with 'history', as was done in 17f81ebe before
they were converted to builtins in 03224ae3. So this reintroduces
the notion of predefined aliases to ksh 93u+m, but only for
interactive shells that are not initialised in POSIX mode.
src/cmd/ksh93/Makefile,
src/cmd/ksh93/Mamfile,
src/cmd/ksh93/include/shtable.h,
src/cmd/ksh93/data/aliases.c:
- Restore aliases.c containing shtab_aliases[], a table specifying
the preset aliases.
src/cmd/ksh93/include/shtable.h,
src/cmd/ksh93/sh/init.c:
- Rename inittree() to sh_inittree() and make it extern, because we
need to use it in main.c (sh_main()).
src/cmd/ksh93/sh/main.c: sh_main():
- Init preset aliases from shtab_aliases[] only if the shell is
interactive and not in POSIX mode.
src/cmd/ksh93/bltins/typeset.c,
src/cmd/ksh93/tests/alias.sh:
- unall(): When unsetting an alias, pass on the NV_NOFREE attribute
to nv_delete() to avoid an erroneous attempt to free a preset
alias from read-only memory. See: 5d50f825
src/cmd/ksh93/data/builtins.c:
- Remove "history" and "r" entries from shtab_builtins[].
- Revert changes to inline fc/hist docs in sh_opthist[].
src/cmd/ksh93/bltins/hist.c: b_hist():
- Remove handling for 'history' and 'r' as builtins.
src/cmd/ksh93/sh.1:
- Update accordingly.
Resolves: https://github.com/ksh93/ksh/issues/125
2020-09-11 18:51:35 +00:00
|
|
|
- The 'history' (== 'hist -l') and 'r' (== 'hist -s') interactive shell
|
|
|
|
|
history commands have reverted to preset aliases and are now only loaded if
|
|
|
|
|
the shell is interactive and not initialised in POSIX mode. This avoids
|
|
|
|
|
unneeded conflicts with external commands by these names, particularly 'r'.
|
|
|
|
|
|
Fix BUG_LOOPRET2 and related return/exit misbehaviour
The 'exit' and 'return' commands without an argument failed to pass
down the exit status of the last-run command when incorporated in a
block with redirection, &&/|| list, 'case' statement, or 'while',
'until' or 'for' loop.
src/cmd/ksh93/bltins/cflow.c:
- Use $?, which is sh.savexit a.k.a. shp->savexit, as the default
exit status value if there is no argument, instead of
shp->oldexit. This fixes the default exit status behaviour to
match POSIX and other shells.
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/include/shell.h:
- Remove now-unused sh.oldexit (a.k.a. shp->oldexit) private struct
member. It appeared to fulfill the same function as sh.savexit,
but in a slightly broken way.
- Move the savexit/$? declaration from the _SH_PRIVATE part of the
struct definition to the public API part. Since $? uses this,
it's clearly a publicly exposed value already, and this is
generally the one to use. (If anything, it's exitval that should
have been private.) This declares savexit right next to exitval,
rewriting the comments to clarify the difference between them.
src/cmd/ksh93/sh/fault.c,
src/cmd/ksh93/sh/subshell.c,
src/cmd/ksh93/sh/xec.c:
- Remove assignments to shp->oldexit.
src/cmd/ksh93/tests/basic.sh:
- Add thorough regression tests for the default exit status
behaviour of 'return' and 'exit' in various lexical contexts.
- Verify that 'for' and 'case' without any command, as well as a
lone redirection, still correctly reset the exit status to 0.
Fixes: #117
2020-09-09 18:02:20 +00:00
|
|
|
2020-09-09:
|
|
|
|
|
|
|
|
|
|
- Fixed BUG_LOOPRET2 and related bugs. The 'exit' and 'return' commands without
|
|
|
|
|
an argument now correctly default to passing down the exit status of the
|
|
|
|
|
last-run command. Tests like the following, in which the last-run command is
|
|
|
|
|
'false', now correctly output 1 instead of 0:
|
|
|
|
|
fn() { return || true; }; false; fn; echo "$?"
|
|
|
|
|
fn() { while return; do true; done; }; false; fn; echo "$?"
|
|
|
|
|
fn() { for i in 1; do return; done; }; false; fn; echo "$?"
|
|
|
|
|
fn() { case 1 in 1) return ;; esac; }; false; fn; echo "$?"
|
|
|
|
|
fn() { { return; } 2>&1; }; false; fn; echo "$?"
|
|
|
|
|
|
2020-09-05 14:20:22 +00:00
|
|
|
2020-09-05:
|
|
|
|
|
|
|
|
|
|
- Fixed erroneous syntax errors in parameter expansions such as ${var:-wor)d}
|
|
|
|
|
or ${var+w(ord}. The parentheses now correctly lose their normal grammatical
|
|
|
|
|
meaning within the braces. Fix by Eric Scrivner backported from ksh2020.
|
|
|
|
|
|
2020-09-04 03:29:52 +00:00
|
|
|
2020-09-04:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused a syntax error to be thrown if the special parameter
|
|
|
|
|
expansions ${!} and ${$} (including braces) were used within a here-document.
|
|
|
|
|
Bug reported by @Saikiran-m on GitHub.
|
|
|
|
|
|
Remove SHOPT_BASH; keep &> redir operator, '-o posix' option
On 16 June there was a call for volunteers to fix the bash
compatibility mode; it has never successfully compiled in 93u+.
Since no one showed up, it is now removed due to lack of interest.
A couple of things are kept, which are now globally enabled:
1. The &>file redirection shorthand (for >file 2>&1). As a matter
of fact, ksh93 already supported this natively, but only while
running rc/profile/login scripts, and it issued a warning. This
makse it globally available and removes the warning, bringing
ksh93 in line with mksh, bash and zsh.
2. The '-o posix' standard compliance option. It is now enabled on
startup if ksh is invoked as 'sh' or if the POSIXLY_CORRECT
variable exists in the environment. To begin with, it disables
the aforementioned &> redirection shorthand. Further compliance
tweaks will be added in subsequent commits. The differences will
be fairly minimal as ksh93 is mostly compliant already.
In all changed files, code was removed that was compiled (more
precisely, failed to compile/link) if the SHOPT_BASH preprocessor
identifier was defined. Below are other changes worth mentioning:
src/cmd/ksh93/sh/bash.c,
src/cmd/ksh93/data/bash_pre_rc.sh:
- Removed.
src/cmd/ksh93/data/lexstates.c,
src/cmd/ksh93/include/shlex.h,
src/cmd/ksh93/sh/lex.c:
- Globally enable &> redirection operator if SH_POSIX not active.
- Remove warning that was issued when &> was used in rc scripts.
src/cmd/ksh93/data/options.c,
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/args.c:
- Keep SH_POSIX option (-o posix).
- Replace SH_TYPE_BASH shell type by SH_TYPE_POSIX.
src/cmd/ksh93/sh/init.c:
- sh_type(): Return SH_TYPE_POSIX shell type if ksh was invoked
as sh (or rsh, restricted sh).
- sh_init(): Enable posix option if the SH_TYPE_POSIX shell type
was detected, or if the CONFORMANCE ast config variable was set
to "standard" (which libast sets on init if POSIXLY_CORRECT
exists in the environment).
src/cmd/ksh93/tests/options.sh,
src/cmd/ksh93/tests/io.sh:
- Replace regression tests for &> and move to io.sh. Since &> is
now for general use, no longer test in an rc script, and don't
check that a warning is issued.
Closes: #9
Progresses: #20
2020-09-01 05:19:19 +00:00
|
|
|
2020-09-01:
|
|
|
|
|
|
|
|
|
|
- The bash-style '&>file' redirection shorthand (for '>file 2>&1') is now
|
|
|
|
|
always recognised and not only when running rc/profile init scripts. It no
|
|
|
|
|
longer issues a warning. This brings ksh93 in line with mksh, bash and zsh.
|
|
|
|
|
|
|
|
|
|
- A long-form shell option '-o posix' has been added, which implements a
|
|
|
|
|
mode for better compatibility with the POSIX standard. It is automatically
|
|
|
|
|
turned on if ksh is invoked under the name 'sh'.
|
|
|
|
|
For now, it:
|
|
|
|
|
* disables the &> redirection shorthand
|
2020-09-01 06:16:51 +00:00
|
|
|
* causes the 'let' arithmetic command to recognise octal numbers by
|
|
|
|
|
leading zeros regardless of the setting of the 'letoctal' option
|
2020-09-01 07:11:27 +00:00
|
|
|
* causes file descriptors > 2 to be left open when invoking another program
|
2020-09-01 07:36:28 +00:00
|
|
|
* makes the <> redirection operator default to stdin instead of stdout
|
|
|
|
|
(this keeps the 2020-05-13 BUG_REDIRIO fix for the POSIX mode while
|
|
|
|
|
restoring traditional ksh93 behaviour for backwards compatibility)
|
-o posix: disable '[ -t ]' == '[ -t 1 ]' hack
On ksh93, 'test -t' is equivalent to 'test -t 1' (and of course
"[ -t ]" is equivalent to "[ -t 1 ]").
This is purely for compatibility with ancient Bourne shell
breakage. No other shell supports this. ksh93 should probably keep
it for backwards compatibility, but it should definitely be
disabled in POSIX mode as it is a violation of the standard; 'test
-t' is an instance of 'test "$string"', which tests if the string
is empty, so it should test if the string '-t' is empty (quod non).
This also replaces the fix for 'test -t 1' in a command
substitution with a better one that avoids forking (re: cafe33f0).
src/cmd/ksh93/sh/parse.c:
- qscan(): If the posix option is active, disable the parser-based
hack that converts a simple "[ -t ]" to "[ -t 1 ]".
src/cmd/ksh93/bltins/test.c:
- e3(): If the posix option is active, disable the part of the
compatibility hack that was used for compound expressions
that end in '-t', e.g. "[ -t 2 -o -t ]".
- test_unop(): Remove the forking fix for "[ -t 1 ]".
src/cmd/ksh93/edit/edit.c:
- tty_check(): This function is used by "[ -t 1 ]" and in other
contexts as well, so a fix here is more comprehensive. Forking
here would cause a segfault, but we don't actually need to. This
adds a fix that simply returns false if we're in a virtual
subshell that is also a command substitution. Since command
substitutions always fork upon redirecting standard output within
them (making them no longer virtual), it is safe to do this.
src/cmd/ksh93/tests/bracket.sh
- Add comprehensive regression tests for test/[/[[ -t variants in
command substitutions, in simple and compound expressions, with
and without redirecting stdout to /dev/tty within the comsub.
- Add tests verifying that -o posix disables the old hack.
- Tweak other tests, including one that globally disabled xtrace.
2020-09-01 19:24:44 +00:00
|
|
|
* disables a noncompliant 'test -t' == 'test -t 1' compatibility hack
|
-o posix: don't import/export variable attributes thru environment
When exporting variables, ksh exports their attributes (such as
'integer' or 'readonly') in a magic environment variable called
"A__z" (string defined in e_envmarker[] in data/msg.c). Child
shells recognise that variable and restore the attributes.
This little-known feature is risky; the environment cannot
necessarily be trusted and that A__z variable is easy to manipulate
before or between ksh invocations, so you can cause a script's
variables to be of the wrong type, or readonly. Backwards
compatibility requires keeping it, at least for now. But it should
be disabled in the posix mode, as it violates POSIX.
To do this, we have to solve a catch-22 in init.c. We must parse
options to know whether to turn on posix mode; it may be specified
as '-o posix' on the command line. The option parsing loop depends
on an initialised environment[*], while environment initialisation
(i.e., importing attributes) should depend on the posix option.
The catch-22 can be solved because initialising just the values
before option parsing is enough to avoid regressions. Importing the
attributes can be delayed until after option parsing. That involves
basically splitting env_init() into two parts while keeping a local
static state variable between them.
src/cmd/ksh93/sh/init.c:
- env_init():
* Split the function in two stages based on a new
'import_attributes' parameter. Import values in the first
stage; import attributes from A__z in the second (if ever).
Make the 'next' variable static as it keeps a state needed for
the attributes import stage.
* Single point of truth, greppability: don't hardcode "A__z" in
separate character comparisons, but use e_envmarker[].
* Fix an indentation error.
- sh_init(): When initialising the environment (env_init), don't
import the attributes from A__z yet; parse options first, then
import attributes only if posix option is not set.
src/cmd/ksh93/sh/name.c:
- sh_envgen(): Don't export variable attributes to A__z if the
posix option is set.
src/cmd/ksh93/tests/attributes.sh:
- Check that variable attributes aren't imported or exported
if the POSIX option is set.
src/cmd/ksh93/sh.1:
- Update.
This was the last item on the TODO list for -o posix for now.
Closes: #20
[*] If environment initialisation is delayed until after option
parsing, bin/shtests shows various regressions, including:
restricted mode breaks; the locale is not initialised properly
so that multibyte variable names break; $SHLVL breaks.
2020-09-05 08:33:50 +00:00
|
|
|
* disables passing an exported variable's attributes (such as integer or
|
|
|
|
|
readonly) to a new ksh process through the environment
|
Remove SHOPT_BASH; keep &> redir operator, '-o posix' option
On 16 June there was a call for volunteers to fix the bash
compatibility mode; it has never successfully compiled in 93u+.
Since no one showed up, it is now removed due to lack of interest.
A couple of things are kept, which are now globally enabled:
1. The &>file redirection shorthand (for >file 2>&1). As a matter
of fact, ksh93 already supported this natively, but only while
running rc/profile/login scripts, and it issued a warning. This
makse it globally available and removes the warning, bringing
ksh93 in line with mksh, bash and zsh.
2. The '-o posix' standard compliance option. It is now enabled on
startup if ksh is invoked as 'sh' or if the POSIXLY_CORRECT
variable exists in the environment. To begin with, it disables
the aforementioned &> redirection shorthand. Further compliance
tweaks will be added in subsequent commits. The differences will
be fairly minimal as ksh93 is mostly compliant already.
In all changed files, code was removed that was compiled (more
precisely, failed to compile/link) if the SHOPT_BASH preprocessor
identifier was defined. Below are other changes worth mentioning:
src/cmd/ksh93/sh/bash.c,
src/cmd/ksh93/data/bash_pre_rc.sh:
- Removed.
src/cmd/ksh93/data/lexstates.c,
src/cmd/ksh93/include/shlex.h,
src/cmd/ksh93/sh/lex.c:
- Globally enable &> redirection operator if SH_POSIX not active.
- Remove warning that was issued when &> was used in rc scripts.
src/cmd/ksh93/data/options.c,
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/args.c:
- Keep SH_POSIX option (-o posix).
- Replace SH_TYPE_BASH shell type by SH_TYPE_POSIX.
src/cmd/ksh93/sh/init.c:
- sh_type(): Return SH_TYPE_POSIX shell type if ksh was invoked
as sh (or rsh, restricted sh).
- sh_init(): Enable posix option if the SH_TYPE_POSIX shell type
was detected, or if the CONFORMANCE ast config variable was set
to "standard" (which libast sets on init if POSIXLY_CORRECT
exists in the environment).
src/cmd/ksh93/tests/options.sh,
src/cmd/ksh93/tests/io.sh:
- Replace regression tests for &> and move to io.sh. Since &> is
now for general use, no longer test in an rc script, and don't
check that a warning is issued.
Closes: #9
Progresses: #20
2020-09-01 05:19:19 +00:00
|
|
|
|
Speed up 'read', fixing macOS hang (take 2)
This fixes a hanging bug that could occur on macOS when using the
'read' command to read from a FIFO and encountering end-of-file
without a final newline character. It also makes the 'read' command
perform 15-25% faster on macOS and Linux.
The previous version (ff385e5a) failed on SunOS/Solaris/Illumos
because those systems apparently don't (fully) support the POSIX
standard recv(2) syscall with MSG_PEEK[*], which is the feature
that iffe detects under the 'socket_peek' identifier. On Illumos,
using that methods causes a compilation failure (unknown identifier
MSG_PEEK); on Solaris 11.4, that method causes multiple regressions
in tests/io.sh, suggesting the method compiles but doesn't work at
all. Instead, SunOS/Solaris/Illumos requires the method using
ioctl(2)+I_PEEK and select(2). No other system that ksh currently
builds on requires this method, so it is now only used on
SunOS/Solaris/Illumos.
So far, this version of sfpkrd() has been tested to work correctly
on Linux, macOS, FreeBSD, NetBSD, OpenBSD, HP-UX, Solaris, and
OmniOS (an Illumos distribution).
It still fails to peek on Cygwin, but in the exact same way it
failed before, so that's no loss.
To test, run the 'io' test set: bin/shtests -p io
src/lib/libast/sfio/sfpkrd.c: sfpkrd():
- Remove long-obsolete Mac OS X and Solaris bug workarounds.
- Remove methods that are no longer needed.
On systems with a POSIX compliant recv(2), the only thing that
is required to avoid regressions is the code that was conditional
upon the socket_peek feature test, which tests for the correct
functioning of the recv(2) syscall. This has now been made
mandatory for non-SunOS/Solaris/Illumos systems (using an #error
directive if it is not detected), with the other methods removed.
The result performs 15-25% faster on macOS and Linux while
passing all the regression tests.
On macOS, avoiding the select(2) method fixes the hanging bug.
On SunOS/Solaris/Illumos (the '__sun' identifier), the method
using ioctl(2)+I_PEEK and select(2) (iffe feature IDs:
stream_peek and lib_select) is preserved.
Resolves: https://github.com/ksh93/ksh/issues/118 (again)
[*] https://pubs.opengroup.org/onlinepubs/9699919799/functions/recv.html
2020-08-19 20:58:46 +00:00
|
|
|
2020-08-19:
|
|
|
|
|
|
|
|
|
|
- Sped up the 'read' command on most systems by 15-25%. Fixed a hanging bug
|
|
|
|
|
on reading from a FIFO that could occur on macOS.
|
|
|
|
|
|
2020-08-17 19:23:39 +00:00
|
|
|
2020-08-17:
|
|
|
|
|
|
|
|
|
|
- 'command -p' incorrectly used the hash table entry (a.k.a. tracked alias)
|
|
|
|
|
for a command if its path was previously hashed. It has now been fixed so
|
|
|
|
|
it never consults the hash table.
|
|
|
|
|
|
Fix 'command -x' on macOS, Linux, Solaris
'command -x' (basically builtin xargs for 'command') worked for
long argument lists on *BSD and HP-UX, but not on macOS and Linux,
where it reliably entered into an infinite loop.
The problem was that it assumed that every byte of the environment
space can be used for arguments, without accounting for alignment
that some OSs do. MacOS seems to be the most wasteful one: it
aligns on 16-byte boundaries and requires some extra bytes per
argument as well.
src/cmd/ksh93/sh/path.c:
- path_xargs(): When calculating how much space to subtract per
argument, add 16 extra bytes to the length of each argument, then
align the result on 16-byte boundaries. The extra 16 bytes is
more than even macOS needs, but hopefully it is future-proof.
- path_spawn(): If path_xargs() does fail, do not enter a retry
loop (which always becomes an infinite loop if the argument list
exceeds OS limitations), but abort with an error message.
2020-08-16 08:08:07 +00:00
|
|
|
2020-08-16:
|
|
|
|
|
|
|
|
|
|
- Fixed 'command -x' on macOS, Linux and Solaris by accounting for a 16-byte
|
|
|
|
|
argument alignment. If execution does fail, it now aborts with an internal
|
|
|
|
|
error message instead of entering an infinite retry loop.
|
|
|
|
|
|
Fix leak and crash upon defining functions in subshells
A memory leak occurred upon leaving a virtual subshell if a
function was defined within it. If this was done more than 32766
(= 2^15-2 = the 'short' max value - 1) times, the shell crashed.
Discussion and reproducer: https://github.com/ksh93/ksh/issues/114
src/cmd/ksh93/sh/subshell.c: table_unset():
- A subshell-defined function was never freed because a broken
check for autoloaded functions (which must not be freed[*]). It
looked for an initial '/' in the canonical path of the script
file that defined the function, but that path is also stored for
regular functions. Now use a check that executes nv_search() in
fpathdict, the same method used in _nv_unset() in name.c for a
regular function unset.
src/cmd/ksh93/bltins/misc.c: b_dot_cmd():
- Fix an additional memory leak introduced in bd88cc7f, that caused
POSIX functions (which are run with b_dot_cmd() like dot scripts)
to leak extra. This fix avoids both the crash fixed there and the
memory leak by introducing a 'tofree' variable remembering the
filename to free. Thanks to Johnothan King for the patch.
src/lib/libast/include/stk.h,
src/lib/libast/misc/stk.c,
src/lib/libast/man/stk.3,
src/lib/libast/man/stak.3:
- Make the stack more resilient by extending the stack reference
counter 'stkref' from (signed) short to unsigned int. On modern
systems with 32-bit ints, this extends the maximum number of
elements on a stack from 2^15-1==32767 to 2^32-1==4294967295.
The ref counter can never be negative, so there is no reason for
signedness. sizeof(int) is defined as the size of a single CPU
word, so this should not affect performance at all.
On a 16-bit system (not that ksh still compiles there), this
doubles the max number of entries to 2^16-1=65535.
src/cmd/ksh93/tests/leaks.sh:
- Add leak regression tests for ksh functions, POSIX functions, dot
scripts run with '.', and dot scripts run with 'source'.
src/cmd/ksh93/tests/path.sh:
- Add an output builtin with a redirect to an autoloaded function
so that a crash[*] is triggered if the check for an autoloaded
function is ever removed from table_unset(), as was done in ksh
93v- (which crashed).
[*] Freeing autoloaded functions after leaving a virtual subshell
causes a crashing bug: https://github.com/att/ast/issues/803
Co-authored-by: Johnothan King <johnothanking@protonmail.com>
Fixes: https://github.com/ksh93/ksh/issues/114
2020-08-13 22:29:27 +00:00
|
|
|
2020-08-13:
|
|
|
|
|
|
|
|
|
|
- Fixed memory leaks and a crashing bug that occurred when defining and
|
|
|
|
|
running functions in subshells.
|
|
|
|
|
|
Fix crash upon running many subshells (#113)
Co-authored-by: Martijn Dekker <martijn@inlv.org>
An intermittent crash occurred after running many thousands of
virtual/non-forked subshells. One reproducer is a crash in the
shbench fibonacci.ksh test, as documented here:
https://github.com/ksh-community/shbench/blob/f3d9e134/bench/fibonacci.ksh#L4-L10
The apparent cause was the signed and insufficiently large 'short'
data type of 'curenv' and related variables which wrapped around to
a negative number when overflowing. These IDs are necessary for the
'wait' builtin to obtain the exit status from a background job.
This fix is inspired by a patch based on ksh 93v-:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-longenv.dif?expand=1
https://src.fedoraproject.org/rpms/ksh/blob/f24/f/ksh-20130628-longer.patch
However, we change the type to 'unsigned int' instead of 'long'. On
all remotely modern systems, ints are 32-bit values, and using this
type avoids a performance degradation on 32-bit sytems. Making them
unsigned prevents an overflow to negative values.
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/include/jobs.h,
src/cmd/ksh93/include/nval.h,
src/cmd/ksh93/include/shell.h:
- Change the types of the static global 'subenv' and the subshell
structure members 'curenv', 'jobenv', 'subenv', 'p_env' and
'subshell' to one consistent type, unsigned int.
src/cmd/ksh93/sh/jobs.c,
src/cmd/ksh93/sh/macro.c:
src/cmd/ksh93/sh/name.c:
src/cmd/ksh93/sh/nvtype.c,
src/cmd/ksh93/sh/subshell.c:
- Updates to match new variable types.
src/cmd/ksh93/tests/subshell.sh:
- Show wrong exit status in message on failure of 'wait' builtin.
2020-08-12 17:50:59 +00:00
|
|
|
2020-08-11:
|
|
|
|
|
|
|
|
|
|
- Fixed an intermittent crash upon running a large number of subshells.
|
|
|
|
|
|
2020-08-10 21:15:53 +00:00
|
|
|
2020-08-10:
|
|
|
|
|
|
|
|
|
|
- A number of fixes have been applied to the printf formatting directives
|
|
|
|
|
%H and %#H (as well as the undocumented equivalents %(html)q and %(url)q):
|
|
|
|
|
1. Both formatters have been made multibyte/UTF-8 aware, and no longer
|
|
|
|
|
delete multibyte characters. Invalid UTF-8 byte sequences are rendered
|
|
|
|
|
as ASCII question marks.
|
|
|
|
|
2. %H no longer wrongly changes spaces to non-breaking spaces ( ).
|
|
|
|
|
3. %H now converts the single quote (') to '%#39;' instead of '''
|
|
|
|
|
which is not a valid entity in all HTML versions.
|
|
|
|
|
4. %#H failed to encode some reserved characters (e.g. '?') while encoding
|
|
|
|
|
some unreserved ones (e.g. '~'). It now percent-encodes all characters
|
|
|
|
|
except those 'unreserved' as per RFC3986 (ASCII alphanumeric plus -._~).
|
|
|
|
|
|
2020-08-11 00:02:31 +00:00
|
|
|
- Fixed a crash that occurred intermittently after running an external
|
|
|
|
|
command from a command substitution expanded from the $PS1 shell prompt.
|
|
|
|
|
|
2020-08-09 23:02:23 +00:00
|
|
|
2020-08-09:
|
|
|
|
|
|
|
|
|
|
- File name generation (a.k.a. pathname expansion, a.k.a. globbing) now
|
|
|
|
|
never matches the special navigational names '.' (current directory) and
|
|
|
|
|
'..' (parent directory). This change makes a pattern like .* useful; it
|
|
|
|
|
now matches all hidden files (dotfiles) in the current directory, without
|
|
|
|
|
the harmful inclusion of '.' and '..'.
|
|
|
|
|
|
redirect: check args before executing redirections (re: 7b82c338)
The 'redirect' builtin command did not error out before executing
any valid redirections. For example, 'redirect ls >foo.txt' issued
an "incorrect syntax" error, but still created 'foo.txt' and left
standard output permanently redirected to it.
src/cmd/ksh93/sh/xec.c: sh_exec():
- If we have redirections (io != NULL), and the command is
SYSREDIR, then check for arguments and error out if there are
any, before calling sh_redirect() to execute redirections.
(Note, the other check for arguments in b_exec() in bltins/misc.c
must be kept, as that applies if there are no redirections.)
src/cmd/ksh93/sh/io.c: sh_redirect():
- Edit comments to better explain what the flag values do.
src/cmd/ksh93/bltins/misc.c:
- Add a dummy b_redirect() function declaration "for the dictionary
generator" as has historically been done for other builtins that
share one C function. I'm not sure what that dictionary generator
is supposed to be, but this also improves greppability.
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/sh.1:
- Fix misleading "I/O redirection arguments" term. I/O redirections
are not arguments at all; no argument parser ever sees them.
src/cmd/ksh93/tests/io.sh:
- Test both conditions that should make 'redirect' produce an
"incorrect syntax" error.
- Test that any redirections are not executed if erroneous
non-redirection arguments exist.
src/cmd/ksh93/tests/builtins.sh:
- "... should show usage info on unrecognized options" test:
Because 'redirect' now refuses to process redirections on error,
the error message was not captured. The fix is to run the builtin
in a braces block and add the redirection to the block.
2020-08-08 22:57:57 +00:00
|
|
|
2020-08-08:
|
|
|
|
|
|
|
|
|
|
- Argument checking in the 'redirect' builtin command (see 2020-06-11) has
|
|
|
|
|
been improved to error out before executing redirections. For example, an
|
|
|
|
|
error like 'redirect ls >foo.txt' now will not create 'foo.txt' and will
|
|
|
|
|
not leave your standard output permanently redirected to it.
|
|
|
|
|
|
2020-08-07 20:09:01 +00:00
|
|
|
2020-08-06:
|
|
|
|
|
|
2020-08-07 01:53:25 +00:00
|
|
|
- Added the '${.sh.pid}' variable as an alternative to Bash's '$BASHPID'.
|
|
|
|
|
This variable is set to the current shell's PID, unlike '$$' (which is
|
|
|
|
|
set to the parent shell's PID). In virtual subshells '${.sh.pid}' is not
|
|
|
|
|
changed from its previous value, while in forked subshells '${.sh.pid}'
|
|
|
|
|
is set to the subshell's process ID.
|
|
|
|
|
|
2020-08-05 17:06:16 +00:00
|
|
|
2020-08-05:
|
|
|
|
|
|
2020-08-05 17:14:30 +00:00
|
|
|
- Fixed a bug in functions that caused ksh to crash when an array with an
|
|
|
|
|
unset method was turned into a multidimensional array.
|
|
|
|
|
|
2020-08-05 17:06:16 +00:00
|
|
|
- Fixed a bug that caused scripts to continue running after over-shifting
|
|
|
|
|
in a function when the function call had a redirection.
|
|
|
|
|
|
Fix shellquoting of invalid multibyte char (re: f9d28935, 8c7c60ec)
This commit fixes two bugs in the generation of $'...' shellquoted
strings:
1. A bug introduced in f9d28935. In UTF-8 locales, a byte that is
invalid in UTF-8, e.g. hex byte 86, would be shellquoted as
\u[86], which is not the same as the correct quoting, \x86.
2. A bug inherited from 93u+. Single bytes (e.g. hex 11) were
always quoted as \x11 and not \x[11], even if a subsequent
character was a hexadecimal digit. However, the parser reads
past two hexadecimal digits, so we got:
$ printf '%q\n' $'\x[11]1'
$'\x111'
$ printf $'\x111' | od -t x1
0000000 c4 91
0000002
After the bug fix, this works correctly:
$ printf '%q\n' $'\x[11]1'
$'\x[11]1'
$ printf $'\x[11]1' | od -t x1
0000000 11 31
0000002
src/cmd/ksh93/sh/string.c: sh_fmtq():
- Make the multibyte code for $'...' more readable, eliminating the
'isbyte' flag.
- When in a multibyte locale, make sure to shellquote both invalid
multibyte characters and unprintable ASCII characters as
hexadecimal bytes (\xNN). This reinstates 93u+ behaviour.
- When quoting bytes, use isxdigit(3) to determine if the next
character is a hex digit, and if so, protect the quoted byte with
square brackets.
src/cmd/ksh93/tests/quoting2.sh:
- Move the 'printf %q' shellquoting regression tests here from
builtins.sh; they test the shellquoting algorithm, not so much
the printf builtin itself.
- Add regression tests for these bugs.
2020-08-05 17:22:22 +00:00
|
|
|
- When generating shellquoted strings (such as with 'printf %q'), the
|
|
|
|
|
hexadecimal value of a quoted unprintable character was not protected with
|
|
|
|
|
square braces, e.g. 0x12 followed by '3' would be quoted as '\x123', which
|
|
|
|
|
is a different value. Such strings are now quoted like '\x[12]3' if the
|
|
|
|
|
next character is a hexadecimal digit.
|
|
|
|
|
|
2020-07-31 16:32:09 +00:00
|
|
|
2020-07-31:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused multidimensional associative arrays to be created
|
|
|
|
|
with an extra array member.
|
|
|
|
|
|
2020-08-01 00:12:45 +00:00
|
|
|
- Fixed a bug that caused the expansions of positional parameters $1 - $9,
|
|
|
|
|
as well as special parameters such as $? and $-, to corrupt any multibyte
|
|
|
|
|
characters immediately following the expansion if a UTF-8 locale is active.
|
|
|
|
|
|
2020-07-30 00:22:11 +00:00
|
|
|
2020-07-29:
|
|
|
|
|
|
|
|
|
|
- On a ksh compiled to use fork(2) to run external commands, a bug has been
|
|
|
|
|
fixed that caused signals (such as SIGINT, Ctrl+C) to be ignored within a
|
|
|
|
|
non-forked subshell after running an external command within that subshell.
|
|
|
|
|
|
2020-07-25 18:46:11 +00:00
|
|
|
2020-07-25:
|
|
|
|
|
|
|
|
|
|
- Fixed BUG_MULTIBIFS: Multibyte characters can now be used as IFS
|
|
|
|
|
delimiters. "$*" was incorrectly joining positional parameters on
|
|
|
|
|
the first byte of a multibyte character. This was due to truncation
|
|
|
|
|
based on the incorrect assumption the IFS would never be larger
|
|
|
|
|
than a single byte.
|
|
|
|
|
|
2020-07-26 01:18:49 +00:00
|
|
|
- Fixed a bug that caused the sleep builtin to continue after being given
|
|
|
|
|
an unrecognized option. 'sleep -: 1' will now show a usage message and
|
|
|
|
|
exit instead of sleep for one second.
|
|
|
|
|
|
2020-07-25 15:39:12 +00:00
|
|
|
- Fixed a bug that caused the 'typeset' variable attributes -a, -A, -l, and
|
|
|
|
|
-u to leak out of a subshell if they were set without assigning a value.
|
|
|
|
|
|
2020-07-23 23:03:57 +00:00
|
|
|
2020-07-23:
|
|
|
|
|
|
2020-07-24 00:20:26 +00:00
|
|
|
- Fixed an infinite loop that could occur when ksh is the system's /bin/sh.
|
|
|
|
|
|
2020-07-23 23:03:57 +00:00
|
|
|
- A command substitution that is run on the same line as a here-document
|
|
|
|
|
will no longer cause a syntax error.
|
|
|
|
|
|
2020-07-22 11:47:04 +00:00
|
|
|
2020-07-22:
|
|
|
|
|
|
|
|
|
|
- Fixed two race conditions when running external commands on
|
|
|
|
|
interactive shells with job control active.
|
|
|
|
|
|
2020-07-20 18:49:32 +00:00
|
|
|
2020-07-20:
|
|
|
|
|
|
|
|
|
|
- If a shell function and a built-in command by the same name exist,
|
|
|
|
|
'whence -a' and 'type -a' now report both.
|
|
|
|
|
|
2020-07-21 03:02:47 +00:00
|
|
|
- Fixed a bug that caused file descriptors opened with 'redirect' or 'exec'
|
|
|
|
|
to survive a subshell environment after exiting it.
|
|
|
|
|
|
2020-07-19 22:42:12 +00:00
|
|
|
2020-07-19:
|
|
|
|
|
|
2020-08-06 23:50:11 +00:00
|
|
|
- Fixed a crash that occurred in the '.' command when using kshdb.
|
2020-07-19 22:42:12 +00:00
|
|
|
|
2020-08-06 23:50:11 +00:00
|
|
|
- Fixed a crash that occurred when attempting to use redirection with an
|
2020-07-19 22:42:12 +00:00
|
|
|
invalid file descriptor.
|
|
|
|
|
|
2020-07-16 17:56:49 +00:00
|
|
|
2020-07-16:
|
|
|
|
|
|
|
|
|
|
- The 'history' and 'r' default aliases have been made regular built-ins,
|
|
|
|
|
leaving zero default aliases.
|
|
|
|
|
|
2020-07-17 04:00:28 +00:00
|
|
|
- Fixed a bug that caused 'sleep -s' to have no effect with intervals longer
|
|
|
|
|
than 30 seconds.
|
|
|
|
|
|
|
|
|
|
- The accuracy of the sleep builtin has been improved. It no longer ignores
|
|
|
|
|
microseconds and doesn't add extra milliseconds when the interval is less
|
|
|
|
|
than 31 seconds.
|
|
|
|
|
|
Convert default typeset aliases to regular builtins
This converts the 'autoload', 'compound', 'float', 'functions',
'integer' and 'nameref' default aliases into regular built-in
commands, so that 'unalias -a' does not remove them. Shell
functions can now use these names, which improves compatibility
with POSIX shell scripts.
src/cmd/ksh93/data/aliases.c:
- Remove default typeset aliases.
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/include/builtins.h:
- Add corresponding built-in command declarations. Typeset-style
commands are now defined by a pointer range, SYSTYPESET ..
SYSTYPESET_END. A couple need their own IDs (SYSCOMPOUND,
SYSNAMEREF) for special-casing in sh/xec.c.
- Update 'typeset --man'.
src/cmd/ksh93/bltins/typeset.c: b_typeset():
- Recognise the new builtin commands by argv[0]. Implement them by
inserting the corresponding 'typeset' options into the argument
list before parsing options. This may seem like a bit of a hack,
but it is simpler, shorter, more future-proof and less
error-prone than manually copying and adapting all the complex
flaggery from the option parsing loop.
src/cmd/ksh93/sh/parse.c,
src/cmd/ksh93/sh/xec.c:
- Recognise typeset-style commands by SYSTYPESET .. SYSTYPESET_END
pointer range.
- Special-case 'compound' (SYSCOMPOUND) and 'nameref' (SYSNAMEREF)
along with recognising the corresponding 'typeset' options.
src/cmd/ksh93/sh.1:
- Update to document the new built-ins.
- Since not all declaration commands are special built-ins now,
identify declaration commands using a double-dagger "\(dd"
character (which renders as '=' in ASCII) and disassociate their
definition from that of special built-ins.
src/cmd/ksh93/tests/variables.sh:
- Adapt a regression test as there is no more 'integer' alias.
2020-07-15 18:52:01 +00:00
|
|
|
2020-07-15:
|
|
|
|
|
|
|
|
|
|
- The 'autoload', 'compound', 'float', 'functions', 'integer' and 'nameref'
|
|
|
|
|
default aliases have been converted into regular built-in commands, so
|
|
|
|
|
that 'unalias -a' does not remove them. Shell functions can now use
|
|
|
|
|
these names, which improves compatibility with POSIX shell scripts.
|
|
|
|
|
|
2020-07-15 22:38:44 +00:00
|
|
|
- The End key escape sequence '^[[F' is now handled in the emacs and vi editing
|
|
|
|
|
modes. The End key moves the cursor to the end of the line (in contrast to
|
|
|
|
|
the Home key doing the opposite).
|
|
|
|
|
|
2020-07-14 21:00:28 +00:00
|
|
|
2020-07-14:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused 'set -b' to have no effect.
|
|
|
|
|
|
2020-07-15 03:11:33 +00:00
|
|
|
- Following the 'time' keyword, the 'times' builtin command now also
|
|
|
|
|
supports millisecond precision.
|
|
|
|
|
|
2020-07-13 18:10:23 +00:00
|
|
|
2020-07-13:
|
|
|
|
|
|
|
|
|
|
- Fixed a fork bomb that could occur when the vi editor was sent SIGTSTP
|
|
|
|
|
while running in a ksh script.
|
|
|
|
|
|
2020-07-14 21:48:04 +00:00
|
|
|
- Appending a lone percent to the end of a format specifier no longer
|
|
|
|
|
causes a syntax error. The extra percent will be treated as a literal
|
|
|
|
|
'%', like in Bash and zsh.
|
|
|
|
|
|
|
|
|
|
- The 'time' keyword now has proper support for millisecond precision.
|
|
|
|
|
Although this feature was previously documented, the 'time' keyword
|
|
|
|
|
only supported up to centisecond precision, which caused a command
|
|
|
|
|
like the one below to return '0.000' on certain operating systems:
|
|
|
|
|
$ TIMEFORMAT='%3R'; time sleep .003
|
|
|
|
|
|
|
|
|
|
- The 'time' keyword now zero-pads seconds less than ten (like mksh).
|
|
|
|
|
|
2020-07-10 16:52:47 +00:00
|
|
|
2020-07-10:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused types created with 'typeset -T' to throw an error
|
|
|
|
|
when used if the type name started with a lowercase 'a'.
|
|
|
|
|
|
2020-07-10 17:26:28 +00:00
|
|
|
- A potential crash due to memory corruption when using many file
|
|
|
|
|
descriptors has been fixed.
|
|
|
|
|
|
2020-07-09 14:09:52 +00:00
|
|
|
2020-07-09:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash on syntax error when sourcing/dotting multiple files.
|
|
|
|
|
|
2020-07-09 15:02:15 +00:00
|
|
|
- Fixed a crash when listing indexed arrays.
|
|
|
|
|
|
Fix hash table memory leak when restoring PATH
There is a bug in path_alias() that may cause a memory leak when
clearing the hash table while setting/restoring PATH.
This applies a fix from Siteshwar Vashist:
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01945.html
Note that, contrary to Siteshwar's analysis linked above, this bug
has nothing directly to do with subshells, forked or otherwise; it
can also be reproduced by temporarily setting PATH for a command,
for example, 'PATH=/dev/null true', and then doing a PATH search.
Modified analysis:
ksh maintains the value of PATH as a linked list. When a local
scope for PATH is created (e.g. in a virtual subshell or when doing
something like PATH=/foo/bar command ...), ksh duplicates PATH by
increasing the refcount for every element in the linked list by
calling the path_dup() and path_alias() functions. However, when
the state of PATH is restored, this refcount is not decreased. Next
time when PATH is reset to a new value, ksh calls the path_delete()
function to delete the linked list that stored the older path. But
the path_delete() function does not free elements whose refcount is
greater than 1, causing a memory leak.
src/cmd/ksh93/sh/path.c: path_alias():
- Decrease refcount and free old item if needed.
(The 'old' variable was already introduced in 99065353, but
its value was never used there; this fixes that as well.)
src/cmd/ksh93/tests/leaks.sh:
- Add regression test. With the bug, setting/restoring PATH
(which clears the hash table) and doing a PATH search 16 times
causes about 1.5 KiB of memory to be leaked.
2020-07-09 17:34:15 +00:00
|
|
|
- Fixed a memory leak when restoring PATH when temporarily setting PATH
|
|
|
|
|
for a command (e.g. PATH=/foo/bar command ...) or in a virtual subshell.
|
|
|
|
|
|
2020-07-09 21:12:04 +00:00
|
|
|
- Combining ((...)) with redirections no longer causes a syntax error
|
|
|
|
|
due to the parser handling '>' incorrectly.
|
|
|
|
|
|
2020-07-09 21:59:45 +00:00
|
|
|
- Fixed a bug that corrupted KIA/CQL cross-reference databases created using
|
|
|
|
|
ksh's -R option; shell warnings were wrongly included in the database file.
|
|
|
|
|
|
Fix UTF-8 shellquoting for xtrace, printf %q, etc.
This fixes an annoying issue in the shell's quoting algorithm
(used for xtrace (set -x), printf %q, and other things) for UTF-8
locales, that caused it to encode perfectly printable UTF-8
characters unnecessarily and inconsistently. For example:
$ (set -x; : 'aeu aéu')
+ : $'aeu a\u[e9]u'
$ (set -x; : 'aéu aeu')
+ : 'aéu aeu'
$ (set -x; : '正常終了 aeu')
+ : '正常終了 aeu'
$ (set -x; : 'aeu 正常終了')
+ : $'aeu \u[6b63]\u[5e38]\u[7d42]\u[4e86]'
This issue was originally reported by lijo george in May 2017:
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01958.html
src/cmd/ksh93/sh/string.c:
- Add is_invisible() function that returns true if a character is a
Unicode invisible (non-graph) character, excluding ASCII space.
Ref.: https://unicode.org/charts/PDF/U2000.pdf
- Use a fallback in is_invisible() if we cannot use the system's
iswprint(3); this is the case for the ksh C.UTF-8 locale if the
OS doesn't support that. Fall back to a hardcoded blacklist of
invisible and control characters and put up with not encoding
nonexistent characters into \u[xxxx] escapes.
Ref.: https://unicode.org/charts/PDF/U2000.pdf
- When deciding whether to switch to $'...' quoting mode (state=2),
use is_invisible() instead of testing for ASCII 0-127 range.
- In $'...' quoting mode, use is_invisible() to decide whether to
encode wide characters into \u[xxxx] escapes.
src/cmd/ksh93/tests/builtins.sh:
- Add regression tests for shellquoting Arabic, Japanese and Latin
UTF-8 characters, to be run only in a UTF-8 locale. The Arabic
sample text[*] contains a couple of direction markers that are
expected to be encoded into \u[xxxx] escapes.
[*] source: https://r12a.github.io/scripts/tutorial/summaries/arabic
2020-07-10 00:38:13 +00:00
|
|
|
- The shell's quoting algorithm (used in xtrace, printf %q, and more) has been
|
|
|
|
|
fixed for UTF-8 (Unicode) locales; it no longer needlessly and inconsistently
|
|
|
|
|
encodes normal printable UTF-8 characters into hexadecimal \u[xxxx] codes.
|
|
|
|
|
|
2020-07-09 04:08:28 +00:00
|
|
|
2020-07-07:
|
|
|
|
|
|
|
|
|
|
- Four of the date formats accepted by 'printf %()T' have had their
|
|
|
|
|
functionality altered to the common behavior of date(1):
|
|
|
|
|
- '%k' and '%l' print the current hour with blank padding, the former
|
|
|
|
|
based on a 24-hour clock and the latter a twelve hour clock. These
|
|
|
|
|
are common extensions present on Linux and *BSD.
|
|
|
|
|
- '%f' prints a date with the format string '%Y.%m.%d-%H:%M:%S' (BusyBox).
|
|
|
|
|
- '%q' prints the quarter of the year (GNU).
|
|
|
|
|
|
2020-07-06 20:51:44 +00:00
|
|
|
2020-07-06:
|
|
|
|
|
|
|
|
|
|
- 'notty' is now written to the ksh auditing file instead of '(null)' if
|
|
|
|
|
the user's tty could not be determined.
|
|
|
|
|
|
2020-07-09 00:09:40 +00:00
|
|
|
- Unsetting an associative array no longer causes a memory leak to occur.
|
|
|
|
|
|
Fix corrupt UTF-8 char processing & shellquoting after aborted read
If the processing of a multibyte character was interrupted in UTF-8
locales, e.g. by reading just one byte of a two-byte character 'ü'
(\303\274) with a command like:
print -nr $'\303\274' | read -n1 g
then the shellquoting algorithm was corrupted in such a way that
the final quote in simple single-quoted string was missing. This
bug may have had other, as yet undiscovered, effects as well. The
problem was with corrupted multibyte character processing and not
with the shell-quoting routine sh_fmtq() itself.
Full trace and discussion at: https://github.com/ksh93/ksh/issues/5
(which is also an attempt to begin to understand the esoteric
workings of the libast mb* macros that process UTF-8 characters).
src/lib/libast/comp/setlocale.c: utf8_mbtowc():
- If called from the mbinit() macro (i.e. if both pointer
parameters are null), reset the global multibyte character
synchronisation state variable. This fixes the problem with
interrupted processing leaving an inconsistent state, provided
that mbinit() is called before processing multibyte characters
(which it is, in most (?) places that do this). Before this fix,
calling mbinit() in UTF-8 locales was a no-op.
src/cmd/ksh93/sh/string.c: sh_fmtq():
- Call mbinit() before potentially processing multibyte characters.
Testing suggests that this could be superfluous, but at worst,
it's harmless; better be sure.
src/cmd/ksh93/tests/builtins.sh:
- Add regression test for shellquoting with 'printf %q' after
interrupting the processing of a multibyte characeter with
'read -n1'. This test only fails in a UTF-8 locale, e.g. when
running: bin/shtests -u builtins SHELL=/buggy/ksh-2012-08-01
Fixes #5.
2020-07-05 17:24:41 +00:00
|
|
|
2020-07-05:
|
|
|
|
|
|
|
|
|
|
- In UTF-8 locales, fix corruption of the shell's internal string quoting
|
|
|
|
|
algorithm (as used by xtrace, 'printf %q', and more) that occurred when
|
|
|
|
|
the processing of a multibyte character was interrupted.
|
|
|
|
|
|
2020-07-03 19:08:00 +00:00
|
|
|
2020-07-03:
|
|
|
|
|
|
|
|
|
|
- Backslashes are no longer escaped in the raw Bourne Shell-like editing
|
|
|
|
|
mode in multibyte locales, i.e. backslashes are no longer treated like
|
|
|
|
|
Control-V if the emacs and vi modes are disabled.
|
|
|
|
|
|
|
|
|
|
- Deleting a backslash in vi mode with Control-H or Backspace now only
|
|
|
|
|
escapes a backslash if it was the previous input. This means erasing a
|
|
|
|
|
string such as 'ab\\\' will only cause the first backslash to escape a
|
|
|
|
|
Backspace as '^?', like in emacs mode.
|
|
|
|
|
|
|
|
|
|
- An odd interaction with Backspace when the last character of a separate
|
|
|
|
|
buffer created with Shift-C was '\' has been fixed. '^?' will no longer
|
|
|
|
|
be output repeatedly when attempting to erase a separate buffer with
|
|
|
|
|
a Backspace. Note that buffers created with Shift-C are not meant to be
|
|
|
|
|
erasable:
|
|
|
|
|
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/vi.html#tag_20_152_13_49
|
|
|
|
|
|
2020-07-04 14:57:47 +00:00
|
|
|
- The 'kill' builtin now supports the SIGINFO signal (on operating systems
|
|
|
|
|
with support for SIGINFO).
|
|
|
|
|
|
2020-07-02 22:29:07 +00:00
|
|
|
2020-07-02:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash that occurred if a directory named '.paths' existed in any
|
|
|
|
|
directory listed in $PATH. The fix was to only read '.paths' if it is a
|
|
|
|
|
regular file or a symlink to a regular file.
|
|
|
|
|
|
2020-07-01 17:14:10 +00:00
|
|
|
2020-06-30:
|
|
|
|
|
|
|
|
|
|
- 'read -u' will no longer crash with a memory fault when given an out of
|
|
|
|
|
range or negative file descriptor.
|
2020-07-02 17:40:15 +00:00
|
|
|
|
|
|
|
|
- The '=~' operator no longer raises an error if a regular expression
|
|
|
|
|
combines the '{x}' quantifier with a sub-expression.
|
Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
|
2020-06-28 22:30:27 +00:00
|
|
|
2020-06-28:
|
|
|
|
|
|
|
|
|
|
- Variables created with 'typeset -RF' no longer cause a memory fault
|
|
|
|
|
when accessed.
|
|
|
|
|
|
2020-06-29 17:08:28 +00:00
|
|
|
- Unsetting an array that was turned into a compound variable will no
|
|
|
|
|
longer cause silent memory corruption.
|
|
|
|
|
|
2020-06-29 18:09:20 +00:00
|
|
|
- Variables created with 'readonly' in functions are now set to the
|
|
|
|
|
specified value instead of nothing. Note that 'readonly' does not
|
|
|
|
|
create a function-local scope, unlike 'typeset -r' which does.
|
|
|
|
|
|
2020-06-26 22:36:29 +00:00
|
|
|
2020-06-26:
|
|
|
|
|
|
|
|
|
|
- Changing to a directory that has a name starting with a '.' will no
|
|
|
|
|
longer fail if preceded by '../' (i.e. 'cd ../.local' will now work).
|
|
|
|
|
|
2020-06-25 21:08:43 +00:00
|
|
|
2020-06-24:
|
|
|
|
|
|
|
|
|
|
- Fixed buggy tab completion of tilde-expanded paths such as
|
|
|
|
|
~/some in 'vi' mode.
|
|
|
|
|
|
2020-06-26 05:19:58 +00:00
|
|
|
- In the raw/default Bourne Shell-like editing mode that occurs when neither
|
|
|
|
|
the 'emacs' nor the 'vi' shell option is active:
|
|
|
|
|
* tab completion is now correctly disabled, instead of enabled and broken;
|
|
|
|
|
* entering tab characters now moves the cursor the correct amount.
|
|
|
|
|
|
2020-06-23 22:02:16 +00:00
|
|
|
2020-06-23:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused combining process substitution with redirection
|
|
|
|
|
to create a bizarre file in the user's current working directory.
|
|
|
|
|
|
|
|
|
|
- Using process substitution while the shell is interactive no longer
|
|
|
|
|
causes the process ID of the asynchronous process to be printed.
|
|
|
|
|
|
2020-06-22 12:59:24 +00:00
|
|
|
2020-06-22:
|
|
|
|
|
|
|
|
|
|
- The 'stop' and 'suspend' default aliases have been converted into regular
|
|
|
|
|
built-in commands, so that 'unalias -a' does not remove them, 'suspend'
|
|
|
|
|
can do a couple of sanity checks, and something like
|
|
|
|
|
cmd=stop; $cmd $!
|
|
|
|
|
will now work. See 'stop --man' and 'suspend --man' for more information.
|
|
|
|
|
|
2020-06-22 17:11:19 +00:00
|
|
|
- Fixed a bug that caused the kill and stop commands to segfault when given
|
|
|
|
|
a non-existent job.
|
|
|
|
|
|
2020-06-22 23:27:05 +00:00
|
|
|
- Nested functions no longer ignore variable assignments that were prefixed
|
|
|
|
|
to their parent function, i.e. 'VAR=foo func' will now set $VAR to 'foo'
|
|
|
|
|
in the scope of any nested function 'func' runs.
|
|
|
|
|
|
2020-06-20 17:08:41 +00:00
|
|
|
2020-06-20:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused setting the following variables as readonly in
|
|
|
|
|
a virtual subshell to affect the environment outside of the subshell:
|
|
|
|
|
$_
|
|
|
|
|
${.sh.name}
|
|
|
|
|
${.sh.subscript}
|
|
|
|
|
${.sh.level}
|
|
|
|
|
$RANDOM
|
|
|
|
|
$LINENO
|
|
|
|
|
|
2021-03-21 14:39:03 +00:00
|
|
|
- Fixed two bugs that caused 'unset .sh.lineno' to always produce a memory
|
|
|
|
|
fault and '(unset .sh.level)' to memory fault when run in nested
|
2020-06-20 17:08:41 +00:00
|
|
|
functions.
|
|
|
|
|
|
2020-06-19 12:01:55 +00:00
|
|
|
2020-06-18:
|
|
|
|
|
|
|
|
|
|
- A two decade old bug that caused 'whence -a' to base the path of
|
|
|
|
|
tracked aliases on the user's current working directory has been
|
2020-06-25 15:21:40 +00:00
|
|
|
fixed. Now the real path to tracked aliases is shown when '-a' is
|
|
|
|
|
passed to the whence command.
|
2020-06-19 12:01:55 +00:00
|
|
|
|
2020-06-18 00:48:51 +00:00
|
|
|
2020-06-17:
|
|
|
|
|
|
|
|
|
|
- A bug in 'unset -f' was fixed that prevented shell functions from
|
|
|
|
|
unsetting themselves while they were running. A POSIX function no longer
|
|
|
|
|
crashes when doing so, and a KornShell-style function no longer silently
|
|
|
|
|
ignores an 'unset -f' on itself. A function of either form now continues
|
|
|
|
|
running after unsetting itself, and is removed at the end of the run.
|
|
|
|
|
|
2020-06-16 11:21:42 +00:00
|
|
|
2020-06-16:
|
|
|
|
|
|
|
|
|
|
- Passing the '-d' flag to the read builtin will no longer cause the '-r'
|
|
|
|
|
flag to be discarded when 'read -r -d' is run.
|
|
|
|
|
|
2020-06-16 21:58:05 +00:00
|
|
|
- Fix BUG_CMDSPASGN: preceding a "special builtin"[*] with 'command' now
|
|
|
|
|
prevents preceding invocation-local variable assignments from becoming global.
|
|
|
|
|
[*] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_14
|
|
|
|
|
|
2020-06-15 07:03:44 +00:00
|
|
|
2020-06-15:
|
|
|
|
|
|
|
|
|
|
- The 'source' alias has been converted into a regular built-in command.
|
|
|
|
|
|
2020-06-15 13:43:37 +00:00
|
|
|
- Functions that set variables in a virtual subshell will no longer affect
|
|
|
|
|
variables of the same name outside of the virtual subshell's environment.
|
|
|
|
|
|
2020-06-15 14:56:11 +00:00
|
|
|
- Terse usage messages written by builtin commands now point the user to
|
|
|
|
|
the --help and --man options for more information.
|
|
|
|
|
|
2020-06-14 16:28:22 +00:00
|
|
|
2020-06-14:
|
2020-06-15 07:03:44 +00:00
|
|
|
|
2020-06-14 16:28:22 +00:00
|
|
|
- 'read -S' is now able to correctly handle strings with double quotes
|
|
|
|
|
nested inside of double quotes.
|
|
|
|
|
|
2020-06-13 21:16:08 +00:00
|
|
|
2020-06-13:
|
2020-06-14 04:28:38 +00:00
|
|
|
|
2020-06-13 21:16:08 +00:00
|
|
|
- Fixed a timezone name determination bug on FreeBSD that caused the
|
2021-03-21 14:39:03 +00:00
|
|
|
output from "LC_ALL=C printf '%T\n' now" to print the wrong time zone name.
|
2020-06-13 21:16:08 +00:00
|
|
|
|
2020-06-11 10:43:17 +00:00
|
|
|
2020-06-11:
|
|
|
|
|
|
2020-06-12 03:18:01 +00:00
|
|
|
- Fixed a bug that caused running 'builtin -d' on a special builtin to
|
|
|
|
|
delete it. The man page for the 'builtin' command documents that special
|
|
|
|
|
builtins cannot be deleted.
|
|
|
|
|
|
2020-06-11 10:43:17 +00:00
|
|
|
- POSIX compliance fix: It is now possible to set shell functions named
|
|
|
|
|
'alias' or 'unalias', overriding the commands by the same names. In
|
|
|
|
|
technical terms, they are now regular builtins, not special builtins.
|
|
|
|
|
|
2020-06-12 02:17:14 +00:00
|
|
|
- The redirect='command exec' alias has been converted to a regular
|
|
|
|
|
'redirect' builtin command that only accepts I/O redirections, which
|
|
|
|
|
persist as in 'exec'. This means that:
|
|
|
|
|
* 'unlias -a' no longer removes the 'redirect' command;
|
|
|
|
|
* users no longer accidentally get logged out of their shells if
|
|
|
|
|
they type something intuitive but wrong, like 'redirect ls >file'.
|
|
|
|
|
|
2020-06-12 04:57:57 +00:00
|
|
|
- The undocumented 'login' and 'newgrp' builtin commands have been removed.
|
|
|
|
|
These replaced your shell session with the external commands by the same
|
|
|
|
|
name, as in 'exec'. If an error occurred (e.g. due to a typo), you would
|
|
|
|
|
end up immediately logged out.
|
|
|
|
|
If you do want this behaviour, you can restore it by setting:
|
|
|
|
|
alias login='exec login'
|
|
|
|
|
alias newgrp='exec newgrp'
|
|
|
|
|
|
2020-06-10 11:00:35 +00:00
|
|
|
2020-06-10:
|
|
|
|
|
|
|
|
|
|
- The 'hash' utility is now a regular builtin instead of an alias to
|
|
|
|
|
'alias -t --'. The functionality of the old command has been removed
|
|
|
|
|
from the alias builtin.
|
|
|
|
|
|
2020-06-25 15:21:40 +00:00
|
|
|
- 'set +r' is no longer able to unset the restricted option. This change
|
|
|
|
|
makes the behavior of 'set +r' identical to 'set +o restricted'.
|
|
|
|
|
|
2020-06-09 15:31:00 +00:00
|
|
|
2020-06-09:
|
|
|
|
|
|
|
|
|
|
- The 'unalias' builtin will now return a non-zero status if it tries
|
|
|
|
|
to remove a previously set alias that is not currently set.
|
|
|
|
|
|
2020-06-07 22:01:24 +00:00
|
|
|
2020-06-08:
|
|
|
|
|
|
2020-06-10 12:16:53 +00:00
|
|
|
- Fix an issue with the up arrow key in Emacs editing mode.
|
|
|
|
|
Emacs editing mode is bugged in ksh93u+ and ksh2020. Let's
|
|
|
|
|
say you were to run the following commands after starting
|
|
|
|
|
a fresh instance of ksh:
|
2021-03-21 14:39:03 +00:00
|
|
|
$ alias foo=true
|
2020-06-10 12:16:53 +00:00
|
|
|
$ unalias foo
|
|
|
|
|
If you type 'a' and then press the up arrow on your keyboard,
|
2021-03-21 14:39:03 +00:00
|
|
|
ksh will complete 'a' to 'alias foo=true' by doing a reverse
|
2020-06-10 12:16:53 +00:00
|
|
|
search for the last command that starts with 'a'.
|
|
|
|
|
Run the alias command again, then type 'u' and press the up
|
2021-03-21 14:39:03 +00:00
|
|
|
arrow key again. If ksh is in Vi mode, you will get 'unalias foo',
|
|
|
|
|
but in Emacs mode you will get 'alias foo=true' again.
|
2020-06-10 12:16:53 +00:00
|
|
|
All subsequent commands were ignored as ksh was saving the first
|
|
|
|
|
command and only based later searches off of it.
|
2020-06-08 10:22:51 +00:00
|
|
|
|
2020-06-07 22:01:24 +00:00
|
|
|
- If 'set -u'/'set -o nounset' is active, then the shell now errors out if a
|
|
|
|
|
nonexistent positional parameter such as $1, $2, ... is accessed, as other
|
|
|
|
|
shells do and POSIX requires. (This does *not* apply to "$@" and "$*".)
|
|
|
|
|
|
2020-06-07 23:05:27 +00:00
|
|
|
- If 'set -u'/'set -o nounset' is active, then the shell now errors out if $!
|
|
|
|
|
is accessed before the shell has launched any background process.
|
|
|
|
|
|
2020-06-08 01:28:36 +00:00
|
|
|
- Removed support for an obscure early 1990s Bell Labs file system research
|
|
|
|
|
project called 3DFS, which has not existed for decades. This removes:
|
|
|
|
|
- an obnoxious default alias 2d='set -f;_2d' that turned off your file name
|
|
|
|
|
wildcard expansion and then tried to run a nonexistent '_2d' command
|
|
|
|
|
- undocumented builtins 'vmap' and 'vpath' that only printed error messages
|
|
|
|
|
- a non-functional -V unary operator for the test and [[ commands
|
|
|
|
|
|
Fix signal handling due to exit status > 256
This fixes two bugs: issuing the 'exit' command with a value > 256
would cause ksh 93u+ to kill itself with the corresponding signal
(try 'exit 265' to SIGKILL your interactive shell), and, if the
last command of a script exits due to a signal, the shell would
repeat that signal to itself, causing any parent ksh to also be
killed.
Discussion:
https://bugzilla.redhat.com/show_bug.cgi?id=1469624
https://rainbow.chard.org/2017/03/21/ksh-deliberately-segfaults-if-the-last-command-in-a-script-crashes/
This commit is loosely based on a patch applied to the 93v- beta
and the abandoned ksh2020, but that patch was incomplete & broken:
$ ksh-2020.0.0 -c 'exit 265'; echo $?
137
Expected: 9. Since the exit was *not* due to a signal, the value
should simply be cropped to the 8 bits supported by the OS.
src/cmd/ksh93/bltins/cflow.c: b_exit():
- For the 'exit' builtin command, bitwise-AND the argument to
'exit' with SH_EXITMASK (8 bits, crop to 0-255) before passing it
on to sh_exit(). This restores the behaviour of <=2011 ksh93
versions and is in line with all other POSIX shells.
It also fixes this bogosity:
$ (exit 265); echo $? # non-forked subshell
265
$ (ulimit -t unlimited; exit 265); echo $? # forked subshell
9
Forked or non-forked should make no difference at all
(see commit message a0e0e29e for why).
src/cmd/ksh93/sh/fault.c: sh_done():
- If the current exit status is equal to the status for the last
signal that was received from a child process, remove the
SH_EXITSIG (9th) bit, so that the shell doesn't kill itself.
- If the shell's last child process exits due to a signal, exit
with a portable 8-bit exit status (128 + signal number). This
avoids the exit status being < 128 by being cropped to 8 bits.
src/cmd/ksh93/tests/signal.sh:
- Add regression test for exit with status > 256.
- Add regression test verifying the shell no longer kills itself.
(cherry picked from commit 98e0fc94393e175ce6adfee390327c320795bf12)
2020-06-08 10:23:37 +00:00
|
|
|
- If the last program run by a ksh script exits with a signal (e.g. crashed),
|
|
|
|
|
ksh itself now exits normally instead of repeating that same signal.
|
|
|
|
|
In addition, using 'exit x' for x > 256 no longer makes ksh issue a signal.
|
|
|
|
|
|
2020-06-06 19:25:59 +00:00
|
|
|
2020-06-06:
|
|
|
|
|
|
|
|
|
|
- The 'times' command is now a builtin command that conforms to POSIX
|
|
|
|
|
instead of an alias for the 'time' command. It displays the accumulated
|
|
|
|
|
user and system CPU times, one line with the times used by the shell and
|
|
|
|
|
another with those used by all of the shell's child processes.
|
|
|
|
|
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_27
|
|
|
|
|
|
POSIX compliance: rm harmful default aliases 'command '/'nohup '
Continuing alias substitution after 'command' (due to the final
space in the alias) is inherently broken and doing so by default is
incompatible with the POSIX standard, as aliases may contain
arbitrary shell grammar.
For instance, until the previous commit, the POSIX standard 'times'
command was an alias: times='{ { time;} 2>&1;}' -- and so, of
course, 'command times' gave a syntax error, although this is
a perfectly valid POSIX idiom that must be supported.
'command' is specified by POSIX as a regular builtin, not an alias.
Therefore it should always bypass aliases just as it bypasses
functions to expose standard builtin and external commands.
I can only imagine that the reason for this command='command '
alias was that some standard commands themselves were implemented
as aliases, and POSIX requires that standard commands are
accessible with the 'command' prefix. But implementing standard
commands as aliases is itself inherently broken. It never worked
for 'command times', as shown; and in any case, removing all
aliases with 'unalias -a' should not get rid of standard commands.
Similarly, the default alias nohup='nohup ' is also harmful.
Anyone who really wants to keep this behaviour can just define
these aliases themselves in their script or ~/.kshrc file.
src/cmd/ksh93/data/aliases.c:
- Remove default alias command='command '.
- Remove default alias nohup='nohup '.
src/cmd/ksh93/sh.1
- Remove the above default aliases from the list.
- Mention that the 'command' builtin does not search for aliases.
(cherry picked from commit 5cfe7c4e2015b7445da24983af5008035c4b6e1e)
2020-06-07 00:36:54 +00:00
|
|
|
- The default aliases command='command ' and nohup='nohup ' have been
|
|
|
|
|
removed because they caused breakage in an attempt to circumvent other
|
|
|
|
|
breakage which is being fixed. In the unlikely even that anyone still
|
|
|
|
|
needs alias substitution to continue on the command argument following
|
|
|
|
|
'command' or 'nohup', it's easy to set these aliases yourself.
|
|
|
|
|
|
Fix unsetting special vars in subshells (re: efa31503, 8b7f8f9b)
This fixes (or at least works around) a bug that caused special
variables such as PATH, LANG, LC_ALL, LINENO, etc. to lose their
effect after being unset in a subshell.
For example:
(unset PATH; PATH=/dev/null; ls); : wrongly ran 'ls'
(unset LC_ALL; LC_ALL=badlocale); : failed to print a diagnostic
This is yet another problem with non-forking/virtual subshells. If
you forced the subshell to fork (one way of doing this is using the
'ulimit' builtin, e.g. ulimit -t unlimited) before unsetting the
special variable, the problem vanished.
I've tried to localise the problem. I suspect the sh_assignok()
function, which is called from unall(), is to blame. This function
is supposed to make a copy of a variable node in the virtual
subshell's variable tree. Apparently, it fails to copy the
associated permanent discipline function settings (stored in the
np->nvfun->disc pointer) that gave these variables their special
effect, and which survive unset. However, my attempts to fix that
have been unsuccessful. If anyone can figure out a fix, please send
a patch/pull request!
Data point: This bug existed in 93u 2011-02-08, but did not yet
exist in M-1993-12-28-s+. So it is a regression.
Meanwhile, pending a proper fix, this commit adds a safe
workaround: it forces a non-forked subshell to fork before
unsetting such a special variable.
src/cmd/ksh93/bltins/typeset.c: unall():
- If we're in a non-forked, non-${ ...; } subshell, then before
unsetting any variable, check for variables with internal
trap/discipline functions, and call sh_subfork() if any are
found. To avoid crashing, this must be done before calling
sh_pushcontext(), so we need to loop through the args separately.
src/cmd/ksh93/tests/variables.sh:
- Remove the 'ulimit' that forced the fork; we do this in C now.
(cherry picked from commit 21b1a67156582e3cbd36936f4af908bb45211a4b)
2020-06-05 13:31:26 +00:00
|
|
|
2020-06-05:
|
|
|
|
|
|
|
|
|
|
- Fix a bug that caused special variables such as PATH, LANG, LC_ALL,
|
|
|
|
|
etc. to lose their effect after being unset in a subshell. For example:
|
|
|
|
|
(unset PATH; PATH=/dev/null; ls); : wrongly ran 'ls'
|
|
|
|
|
(unset LC_ALL; LC_ALL=badlocale); : failed to print a diagnostic
|
2020-06-24 17:38:09 +00:00
|
|
|
This also fixes BUG_KUNSETIFS: unsetting IFS in a subshell failed if IFS
|
|
|
|
|
was set to the empty value in the parent shell.
|
Fix unsetting special vars in subshells (re: efa31503, 8b7f8f9b)
This fixes (or at least works around) a bug that caused special
variables such as PATH, LANG, LC_ALL, LINENO, etc. to lose their
effect after being unset in a subshell.
For example:
(unset PATH; PATH=/dev/null; ls); : wrongly ran 'ls'
(unset LC_ALL; LC_ALL=badlocale); : failed to print a diagnostic
This is yet another problem with non-forking/virtual subshells. If
you forced the subshell to fork (one way of doing this is using the
'ulimit' builtin, e.g. ulimit -t unlimited) before unsetting the
special variable, the problem vanished.
I've tried to localise the problem. I suspect the sh_assignok()
function, which is called from unall(), is to blame. This function
is supposed to make a copy of a variable node in the virtual
subshell's variable tree. Apparently, it fails to copy the
associated permanent discipline function settings (stored in the
np->nvfun->disc pointer) that gave these variables their special
effect, and which survive unset. However, my attempts to fix that
have been unsuccessful. If anyone can figure out a fix, please send
a patch/pull request!
Data point: This bug existed in 93u 2011-02-08, but did not yet
exist in M-1993-12-28-s+. So it is a regression.
Meanwhile, pending a proper fix, this commit adds a safe
workaround: it forces a non-forked subshell to fork before
unsetting such a special variable.
src/cmd/ksh93/bltins/typeset.c: unall():
- If we're in a non-forked, non-${ ...; } subshell, then before
unsetting any variable, check for variables with internal
trap/discipline functions, and call sh_subfork() if any are
found. To avoid crashing, this must be done before calling
sh_pushcontext(), so we need to loop through the args separately.
src/cmd/ksh93/tests/variables.sh:
- Remove the 'ulimit' that forced the fork; we do this in C now.
(cherry picked from commit 21b1a67156582e3cbd36936f4af908bb45211a4b)
2020-06-05 13:31:26 +00:00
|
|
|
|
2020-06-05 14:03:07 +00:00
|
|
|
- Fix crashes on some systems, including at least a crash in 'print -v' on
|
|
|
|
|
macOS, by eliminating an invalid/undefined use of memccpy() on overlapping
|
|
|
|
|
buffers in the commonly used sfputr() function.
|
|
|
|
|
|
Fix ${.sh.subshell} counter to actually count level of subshells
This counter is documented as follows:
"The current depth for subshells and command substitution."
But before this commit, the actual behaviour was that the counter
was reset to zero whenever a subshell forked for any reason: a
pipe, background job, running 'ulimit', redirecting stdout in a
command substitution, and more. This behaviour was:
1. Not consistent with the documentation. Non-forked (a.k.a.
virtual) subshells are an internal implementation detail which
scripts should not have to be concerned with. The manual page
doesn't mention them at all.
2. Inherently broken. Since a subshell may fork for any number of
reasons, even mid-run, and those reasons may change with
bugfixes and further development, scripts have never actually
been able to rely on the value of ${.sh.subshell}.
So, this commit fixes the counter to count the levels of all
subshells, both virtual and forked.
src/cmd/ksh93/sh/xec.c: _sh_fork():
- Increase ${.sh.subshell} whenever we fork.
src/cmd/ksh93/sh/subshell.c:
- sh_subfork():
* Fix comment to properly explain what it does. It doesn't
"create" a subshell, it forks off an existing virtual subshell.
* Don't zero ${.sh.subshell}. Instead, since sh_fork() increases
it but we're forking an existing subshell, undo the increase.
- sh_subshell():
* Remove 'int16_t subshell' variable. It was unnecessary and
mostly unused. It was also the wrong type: it was assigned the
value from shp->subshell which is of type short.
* Increase and decrease the level of virtual subshells and
${.sh.subshell} independently.
src/cmd/ksh93/tests/variables.sh:
- Add regression tests for ${.sh.subshell} in virtual and forked
subshells of several kinds: comsub, parentheses, pipe, bg job.
- Undo wrong error test count fix from 04b4aef0.
(cherry picked from commit a0e0e29e7e0dbf21e4b3958ae02bde6665fb2696)
2020-06-05 19:39:47 +00:00
|
|
|
- Fix the ${.sh.subshell} level counter; it is no longer reset to zero when a
|
|
|
|
|
non-forked subshell happens to fork into a separate process for some reason
|
|
|
|
|
(an internal implementation detail that should be unnoticeable to scripts).
|
|
|
|
|
|
Fix BUG_KBGPID: $! was not updated under certain conditions
The $! special parameter was not set if a background job
(somecommand &) or co-process (somecommand |&) was launched as the
only command within a braces block with an attached redirection,
for example:
{
somecommand &
} >&2
With the bug, $! was unchanged; now it contains the PID of
somecommand.
Ref.: https://github.com/att/ast/issues/1357
src/cmd/ksh93/sh/parse.c: item():
- When processing redirections following a compound command, always
create a parent node with the TSETIO (I/O redirection) token.
Before this commit, if the last command was of type TFORK (and
the last command only tested as TFORK if the bg job or coprocess
was the only command in a braces block, because the ksh parser
optimises away the braces in that case), then the parent node was
created with the TFORK token instead.
I have no idea what David Korn's intention was with that, but
this is clearly very wrong. Creating another TFORK node when
parsing the redirection caused sh_exec() in sh/xec.c to execute
the redirection in an extra forked, non-background subshell.
Since redirections are executed before anything else, this
subshell is what then launched the background job between the
braces, so $! (a.k.a. shp->bckpid) was updated in that subshell
only, and never in the main shell. The extra subshell also
prevented the background job from being noticed by job control
on interactive shells.
So, the fix is simply to remove the broken test for TFORK.
src/cmd/ksh93/tests/variables.sh:
- Add regression tests for a bg job and a co-process as the only
command within a braces block with attached redirection.
(cherry picked from commit ffe5df30e69f7b596941a98498014d8e838861f2)
2020-06-04 03:19:59 +00:00
|
|
|
2020-06-04:
|
|
|
|
|
|
|
|
|
|
- Fix BUG_KBGPID: the $! special parameter was not set if a background job
|
|
|
|
|
(somecommand &) or co-process (somecommand |&) was launched as the only
|
|
|
|
|
command within a braces block with an attached redirection, for example:
|
|
|
|
|
{
|
|
|
|
|
somecommand &
|
|
|
|
|
} >&2
|
|
|
|
|
With the bug, $! was unchanged; now it contains the PID of somecommand.
|
|
|
|
|
|
2020-05-31 13:29:15 +00:00
|
|
|
2020-05-31:
|
|
|
|
|
|
|
|
|
|
- Fix a bug in autoloading functions. Directories in the path search list
|
|
|
|
|
which should be skipped (e.g. because they don't exist) did not interact
|
|
|
|
|
correctly with autoloaded functions, so that a function to autoload was
|
|
|
|
|
not always found correctly.
|
|
|
|
|
Details: https://github.com/att/ast/issues/1454
|
|
|
|
|
|
2020-05-30 14:21:30 +00:00
|
|
|
2020-05-30:
|
|
|
|
|
|
|
|
|
|
- Fix POSIX compliance of 'test'/'[' exit status on error. The command now
|
|
|
|
|
returns status 2 instead of 1 when given an invalid number or arithmetic
|
|
|
|
|
expression, e.g.:
|
|
|
|
|
[ 123 -eq 123x ]; echo $?
|
|
|
|
|
now outputs 2 instead of 1.
|
|
|
|
|
|
Fix redefining & unsetting functions in subshells (BUG_FNSUBSH)
Functions can now be correctly redefined and unset in subshell
environments (such as ( ... ), $(command substitutions), etc).
Before this fix, attempts to do this were silently ignored (!!!),
causing the wrong code (i.e.: the function by the same name from
the parent shell environment) to be executed.
Redefining and unsetting functions within "shared" command
substitutions of the form '${ ...; }' is also fixed.
Prior discussion: https://github.com/att/ast/issues/73
src/cmd/ksh93/sh/parse.c:
- A fix from George Koelher (URL above). He writes:
| The parser can set t->comnamp to the wrong function.
| Suppose that the shell has executed
| foo() { echo WRONG; }
| and is now parsing
| (foo() { echo ok; } && foo)
| The parser was setting t->comnamp to the wrong foo. [This
| fix] doesn't set t->comnamp unless it was a builtin. Now the
| subshell can't call t->comnamp, so it looks for foo and finds
| the ok foo in the subshell's function tree.
src/cmd/ksh93/bltins/typeset.c:
- Unsetting functions in a virtual/non-forked subshell still
doesn't work: nv_open() fails to find the function. To work
around this problem, make 'unset -f' fork the subshell into its
own process with sh_subfork().
- The workaround exposed another bug: if we unset a function in a
subshell tree that overrode a function by the same name in the
main shell, then nv_delete() exposes the function from the main
shell scope. Since 'unset -f' now always forks a subshell, the
fix is to simply walk though troot's parent views and delete any
such zombie functions as well. (Without this, the 4 'more fun'
tests in tests/subshell.sh fail.)
src/cmd/ksh93/sh/subshell.c: sh_subfuntree():
- Fix function (re)definitions and unsetting in "shared" command
substitutions of the form '${ commandlist; }' (i.e.: if
sp->shp->subshare is true). Though internally this is a weird
form of virtual subshell, the manual page says it does not
execute in a subshell (meaning, all changes must survive it), so
a subshell function tree must not be created for these.
src/cmd/ksh93/tests/subshell.sh:
- Add regression tests related to these bugfixes. Test unsetting
and redefining a function in all three forms of virtual subshell.
(cherry picked from commit dde387825ab1bbd9f2eafc5dc38d5fd0bf9c3652)
2020-05-29 07:27:20 +00:00
|
|
|
2020-05-29:
|
|
|
|
|
|
|
|
|
|
- Fix BUG_FNSUBSH: functions can now be correctly redefined and unset in
|
|
|
|
|
subshell environments (such as ( ... ), $(command substitutions), etc).
|
|
|
|
|
Before this fix, this was silently ignored, causing the function by the
|
|
|
|
|
same name from the parent shell environment to be executed instead.
|
|
|
|
|
fn() { echo mainsh; }
|
|
|
|
|
(fn() { echo subsh; }; fn); fn
|
|
|
|
|
This now correctly outputs "subsh mainsh" instead of "mainsh mainsh".
|
|
|
|
|
ls() { echo "ls executed"; }
|
|
|
|
|
(unset -f ls; ls); ls
|
|
|
|
|
This now correctly lists your directory and then prints "ls executed",
|
|
|
|
|
instead of printing "ls executed" twice.
|
|
|
|
|
|
Fix unsetting aliases in subshells
Aliases can now be correctly unset within subshell environments
(such as ( ... ), $(command substitutions), etc), as well as
non-subshell "shared" command substitutions (${ ...; }). Before,
attempts to unset aliases within these were silently ignored.
Prior discussion: https://github.com/att/ast/issues/108
Subshell alias trees are only referenced in a few places in the
code, *and* have always been broken, so this commit gets rid of the
whole notion of a subshell alias tree. Instead, there is now just
one flat alias tree, and subshells fork into a separate process
when aliases are set or unset within them. It is not really
conceivable that this could be a performance-sensitive operation,
or even a common one, so this is a clean fix with no downside.
src/cmd/ksh93/include/defs.h:
- Remove sh_subaliastree() definition.
src/cmd/ksh93/sh/subshell.c:
- Remove salias element (pointer to subshell alias tree) from
subshell struct.
- Remove sh_subaliastree() function.
- sh_subshell(): Remove alias subshell tree cleanup.
src/cmd/ksh93/bltins/typeset.c:
- b_alias(): If in subshell, fork before setting alias.
- b_unalias(): If in subshell, fork before unsetting alias.
- unall(): Remove sh_subaliastree() call.
src/cmd/ksh93/sh/name.c:
- nv_open(): Remove sh_subaliastree() call.
src/cmd/ksh93/tests/subshell.sh:
- Add regression tests for unsetting or redefining aliases within
subshells.
(cherry picked from commit 12a15605b9521a2564a6e657905705a060e89095)
2020-05-29 07:27:53 +00:00
|
|
|
- Fix a similar bug with aliases. These can now be correctly unset
|
|
|
|
|
in subshell environments.
|
|
|
|
|
|
2020-05-20 21:42:00 +00:00
|
|
|
2020-05-21:
|
|
|
|
|
|
|
|
|
|
- Fix truncating of files with the combined redirections '<>;file' and
|
|
|
|
|
'<#pattern'. The bug was caused by out-of-sync streams.
|
|
|
|
|
Details and discussion: https://github.com/att/ast/issues/61
|
|
|
|
|
|
2020-05-30 14:55:30 +00:00
|
|
|
- Patched code injection vulnerability CVE-2019-14868. As a result, you can
|
2020-05-21 12:27:51 +00:00
|
|
|
no longer use expressions in imported numeric environment variables; only
|
|
|
|
|
integer literals are allowed.
|
|
|
|
|
|
Fix bugs in testing if a parameter is set
This fixes three related bugs:
1. Expansions like ${var+set} remained static when used within a
'for', 'while' or 'until' loop; the expansions din't change
along with the state of the variable, so they could not be used
to check whether a variable is set within a loop if the state of
that variable changed in the course of the loop. (BUG_ISSETLOOP)
2. ${IFS+s} always yielded 's', and [[ -v IFS ]] always yielded
true, even if IFS is unset. (BUG_IFSISSET)
3. IFS was incorrectly exempt from '-u' ('-o nounset') checks.
src/cmd/ksh93/sh/macro.c: varsub():
- When getting a node pointer (np) to the parameter to test,
special-case IFS by checking if it has a value and not setting
the pointer if not. The node to IFS always exists, even after
'unset -v IFS', so before this fix it always followed the code
path for a parameter that is set. This fixes BUG_IFSISSET for
${IFS+s} and also fixes set -u (-o nounset) with IFS.
- Before using the 'nv_isnull' macro to check if a regular variable
is set, call nv_optimize() if needed. This fixes BUG_ISSETLOOP.
Idea from Kurtis Rader: https://github.com/att/ast/issues/1090
Of course this only works if SHOPT_OPTIMIZE==1 (the default),
but if not, then this bug is not triggered in the first place.
- Add some comments for future reference.
src/cmd/ksh93/bltins/test.c: test_unop():
- Fix BUG_IFSISSET for [[ -v IFS ]]. The nv_optimize() method
doesn't seem to have any effect here, so the only way that I can
figure out is to special-case IFS, nv_getval()'ing it to check if
IFS has a value in the current scope.
src/cmd/ksh93/tests/variables.sh:
- Add regression tests for checking if a varariable is set within a
loop, within and outside a function with that variable made local
(to check if the scope is honoured). Repeat these tests for a
regular variable and for IFS, for ${foo+set} and [[ -v foo ]].
(cherry picked from commit a2cf79cb98fa3e47eca85d9049d1d831636c9b16)
2020-05-20 14:50:43 +00:00
|
|
|
2020-05-20:
|
|
|
|
|
|
|
|
|
|
- Fix BUG_ISSETLOOP. Expansions like ${var+set} remained static when used
|
|
|
|
|
within a 'for', 'while' or 'until' loop; the expansions din't change along
|
|
|
|
|
with the state of the variable, so they could not be used to check whether a
|
|
|
|
|
variable is set within a loop if the state of that variable changed in the
|
|
|
|
|
course of the loop.
|
|
|
|
|
|
|
|
|
|
- Fix BUG_IFSISSET. ${IFS+s} always yielded 's', and [[ -v IFS ]] always
|
|
|
|
|
yielded true, even if IFS is unset. This applied to IFS only.
|
|
|
|
|
|
2020-05-19 14:49:56 +00:00
|
|
|
2020-05-19:
|
|
|
|
|
|
|
|
|
|
- Fix 'command -p'. The -p option causes the operating system's standard
|
|
|
|
|
utilities path (as output by 'getconf PATH') to be searched instead of $PATH.
|
|
|
|
|
Before this fix, this was broken on non-interactive shells as the internal
|
|
|
|
|
variable holding the default PATH value was not correctly initialised.
|
|
|
|
|
|
Fix 'test -t 1' in $(command substitutions)
Standard output (FD 1) tested as being on a terminal within a
command substitution, which makes no sense as the command
substitution is supposed to be catching standard output.
ksh -c 'v=$(echo begincomsub
[ -t 1 ] && echo oops
echo endcomsub)
echo "$v"'
This should not output "oops".
This is one of the many bugs with ksh93 virtual (non-forked)
subshells. On the abandoned Vashist/Rader ksh2020 branch, this bug
was fixed by changing quite a lot of code, which introduced and/or
exposed another bug:
https://github.com/att/ast/issues/1079
https://github.com/att/ast/commit/8e1e405e
https://github.com/att/ast/issues/1088
That issue was unresolved when the ksh2020 branch was abandoned.
The safer and more conservative fix is simply forcing the subshell
to fork if we're in a non-forked command substitution and testing
'-t 1'. It is hard to imagine a situation where this would cause a
noticable performance hit.
Note that this fix does not affect ksh93-specific "shared"
non-subshell ${ command substitutions; } which are executed in the
main shell environment, so that variables survive, etcetera.
'test -t 1' continues to wrongly return true there, but command
substitutions of that form cannot be forked because that would
defeat their purpose.
src/cmd/ksh93/bltins/test.c:
- Fix 'test -t 1', '[ -t 1 ]' and '[[ -t 1 ]]' by forking the
current subshell if it is a virtual/non-forked subshell
(shp->subshell), and a command substitution (shp->comsub), but
NOT a "shared" ${ command substitution; } (!shp->subshare).
src/cmd/ksh93/tests/bracket.sh:
- Add two regression tests for this issue, which were adapted from
the Vashist/Rader ksh2020 branch.
NEWS, src/cmd/ksh93/include/version.h:
- Update.
(cherry picked from commit b8ef05e457ead65b83417699b8dd8632f855e2fa)
2020-05-16 19:06:49 +00:00
|
|
|
2020-05-16:
|
|
|
|
|
|
|
|
|
|
- Fix 'test -t 1', '[ -t 1 ]', '[[ -t 1 ]]' in command substitutions.
|
|
|
|
|
Standard output (file descriptor 1) tested as being on a terminal within a
|
|
|
|
|
command substitution, which makes no sense as the command substitution is
|
|
|
|
|
supposed to be catching standard output.
|
|
|
|
|
v=$(echo begincomsub
|
|
|
|
|
[ -t 1 ] && echo oops
|
|
|
|
|
echo endcomsub)
|
|
|
|
|
echo "$v"
|
|
|
|
|
This now does not output "oops".
|
|
|
|
|
|
2020-05-14 10:36:16 +00:00
|
|
|
2020-05-14:
|
|
|
|
|
|
|
|
|
|
- Fix syncing history when print -s -f is used. For example, the
|
|
|
|
|
following now correctly adds a 'cd' command to the history:
|
|
|
|
|
print -s -f 'cd -- %q\n' "$PWD"
|
|
|
|
|
Ref.: https://github.com/att/ast/issues/425
|
|
|
|
|
https://github.com/att/ast/pull/442
|
|
|
|
|
|
2020-06-09 19:57:05 +00:00
|
|
|
- Fix BUG_PUTIOERR: Output builtins now correctly detect
|
2020-05-16 14:04:35 +00:00
|
|
|
input/output errors. This allows scripts to check for a nonzero exit
|
|
|
|
|
status on the 'print', 'printf' and 'echo' builtins and prevent possible
|
|
|
|
|
infinite loops if SIGPIPE is ignored.
|
|
|
|
|
|
2020-05-14 16:41:26 +00:00
|
|
|
- Add a convenient bin/run_ksh_tests script to the source tree that
|
|
|
|
|
sets up the necessary environment and runs the ksh regression tests.
|
|
|
|
|
|
Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
2020-05-13:
|
|
|
|
|
|
|
|
|
|
- Fix BUG_CASELIT: an undocumented 'case' pattern matching misbehaviour that
|
|
|
|
|
goes back to the original Bourne shell, but wasn't discovered until 2018.
|
|
|
|
|
If a pattern doesn't match as a pattern, it was tried again as a literal
|
|
|
|
|
string. This broke common validation use cases, e.g.:
|
|
|
|
|
n='[0-9]'
|
|
|
|
|
case $n in
|
|
|
|
|
( [0-9] ) echo "$n is a number" ;;
|
|
|
|
|
esac
|
|
|
|
|
would output "[0-9] is a number" as the literal string fallback matches the
|
|
|
|
|
pattern. As this misbehaviour was never documented anywhere (not for Bourne,
|
|
|
|
|
ksh88, or ksh93), and it was never replicated in other shells (not even in
|
|
|
|
|
ksh88 clones pdksh and mksh), it is unlikely any scripts rely on it.
|
|
|
|
|
Of course, a literal string fallback, should it be needed, is trivial to
|
|
|
|
|
implement correctly without this breakage:
|
|
|
|
|
case $n in
|
|
|
|
|
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
|
|
|
|
|
esac
|
2020-05-13 14:00:33 +00:00
|
|
|
Ref.: https://github.com/att/ast/issues/476
|
|
|
|
|
|
|
|
|
|
- Fix BUG_REDIRIO: ksh used to redirect standard output by default when no
|
|
|
|
|
file descriptor was specified with the rarely used '<>' reading/writing
|
|
|
|
|
redirection operator. It now redirects standard input by default, as POSIX
|
|
|
|
|
specifies and as all other POSIX shells do. To redirect standard output
|
|
|
|
|
for reading and writing, you now need '1<>'.
|
|
|
|
|
Ref.: https://github.com/att/ast/issues/75
|
|
|
|
|
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_07
|
