Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
This documents significant changes in the 93u+m branch of AT&T ksh93.
|
2020-06-14 04:28:38 +00:00
|
|
|
For full details, see the git log at: https://github.com/ksh93/ksh
|
Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
|
|
|
|
|
Any uppercase BUG_* names are modernish shell bug IDs.
|
2020-07-01 17:14:10 +00:00
|
|
|
|
2020-09-29 22:34:02 +00:00
|
|
|
2020-09-30:
|
|
|
|
|
|
|
|
|
|
- Fixed: 'typeset -xu' and 'typeset -xl' (export + change case) failed to
|
|
|
|
|
change the case of a variable's value in certain conditions.
|
|
|
|
|
|
Fix pipefail with (errexit or ERR trap) regression
ksh 93u+ introduced a regression in the combination of the
'set -o pipefail' and 'set -e'/'set -o errexit' options:
$ ksh93 -o errexit -o pipefail -c \
'(exit 3) | true; echo "still here despite $? status"'
still here despite 3 status
The bug is in how the the huge sh_exec() function in xec.c handles
the 'echeck' flag. Near the end of sh_exec(), this flag triggers a
sh_chktrap() call to check whether to trigger any traps, including
the ERR trap -- and that same function also handles the errexit
option, which is basically the same as 'trap "exit" ERR'.
We can learn more easily how sh_exec() works by inserting debug
warnings in all its 'switch(type&COMMSK)' cases, like:
case TCOM:
errormsg(SH_DICT,ERROR_warn(0),"[DEBUG] TCOM");
... and same for all the others. With that done, the output
of a very simple dummy pipeline looks as follows:
$ arch/*/bin/ksh -c 'true | true | true'
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TFIL
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TFORK
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TFORK
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TSETIO
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TCOM
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TCOM
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TCOM
So, it looks like sh_exec() handles this pipeline as follows:
TFIL
|_____TFORK
| |_____TCOM
|_____TFORK
| |_____TCOM
|_____TSETIO
|_____TCOM
Each time a pipeline like command1 | command2 | ... is executed,
sh_exec() is invoked with type TFIL; this then recursively invokes
sh_exec() to handle the individual elements. The last element of
the pipe triggers a sh_exec() run with type TSETIO; since it is run
in the current shell environment, it is effectively treated as a
command with an input redirection. All the previous elements are of
type TFORK instead, because they are executed asynchronously in
separate, forked subshell processes. Finally, the TFORK or TSETIO
code then recursively calls sh_exec() again with type TCOM to
actually execute the commands.
When reading the code, we find that the 'echeck' flag is set as
part of the TSETIO code. This makes sense of why only an error in
the last element of the pipe triggers the errexit/ERR trap action.
So that's the bug: the flag is set in the wrong place.
This can be fixed by setting that flag in the TFIL handling code
instead, as this is what calls everything else and collects all the
exit statuses. So the sh_chktrap() call is now executed after
handling the entire pipeline, at the TFIL recursion level.
This also allows getting rid of the special-casing in the buggy
TSETIO version. The SH_ERREXIT state is restored at the end of each
sh_exec() call, so since we're now doing this at a lower recursion
level, it will already have been restored.
src/cmd/ksh93/sh/xec.c: sh_exec():
- Fix the bug as per the above.
src/cmd/ksh93/tests/options.sh:
- Add tests for errexit and ERR trap combined with pipefail.
src/cmd/ksh93/tests/basic.sh:
- Tweak a couple of tests that reported a trap wasn't triggered
even if it was actually triggered more than once.
Fixes: https://github.com/ksh93/ksh/issues/121
Thanks to Stéphane Chazelas for the bug report.
2020-09-30 14:13:34 +00:00
|
|
|
- A ksh 93u+ regression was fixed in the combination of ERR trap handling and
|
|
|
|
|
the 'pipefail' option. A pipeline now triggers the ERR trap correctly again
|
|
|
|
|
if the 'pipefail' option is active and any of the pipeline elements return a
|
|
|
|
|
nonzero exit status. Similarly, if both the 'errexit' and 'pipefail' options
|
|
|
|
|
are active, ksh now correctly exits if any pipeline element returns nonzero.
|
|
|
|
|
|
2020-10-01 04:13:00 +00:00
|
|
|
- Autoloading a function no longer causes the calling script's $LINENO to be
|
|
|
|
|
off by the number of lines in the function definition file that was loaded.
|
|
|
|
|
This also corrects line numbers in warnings and error messages.
|
|
|
|
|
|
Fix signal/trap behaviour in ksh functions (rhbz#1454804)
Prior discussion:
https://bugzilla.redhat.com/1454804
On 2017-05-23 13:33:25 UTC, Paulo Andrade wrote:
> In previous ksh versions, when exiting the scope of a ksh
> (not posix) function, it would restore the trap table of
> the "calling context" and if the reason the function exited
> was a signal, it would call sh_fault() passing as argument
> the signal value.
> Newer ksh checks it, but calls kill(getpid(), signal_number)
> after restoring the trap table, but only calls for SIGINT and
> SIGQUIT.
[...]
> The old way appears to have been more appropriate, but there
> must be a reason to only pass SIGINT and SIGQUIT as it is an
> explicit patch.
The last paragraph is where I differ. This would not be the first
example of outright breakage that appeared to be added deliberately
and that 93u+m has fixed or removed, see e.g. 8477d2ce ('printf %H'
had code that deleted all multibyte characters), cefe087d, or
781f0a39. Sometimes it seems the developers added a little
experiment and then forgot all about it, so it became a misfeature.
In this instance, the correct pre-2012 ksh behaviour is still
explicitly documented in (k)sh.1: "A trap condition that is not
caught or ignored by the function causes the function to terminate
and the condition to be passed on to the caller". Meaning, if there
is no function-local trap, the signal defaults to the parent scope.
There is no language that limits this to SIGINT and SIGQUIT only.
It also makes no sense at all to do so -- signals such as SIGPIPE,
SIGTERM, or SIGSEGV need to be caught by default and to do
otherwise results in misbehaviour by default.
src/cmd/ksh93/sh/xec.c: sh_funscope():
- When resending a signal after restoring the global traps state,
remove the spurious check that limits this to SIGINT and SIGQUIT.
- Replace it with a check for nsig!=0, as that means there were
parent trap states to restore. Otherwise 'kill' may be called
with an invalid signal argument, causing a crash on macOS.
src/cmd/ksh93/tests/signal.sh:
- Update a test to check that a function-local SIGTERM trap is
triggered correctly when signalled from another process.
- Complete the tests for 3aee10d7; this bug needed fixing before
we could test that previous fix in a ksh function scope.
- Add a test for triggering global traps from ksh functions,
testing multiple POSIX-standard signals.
2020-09-29 00:28:08 +00:00
|
|
|
2020-09-28:
|
|
|
|
|
|
|
|
|
|
- While executing a ksh-style function, ksh 93u+ ignored all signals for which
|
|
|
|
|
the function had not set a local trap, except for SIGINT and SIGQUIT. This
|
|
|
|
|
was contrary to the manual, which states that a "trap condition that is not
|
|
|
|
|
caught or ignored by the function causes the function to terminate and the
|
|
|
|
|
condition to be passed on to the caller". This has now been fixed in 93u+m to
|
|
|
|
|
match the documentation, so that e.g. global traps work as expected again.
|
|
|
|
|
|
2020-09-27 19:26:09 +00:00
|
|
|
2020-09-27:
|
|
|
|
|
|
2020-09-28 02:47:53 +00:00
|
|
|
- The shell's lexical analysis of a 'case' statement within a do...done block
|
2020-09-27 19:26:09 +00:00
|
|
|
within a command substitution of the form $(...) has been fixed so that code
|
|
|
|
|
like the following no longer throws a spurious syntax error:
|
|
|
|
|
x=$(for i in 1; do case $i in word) true;; esac; done)
|
|
|
|
|
Previously, this required a leading parenthesis before 'word', although the
|
|
|
|
|
syntax error claimed that the ';;' was unexpected.
|
|
|
|
|
|
2020-09-26 18:26:18 +00:00
|
|
|
2020-09-26:
|
|
|
|
|
|
|
|
|
|
- 'whence -f' now completely ignores the existence of functions, as documented.
|
|
|
|
|
|
2020-09-26 18:28:55 +00:00
|
|
|
- ksh now does not import environment variables whose names are not valid in
|
|
|
|
|
the shell language, as it would be impossible to change or unset them.
|
|
|
|
|
However, they stay in the environment to be passed to child processes.
|
|
|
|
|
|
Fix argv rewrite on invoking hashbangless script (rhbz#1047506)
The fixargs() function is invoked when ksh needs to run a script
without a #!/hashbang/path. Instead of letting the kernel invoke a
shell, ksh exfile()s the script itself from sh_main(). In the
forked child, it calls fixargs() to set the argument list in the
environment to the args of the new script, so that 'ps' and
/proc/PID/cmdline show the expected output.
But fixargs() is broken because, on systems other than HP-UX (on
which ksh uses pstat(2)), ksh simply inserts a terminating zero.
The arguments list is not a zero-terminated C string. Unix systems
expect the entire arguments buffer to be zeroed out, otherwise 'ps'
and /proc/*/cmdline will have fragments of previous command lines
in the output.
The Red Hat patch for this bug is:
https://src.fedoraproject.org/rpms/ksh/blob/642af4d6/f/ksh-20120801-argvfix.patch
However, that fix is incomplete because 'command_len' was also
hardcoded to be limited to 64 characters (!), which still gave
invalid 'ps' output if the erased command line was longer.
src/cmd/ksh93/sh/main.c: fixargs():
- Remove CMD_LENGTH macro which was defined as 64.
- Remove code that limited the erasure of the arguments buffer to
CMD_LENGTH characters. That code also had quite a dodgy strdup()
call -- it copies arguments to the heap, but they are never freed
(or even used), so it's a memory leak. Also, none of this is
ever done if the length is calculated using pstat(2) on HP-UX,
which is a clear indication that it's unnecessary.
(I think this code block must have been some experiment they
forgot to remove. One reason why I think so is that a 64 byte
arguments limit never made sense, even in the 1980s when they
wrote ksh on 80-column CRT displays. Another indication of this
is that fixing it didn't require adding anything; the code to do
the right thing was already there, it was just being overridden.)
- Zero out the full arguments length as in the Red Hat patch.
src/cmd/ksh93/tests/basic.sh:
- Add test. It's sort of involved because 'ps' is one of the least
portable commands in practice, in spite of standardisation.
2020-09-25 06:19:43 +00:00
|
|
|
2020-09-25:
|
|
|
|
|
|
whence -v/-a: report path to autoloadable functions
Since at least 1999, whence -v on pdksh (and its successor mksh)
reports the path where an autoloadable function may be found:
$ mkdir ~/fun; FPATH=~/fun
$ echo 'myfn() { echo hi; }' >~/fun/myfn
$ whence -v myfn
myfn is a undefined (autoload from /home/user/fun/myfn) function
Whereas ksh93 only reports, rather uselessly:
myfn is an undefined function
As of this commit, whence -v/-a on ksh 93u+m does the same as
pdksh, but with correct grammar:
myfn is an undefined function (autoload from /home/user/fun/myfn)
This may be a small violation of my own "no new features" policy
for 93u+m, but I couldn't resist. This omission has been annoying
me, and it's just embarrassing to lack a pdksh feature :)
src/cmd/ksh93/include/path.h,
src/cmd/ksh93/data/msg.c:
- Add e_autoloadfrom[] = " (autoload from %s)" message.
src/cmd/ksh93/bltins/whence.c: whence():
- Report the path (if any) when reporting an undefined function.
This needs to be done in two places:
1. When a function has been explicitly marked undefined with
'autoload', we need to do a quick path_search() loop to find
the path. (These undefined functions take precedence over
regular commands, so are reported first.)
2. When a function is not explicitly autoloaded but merely
available in $FPATH, that path search was already done, so all
we need to do is report it. (These are reported last.)
Note that the output remains as on 93u+ if no function definition
file is found on $FPATH. This is also like pdksh/mksh.
src/cmd/ksh93/data/builtins.c:
- Bump 'whence' version date. The inline docs never detailed very
exactly what 'whence -v' reports, so no need for further edits.
src/cmd/ksh93/tests/path.sh:
- Regress-test the new whence behaviour plus actual autoloading,
including the command override behaviour of autoloaded functions.
2020-09-25 13:39:08 +00:00
|
|
|
- whence -v/-a now reports the path to the file that an "undefined" (i.e.
|
|
|
|
|
autoloadable) function will be loaded from when invoked, if found in $FPATH.
|
|
|
|
|
|
Fix argv rewrite on invoking hashbangless script (rhbz#1047506)
The fixargs() function is invoked when ksh needs to run a script
without a #!/hashbang/path. Instead of letting the kernel invoke a
shell, ksh exfile()s the script itself from sh_main(). In the
forked child, it calls fixargs() to set the argument list in the
environment to the args of the new script, so that 'ps' and
/proc/PID/cmdline show the expected output.
But fixargs() is broken because, on systems other than HP-UX (on
which ksh uses pstat(2)), ksh simply inserts a terminating zero.
The arguments list is not a zero-terminated C string. Unix systems
expect the entire arguments buffer to be zeroed out, otherwise 'ps'
and /proc/*/cmdline will have fragments of previous command lines
in the output.
The Red Hat patch for this bug is:
https://src.fedoraproject.org/rpms/ksh/blob/642af4d6/f/ksh-20120801-argvfix.patch
However, that fix is incomplete because 'command_len' was also
hardcoded to be limited to 64 characters (!), which still gave
invalid 'ps' output if the erased command line was longer.
src/cmd/ksh93/sh/main.c: fixargs():
- Remove CMD_LENGTH macro which was defined as 64.
- Remove code that limited the erasure of the arguments buffer to
CMD_LENGTH characters. That code also had quite a dodgy strdup()
call -- it copies arguments to the heap, but they are never freed
(or even used), so it's a memory leak. Also, none of this is
ever done if the length is calculated using pstat(2) on HP-UX,
which is a clear indication that it's unnecessary.
(I think this code block must have been some experiment they
forgot to remove. One reason why I think so is that a 64 byte
arguments limit never made sense, even in the 1980s when they
wrote ksh on 80-column CRT displays. Another indication of this
is that fixing it didn't require adding anything; the code to do
the right thing was already there, it was just being overridden.)
- Zero out the full arguments length as in the Red Hat patch.
src/cmd/ksh93/tests/basic.sh:
- Add test. It's sort of involved because 'ps' is one of the least
portable commands in practice, in spite of standardisation.
2020-09-25 06:19:43 +00:00
|
|
|
- When ksh invoked a shell script that does not have a leading
|
|
|
|
|
#!/hashbang/path, 'ps' and /proc/<PID>/cmdline showed corrupted output if
|
|
|
|
|
the new script's command line was shorter than that of the invoking script.
|
|
|
|
|
This has been fixed by wiping the arguments buffer correctly.
|
|
|
|
|
|
2020-09-24 05:36:44 +00:00
|
|
|
2020-09-24:
|
|
|
|
|
|
|
|
|
|
- An omission made it impossible to turn off brace expansion within command
|
|
|
|
|
substitutions (`...`, $(...) or ${ ...; }) as the code for parsing these
|
|
|
|
|
did not check the -B/braceexpand option. This check has now been added.
|
|
|
|
|
|
2020-09-22 23:56:09 +00:00
|
|
|
2020-09-23:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash that could occur when running a pipeline containing
|
|
|
|
|
backtick-style command substitutions with job control enabled.
|
|
|
|
|
|
Fix typeset -l/-u crash on special vars (rhbz#1083713)
When using typeset -l or -u on a variable that cannot be changed
when the shell is in restricted mode, ksh crashed.
This fixed is inspired by this Red Hat fix, which is incomplete:
https://src.fedoraproject.org/rpms/ksh/blob/642af4d6/f/ksh-20120801-tpstl.patch
The crash was caused by the nv_shell() function. It walks though a
discipline function tree to get the pointer to the interpreter
associated with it. Evidently, the problem is that some pointer in
that walk is not set correctly for all special variables.
Thing is, ksh only has one shell language interpreter, and only one
global data structure (called 'sh') to keep its main state[*]. Yet,
the code is full of 'shp' pointers to that structure. Most (not
all) functions pass that pointer around to each other, accessing
that struct indirectly, ostensibly to account for the non-existent
possibility that there might be more than one interpreter state.
The "why" of that is an interesting cause for speculation that I
may get to sometime. For now, it is enough to know that, in the
code as it is, it matters not one iota what pointer to the shell
interpreter state is used; they all point to the same thing (unless
it's broken, as in this bug).
So, rather than fixing nv_shell() and/or associated pointer
assignments, this commit simply removes it, and replaces it with
calls to sh_getinterp(), which always returns a pointer to sh (see
init.c, where that function is defined as literally 'return &sh').
[*] Defined in shell.h, with the _SH_PRIVATE part in defs.h
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/name.c:
- Remove nv_shell().
src/cmd/ksh93/sh/init.c:
- In all the discipline functions for special variables, initialise
shp using sh_getinterp() instead of nv_shell().
src/cmd/ksh93/tests/variables.sh:
- Add regression test for typeset -l/-u on all special variables.
2020-09-23 17:15:45 +00:00
|
|
|
- Fixed a crash that occurred when using 'typeset -u' or 'typeset -l' on a
|
|
|
|
|
special variable such as PATH, ENV or SHELL.
|
|
|
|
|
|
2020-09-21 23:42:15 +00:00
|
|
|
2020-09-21:
|
|
|
|
|
|
|
|
|
|
- A bug was fixed that caused command substitutions embedded in here-documents
|
|
|
|
|
to lose the output of the commands they ran. This bug occurred when ksh was
|
|
|
|
|
compiled with the SHOPT_SPAWN compile-time option.
|
|
|
|
|
|
2020-09-22 00:45:59 +00:00
|
|
|
- Bugfix: var=$(< file) now reads the file even if the standard inout, standard
|
|
|
|
|
output and/or standard error file descriptors are closed.
|
|
|
|
|
|
Multiple 'whence' and path search fixes
Hopefully this doesn't introduce new bugs, but it does fix at
least the following:
1. When whence -v/-a found an "undefined" (i.e. autoloadable)
function in $FPATH, it actually loaded the function as a side
effect of reporting on its existence (!). Now it only reports.
2. 'whence' will now canonicalise paths properly. Examples:
$ whence ///usr/lib/../bin//./env
/usr/bin/env
$ (cd /; whence -v dev/../usr/bin//./env)
dev/../usr/bin//./env is /usr/bin/env
3. 'whence' no longer prefixes a spurious double slash when doing
something like 'cd / && whence bin/echo'. On Cygwin, an initial
double slash denotes a network server, so this was not just a
cosmetic problem.
4. 'whence -a' now reports a "tracked alias" (a.k.a. hash table
entry, i.e. cached $PATH search) even if an actual alias by the
same name exists. This needed fixing because in fact the hash
table entry continues to be used when bypassing the alias.
Aliases and "tracked aliases" are not remotely the same thing;
confusing nomenclature is not a reason to report wrong results.
5. When using 'hash' or 'alias -t' on a command that is also a
builtin to force caching a $PATH search for the external
command, 'whence -a' double-reported the path:
$ hash printf; whence -a printf
printf is a shell builtin
printf is /usr/bin/printf
printf is a tracked alias for /usr/bin/printf
This is now fixed so that the second output line is gone.
Plus, if there were multiple versions of the command on $PATH,
the tracked alias was reported at the end, which is the wrong
order. This is also fixed.
src/cmd/ksh93/bltins/whence.c: whence():
- Refactor the do...while loop that handles whence -v/-a for path
searches in such a way that the code actually makes sense and
stops looking like higher esotericism. Just doing this fixed #2,
#4 and #5 above (the latter two before I even noticed them). For
instance, the path_fullname() call to canonicalise paths was
already there; it was just never used.
- Remove broken 'notrack' flaggery for deciding whether to report a
hash table entry a.k.a. "tracked alias"; instead, check the hash
table (shp->track_tree).
src/cmd/ksh93/sh/path.c:
- path_search(): Re #3: When prefixing the PWD, first check if
we're in '/' and if so, don't prefix it; otherwise, adding the
next slash causes an initial double slash. (Since '/' is the only
valid single-character absolute path, all we need to do is check
if the second character pwd[1] is non-null.)
- path_search(): Re #1: Stop autoloading when called by 'whence':
* The 'flag==2' check to avoid autoloading a function was
broken. The flag value is 2 on the first whence() loop
iteration, but 3 on subsequent ones. Change to 'flag >= 2'.
* However, this only fixes it if the function file does not have
the x permission bit, as executable files are handled by
path_absolute() which unconditionally autoloads functions!
So, pass on our flag parameter when callling path_absolute().
- path_absolute(): Re #1: Add flag parameter. Do not autoload
functions if flag >= 2.
src/cmd/ksh93/include/path.h,
src/cmd/ksh93/bltins/typeset.c,
src/cmd/ksh93/sh/main.c,
src/cmd/ksh93/sh/xec.c:
- Re #1: Update path_absolute() calls, adding a 0 flag parameter.
src/cmd/ksh93/include/name.h:
- Remove now-unused pathcomp member from union Value. It was
introduced in 99065353 to allow examining the value of a tracked
alias. This commit uses nv_getval() instead.
src/cmd/ksh93/tests/builtins.sh,
src/cmd/ksh93/tests/path.sh:
- Add and tweak various related tests.
Fixes: https://github.com/ksh93/ksh/issues/84
2020-09-20 05:56:09 +00:00
|
|
|
2020-09-20:
|
|
|
|
|
|
|
|
|
|
- Bugfix: when whence -v/-a found an "undefined" (i.e. autoloadable) function
|
|
|
|
|
in $FPATH, it actually loaded the function as a side effect of reporting on
|
|
|
|
|
its existence. Now it only reports, as documented.
|
|
|
|
|
|
|
|
|
|
- 'whence' will now canonicalise paths properly, resolving '.' and '..'
|
|
|
|
|
elements in paths given to it. It also no longer prefixes a spurious
|
|
|
|
|
double slash when doing something like 'cd / && whence bin/echo'.
|
|
|
|
|
|
2020-09-18 18:32:34 +00:00
|
|
|
2020-09-18:
|
|
|
|
|
|
|
|
|
|
- Setting the 'posix' option now turns off the 'braceexpand' option, as brace
|
|
|
|
|
expansion is not specified by POSIX and potentially incompatible with sh
|
|
|
|
|
scripts. In addition, 'set -o posix' now turns on the 'letoctal' option
|
|
|
|
|
instead of controlling that behaviour directly. 'set +o posix' does the
|
|
|
|
|
reverse of these.
|
|
|
|
|
|
emacs, vi: Support repeat parameters to VT220 keys (re: f2a3f4e3)
In the vi and emacs line editors, repeat count parameters can now
also be used for the arrow keys and the forward-delete key. E.g.,
in emacs mode, <ESC> 7 <left-arrow> will now move the cursor seven
positions to the left. In vi control mode, this would be entered
as: 7 <left-arrow>.
src/cmd/ksh93/edit/emacs.c:
- ed_emacsread(): Upon getting ^[ (ESC), save current repeat count
in a new variable; restore and reset it upon the next character.
- escape(): Minor bugfix: when processing a ^[[x sequence where 'x'
is a character other than '~' (which would be DEL), also reinsert
the final character into the buffer so scripts can detect them.
src/cmd/ksh93/edit/vi.c:
- cntlmode(): Do not reset the repeat count if the command is '[',
the character following ESC in VT220 escape sequences.
- mvcursor():
* Do not use getcount() to get the character following '[', as
that was parsing repetition parameters in the wrong place.
There wouldn't be any, so this would reset the repeat count.
* After that, no more need for the special-casing of ^[[3~ (DEL)
introduced in f2a3f4e3. Move it to within the 'switch' block.
* When handling left and right arrows and Home and End keys, do
not modify cursor directly but ed_ungetchar() the corresponding
traditional command keys as with the rest. Otherwise a repeat
count parameter would now wrongly survive those keys.
src/cmd/ksh93/sh.1:
- Document control character notation used for vi mode docs.
- Since vi control mode beeps and aborts on ESC except if a
subsequent [ is already in the input buffer upon receiving ESC,
document that VT220 escape sequences only preserve repeat counts
when entered into the input buffer all at once.
- Don't skip the initial ESC in the documentation of the VT220
escape sequences. In control mode, skipping the initial ESC still
works as before, but that is now undocumented, as it's really
nothing more than an artefact of VT220 escape processing.
- Move the two long paragraphs on '-o viraw' and canonical (i.e.
line-based) input processing from the vi editor introduction to
the options section under 'viraw'. It is much too arcane for the
intro, and besides, ksh 93u+ (and hence also 93u+m) has
SHOPT_VIRAW enabled by default, so the shell is compiled to force
this option on at all times, making it even less relevant for
most users.
2020-09-17 15:28:03 +00:00
|
|
|
2020-09-17:
|
|
|
|
|
|
|
|
|
|
- In the vi and emacs line editors, repeat count parameters can now also be
|
|
|
|
|
used for the arrow keys and the forward-delete key. E.g., in emacs mode,
|
|
|
|
|
<ESC> 7 <left-arrow> will now move the cursor seven positions to the left.
|
|
|
|
|
In vi control mode, this would be entered as: 7 <left-arrow>.
|
|
|
|
|
|
2020-09-18 01:03:39 +00:00
|
|
|
- When a background job on an interactive shell received SIGINT or SIGPIPE, the
|
|
|
|
|
job termination message was empty. It now shows "Interrupt" or "Broken Pipe".
|
|
|
|
|
|
A few job control (-m, -o monitor) fixes (rhbz#960034)
This patch from Red Hat fixes the following:
1. ksh was ignoring the -m (-o monitor) option when specified on
the invocation command line.
2. Scripts did not properly terminate their background processes
on Ctrl+C if the -m option was turned off. Reproducer:
xterm &
read junk
When run as a script without turning on -m, pressing Ctrl+C
should terminate the xterm, and now does.
3. Scripts no longer attempt to set the terminal foreground process
group ID, as only interactive shells should be doing that.
This makes some progress on https://github.com/ksh93/ksh/issues/119
but we're a long way from fixing all of that.
src/cmd/ksh93/sh/main.c: exfile():
- On non-interactive shells, do not turn off the monitor option.
Instead, if it was turned on, turn on the SH_MONITOR state flag.
src/cmd/ksh93/edit/edit.c: ed_getchar():
- On Ctrl+C, issue SIGINT to the current process group using
killpg(2) instead of going via sh_fault(), which handles a
signal only for the current shell process.
src/cmd/ksh93/sh/jobs.c: job_reap(), job_reset(),
src/cmd/ksh93/sh/xec.c: sh_exec():
- Only attempt to set the terminal foreground process group ID
using tcsetpgrp(3) if the shell is interactive.
Original patch: https://src.fedoraproject.org/rpms/ksh/blob/642af4d6/f/ksh-20120801-kshmfix.patch
This was applied to Red Hat's ksh 93u+ on 8 July 2013.
2020-09-18 02:42:27 +00:00
|
|
|
- The -m (-o monitor) option is no longer ignored when specified on the shell
|
|
|
|
|
invocation command line.
|
|
|
|
|
|
|
|
|
|
- A script that is interrupted with Ctrl+C now terminates its background jobs
|
|
|
|
|
as expected, unless the -m (-o monitor) option was turned on.
|
|
|
|
|
|
2020-09-14 10:15:05 +00:00
|
|
|
2020-09-14:
|
|
|
|
|
|
|
|
|
|
- Corrected rounding of floating point values by ksh's printf %f formatting
|
|
|
|
|
operator. Fix contributed by @hyenias.
|
|
|
|
|
|
Handle forward-delete key in emacs and vi editors
On every modern system, the forward-delete key on PC/Mac keyboards
generates the VT220 sequence ESC [ 3 ~. Every other shell with an
editor handles this now, ksh93 seems to be the last not to.
src/cmd/ksh93/edit/emacs.c: escape():
- Handle the ^[[3 as part of normal escape processing, then read an
extra character to check for the final '~'. If detected, insert
an ERASECHAR key event.
src/cmd/ksh93/edit/vi.c: mvcursor():
- Replace the ^[[3~ sequence by an 'x' command. We have to
special-case its processing, because vi mode parses numbers as
repetition operators. The escape sequence contains a number,
making it incompatible with normal command handling. This means
number repetitions don't work with the forward-delete key. If
that annoys anyone enough to fix it, a patch would be welcome.
For now, it will do to make the forward-delete key stop
exhibiting bizarre behaviour (beep + change case + move forward).
src/cmd/ksh93/sh.1
- Copy-edit emacs documentation for VT220-style sequences; map them
to their actual key, otherwise it's meaningless to the reader.
- Document the new forward-delete key behaviour for emacs mode.
- Leave the forward-delete key for vi mode undocumented for now, as
repetitions don't work, so it doesn't really match the vi canon.
(OTOH, it doesn't work in vim, either...)
2020-09-14 22:49:05 +00:00
|
|
|
- The forward-delete key now works as expected in emacs and vi editing modes.
|
|
|
|
|
|
Fix 'command' expansion bug and POSIX compliance
The 'command' name can now result from an expansion, e.g.:
c=command; "$c" ls
set -- command ls; "$@"
both work now. This fixes BUG_CMDEXPAN.
If -o posix is on, 'command' now disables not only the "special"
but also the "declaration" properties of builtin commands that it
invokes. This is because POSIX specifies 'command' as a simple
regular builtin, and any command name following 'command' is just
an argument to the 'command' command, so there is nothing that
allows any further arguments (such as assignment-arguments) to be
treated specially by the parser. So, if and only if -o posix is on:
a. Arguments that start with a variable name followed by '=' are
always treated as regular words subject to normal shell syntax.
b. Since assignment-arguments are not processed as assignments
before the command itself, 'command' can now stop the shell from
exiting (as required by the standard) if a command that it
invokes (such as 'export') tries to modify a readonly variable.
This fixes BUG_CMDSPEXIT.
Most of 'command' is integrated in the parser and parse tree
executer, so that is where it needed fixing.
src/cmd/ksh93/sh/parse.c: simple():
- If the posix option is on, do not skip past SYSCOMMAND so that
any declaration builtin commands that are arguments to 'command'
are not detected and thus not treated specially at parsetime.
src/cmd/ksh93/sh/xec.c: sh_exec():
- When detecting SYSCOMMAND in order to skip past it, not only
compare the Namval_t pointer 'np' to SYSCOMMAND, but also handle
the case where that pointer is NULL, as when the command name
results from an expansion. In that case, search the function tree
shp->fun_tree for the name and see if that yields the SYSCOMMAND
pointer. fun_tree is initialised with a dtview to bltin_tree, so
searching fun_tree instead allows for overriding 'command' with a
shell function (which the POSIX standard requires us to allow).
src/cmd/ksh93/sh.1,
src/cmd/ksh93/data/builtins.c:
- Update documentation to match these changes.
- Various related edits and improvements.
src/cmd/ksh93/tests/builtins.sh:
- Check that 'command' works if resulting from an expansion.
- Check that 'command' can be overridden by a shell function.
2020-09-11 07:33:29 +00:00
|
|
|
2020-09-11:
|
|
|
|
|
|
|
|
|
|
- The 'command' regular builtin utility (which runs a simple command, removing
|
|
|
|
|
special properties) has been made fully POSIX compliant.
|
|
|
|
|
1. The 'command' name can now result from an expansion (fixing BUG_CMDEXPAN),
|
|
|
|
|
e.g. 'c=command; "$c" ls' and 'set -- command ls; "$@"' now work.
|
|
|
|
|
2. If and only if the POSIX mode (the new -o posix shell option) is active,
|
|
|
|
|
then the 'command' utility now disables not only "special" but also
|
|
|
|
|
"declaration" properties of builtin commands that it invokes, meaning:
|
|
|
|
|
a. arguments that start with a variable name followed by '=' are
|
|
|
|
|
always treated as regular words subject to normal shell syntax;
|
|
|
|
|
b. 'command' can now stop the shell from exiting if a command that it
|
|
|
|
|
invokes tries to modify a readonly variable (fixing BUG_CMDSPEXIT).
|
|
|
|
|
|
Reinstate 'r' and 'history' as preset aliases for interactive ksh
Following a community discussion, it became clear that 'r' is
particularly problematic as a regular builtin, as the name can and
does conflict with at least one legit external command by that
name. There was a consensus against removing it altogether and
letting users set the alias in their login scripts. However,
aliases are easier to bypass, remove or rename than builtins are.
My compromise is to reinstate 'r' as a preset alias on interactive
shells only, along with 'history', as was done in 17f81ebe before
they were converted to builtins in 03224ae3. So this reintroduces
the notion of predefined aliases to ksh 93u+m, but only for
interactive shells that are not initialised in POSIX mode.
src/cmd/ksh93/Makefile,
src/cmd/ksh93/Mamfile,
src/cmd/ksh93/include/shtable.h,
src/cmd/ksh93/data/aliases.c:
- Restore aliases.c containing shtab_aliases[], a table specifying
the preset aliases.
src/cmd/ksh93/include/shtable.h,
src/cmd/ksh93/sh/init.c:
- Rename inittree() to sh_inittree() and make it extern, because we
need to use it in main.c (sh_main()).
src/cmd/ksh93/sh/main.c: sh_main():
- Init preset aliases from shtab_aliases[] only if the shell is
interactive and not in POSIX mode.
src/cmd/ksh93/bltins/typeset.c,
src/cmd/ksh93/tests/alias.sh:
- unall(): When unsetting an alias, pass on the NV_NOFREE attribute
to nv_delete() to avoid an erroneous attempt to free a preset
alias from read-only memory. See: 5d50f825
src/cmd/ksh93/data/builtins.c:
- Remove "history" and "r" entries from shtab_builtins[].
- Revert changes to inline fc/hist docs in sh_opthist[].
src/cmd/ksh93/bltins/hist.c: b_hist():
- Remove handling for 'history' and 'r' as builtins.
src/cmd/ksh93/sh.1:
- Update accordingly.
Resolves: https://github.com/ksh93/ksh/issues/125
2020-09-11 18:51:35 +00:00
|
|
|
- The 'history' (== 'hist -l') and 'r' (== 'hist -s') interactive shell
|
|
|
|
|
history commands have reverted to preset aliases and are now only loaded if
|
|
|
|
|
the shell is interactive and not initialised in POSIX mode. This avoids
|
|
|
|
|
unneeded conflicts with external commands by these names, particularly 'r'.
|
|
|
|
|
|
Fix BUG_LOOPRET2 and related return/exit misbehaviour
The 'exit' and 'return' commands without an argument failed to pass
down the exit status of the last-run command when incorporated in a
block with redirection, &&/|| list, 'case' statement, or 'while',
'until' or 'for' loop.
src/cmd/ksh93/bltins/cflow.c:
- Use $?, which is sh.savexit a.k.a. shp->savexit, as the default
exit status value if there is no argument, instead of
shp->oldexit. This fixes the default exit status behaviour to
match POSIX and other shells.
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/include/shell.h:
- Remove now-unused sh.oldexit (a.k.a. shp->oldexit) private struct
member. It appeared to fulfill the same function as sh.savexit,
but in a slightly broken way.
- Move the savexit/$? declaration from the _SH_PRIVATE part of the
struct definition to the public API part. Since $? uses this,
it's clearly a publicly exposed value already, and this is
generally the one to use. (If anything, it's exitval that should
have been private.) This declares savexit right next to exitval,
rewriting the comments to clarify the difference between them.
src/cmd/ksh93/sh/fault.c,
src/cmd/ksh93/sh/subshell.c,
src/cmd/ksh93/sh/xec.c:
- Remove assignments to shp->oldexit.
src/cmd/ksh93/tests/basic.sh:
- Add thorough regression tests for the default exit status
behaviour of 'return' and 'exit' in various lexical contexts.
- Verify that 'for' and 'case' without any command, as well as a
lone redirection, still correctly reset the exit status to 0.
Fixes: #117
2020-09-09 18:02:20 +00:00
|
|
|
2020-09-09:
|
|
|
|
|
|
|
|
|
|
- Fixed BUG_LOOPRET2 and related bugs. The 'exit' and 'return' commands without
|
|
|
|
|
an argument now correctly default to passing down the exit status of the
|
|
|
|
|
last-run command. Tests like the following, in which the last-run command is
|
|
|
|
|
'false', now correctly output 1 instead of 0:
|
|
|
|
|
fn() { return || true; }; false; fn; echo "$?"
|
|
|
|
|
fn() { while return; do true; done; }; false; fn; echo "$?"
|
|
|
|
|
fn() { for i in 1; do return; done; }; false; fn; echo "$?"
|
|
|
|
|
fn() { case 1 in 1) return ;; esac; }; false; fn; echo "$?"
|
|
|
|
|
fn() { { return; } 2>&1; }; false; fn; echo "$?"
|
|
|
|
|
|
2020-09-05 14:20:22 +00:00
|
|
|
2020-09-05:
|
|
|
|
|
|
|
|
|
|
- Fixed erroneous syntax errors in parameter expansions such as ${var:-wor)d}
|
|
|
|
|
or ${var+w(ord}. The parentheses now correctly lose their normal grammatical
|
|
|
|
|
meaning within the braces. Fix by Eric Scrivner backported from ksh2020.
|
|
|
|
|
|
2020-09-04 03:29:52 +00:00
|
|
|
2020-09-04:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused a syntax error to be thrown if the special parameter
|
|
|
|
|
expansions ${!} and ${$} (including braces) were used within a here-document.
|
|
|
|
|
Bug reported by @Saikiran-m on GitHub.
|
|
|
|
|
|
Remove SHOPT_BASH; keep &> redir operator, '-o posix' option
On 16 June there was a call for volunteers to fix the bash
compatibility mode; it has never successfully compiled in 93u+.
Since no one showed up, it is now removed due to lack of interest.
A couple of things are kept, which are now globally enabled:
1. The &>file redirection shorthand (for >file 2>&1). As a matter
of fact, ksh93 already supported this natively, but only while
running rc/profile/login scripts, and it issued a warning. This
makse it globally available and removes the warning, bringing
ksh93 in line with mksh, bash and zsh.
2. The '-o posix' standard compliance option. It is now enabled on
startup if ksh is invoked as 'sh' or if the POSIXLY_CORRECT
variable exists in the environment. To begin with, it disables
the aforementioned &> redirection shorthand. Further compliance
tweaks will be added in subsequent commits. The differences will
be fairly minimal as ksh93 is mostly compliant already.
In all changed files, code was removed that was compiled (more
precisely, failed to compile/link) if the SHOPT_BASH preprocessor
identifier was defined. Below are other changes worth mentioning:
src/cmd/ksh93/sh/bash.c,
src/cmd/ksh93/data/bash_pre_rc.sh:
- Removed.
src/cmd/ksh93/data/lexstates.c,
src/cmd/ksh93/include/shlex.h,
src/cmd/ksh93/sh/lex.c:
- Globally enable &> redirection operator if SH_POSIX not active.
- Remove warning that was issued when &> was used in rc scripts.
src/cmd/ksh93/data/options.c,
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/args.c:
- Keep SH_POSIX option (-o posix).
- Replace SH_TYPE_BASH shell type by SH_TYPE_POSIX.
src/cmd/ksh93/sh/init.c:
- sh_type(): Return SH_TYPE_POSIX shell type if ksh was invoked
as sh (or rsh, restricted sh).
- sh_init(): Enable posix option if the SH_TYPE_POSIX shell type
was detected, or if the CONFORMANCE ast config variable was set
to "standard" (which libast sets on init if POSIXLY_CORRECT
exists in the environment).
src/cmd/ksh93/tests/options.sh,
src/cmd/ksh93/tests/io.sh:
- Replace regression tests for &> and move to io.sh. Since &> is
now for general use, no longer test in an rc script, and don't
check that a warning is issued.
Closes: #9
Progresses: #20
2020-09-01 05:19:19 +00:00
|
|
|
2020-09-01:
|
|
|
|
|
|
|
|
|
|
- The bash-style '&>file' redirection shorthand (for '>file 2>&1') is now
|
|
|
|
|
always recognised and not only when running rc/profile init scripts. It no
|
|
|
|
|
longer issues a warning. This brings ksh93 in line with mksh, bash and zsh.
|
|
|
|
|
|
|
|
|
|
- A long-form shell option '-o posix' has been added, which implements a
|
|
|
|
|
mode for better compatibility with the POSIX standard. It is automatically
|
|
|
|
|
turned on if ksh is invoked under the name 'sh'.
|
|
|
|
|
For now, it:
|
|
|
|
|
* disables the &> redirection shorthand
|
2020-09-01 06:16:51 +00:00
|
|
|
* causes the 'let' arithmetic command to recognise octal numbers by
|
|
|
|
|
leading zeros regardless of the setting of the 'letoctal' option
|
2020-09-01 07:11:27 +00:00
|
|
|
* causes file descriptors > 2 to be left open when invoking another program
|
2020-09-01 07:36:28 +00:00
|
|
|
* makes the <> redirection operator default to stdin instead of stdout
|
|
|
|
|
(this keeps the 2020-05-13 BUG_REDIRIO fix for the POSIX mode while
|
|
|
|
|
restoring traditional ksh93 behaviour for backwards compatibility)
|
-o posix: disable '[ -t ]' == '[ -t 1 ]' hack
On ksh93, 'test -t' is equivalent to 'test -t 1' (and of course
"[ -t ]" is equivalent to "[ -t 1 ]").
This is purely for compatibility with ancient Bourne shell
breakage. No other shell supports this. ksh93 should probably keep
it for backwards compatibility, but it should definitely be
disabled in POSIX mode as it is a violation of the standard; 'test
-t' is an instance of 'test "$string"', which tests if the string
is empty, so it should test if the string '-t' is empty (quod non).
This also replaces the fix for 'test -t 1' in a command
substitution with a better one that avoids forking (re: cafe33f0).
src/cmd/ksh93/sh/parse.c:
- qscan(): If the posix option is active, disable the parser-based
hack that converts a simple "[ -t ]" to "[ -t 1 ]".
src/cmd/ksh93/bltins/test.c:
- e3(): If the posix option is active, disable the part of the
compatibility hack that was used for compound expressions
that end in '-t', e.g. "[ -t 2 -o -t ]".
- test_unop(): Remove the forking fix for "[ -t 1 ]".
src/cmd/ksh93/edit/edit.c:
- tty_check(): This function is used by "[ -t 1 ]" and in other
contexts as well, so a fix here is more comprehensive. Forking
here would cause a segfault, but we don't actually need to. This
adds a fix that simply returns false if we're in a virtual
subshell that is also a command substitution. Since command
substitutions always fork upon redirecting standard output within
them (making them no longer virtual), it is safe to do this.
src/cmd/ksh93/tests/bracket.sh
- Add comprehensive regression tests for test/[/[[ -t variants in
command substitutions, in simple and compound expressions, with
and without redirecting stdout to /dev/tty within the comsub.
- Add tests verifying that -o posix disables the old hack.
- Tweak other tests, including one that globally disabled xtrace.
2020-09-01 19:24:44 +00:00
|
|
|
* disables a noncompliant 'test -t' == 'test -t 1' compatibility hack
|
-o posix: don't import/export variable attributes thru environment
When exporting variables, ksh exports their attributes (such as
'integer' or 'readonly') in a magic environment variable called
"A__z" (string defined in e_envmarker[] in data/msg.c). Child
shells recognise that variable and restore the attributes.
This little-known feature is risky; the environment cannot
necessarily be trusted and that A__z variable is easy to manipulate
before or between ksh invocations, so you can cause a script's
variables to be of the wrong type, or readonly. Backwards
compatibility requires keeping it, at least for now. But it should
be disabled in the posix mode, as it violates POSIX.
To do this, we have to solve a catch-22 in init.c. We must parse
options to know whether to turn on posix mode; it may be specified
as '-o posix' on the command line. The option parsing loop depends
on an initialised environment[*], while environment initialisation
(i.e., importing attributes) should depend on the posix option.
The catch-22 can be solved because initialising just the values
before option parsing is enough to avoid regressions. Importing the
attributes can be delayed until after option parsing. That involves
basically splitting env_init() into two parts while keeping a local
static state variable between them.
src/cmd/ksh93/sh/init.c:
- env_init():
* Split the function in two stages based on a new
'import_attributes' parameter. Import values in the first
stage; import attributes from A__z in the second (if ever).
Make the 'next' variable static as it keeps a state needed for
the attributes import stage.
* Single point of truth, greppability: don't hardcode "A__z" in
separate character comparisons, but use e_envmarker[].
* Fix an indentation error.
- sh_init(): When initialising the environment (env_init), don't
import the attributes from A__z yet; parse options first, then
import attributes only if posix option is not set.
src/cmd/ksh93/sh/name.c:
- sh_envgen(): Don't export variable attributes to A__z if the
posix option is set.
src/cmd/ksh93/tests/attributes.sh:
- Check that variable attributes aren't imported or exported
if the POSIX option is set.
src/cmd/ksh93/sh.1:
- Update.
This was the last item on the TODO list for -o posix for now.
Closes: #20
[*] If environment initialisation is delayed until after option
parsing, bin/shtests shows various regressions, including:
restricted mode breaks; the locale is not initialised properly
so that multibyte variable names break; $SHLVL breaks.
2020-09-05 08:33:50 +00:00
|
|
|
* disables passing an exported variable's attributes (such as integer or
|
|
|
|
|
readonly) to a new ksh process through the environment
|
Remove SHOPT_BASH; keep &> redir operator, '-o posix' option
On 16 June there was a call for volunteers to fix the bash
compatibility mode; it has never successfully compiled in 93u+.
Since no one showed up, it is now removed due to lack of interest.
A couple of things are kept, which are now globally enabled:
1. The &>file redirection shorthand (for >file 2>&1). As a matter
of fact, ksh93 already supported this natively, but only while
running rc/profile/login scripts, and it issued a warning. This
makse it globally available and removes the warning, bringing
ksh93 in line with mksh, bash and zsh.
2. The '-o posix' standard compliance option. It is now enabled on
startup if ksh is invoked as 'sh' or if the POSIXLY_CORRECT
variable exists in the environment. To begin with, it disables
the aforementioned &> redirection shorthand. Further compliance
tweaks will be added in subsequent commits. The differences will
be fairly minimal as ksh93 is mostly compliant already.
In all changed files, code was removed that was compiled (more
precisely, failed to compile/link) if the SHOPT_BASH preprocessor
identifier was defined. Below are other changes worth mentioning:
src/cmd/ksh93/sh/bash.c,
src/cmd/ksh93/data/bash_pre_rc.sh:
- Removed.
src/cmd/ksh93/data/lexstates.c,
src/cmd/ksh93/include/shlex.h,
src/cmd/ksh93/sh/lex.c:
- Globally enable &> redirection operator if SH_POSIX not active.
- Remove warning that was issued when &> was used in rc scripts.
src/cmd/ksh93/data/options.c,
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/args.c:
- Keep SH_POSIX option (-o posix).
- Replace SH_TYPE_BASH shell type by SH_TYPE_POSIX.
src/cmd/ksh93/sh/init.c:
- sh_type(): Return SH_TYPE_POSIX shell type if ksh was invoked
as sh (or rsh, restricted sh).
- sh_init(): Enable posix option if the SH_TYPE_POSIX shell type
was detected, or if the CONFORMANCE ast config variable was set
to "standard" (which libast sets on init if POSIXLY_CORRECT
exists in the environment).
src/cmd/ksh93/tests/options.sh,
src/cmd/ksh93/tests/io.sh:
- Replace regression tests for &> and move to io.sh. Since &> is
now for general use, no longer test in an rc script, and don't
check that a warning is issued.
Closes: #9
Progresses: #20
2020-09-01 05:19:19 +00:00
|
|
|
|
Speed up 'read', fixing macOS hang (take 2)
This fixes a hanging bug that could occur on macOS when using the
'read' command to read from a FIFO and encountering end-of-file
without a final newline character. It also makes the 'read' command
perform 15-25% faster on macOS and Linux.
The previous version (ff385e5a) failed on SunOS/Solaris/Illumos
because those systems apparently don't (fully) support the POSIX
standard recv(2) syscall with MSG_PEEK[*], which is the feature
that iffe detects under the 'socket_peek' identifier. On Illumos,
using that methods causes a compilation failure (unknown identifier
MSG_PEEK); on Solaris 11.4, that method causes multiple regressions
in tests/io.sh, suggesting the method compiles but doesn't work at
all. Instead, SunOS/Solaris/Illumos requires the method using
ioctl(2)+I_PEEK and select(2). No other system that ksh currently
builds on requires this method, so it is now only used on
SunOS/Solaris/Illumos.
So far, this version of sfpkrd() has been tested to work correctly
on Linux, macOS, FreeBSD, NetBSD, OpenBSD, HP-UX, Solaris, and
OmniOS (an Illumos distribution).
It still fails to peek on Cygwin, but in the exact same way it
failed before, so that's no loss.
To test, run the 'io' test set: bin/shtests -p io
src/lib/libast/sfio/sfpkrd.c: sfpkrd():
- Remove long-obsolete Mac OS X and Solaris bug workarounds.
- Remove methods that are no longer needed.
On systems with a POSIX compliant recv(2), the only thing that
is required to avoid regressions is the code that was conditional
upon the socket_peek feature test, which tests for the correct
functioning of the recv(2) syscall. This has now been made
mandatory for non-SunOS/Solaris/Illumos systems (using an #error
directive if it is not detected), with the other methods removed.
The result performs 15-25% faster on macOS and Linux while
passing all the regression tests.
On macOS, avoiding the select(2) method fixes the hanging bug.
On SunOS/Solaris/Illumos (the '__sun' identifier), the method
using ioctl(2)+I_PEEK and select(2) (iffe feature IDs:
stream_peek and lib_select) is preserved.
Resolves: https://github.com/ksh93/ksh/issues/118 (again)
[*] https://pubs.opengroup.org/onlinepubs/9699919799/functions/recv.html
2020-08-19 20:58:46 +00:00
|
|
|
2020-08-19:
|
|
|
|
|
|
|
|
|
|
- Sped up the 'read' command on most systems by 15-25%. Fixed a hanging bug
|
|
|
|
|
on reading from a FIFO that could occur on macOS.
|
|
|
|
|
|
2020-08-17 19:23:39 +00:00
|
|
|
2020-08-17:
|
|
|
|
|
|
|
|
|
|
- 'command -p' incorrectly used the hash table entry (a.k.a. tracked alias)
|
|
|
|
|
for a command if its path was previously hashed. It has now been fixed so
|
|
|
|
|
it never consults the hash table.
|
|
|
|
|
|
Fix 'command -x' on macOS, Linux, Solaris
'command -x' (basically builtin xargs for 'command') worked for
long argument lists on *BSD and HP-UX, but not on macOS and Linux,
where it reliably entered into an infinite loop.
The problem was that it assumed that every byte of the environment
space can be used for arguments, without accounting for alignment
that some OSs do. MacOS seems to be the most wasteful one: it
aligns on 16-byte boundaries and requires some extra bytes per
argument as well.
src/cmd/ksh93/sh/path.c:
- path_xargs(): When calculating how much space to subtract per
argument, add 16 extra bytes to the length of each argument, then
align the result on 16-byte boundaries. The extra 16 bytes is
more than even macOS needs, but hopefully it is future-proof.
- path_spawn(): If path_xargs() does fail, do not enter a retry
loop (which always becomes an infinite loop if the argument list
exceeds OS limitations), but abort with an error message.
2020-08-16 08:08:07 +00:00
|
|
|
2020-08-16:
|
|
|
|
|
|
|
|
|
|
- Fixed 'command -x' on macOS, Linux and Solaris by accounting for a 16-byte
|
|
|
|
|
argument alignment. If execution does fail, it now aborts with an internal
|
|
|
|
|
error message instead of entering an infinite retry loop.
|
|
|
|
|
|
Fix leak and crash upon defining functions in subshells
A memory leak occurred upon leaving a virtual subshell if a
function was defined within it. If this was done more than 32766
(= 2^15-2 = the 'short' max value - 1) times, the shell crashed.
Discussion and reproducer: https://github.com/ksh93/ksh/issues/114
src/cmd/ksh93/sh/subshell.c: table_unset():
- A subshell-defined function was never freed because a broken
check for autoloaded functions (which must not be freed[*]). It
looked for an initial '/' in the canonical path of the script
file that defined the function, but that path is also stored for
regular functions. Now use a check that executes nv_search() in
fpathdict, the same method used in _nv_unset() in name.c for a
regular function unset.
src/cmd/ksh93/bltins/misc.c: b_dot_cmd():
- Fix an additional memory leak introduced in bd88cc7f, that caused
POSIX functions (which are run with b_dot_cmd() like dot scripts)
to leak extra. This fix avoids both the crash fixed there and the
memory leak by introducing a 'tofree' variable remembering the
filename to free. Thanks to Johnothan King for the patch.
src/lib/libast/include/stk.h,
src/lib/libast/misc/stk.c,
src/lib/libast/man/stk.3,
src/lib/libast/man/stak.3:
- Make the stack more resilient by extending the stack reference
counter 'stkref' from (signed) short to unsigned int. On modern
systems with 32-bit ints, this extends the maximum number of
elements on a stack from 2^15-1==32767 to 2^32-1==4294967295.
The ref counter can never be negative, so there is no reason for
signedness. sizeof(int) is defined as the size of a single CPU
word, so this should not affect performance at all.
On a 16-bit system (not that ksh still compiles there), this
doubles the max number of entries to 2^16-1=65535.
src/cmd/ksh93/tests/leaks.sh:
- Add leak regression tests for ksh functions, POSIX functions, dot
scripts run with '.', and dot scripts run with 'source'.
src/cmd/ksh93/tests/path.sh:
- Add an output builtin with a redirect to an autoloaded function
so that a crash[*] is triggered if the check for an autoloaded
function is ever removed from table_unset(), as was done in ksh
93v- (which crashed).
[*] Freeing autoloaded functions after leaving a virtual subshell
causes a crashing bug: https://github.com/att/ast/issues/803
Co-authored-by: Johnothan King <johnothanking@protonmail.com>
Fixes: https://github.com/ksh93/ksh/issues/114
2020-08-13 22:29:27 +00:00
|
|
|
2020-08-13:
|
|
|
|
|
|
|
|
|
|
- Fixed memory leaks and a crashing bug that occurred when defining and
|
|
|
|
|
running functions in subshells.
|
|
|
|
|
|
Fix crash upon running many subshells (#113)
Co-authored-by: Martijn Dekker <martijn@inlv.org>
An intermittent crash occurred after running many thousands of
virtual/non-forked subshells. One reproducer is a crash in the
shbench fibonacci.ksh test, as documented here:
https://github.com/ksh-community/shbench/blob/f3d9e134/bench/fibonacci.ksh#L4-L10
The apparent cause was the signed and insufficiently large 'short'
data type of 'curenv' and related variables which wrapped around to
a negative number when overflowing. These IDs are necessary for the
'wait' builtin to obtain the exit status from a background job.
This fix is inspired by a patch based on ksh 93v-:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-longenv.dif?expand=1
https://src.fedoraproject.org/rpms/ksh/blob/f24/f/ksh-20130628-longer.patch
However, we change the type to 'unsigned int' instead of 'long'. On
all remotely modern systems, ints are 32-bit values, and using this
type avoids a performance degradation on 32-bit sytems. Making them
unsigned prevents an overflow to negative values.
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/include/jobs.h,
src/cmd/ksh93/include/nval.h,
src/cmd/ksh93/include/shell.h:
- Change the types of the static global 'subenv' and the subshell
structure members 'curenv', 'jobenv', 'subenv', 'p_env' and
'subshell' to one consistent type, unsigned int.
src/cmd/ksh93/sh/jobs.c,
src/cmd/ksh93/sh/macro.c:
src/cmd/ksh93/sh/name.c:
src/cmd/ksh93/sh/nvtype.c,
src/cmd/ksh93/sh/subshell.c:
- Updates to match new variable types.
src/cmd/ksh93/tests/subshell.sh:
- Show wrong exit status in message on failure of 'wait' builtin.
2020-08-12 17:50:59 +00:00
|
|
|
2020-08-11:
|
|
|
|
|
|
|
|
|
|
- Fixed an intermittent crash upon running a large number of subshells.
|
|
|
|
|
|
2020-08-10 21:15:53 +00:00
|
|
|
2020-08-10:
|
|
|
|
|
|
|
|
|
|
- A number of fixes have been applied to the printf formatting directives
|
|
|
|
|
%H and %#H (as well as the undocumented equivalents %(html)q and %(url)q):
|
|
|
|
|
1. Both formatters have been made multibyte/UTF-8 aware, and no longer
|
|
|
|
|
delete multibyte characters. Invalid UTF-8 byte sequences are rendered
|
|
|
|
|
as ASCII question marks.
|
|
|
|
|
2. %H no longer wrongly changes spaces to non-breaking spaces ( ).
|
|
|
|
|
3. %H now converts the single quote (') to '%#39;' instead of '''
|
|
|
|
|
which is not a valid entity in all HTML versions.
|
|
|
|
|
4. %#H failed to encode some reserved characters (e.g. '?') while encoding
|
|
|
|
|
some unreserved ones (e.g. '~'). It now percent-encodes all characters
|
|
|
|
|
except those 'unreserved' as per RFC3986 (ASCII alphanumeric plus -._~).
|
|
|
|
|
|
2020-08-11 00:02:31 +00:00
|
|
|
- Fixed a crash that occurred intermittently after running an external
|
|
|
|
|
command from a command substitution expanded from the $PS1 shell prompt.
|
|
|
|
|
|
2020-08-09 23:02:23 +00:00
|
|
|
2020-08-09:
|
|
|
|
|
|
|
|
|
|
- File name generation (a.k.a. pathname expansion, a.k.a. globbing) now
|
|
|
|
|
never matches the special navigational names '.' (current directory) and
|
|
|
|
|
'..' (parent directory). This change makes a pattern like .* useful; it
|
|
|
|
|
now matches all hidden files (dotfiles) in the current directory, without
|
|
|
|
|
the harmful inclusion of '.' and '..'.
|
|
|
|
|
|
redirect: check args before executing redirections (re: 7b82c338)
The 'redirect' builtin command did not error out before executing
any valid redirections. For example, 'redirect ls >foo.txt' issued
an "incorrect syntax" error, but still created 'foo.txt' and left
standard output permanently redirected to it.
src/cmd/ksh93/sh/xec.c: sh_exec():
- If we have redirections (io != NULL), and the command is
SYSREDIR, then check for arguments and error out if there are
any, before calling sh_redirect() to execute redirections.
(Note, the other check for arguments in b_exec() in bltins/misc.c
must be kept, as that applies if there are no redirections.)
src/cmd/ksh93/sh/io.c: sh_redirect():
- Edit comments to better explain what the flag values do.
src/cmd/ksh93/bltins/misc.c:
- Add a dummy b_redirect() function declaration "for the dictionary
generator" as has historically been done for other builtins that
share one C function. I'm not sure what that dictionary generator
is supposed to be, but this also improves greppability.
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/sh.1:
- Fix misleading "I/O redirection arguments" term. I/O redirections
are not arguments at all; no argument parser ever sees them.
src/cmd/ksh93/tests/io.sh:
- Test both conditions that should make 'redirect' produce an
"incorrect syntax" error.
- Test that any redirections are not executed if erroneous
non-redirection arguments exist.
src/cmd/ksh93/tests/builtins.sh:
- "... should show usage info on unrecognized options" test:
Because 'redirect' now refuses to process redirections on error,
the error message was not captured. The fix is to run the builtin
in a braces block and add the redirection to the block.
2020-08-08 22:57:57 +00:00
|
|
|
2020-08-08:
|
|
|
|
|
|
|
|
|
|
- Argument checking in the 'redirect' builtin command (see 2020-06-11) has
|
|
|
|
|
been improved to error out before executing redirections. For example, an
|
|
|
|
|
error like 'redirect ls >foo.txt' now will not create 'foo.txt' and will
|
|
|
|
|
not leave your standard output permanently redirected to it.
|
|
|
|
|
|
2020-08-07 20:09:01 +00:00
|
|
|
2020-08-06:
|
|
|
|
|
|
2020-08-07 01:53:25 +00:00
|
|
|
- Added the '${.sh.pid}' variable as an alternative to Bash's '$BASHPID'.
|
|
|
|
|
This variable is set to the current shell's PID, unlike '$$' (which is
|
|
|
|
|
set to the parent shell's PID). In virtual subshells '${.sh.pid}' is not
|
|
|
|
|
changed from its previous value, while in forked subshells '${.sh.pid}'
|
|
|
|
|
is set to the subshell's process ID.
|
|
|
|
|
|
2020-08-05 17:06:16 +00:00
|
|
|
2020-08-05:
|
|
|
|
|
|
2020-08-05 17:14:30 +00:00
|
|
|
- Fixed a bug in functions that caused ksh to crash when an array with an
|
|
|
|
|
unset method was turned into a multidimensional array.
|
|
|
|
|
|
2020-08-05 17:06:16 +00:00
|
|
|
- Fixed a bug that caused scripts to continue running after over-shifting
|
|
|
|
|
in a function when the function call had a redirection.
|
|
|
|
|
|
Fix shellquoting of invalid multibyte char (re: f9d28935, 8c7c60ec)
This commit fixes two bugs in the generation of $'...' shellquoted
strings:
1. A bug introduced in f9d28935. In UTF-8 locales, a byte that is
invalid in UTF-8, e.g. hex byte 86, would be shellquoted as
\u[86], which is not the same as the correct quoting, \x86.
2. A bug inherited from 93u+. Single bytes (e.g. hex 11) were
always quoted as \x11 and not \x[11], even if a subsequent
character was a hexadecimal digit. However, the parser reads
past two hexadecimal digits, so we got:
$ printf '%q\n' $'\x[11]1'
$'\x111'
$ printf $'\x111' | od -t x1
0000000 c4 91
0000002
After the bug fix, this works correctly:
$ printf '%q\n' $'\x[11]1'
$'\x[11]1'
$ printf $'\x[11]1' | od -t x1
0000000 11 31
0000002
src/cmd/ksh93/sh/string.c: sh_fmtq():
- Make the multibyte code for $'...' more readable, eliminating the
'isbyte' flag.
- When in a multibyte locale, make sure to shellquote both invalid
multibyte characters and unprintable ASCII characters as
hexadecimal bytes (\xNN). This reinstates 93u+ behaviour.
- When quoting bytes, use isxdigit(3) to determine if the next
character is a hex digit, and if so, protect the quoted byte with
square brackets.
src/cmd/ksh93/tests/quoting2.sh:
- Move the 'printf %q' shellquoting regression tests here from
builtins.sh; they test the shellquoting algorithm, not so much
the printf builtin itself.
- Add regression tests for these bugs.
2020-08-05 17:22:22 +00:00
|
|
|
- When generating shellquoted strings (such as with 'printf %q'), the
|
|
|
|
|
hexadecimal value of a quoted unprintable character was not protected with
|
|
|
|
|
square braces, e.g. 0x12 followed by '3' would be quoted as '\x123', which
|
|
|
|
|
is a different value. Such strings are now quoted like '\x[12]3' if the
|
|
|
|
|
next character is a hexadecimal digit.
|
|
|
|
|
|
2020-07-31 16:32:09 +00:00
|
|
|
2020-07-31:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused multidimensional associative arrays to be created
|
|
|
|
|
with an extra array member.
|
|
|
|
|
|
2020-08-01 00:12:45 +00:00
|
|
|
- Fixed a bug that caused the expansions of positional parameters $1 - $9,
|
|
|
|
|
as well as special parameters such as $? and $-, to corrupt any multibyte
|
|
|
|
|
characters immediately following the expansion if a UTF-8 locale is active.
|
|
|
|
|
|
2020-07-30 00:22:11 +00:00
|
|
|
2020-07-29:
|
|
|
|
|
|
|
|
|
|
- On a ksh compiled to use fork(2) to run external commands, a bug has been
|
|
|
|
|
fixed that caused signals (such as SIGINT, Ctrl+C) to be ignored within a
|
|
|
|
|
non-forked subshell after running an external command within that subshell.
|
|
|
|
|
|
2020-07-25 18:46:11 +00:00
|
|
|
2020-07-25:
|
|
|
|
|
|
|
|
|
|
- Fixed BUG_MULTIBIFS: Multibyte characters can now be used as IFS
|
|
|
|
|
delimiters. "$*" was incorrectly joining positional parameters on
|
|
|
|
|
the first byte of a multibyte character. This was due to truncation
|
|
|
|
|
based on the incorrect assumption the IFS would never be larger
|
|
|
|
|
than a single byte.
|
|
|
|
|
|
2020-07-26 01:18:49 +00:00
|
|
|
- Fixed a bug that caused the sleep builtin to continue after being given
|
|
|
|
|
an unrecognized option. 'sleep -: 1' will now show a usage message and
|
|
|
|
|
exit instead of sleep for one second.
|
|
|
|
|
|
2020-07-25 15:39:12 +00:00
|
|
|
- Fixed a bug that caused the 'typeset' variable attributes -a, -A, -l, and
|
|
|
|
|
-u to leak out of a subshell if they were set without assigning a value.
|
|
|
|
|
|
2020-07-23 23:03:57 +00:00
|
|
|
2020-07-23:
|
|
|
|
|
|
2020-07-24 00:20:26 +00:00
|
|
|
- Fixed an infinite loop that could occur when ksh is the system's /bin/sh.
|
|
|
|
|
|
2020-07-23 23:03:57 +00:00
|
|
|
- A command substitution that is run on the same line as a here-document
|
|
|
|
|
will no longer cause a syntax error.
|
|
|
|
|
|
2020-07-22 11:47:04 +00:00
|
|
|
2020-07-22:
|
|
|
|
|
|
|
|
|
|
- Fixed two race conditions when running external commands on
|
|
|
|
|
interactive shells with job control active.
|
|
|
|
|
|
2020-07-20 18:49:32 +00:00
|
|
|
2020-07-20:
|
|
|
|
|
|
|
|
|
|
- If a shell function and a built-in command by the same name exist,
|
|
|
|
|
'whence -a' and 'type -a' now report both.
|
|
|
|
|
|
2020-07-21 03:02:47 +00:00
|
|
|
- Fixed a bug that caused file descriptors opened with 'redirect' or 'exec'
|
|
|
|
|
to survive a subshell environment after exiting it.
|
|
|
|
|
|
2020-07-19 22:42:12 +00:00
|
|
|
2020-07-19:
|
|
|
|
|
|
2020-08-06 23:50:11 +00:00
|
|
|
- Fixed a crash that occurred in the '.' command when using kshdb.
|
2020-07-19 22:42:12 +00:00
|
|
|
|
2020-08-06 23:50:11 +00:00
|
|
|
- Fixed a crash that occurred when attempting to use redirection with an
|
2020-07-19 22:42:12 +00:00
|
|
|
invalid file descriptor.
|
|
|
|
|
|
2020-07-16 17:56:49 +00:00
|
|
|
2020-07-16:
|
|
|
|
|
|
|
|
|
|
- The 'history' and 'r' default aliases have been made regular built-ins,
|
|
|
|
|
leaving zero default aliases.
|
|
|
|
|
|
2020-07-17 04:00:28 +00:00
|
|
|
- Fixed a bug that caused 'sleep -s' to have no effect with intervals longer
|
|
|
|
|
than 30 seconds.
|
|
|
|
|
|
|
|
|
|
- The accuracy of the sleep builtin has been improved. It no longer ignores
|
|
|
|
|
microseconds and doesn't add extra milliseconds when the interval is less
|
|
|
|
|
than 31 seconds.
|
|
|
|
|
|
Convert default typeset aliases to regular builtins
This converts the 'autoload', 'compound', 'float', 'functions',
'integer' and 'nameref' default aliases into regular built-in
commands, so that 'unalias -a' does not remove them. Shell
functions can now use these names, which improves compatibility
with POSIX shell scripts.
src/cmd/ksh93/data/aliases.c:
- Remove default typeset aliases.
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/include/builtins.h:
- Add corresponding built-in command declarations. Typeset-style
commands are now defined by a pointer range, SYSTYPESET ..
SYSTYPESET_END. A couple need their own IDs (SYSCOMPOUND,
SYSNAMEREF) for special-casing in sh/xec.c.
- Update 'typeset --man'.
src/cmd/ksh93/bltins/typeset.c: b_typeset():
- Recognise the new builtin commands by argv[0]. Implement them by
inserting the corresponding 'typeset' options into the argument
list before parsing options. This may seem like a bit of a hack,
but it is simpler, shorter, more future-proof and less
error-prone than manually copying and adapting all the complex
flaggery from the option parsing loop.
src/cmd/ksh93/sh/parse.c,
src/cmd/ksh93/sh/xec.c:
- Recognise typeset-style commands by SYSTYPESET .. SYSTYPESET_END
pointer range.
- Special-case 'compound' (SYSCOMPOUND) and 'nameref' (SYSNAMEREF)
along with recognising the corresponding 'typeset' options.
src/cmd/ksh93/sh.1:
- Update to document the new built-ins.
- Since not all declaration commands are special built-ins now,
identify declaration commands using a double-dagger "\(dd"
character (which renders as '=' in ASCII) and disassociate their
definition from that of special built-ins.
src/cmd/ksh93/tests/variables.sh:
- Adapt a regression test as there is no more 'integer' alias.
2020-07-15 18:52:01 +00:00
|
|
|
2020-07-15:
|
|
|
|
|
|
|
|
|
|
- The 'autoload', 'compound', 'float', 'functions', 'integer' and 'nameref'
|
|
|
|
|
default aliases have been converted into regular built-in commands, so
|
|
|
|
|
that 'unalias -a' does not remove them. Shell functions can now use
|
|
|
|
|
these names, which improves compatibility with POSIX shell scripts.
|
|
|
|
|
|
2020-07-15 22:38:44 +00:00
|
|
|
- The End key escape sequence '^[[F' is now handled in the emacs and vi editing
|
|
|
|
|
modes. The End key moves the cursor to the end of the line (in contrast to
|
|
|
|
|
the Home key doing the opposite).
|
|
|
|
|
|
2020-07-14 21:00:28 +00:00
|
|
|
2020-07-14:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused 'set -b' to have no effect.
|
|
|
|
|
|
2020-07-15 03:11:33 +00:00
|
|
|
- Following the 'time' keyword, the 'times' builtin command now also
|
|
|
|
|
supports millisecond precision.
|
|
|
|
|
|
2020-07-13 18:10:23 +00:00
|
|
|
2020-07-13:
|
|
|
|
|
|
|
|
|
|
- Fixed a fork bomb that could occur when the vi editor was sent SIGTSTP
|
|
|
|
|
while running in a ksh script.
|
|
|
|
|
|
2020-07-14 21:48:04 +00:00
|
|
|
- Appending a lone percent to the end of a format specifier no longer
|
|
|
|
|
causes a syntax error. The extra percent will be treated as a literal
|
|
|
|
|
'%', like in Bash and zsh.
|
|
|
|
|
|
|
|
|
|
- The 'time' keyword now has proper support for millisecond precision.
|
|
|
|
|
Although this feature was previously documented, the 'time' keyword
|
|
|
|
|
only supported up to centisecond precision, which caused a command
|
|
|
|
|
like the one below to return '0.000' on certain operating systems:
|
|
|
|
|
$ TIMEFORMAT='%3R'; time sleep .003
|
|
|
|
|
|
|
|
|
|
- The 'time' keyword now zero-pads seconds less than ten (like mksh).
|
|
|
|
|
|
2020-07-10 16:52:47 +00:00
|
|
|
2020-07-10:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused types created with 'typeset -T' to throw an error
|
|
|
|
|
when used if the type name started with a lowercase 'a'.
|
|
|
|
|
|
2020-07-10 17:26:28 +00:00
|
|
|
- A potential crash due to memory corruption when using many file
|
|
|
|
|
descriptors has been fixed.
|
|
|
|
|
|
2020-07-09 14:09:52 +00:00
|
|
|
2020-07-09:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash on syntax error when sourcing/dotting multiple files.
|
|
|
|
|
|
2020-07-09 15:02:15 +00:00
|
|
|
- Fixed a crash when listing indexed arrays.
|
|
|
|
|
|
Fix hash table memory leak when restoring PATH
There is a bug in path_alias() that may cause a memory leak when
clearing the hash table while setting/restoring PATH.
This applies a fix from Siteshwar Vashist:
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01945.html
Note that, contrary to Siteshwar's analysis linked above, this bug
has nothing directly to do with subshells, forked or otherwise; it
can also be reproduced by temporarily setting PATH for a command,
for example, 'PATH=/dev/null true', and then doing a PATH search.
Modified analysis:
ksh maintains the value of PATH as a linked list. When a local
scope for PATH is created (e.g. in a virtual subshell or when doing
something like PATH=/foo/bar command ...), ksh duplicates PATH by
increasing the refcount for every element in the linked list by
calling the path_dup() and path_alias() functions. However, when
the state of PATH is restored, this refcount is not decreased. Next
time when PATH is reset to a new value, ksh calls the path_delete()
function to delete the linked list that stored the older path. But
the path_delete() function does not free elements whose refcount is
greater than 1, causing a memory leak.
src/cmd/ksh93/sh/path.c: path_alias():
- Decrease refcount and free old item if needed.
(The 'old' variable was already introduced in 99065353, but
its value was never used there; this fixes that as well.)
src/cmd/ksh93/tests/leaks.sh:
- Add regression test. With the bug, setting/restoring PATH
(which clears the hash table) and doing a PATH search 16 times
causes about 1.5 KiB of memory to be leaked.
2020-07-09 17:34:15 +00:00
|
|
|
- Fixed a memory leak when restoring PATH when temporarily setting PATH
|
|
|
|
|
for a command (e.g. PATH=/foo/bar command ...) or in a virtual subshell.
|
|
|
|
|
|
2020-07-09 21:12:04 +00:00
|
|
|
- Combining ((...)) with redirections no longer causes a syntax error
|
|
|
|
|
due to the parser handling '>' incorrectly.
|
|
|
|
|
|
2020-07-09 21:59:45 +00:00
|
|
|
- Fixed a bug that corrupted KIA/CQL cross-reference databases created using
|
|
|
|
|
ksh's -R option; shell warnings were wrongly included in the database file.
|
|
|
|
|
|
Fix UTF-8 shellquoting for xtrace, printf %q, etc.
This fixes an annoying issue in the shell's quoting algorithm
(used for xtrace (set -x), printf %q, and other things) for UTF-8
locales, that caused it to encode perfectly printable UTF-8
characters unnecessarily and inconsistently. For example:
$ (set -x; : 'aeu aéu')
+ : $'aeu a\u[e9]u'
$ (set -x; : 'aéu aeu')
+ : 'aéu aeu'
$ (set -x; : '正常終了 aeu')
+ : '正常終了 aeu'
$ (set -x; : 'aeu 正常終了')
+ : $'aeu \u[6b63]\u[5e38]\u[7d42]\u[4e86]'
This issue was originally reported by lijo george in May 2017:
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01958.html
src/cmd/ksh93/sh/string.c:
- Add is_invisible() function that returns true if a character is a
Unicode invisible (non-graph) character, excluding ASCII space.
Ref.: https://unicode.org/charts/PDF/U2000.pdf
- Use a fallback in is_invisible() if we cannot use the system's
iswprint(3); this is the case for the ksh C.UTF-8 locale if the
OS doesn't support that. Fall back to a hardcoded blacklist of
invisible and control characters and put up with not encoding
nonexistent characters into \u[xxxx] escapes.
Ref.: https://unicode.org/charts/PDF/U2000.pdf
- When deciding whether to switch to $'...' quoting mode (state=2),
use is_invisible() instead of testing for ASCII 0-127 range.
- In $'...' quoting mode, use is_invisible() to decide whether to
encode wide characters into \u[xxxx] escapes.
src/cmd/ksh93/tests/builtins.sh:
- Add regression tests for shellquoting Arabic, Japanese and Latin
UTF-8 characters, to be run only in a UTF-8 locale. The Arabic
sample text[*] contains a couple of direction markers that are
expected to be encoded into \u[xxxx] escapes.
[*] source: https://r12a.github.io/scripts/tutorial/summaries/arabic
2020-07-10 00:38:13 +00:00
|
|
|
- The shell's quoting algorithm (used in xtrace, printf %q, and more) has been
|
|
|
|
|
fixed for UTF-8 (Unicode) locales; it no longer needlessly and inconsistently
|
|
|
|
|
encodes normal printable UTF-8 characters into hexadecimal \u[xxxx] codes.
|
|
|
|
|
|
2020-07-09 04:08:28 +00:00
|
|
|
2020-07-07:
|
|
|
|
|
|
|
|
|
|
- Four of the date formats accepted by 'printf %()T' have had their
|
|
|
|
|
functionality altered to the common behavior of date(1):
|
|
|
|
|
- '%k' and '%l' print the current hour with blank padding, the former
|
|
|
|
|
based on a 24-hour clock and the latter a twelve hour clock. These
|
|
|
|
|
are common extensions present on Linux and *BSD.
|
|
|
|
|
- '%f' prints a date with the format string '%Y.%m.%d-%H:%M:%S' (BusyBox).
|
|
|
|
|
- '%q' prints the quarter of the year (GNU).
|
|
|
|
|
|
2020-07-06 20:51:44 +00:00
|
|
|
2020-07-06:
|
|
|
|
|
|
|
|
|
|
- 'notty' is now written to the ksh auditing file instead of '(null)' if
|
|
|
|
|
the user's tty could not be determined.
|
|
|
|
|
|
2020-07-09 00:09:40 +00:00
|
|
|
- Unsetting an associative array no longer causes a memory leak to occur.
|
|
|
|
|
|
Fix corrupt UTF-8 char processing & shellquoting after aborted read
If the processing of a multibyte character was interrupted in UTF-8
locales, e.g. by reading just one byte of a two-byte character 'ü'
(\303\274) with a command like:
print -nr $'\303\274' | read -n1 g
then the shellquoting algorithm was corrupted in such a way that
the final quote in simple single-quoted string was missing. This
bug may have had other, as yet undiscovered, effects as well. The
problem was with corrupted multibyte character processing and not
with the shell-quoting routine sh_fmtq() itself.
Full trace and discussion at: https://github.com/ksh93/ksh/issues/5
(which is also an attempt to begin to understand the esoteric
workings of the libast mb* macros that process UTF-8 characters).
src/lib/libast/comp/setlocale.c: utf8_mbtowc():
- If called from the mbinit() macro (i.e. if both pointer
parameters are null), reset the global multibyte character
synchronisation state variable. This fixes the problem with
interrupted processing leaving an inconsistent state, provided
that mbinit() is called before processing multibyte characters
(which it is, in most (?) places that do this). Before this fix,
calling mbinit() in UTF-8 locales was a no-op.
src/cmd/ksh93/sh/string.c: sh_fmtq():
- Call mbinit() before potentially processing multibyte characters.
Testing suggests that this could be superfluous, but at worst,
it's harmless; better be sure.
src/cmd/ksh93/tests/builtins.sh:
- Add regression test for shellquoting with 'printf %q' after
interrupting the processing of a multibyte characeter with
'read -n1'. This test only fails in a UTF-8 locale, e.g. when
running: bin/shtests -u builtins SHELL=/buggy/ksh-2012-08-01
Fixes #5.
2020-07-05 17:24:41 +00:00
|
|
|
2020-07-05:
|
|
|
|
|
|
|
|
|
|
- In UTF-8 locales, fix corruption of the shell's internal string quoting
|
|
|
|
|
algorithm (as used by xtrace, 'printf %q', and more) that occurred when
|
|
|
|
|
the processing of a multibyte character was interrupted.
|
|
|
|
|
|
2020-07-03 19:08:00 +00:00
|
|
|
2020-07-03:
|
|
|
|
|
|
|
|
|
|
- Backslashes are no longer escaped in the raw Bourne Shell-like editing
|
|
|
|
|
mode in multibyte locales, i.e. backslashes are no longer treated like
|
|
|
|
|
Control-V if the emacs and vi modes are disabled.
|
|
|
|
|
|
|
|
|
|
- Deleting a backslash in vi mode with Control-H or Backspace now only
|
|
|
|
|
escapes a backslash if it was the previous input. This means erasing a
|
|
|
|
|
string such as 'ab\\\' will only cause the first backslash to escape a
|
|
|
|
|
Backspace as '^?', like in emacs mode.
|
|
|
|
|
|
|
|
|
|
- An odd interaction with Backspace when the last character of a separate
|
|
|
|
|
buffer created with Shift-C was '\' has been fixed. '^?' will no longer
|
|
|
|
|
be output repeatedly when attempting to erase a separate buffer with
|
|
|
|
|
a Backspace. Note that buffers created with Shift-C are not meant to be
|
|
|
|
|
erasable:
|
|
|
|
|
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/vi.html#tag_20_152_13_49
|
|
|
|
|
|
2020-07-04 14:57:47 +00:00
|
|
|
- The 'kill' builtin now supports the SIGINFO signal (on operating systems
|
|
|
|
|
with support for SIGINFO).
|
|
|
|
|
|
2020-07-02 22:29:07 +00:00
|
|
|
2020-07-02:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash that occurred if a directory named '.paths' existed in any
|
|
|
|
|
directory listed in $PATH. The fix was to only read '.paths' if it is a
|
|
|
|
|
regular file or a symlink to a regular file.
|
|
|
|
|
|
2020-07-01 17:14:10 +00:00
|
|
|
2020-06-30:
|
|
|
|
|
|
|
|
|
|
- 'read -u' will no longer crash with a memory fault when given an out of
|
|
|
|
|
range or negative file descriptor.
|
2020-07-02 17:40:15 +00:00
|
|
|
|
|
|
|
|
- The '=~' operator no longer raises an error if a regular expression
|
|
|
|
|
combines the '{x}' quantifier with a sub-expression.
|
Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
|
2020-06-28 22:30:27 +00:00
|
|
|
2020-06-28:
|
|
|
|
|
|
|
|
|
|
- Variables created with 'typeset -RF' no longer cause a memory fault
|
|
|
|
|
when accessed.
|
|
|
|
|
|
2020-06-29 17:08:28 +00:00
|
|
|
- Unsetting an array that was turned into a compound variable will no
|
|
|
|
|
longer cause silent memory corruption.
|
|
|
|
|
|
2020-06-29 18:09:20 +00:00
|
|
|
- Variables created with 'readonly' in functions are now set to the
|
|
|
|
|
specified value instead of nothing. Note that 'readonly' does not
|
|
|
|
|
create a function-local scope, unlike 'typeset -r' which does.
|
|
|
|
|
|
2020-06-26 22:36:29 +00:00
|
|
|
2020-06-26:
|
|
|
|
|
|
|
|
|
|
- Changing to a directory that has a name starting with a '.' will no
|
|
|
|
|
longer fail if preceded by '../' (i.e. 'cd ../.local' will now work).
|
|
|
|
|
|
2020-06-25 21:08:43 +00:00
|
|
|
2020-06-24:
|
|
|
|
|
|
|
|
|
|
- Fixed buggy tab completion of tilde-expanded paths such as
|
|
|
|
|
~/some in 'vi' mode.
|
|
|
|
|
|
2020-06-26 05:19:58 +00:00
|
|
|
- In the raw/default Bourne Shell-like editing mode that occurs when neither
|
|
|
|
|
the 'emacs' nor the 'vi' shell option is active:
|
|
|
|
|
* tab completion is now correctly disabled, instead of enabled and broken;
|
|
|
|
|
* entering tab characters now moves the cursor the correct amount.
|
|
|
|
|
|
2020-06-23 22:02:16 +00:00
|
|
|
2020-06-23:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused combining process substitution with redirection
|
|
|
|
|
to create a bizarre file in the user's current working directory.
|
|
|
|
|
|
|
|
|
|
- Using process substitution while the shell is interactive no longer
|
|
|
|
|
causes the process ID of the asynchronous process to be printed.
|
|
|
|
|
|
2020-06-22 12:59:24 +00:00
|
|
|
2020-06-22:
|
|
|
|
|
|
|
|
|
|
- The 'stop' and 'suspend' default aliases have been converted into regular
|
|
|
|
|
built-in commands, so that 'unalias -a' does not remove them, 'suspend'
|
|
|
|
|
can do a couple of sanity checks, and something like
|
|
|
|
|
cmd=stop; $cmd $!
|
|
|
|
|
will now work. See 'stop --man' and 'suspend --man' for more information.
|
|
|
|
|
|
2020-06-22 17:11:19 +00:00
|
|
|
- Fixed a bug that caused the kill and stop commands to segfault when given
|
|
|
|
|
a non-existent job.
|
|
|
|
|
|
2020-06-22 23:27:05 +00:00
|
|
|
- Nested functions no longer ignore variable assignments that were prefixed
|
|
|
|
|
to their parent function, i.e. 'VAR=foo func' will now set $VAR to 'foo'
|
|
|
|
|
in the scope of any nested function 'func' runs.
|
|
|
|
|
|
2020-06-20 17:08:41 +00:00
|
|
|
2020-06-20:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused setting the following variables as readonly in
|
|
|
|
|
a virtual subshell to affect the environment outside of the subshell:
|
|
|
|
|
$_
|
|
|
|
|
${.sh.name}
|
|
|
|
|
${.sh.subscript}
|
|
|
|
|
${.sh.level}
|
|
|
|
|
$RANDOM
|
|
|
|
|
$LINENO
|
|
|
|
|
|
|
|
|
|
- Fixed two bugs that caused `unset .sh.lineno` to always produce a memory
|
|
|
|
|
fault and `(unset .sh.level)` to memory fault when run in nested
|
|
|
|
|
functions.
|
|
|
|
|
|
2020-06-19 12:01:55 +00:00
|
|
|
2020-06-18:
|
|
|
|
|
|
|
|
|
|
- A two decade old bug that caused 'whence -a' to base the path of
|
|
|
|
|
tracked aliases on the user's current working directory has been
|
2020-06-25 15:21:40 +00:00
|
|
|
fixed. Now the real path to tracked aliases is shown when '-a' is
|
|
|
|
|
passed to the whence command.
|
2020-06-19 12:01:55 +00:00
|
|
|
|
2020-06-18 00:48:51 +00:00
|
|
|
2020-06-17:
|
|
|
|
|
|
|
|
|
|
- A bug in 'unset -f' was fixed that prevented shell functions from
|
|
|
|
|
unsetting themselves while they were running. A POSIX function no longer
|
|
|
|
|
crashes when doing so, and a KornShell-style function no longer silently
|
|
|
|
|
ignores an 'unset -f' on itself. A function of either form now continues
|
|
|
|
|
running after unsetting itself, and is removed at the end of the run.
|
|
|
|
|
|
2020-06-16 11:21:42 +00:00
|
|
|
2020-06-16:
|
|
|
|
|
|
|
|
|
|
- Passing the '-d' flag to the read builtin will no longer cause the '-r'
|
|
|
|
|
flag to be discarded when 'read -r -d' is run.
|
|
|
|
|
|
2020-06-16 21:58:05 +00:00
|
|
|
- Fix BUG_CMDSPASGN: preceding a "special builtin"[*] with 'command' now
|
|
|
|
|
prevents preceding invocation-local variable assignments from becoming global.
|
|
|
|
|
[*] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_14
|
|
|
|
|
|
2020-06-15 07:03:44 +00:00
|
|
|
2020-06-15:
|
|
|
|
|
|
|
|
|
|
- The 'source' alias has been converted into a regular built-in command.
|
|
|
|
|
|
2020-06-15 13:43:37 +00:00
|
|
|
- Functions that set variables in a virtual subshell will no longer affect
|
|
|
|
|
variables of the same name outside of the virtual subshell's environment.
|
|
|
|
|
|
2020-06-15 14:56:11 +00:00
|
|
|
- Terse usage messages written by builtin commands now point the user to
|
|
|
|
|
the --help and --man options for more information.
|
|
|
|
|
|
2020-06-14 16:28:22 +00:00
|
|
|
2020-06-14:
|
2020-06-15 07:03:44 +00:00
|
|
|
|
2020-06-14 16:28:22 +00:00
|
|
|
- 'read -S' is now able to correctly handle strings with double quotes
|
|
|
|
|
nested inside of double quotes.
|
|
|
|
|
|
2020-06-13 21:16:08 +00:00
|
|
|
2020-06-13:
|
2020-06-14 04:28:38 +00:00
|
|
|
|
2020-06-13 21:16:08 +00:00
|
|
|
- Fixed a timezone name determination bug on FreeBSD that caused the
|
|
|
|
|
output from `LC_ALL=C printf '%T' now` to print the wrong time zone name.
|
|
|
|
|
|
2020-06-11 10:43:17 +00:00
|
|
|
2020-06-11:
|
|
|
|
|
|
2020-06-12 03:18:01 +00:00
|
|
|
- Fixed a bug that caused running 'builtin -d' on a special builtin to
|
|
|
|
|
delete it. The man page for the 'builtin' command documents that special
|
|
|
|
|
builtins cannot be deleted.
|
|
|
|
|
|
2020-06-11 10:43:17 +00:00
|
|
|
- POSIX compliance fix: It is now possible to set shell functions named
|
|
|
|
|
'alias' or 'unalias', overriding the commands by the same names. In
|
|
|
|
|
technical terms, they are now regular builtins, not special builtins.
|
|
|
|
|
|
2020-06-12 02:17:14 +00:00
|
|
|
- The redirect='command exec' alias has been converted to a regular
|
|
|
|
|
'redirect' builtin command that only accepts I/O redirections, which
|
|
|
|
|
persist as in 'exec'. This means that:
|
|
|
|
|
* 'unlias -a' no longer removes the 'redirect' command;
|
|
|
|
|
* users no longer accidentally get logged out of their shells if
|
|
|
|
|
they type something intuitive but wrong, like 'redirect ls >file'.
|
|
|
|
|
|
2020-06-12 04:57:57 +00:00
|
|
|
- The undocumented 'login' and 'newgrp' builtin commands have been removed.
|
|
|
|
|
These replaced your shell session with the external commands by the same
|
|
|
|
|
name, as in 'exec'. If an error occurred (e.g. due to a typo), you would
|
|
|
|
|
end up immediately logged out.
|
|
|
|
|
If you do want this behaviour, you can restore it by setting:
|
|
|
|
|
alias login='exec login'
|
|
|
|
|
alias newgrp='exec newgrp'
|
|
|
|
|
|
2020-06-10 11:00:35 +00:00
|
|
|
2020-06-10:
|
|
|
|
|
|
|
|
|
|
- The 'hash' utility is now a regular builtin instead of an alias to
|
|
|
|
|
'alias -t --'. The functionality of the old command has been removed
|
|
|
|
|
from the alias builtin.
|
|
|
|
|
|
|
|
|
|
- Changing the hash table in a subshell will no longer affect the parent
|
|
|
|
|
shell's hash table. This fix applies to the hash utility and when the
|
|
|
|
|
PATH is reset manually.
|
|
|
|
|
|
2020-06-25 15:21:40 +00:00
|
|
|
- 'set +r' is no longer able to unset the restricted option. This change
|
|
|
|
|
makes the behavior of 'set +r' identical to 'set +o restricted'.
|
|
|
|
|
|
2020-06-09 15:31:00 +00:00
|
|
|
2020-06-09:
|
|
|
|
|
|
|
|
|
|
- The 'unalias' builtin will now return a non-zero status if it tries
|
|
|
|
|
to remove a previously set alias that is not currently set.
|
|
|
|
|
|
2020-06-07 22:01:24 +00:00
|
|
|
2020-06-08:
|
|
|
|
|
|
2020-06-10 12:16:53 +00:00
|
|
|
- Fix an issue with the up arrow key in Emacs editing mode.
|
|
|
|
|
Emacs editing mode is bugged in ksh93u+ and ksh2020. Let's
|
|
|
|
|
say you were to run the following commands after starting
|
|
|
|
|
a fresh instance of ksh:
|
|
|
|
|
$ alias foo='true'
|
|
|
|
|
$ unalias foo
|
|
|
|
|
If you type 'a' and then press the up arrow on your keyboard,
|
|
|
|
|
ksh will complete 'a' to `alias foo='true'` by doing a reverse
|
|
|
|
|
search for the last command that starts with 'a'.
|
|
|
|
|
Run the alias command again, then type 'u' and press the up
|
|
|
|
|
arrow key again. If ksh is in Vi mode, you will get `unalias foo`,
|
|
|
|
|
but in Emacs mode you will get `alias foo='true'` again.
|
|
|
|
|
All subsequent commands were ignored as ksh was saving the first
|
|
|
|
|
command and only based later searches off of it.
|
2020-06-08 10:22:51 +00:00
|
|
|
|
2020-06-07 22:01:24 +00:00
|
|
|
- If 'set -u'/'set -o nounset' is active, then the shell now errors out if a
|
|
|
|
|
nonexistent positional parameter such as $1, $2, ... is accessed, as other
|
|
|
|
|
shells do and POSIX requires. (This does *not* apply to "$@" and "$*".)
|
|
|
|
|
|
2020-06-07 23:05:27 +00:00
|
|
|
- If 'set -u'/'set -o nounset' is active, then the shell now errors out if $!
|
|
|
|
|
is accessed before the shell has launched any background process.
|
|
|
|
|
|
2020-06-08 01:28:36 +00:00
|
|
|
- Removed support for an obscure early 1990s Bell Labs file system research
|
|
|
|
|
project called 3DFS, which has not existed for decades. This removes:
|
|
|
|
|
- an obnoxious default alias 2d='set -f;_2d' that turned off your file name
|
|
|
|
|
wildcard expansion and then tried to run a nonexistent '_2d' command
|
|
|
|
|
- undocumented builtins 'vmap' and 'vpath' that only printed error messages
|
|
|
|
|
- a non-functional -V unary operator for the test and [[ commands
|
|
|
|
|
|
Fix signal handling due to exit status > 256
This fixes two bugs: issuing the 'exit' command with a value > 256
would cause ksh 93u+ to kill itself with the corresponding signal
(try 'exit 265' to SIGKILL your interactive shell), and, if the
last command of a script exits due to a signal, the shell would
repeat that signal to itself, causing any parent ksh to also be
killed.
Discussion:
https://bugzilla.redhat.com/show_bug.cgi?id=1469624
https://rainbow.chard.org/2017/03/21/ksh-deliberately-segfaults-if-the-last-command-in-a-script-crashes/
This commit is loosely based on a patch applied to the 93v- beta
and the abandoned ksh2020, but that patch was incomplete & broken:
$ ksh-2020.0.0 -c 'exit 265'; echo $?
137
Expected: 9. Since the exit was *not* due to a signal, the value
should simply be cropped to the 8 bits supported by the OS.
src/cmd/ksh93/bltins/cflow.c: b_exit():
- For the 'exit' builtin command, bitwise-AND the argument to
'exit' with SH_EXITMASK (8 bits, crop to 0-255) before passing it
on to sh_exit(). This restores the behaviour of <=2011 ksh93
versions and is in line with all other POSIX shells.
It also fixes this bogosity:
$ (exit 265); echo $? # non-forked subshell
265
$ (ulimit -t unlimited; exit 265); echo $? # forked subshell
9
Forked or non-forked should make no difference at all
(see commit message a0e0e29e for why).
src/cmd/ksh93/sh/fault.c: sh_done():
- If the current exit status is equal to the status for the last
signal that was received from a child process, remove the
SH_EXITSIG (9th) bit, so that the shell doesn't kill itself.
- If the shell's last child process exits due to a signal, exit
with a portable 8-bit exit status (128 + signal number). This
avoids the exit status being < 128 by being cropped to 8 bits.
src/cmd/ksh93/tests/signal.sh:
- Add regression test for exit with status > 256.
- Add regression test verifying the shell no longer kills itself.
(cherry picked from commit 98e0fc94393e175ce6adfee390327c320795bf12)
2020-06-08 10:23:37 +00:00
|
|
|
- If the last program run by a ksh script exits with a signal (e.g. crashed),
|
|
|
|
|
ksh itself now exits normally instead of repeating that same signal.
|
|
|
|
|
In addition, using 'exit x' for x > 256 no longer makes ksh issue a signal.
|
|
|
|
|
|
2020-06-06 19:25:59 +00:00
|
|
|
2020-06-06:
|
|
|
|
|
|
|
|
|
|
- The 'times' command is now a builtin command that conforms to POSIX
|
|
|
|
|
instead of an alias for the 'time' command. It displays the accumulated
|
|
|
|
|
user and system CPU times, one line with the times used by the shell and
|
|
|
|
|
another with those used by all of the shell's child processes.
|
|
|
|
|
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_27
|
|
|
|
|
|
POSIX compliance: rm harmful default aliases 'command '/'nohup '
Continuing alias substitution after 'command' (due to the final
space in the alias) is inherently broken and doing so by default is
incompatible with the POSIX standard, as aliases may contain
arbitrary shell grammar.
For instance, until the previous commit, the POSIX standard 'times'
command was an alias: times='{ { time;} 2>&1;}' -- and so, of
course, 'command times' gave a syntax error, although this is
a perfectly valid POSIX idiom that must be supported.
'command' is specified by POSIX as a regular builtin, not an alias.
Therefore it should always bypass aliases just as it bypasses
functions to expose standard builtin and external commands.
I can only imagine that the reason for this command='command '
alias was that some standard commands themselves were implemented
as aliases, and POSIX requires that standard commands are
accessible with the 'command' prefix. But implementing standard
commands as aliases is itself inherently broken. It never worked
for 'command times', as shown; and in any case, removing all
aliases with 'unalias -a' should not get rid of standard commands.
Similarly, the default alias nohup='nohup ' is also harmful.
Anyone who really wants to keep this behaviour can just define
these aliases themselves in their script or ~/.kshrc file.
src/cmd/ksh93/data/aliases.c:
- Remove default alias command='command '.
- Remove default alias nohup='nohup '.
src/cmd/ksh93/sh.1
- Remove the above default aliases from the list.
- Mention that the 'command' builtin does not search for aliases.
(cherry picked from commit 5cfe7c4e2015b7445da24983af5008035c4b6e1e)
2020-06-07 00:36:54 +00:00
|
|
|
- The default aliases command='command ' and nohup='nohup ' have been
|
|
|
|
|
removed because they caused breakage in an attempt to circumvent other
|
|
|
|
|
breakage which is being fixed. In the unlikely even that anyone still
|
|
|
|
|
needs alias substitution to continue on the command argument following
|
|
|
|
|
'command' or 'nohup', it's easy to set these aliases yourself.
|
|
|
|
|
|
Fix unsetting special vars in subshells (re: efa31503, 8b7f8f9b)
This fixes (or at least works around) a bug that caused special
variables such as PATH, LANG, LC_ALL, LINENO, etc. to lose their
effect after being unset in a subshell.
For example:
(unset PATH; PATH=/dev/null; ls); : wrongly ran 'ls'
(unset LC_ALL; LC_ALL=badlocale); : failed to print a diagnostic
This is yet another problem with non-forking/virtual subshells. If
you forced the subshell to fork (one way of doing this is using the
'ulimit' builtin, e.g. ulimit -t unlimited) before unsetting the
special variable, the problem vanished.
I've tried to localise the problem. I suspect the sh_assignok()
function, which is called from unall(), is to blame. This function
is supposed to make a copy of a variable node in the virtual
subshell's variable tree. Apparently, it fails to copy the
associated permanent discipline function settings (stored in the
np->nvfun->disc pointer) that gave these variables their special
effect, and which survive unset. However, my attempts to fix that
have been unsuccessful. If anyone can figure out a fix, please send
a patch/pull request!
Data point: This bug existed in 93u 2011-02-08, but did not yet
exist in M-1993-12-28-s+. So it is a regression.
Meanwhile, pending a proper fix, this commit adds a safe
workaround: it forces a non-forked subshell to fork before
unsetting such a special variable.
src/cmd/ksh93/bltins/typeset.c: unall():
- If we're in a non-forked, non-${ ...; } subshell, then before
unsetting any variable, check for variables with internal
trap/discipline functions, and call sh_subfork() if any are
found. To avoid crashing, this must be done before calling
sh_pushcontext(), so we need to loop through the args separately.
src/cmd/ksh93/tests/variables.sh:
- Remove the 'ulimit' that forced the fork; we do this in C now.
(cherry picked from commit 21b1a67156582e3cbd36936f4af908bb45211a4b)
2020-06-05 13:31:26 +00:00
|
|
|
2020-06-05:
|
|
|
|
|
|
|
|
|
|
- Fix a bug that caused special variables such as PATH, LANG, LC_ALL,
|
|
|
|
|
etc. to lose their effect after being unset in a subshell. For example:
|
|
|
|
|
(unset PATH; PATH=/dev/null; ls); : wrongly ran 'ls'
|
|
|
|
|
(unset LC_ALL; LC_ALL=badlocale); : failed to print a diagnostic
|
2020-06-24 17:38:09 +00:00
|
|
|
This also fixes BUG_KUNSETIFS: unsetting IFS in a subshell failed if IFS
|
|
|
|
|
was set to the empty value in the parent shell.
|
Fix unsetting special vars in subshells (re: efa31503, 8b7f8f9b)
This fixes (or at least works around) a bug that caused special
variables such as PATH, LANG, LC_ALL, LINENO, etc. to lose their
effect after being unset in a subshell.
For example:
(unset PATH; PATH=/dev/null; ls); : wrongly ran 'ls'
(unset LC_ALL; LC_ALL=badlocale); : failed to print a diagnostic
This is yet another problem with non-forking/virtual subshells. If
you forced the subshell to fork (one way of doing this is using the
'ulimit' builtin, e.g. ulimit -t unlimited) before unsetting the
special variable, the problem vanished.
I've tried to localise the problem. I suspect the sh_assignok()
function, which is called from unall(), is to blame. This function
is supposed to make a copy of a variable node in the virtual
subshell's variable tree. Apparently, it fails to copy the
associated permanent discipline function settings (stored in the
np->nvfun->disc pointer) that gave these variables their special
effect, and which survive unset. However, my attempts to fix that
have been unsuccessful. If anyone can figure out a fix, please send
a patch/pull request!
Data point: This bug existed in 93u 2011-02-08, but did not yet
exist in M-1993-12-28-s+. So it is a regression.
Meanwhile, pending a proper fix, this commit adds a safe
workaround: it forces a non-forked subshell to fork before
unsetting such a special variable.
src/cmd/ksh93/bltins/typeset.c: unall():
- If we're in a non-forked, non-${ ...; } subshell, then before
unsetting any variable, check for variables with internal
trap/discipline functions, and call sh_subfork() if any are
found. To avoid crashing, this must be done before calling
sh_pushcontext(), so we need to loop through the args separately.
src/cmd/ksh93/tests/variables.sh:
- Remove the 'ulimit' that forced the fork; we do this in C now.
(cherry picked from commit 21b1a67156582e3cbd36936f4af908bb45211a4b)
2020-06-05 13:31:26 +00:00
|
|
|
|
2020-06-05 14:03:07 +00:00
|
|
|
- Fix crashes on some systems, including at least a crash in 'print -v' on
|
|
|
|
|
macOS, by eliminating an invalid/undefined use of memccpy() on overlapping
|
|
|
|
|
buffers in the commonly used sfputr() function.
|
|
|
|
|
|
Fix ${.sh.subshell} counter to actually count level of subshells
This counter is documented as follows:
"The current depth for subshells and command substitution."
But before this commit, the actual behaviour was that the counter
was reset to zero whenever a subshell forked for any reason: a
pipe, background job, running 'ulimit', redirecting stdout in a
command substitution, and more. This behaviour was:
1. Not consistent with the documentation. Non-forked (a.k.a.
virtual) subshells are an internal implementation detail which
scripts should not have to be concerned with. The manual page
doesn't mention them at all.
2. Inherently broken. Since a subshell may fork for any number of
reasons, even mid-run, and those reasons may change with
bugfixes and further development, scripts have never actually
been able to rely on the value of ${.sh.subshell}.
So, this commit fixes the counter to count the levels of all
subshells, both virtual and forked.
src/cmd/ksh93/sh/xec.c: _sh_fork():
- Increase ${.sh.subshell} whenever we fork.
src/cmd/ksh93/sh/subshell.c:
- sh_subfork():
* Fix comment to properly explain what it does. It doesn't
"create" a subshell, it forks off an existing virtual subshell.
* Don't zero ${.sh.subshell}. Instead, since sh_fork() increases
it but we're forking an existing subshell, undo the increase.
- sh_subshell():
* Remove 'int16_t subshell' variable. It was unnecessary and
mostly unused. It was also the wrong type: it was assigned the
value from shp->subshell which is of type short.
* Increase and decrease the level of virtual subshells and
${.sh.subshell} independently.
src/cmd/ksh93/tests/variables.sh:
- Add regression tests for ${.sh.subshell} in virtual and forked
subshells of several kinds: comsub, parentheses, pipe, bg job.
- Undo wrong error test count fix from 04b4aef0.
(cherry picked from commit a0e0e29e7e0dbf21e4b3958ae02bde6665fb2696)
2020-06-05 19:39:47 +00:00
|
|
|
- Fix the ${.sh.subshell} level counter; it is no longer reset to zero when a
|
|
|
|
|
non-forked subshell happens to fork into a separate process for some reason
|
|
|
|
|
(an internal implementation detail that should be unnoticeable to scripts).
|
|
|
|
|
|
Fix BUG_KBGPID: $! was not updated under certain conditions
The $! special parameter was not set if a background job
(somecommand &) or co-process (somecommand |&) was launched as the
only command within a braces block with an attached redirection,
for example:
{
somecommand &
} >&2
With the bug, $! was unchanged; now it contains the PID of
somecommand.
Ref.: https://github.com/att/ast/issues/1357
src/cmd/ksh93/sh/parse.c: item():
- When processing redirections following a compound command, always
create a parent node with the TSETIO (I/O redirection) token.
Before this commit, if the last command was of type TFORK (and
the last command only tested as TFORK if the bg job or coprocess
was the only command in a braces block, because the ksh parser
optimises away the braces in that case), then the parent node was
created with the TFORK token instead.
I have no idea what David Korn's intention was with that, but
this is clearly very wrong. Creating another TFORK node when
parsing the redirection caused sh_exec() in sh/xec.c to execute
the redirection in an extra forked, non-background subshell.
Since redirections are executed before anything else, this
subshell is what then launched the background job between the
braces, so $! (a.k.a. shp->bckpid) was updated in that subshell
only, and never in the main shell. The extra subshell also
prevented the background job from being noticed by job control
on interactive shells.
So, the fix is simply to remove the broken test for TFORK.
src/cmd/ksh93/tests/variables.sh:
- Add regression tests for a bg job and a co-process as the only
command within a braces block with attached redirection.
(cherry picked from commit ffe5df30e69f7b596941a98498014d8e838861f2)
2020-06-04 03:19:59 +00:00
|
|
|
2020-06-04:
|
|
|
|
|
|
|
|
|
|
- Fix BUG_KBGPID: the $! special parameter was not set if a background job
|
|
|
|
|
(somecommand &) or co-process (somecommand |&) was launched as the only
|
|
|
|
|
command within a braces block with an attached redirection, for example:
|
|
|
|
|
{
|
|
|
|
|
somecommand &
|
|
|
|
|
} >&2
|
|
|
|
|
With the bug, $! was unchanged; now it contains the PID of somecommand.
|
|
|
|
|
|
2020-05-31 13:29:15 +00:00
|
|
|
2020-05-31:
|
|
|
|
|
|
|
|
|
|
- Fix a bug in autoloading functions. Directories in the path search list
|
|
|
|
|
which should be skipped (e.g. because they don't exist) did not interact
|
|
|
|
|
correctly with autoloaded functions, so that a function to autoload was
|
|
|
|
|
not always found correctly.
|
|
|
|
|
Details: https://github.com/att/ast/issues/1454
|
|
|
|
|
|
2020-05-30 14:21:30 +00:00
|
|
|
2020-05-30:
|
|
|
|
|
|
|
|
|
|
- Fix POSIX compliance of 'test'/'[' exit status on error. The command now
|
|
|
|
|
returns status 2 instead of 1 when given an invalid number or arithmetic
|
|
|
|
|
expression, e.g.:
|
|
|
|
|
[ 123 -eq 123x ]; echo $?
|
|
|
|
|
now outputs 2 instead of 1.
|
|
|
|
|
|
Fix redefining & unsetting functions in subshells (BUG_FNSUBSH)
Functions can now be correctly redefined and unset in subshell
environments (such as ( ... ), $(command substitutions), etc).
Before this fix, attempts to do this were silently ignored (!!!),
causing the wrong code (i.e.: the function by the same name from
the parent shell environment) to be executed.
Redefining and unsetting functions within "shared" command
substitutions of the form '${ ...; }' is also fixed.
Prior discussion: https://github.com/att/ast/issues/73
src/cmd/ksh93/sh/parse.c:
- A fix from George Koelher (URL above). He writes:
| The parser can set t->comnamp to the wrong function.
| Suppose that the shell has executed
| foo() { echo WRONG; }
| and is now parsing
| (foo() { echo ok; } && foo)
| The parser was setting t->comnamp to the wrong foo. [This
| fix] doesn't set t->comnamp unless it was a builtin. Now the
| subshell can't call t->comnamp, so it looks for foo and finds
| the ok foo in the subshell's function tree.
src/cmd/ksh93/bltins/typeset.c:
- Unsetting functions in a virtual/non-forked subshell still
doesn't work: nv_open() fails to find the function. To work
around this problem, make 'unset -f' fork the subshell into its
own process with sh_subfork().
- The workaround exposed another bug: if we unset a function in a
subshell tree that overrode a function by the same name in the
main shell, then nv_delete() exposes the function from the main
shell scope. Since 'unset -f' now always forks a subshell, the
fix is to simply walk though troot's parent views and delete any
such zombie functions as well. (Without this, the 4 'more fun'
tests in tests/subshell.sh fail.)
src/cmd/ksh93/sh/subshell.c: sh_subfuntree():
- Fix function (re)definitions and unsetting in "shared" command
substitutions of the form '${ commandlist; }' (i.e.: if
sp->shp->subshare is true). Though internally this is a weird
form of virtual subshell, the manual page says it does not
execute in a subshell (meaning, all changes must survive it), so
a subshell function tree must not be created for these.
src/cmd/ksh93/tests/subshell.sh:
- Add regression tests related to these bugfixes. Test unsetting
and redefining a function in all three forms of virtual subshell.
(cherry picked from commit dde387825ab1bbd9f2eafc5dc38d5fd0bf9c3652)
2020-05-29 07:27:20 +00:00
|
|
|
2020-05-29:
|
|
|
|
|
|
|
|
|
|
- Fix BUG_FNSUBSH: functions can now be correctly redefined and unset in
|
|
|
|
|
subshell environments (such as ( ... ), $(command substitutions), etc).
|
|
|
|
|
Before this fix, this was silently ignored, causing the function by the
|
|
|
|
|
same name from the parent shell environment to be executed instead.
|
|
|
|
|
fn() { echo mainsh; }
|
|
|
|
|
(fn() { echo subsh; }; fn); fn
|
|
|
|
|
This now correctly outputs "subsh mainsh" instead of "mainsh mainsh".
|
|
|
|
|
ls() { echo "ls executed"; }
|
|
|
|
|
(unset -f ls; ls); ls
|
|
|
|
|
This now correctly lists your directory and then prints "ls executed",
|
|
|
|
|
instead of printing "ls executed" twice.
|
|
|
|
|
|
Fix unsetting aliases in subshells
Aliases can now be correctly unset within subshell environments
(such as ( ... ), $(command substitutions), etc), as well as
non-subshell "shared" command substitutions (${ ...; }). Before,
attempts to unset aliases within these were silently ignored.
Prior discussion: https://github.com/att/ast/issues/108
Subshell alias trees are only referenced in a few places in the
code, *and* have always been broken, so this commit gets rid of the
whole notion of a subshell alias tree. Instead, there is now just
one flat alias tree, and subshells fork into a separate process
when aliases are set or unset within them. It is not really
conceivable that this could be a performance-sensitive operation,
or even a common one, so this is a clean fix with no downside.
src/cmd/ksh93/include/defs.h:
- Remove sh_subaliastree() definition.
src/cmd/ksh93/sh/subshell.c:
- Remove salias element (pointer to subshell alias tree) from
subshell struct.
- Remove sh_subaliastree() function.
- sh_subshell(): Remove alias subshell tree cleanup.
src/cmd/ksh93/bltins/typeset.c:
- b_alias(): If in subshell, fork before setting alias.
- b_unalias(): If in subshell, fork before unsetting alias.
- unall(): Remove sh_subaliastree() call.
src/cmd/ksh93/sh/name.c:
- nv_open(): Remove sh_subaliastree() call.
src/cmd/ksh93/tests/subshell.sh:
- Add regression tests for unsetting or redefining aliases within
subshells.
(cherry picked from commit 12a15605b9521a2564a6e657905705a060e89095)
2020-05-29 07:27:53 +00:00
|
|
|
- Fix a similar bug with aliases. These can now be correctly unset
|
|
|
|
|
in subshell environments.
|
|
|
|
|
|
2020-05-20 21:42:00 +00:00
|
|
|
2020-05-21:
|
|
|
|
|
|
|
|
|
|
- Fix truncating of files with the combined redirections '<>;file' and
|
|
|
|
|
'<#pattern'. The bug was caused by out-of-sync streams.
|
|
|
|
|
Details and discussion: https://github.com/att/ast/issues/61
|
|
|
|
|
|
2020-05-30 14:55:30 +00:00
|
|
|
- Patched code injection vulnerability CVE-2019-14868. As a result, you can
|
2020-05-21 12:27:51 +00:00
|
|
|
no longer use expressions in imported numeric environment variables; only
|
|
|
|
|
integer literals are allowed.
|
|
|
|
|
|
Fix bugs in testing if a parameter is set
This fixes three related bugs:
1. Expansions like ${var+set} remained static when used within a
'for', 'while' or 'until' loop; the expansions din't change
along with the state of the variable, so they could not be used
to check whether a variable is set within a loop if the state of
that variable changed in the course of the loop. (BUG_ISSETLOOP)
2. ${IFS+s} always yielded 's', and [[ -v IFS ]] always yielded
true, even if IFS is unset. (BUG_IFSISSET)
3. IFS was incorrectly exempt from '-u' ('-o nounset') checks.
src/cmd/ksh93/sh/macro.c: varsub():
- When getting a node pointer (np) to the parameter to test,
special-case IFS by checking if it has a value and not setting
the pointer if not. The node to IFS always exists, even after
'unset -v IFS', so before this fix it always followed the code
path for a parameter that is set. This fixes BUG_IFSISSET for
${IFS+s} and also fixes set -u (-o nounset) with IFS.
- Before using the 'nv_isnull' macro to check if a regular variable
is set, call nv_optimize() if needed. This fixes BUG_ISSETLOOP.
Idea from Kurtis Rader: https://github.com/att/ast/issues/1090
Of course this only works if SHOPT_OPTIMIZE==1 (the default),
but if not, then this bug is not triggered in the first place.
- Add some comments for future reference.
src/cmd/ksh93/bltins/test.c: test_unop():
- Fix BUG_IFSISSET for [[ -v IFS ]]. The nv_optimize() method
doesn't seem to have any effect here, so the only way that I can
figure out is to special-case IFS, nv_getval()'ing it to check if
IFS has a value in the current scope.
src/cmd/ksh93/tests/variables.sh:
- Add regression tests for checking if a varariable is set within a
loop, within and outside a function with that variable made local
(to check if the scope is honoured). Repeat these tests for a
regular variable and for IFS, for ${foo+set} and [[ -v foo ]].
(cherry picked from commit a2cf79cb98fa3e47eca85d9049d1d831636c9b16)
2020-05-20 14:50:43 +00:00
|
|
|
2020-05-20:
|
|
|
|
|
|
|
|
|
|
- Fix BUG_ISSETLOOP. Expansions like ${var+set} remained static when used
|
|
|
|
|
within a 'for', 'while' or 'until' loop; the expansions din't change along
|
|
|
|
|
with the state of the variable, so they could not be used to check whether a
|
|
|
|
|
variable is set within a loop if the state of that variable changed in the
|
|
|
|
|
course of the loop.
|
|
|
|
|
|
|
|
|
|
- Fix BUG_IFSISSET. ${IFS+s} always yielded 's', and [[ -v IFS ]] always
|
|
|
|
|
yielded true, even if IFS is unset. This applied to IFS only.
|
|
|
|
|
|
2020-05-19 14:49:56 +00:00
|
|
|
2020-05-19:
|
|
|
|
|
|
|
|
|
|
- Fix 'command -p'. The -p option causes the operating system's standard
|
|
|
|
|
utilities path (as output by 'getconf PATH') to be searched instead of $PATH.
|
|
|
|
|
Before this fix, this was broken on non-interactive shells as the internal
|
|
|
|
|
variable holding the default PATH value was not correctly initialised.
|
|
|
|
|
|
Fix 'test -t 1' in $(command substitutions)
Standard output (FD 1) tested as being on a terminal within a
command substitution, which makes no sense as the command
substitution is supposed to be catching standard output.
ksh -c 'v=$(echo begincomsub
[ -t 1 ] && echo oops
echo endcomsub)
echo "$v"'
This should not output "oops".
This is one of the many bugs with ksh93 virtual (non-forked)
subshells. On the abandoned Vashist/Rader ksh2020 branch, this bug
was fixed by changing quite a lot of code, which introduced and/or
exposed another bug:
https://github.com/att/ast/issues/1079
https://github.com/att/ast/commit/8e1e405e
https://github.com/att/ast/issues/1088
That issue was unresolved when the ksh2020 branch was abandoned.
The safer and more conservative fix is simply forcing the subshell
to fork if we're in a non-forked command substitution and testing
'-t 1'. It is hard to imagine a situation where this would cause a
noticable performance hit.
Note that this fix does not affect ksh93-specific "shared"
non-subshell ${ command substitutions; } which are executed in the
main shell environment, so that variables survive, etcetera.
'test -t 1' continues to wrongly return true there, but command
substitutions of that form cannot be forked because that would
defeat their purpose.
src/cmd/ksh93/bltins/test.c:
- Fix 'test -t 1', '[ -t 1 ]' and '[[ -t 1 ]]' by forking the
current subshell if it is a virtual/non-forked subshell
(shp->subshell), and a command substitution (shp->comsub), but
NOT a "shared" ${ command substitution; } (!shp->subshare).
src/cmd/ksh93/tests/bracket.sh:
- Add two regression tests for this issue, which were adapted from
the Vashist/Rader ksh2020 branch.
NEWS, src/cmd/ksh93/include/version.h:
- Update.
(cherry picked from commit b8ef05e457ead65b83417699b8dd8632f855e2fa)
2020-05-16 19:06:49 +00:00
|
|
|
2020-05-16:
|
|
|
|
|
|
|
|
|
|
- Fix 'test -t 1', '[ -t 1 ]', '[[ -t 1 ]]' in command substitutions.
|
|
|
|
|
Standard output (file descriptor 1) tested as being on a terminal within a
|
|
|
|
|
command substitution, which makes no sense as the command substitution is
|
|
|
|
|
supposed to be catching standard output.
|
|
|
|
|
v=$(echo begincomsub
|
|
|
|
|
[ -t 1 ] && echo oops
|
|
|
|
|
echo endcomsub)
|
|
|
|
|
echo "$v"
|
|
|
|
|
This now does not output "oops".
|
|
|
|
|
|
2020-05-14 10:36:16 +00:00
|
|
|
2020-05-14:
|
|
|
|
|
|
|
|
|
|
- Fix syncing history when print -s -f is used. For example, the
|
|
|
|
|
following now correctly adds a 'cd' command to the history:
|
|
|
|
|
print -s -f 'cd -- %q\n' "$PWD"
|
|
|
|
|
Ref.: https://github.com/att/ast/issues/425
|
|
|
|
|
https://github.com/att/ast/pull/442
|
|
|
|
|
|
2020-06-09 19:57:05 +00:00
|
|
|
- Fix BUG_PUTIOERR: Output builtins now correctly detect
|
2020-05-16 14:04:35 +00:00
|
|
|
input/output errors. This allows scripts to check for a nonzero exit
|
|
|
|
|
status on the 'print', 'printf' and 'echo' builtins and prevent possible
|
|
|
|
|
infinite loops if SIGPIPE is ignored.
|
|
|
|
|
|
2020-05-14 16:41:26 +00:00
|
|
|
- Add a convenient bin/run_ksh_tests script to the source tree that
|
|
|
|
|
sets up the necessary environment and runs the ksh regression tests.
|
|
|
|
|
|
Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
2020-05-13:
|
|
|
|
|
|
|
|
|
|
- Fix BUG_CASELIT: an undocumented 'case' pattern matching misbehaviour that
|
|
|
|
|
goes back to the original Bourne shell, but wasn't discovered until 2018.
|
|
|
|
|
If a pattern doesn't match as a pattern, it was tried again as a literal
|
|
|
|
|
string. This broke common validation use cases, e.g.:
|
|
|
|
|
n='[0-9]'
|
|
|
|
|
case $n in
|
|
|
|
|
( [0-9] ) echo "$n is a number" ;;
|
|
|
|
|
esac
|
|
|
|
|
would output "[0-9] is a number" as the literal string fallback matches the
|
|
|
|
|
pattern. As this misbehaviour was never documented anywhere (not for Bourne,
|
|
|
|
|
ksh88, or ksh93), and it was never replicated in other shells (not even in
|
|
|
|
|
ksh88 clones pdksh and mksh), it is unlikely any scripts rely on it.
|
|
|
|
|
Of course, a literal string fallback, should it be needed, is trivial to
|
|
|
|
|
implement correctly without this breakage:
|
|
|
|
|
case $n in
|
|
|
|
|
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
|
|
|
|
|
esac
|
2020-05-13 14:00:33 +00:00
|
|
|
Ref.: https://github.com/att/ast/issues/476
|
|
|
|
|
|
|
|
|
|
- Fix BUG_REDIRIO: ksh used to redirect standard output by default when no
|
|
|
|
|
file descriptor was specified with the rarely used '<>' reading/writing
|
|
|
|
|
redirection operator. It now redirects standard input by default, as POSIX
|
|
|
|
|
specifies and as all other POSIX shells do. To redirect standard output
|
|
|
|
|
for reading and writing, you now need '1<>'.
|
|
|
|
|
Ref.: https://github.com/att/ast/issues/75
|
|
|
|
|
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_07
|
