Expose --histreedit and --histverify options (re: 921bbcae)
This adds two long-form shell options that modify history expansion
(-H/--histexpand). If --histreedit is on and a history expansion
fails, the command line is reloaded into the next prompt's edit
buffer, allowing corrections. If --histverify is on, the results of
a history expansion are not immediately executed but instead loaded
into the next prompt's edit buffer, allowing further changes.
SH_HISTREEDIT and SH_HISTVERIFY were already handled all along in
slowread() in io.c and via 'reedit' arguments to functions called
there, but could not be turned on as they were only ever exposed as
shopt options in the removed bash compatibility mode (in theory
only, as it failed to compile). I had overlooked them until now.
So, since the code is there, let's expose these options through the
normal long options interface. They're working fine, and activating
them actually makes history expansion tolerable to use.
src/cmd/ksh93/data/options.c:
- Make these options available as "histreedit" and "histverify".
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/sh.1:
- Document the "new" options.
src/cmd/ksh93/include/defs.h:
- Remove unused SH_HISTAPPEND and SH_HISTORY2 macros which were
part of the removed bash compatibility code. Note that ksh does
not need a histappend option as it never overwrites the history
file (in the bash mode, this shopt option was a no-op).
2022-01-12 14:55:02 +00:00
|
|
|
This documents significant changes in the 1.0 branch of ksh 93u+m.
|
|
|
|
|
For full details, see the git log at: https://github.com/ksh93/ksh/tree/1.0
|
2022-08-01 19:26:14 +00:00
|
|
|
Uppercase BUG_* IDs are shell bug IDs as used by the Modernish shell library.
|
Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
|
2022-08-01 19:26:14 +00:00
|
|
|
2022-08-01:
|
|
|
|
|
_ _ ___ _____ ___ ___ ___
|
|
|
|
|
| | _____| |__ / _ \___ / _ _ _ _ __ ___ / / | / _ \ / _ \
|
|
|
|
|
| |/ / __| '_ \ | (_) ||_ \| | | |_| |_| '_ ` _ \ / /| || | | | | | |
|
|
|
|
|
| <\__ \ | | | \__, |__) | |_| |_ _| | | | | |/ / | || |_| | |_| |
|
|
|
|
|
|_|\_\___/_| |_| /_/____/ \__,_| |_| |_| |_| |_/_/ |_(_)___(_)___/
|
|
|
|
|
|
|
|
|
|
- Exactly ten years after ksh 93u+ 2012-08-01, we now have a new ksh release.
|
|
|
|
|
Many thanks to all the direct and indirect contributors for their hard work!
|
2020-07-01 17:14:10 +00:00
|
|
|
|
2022-07-28 20:04:18 +00:00
|
|
|
2022-07-28:
|
|
|
|
|
|
|
|
|
|
- Fixed BUG_IFSGLOBS ("$*" does pattern matching if the first character of
|
|
|
|
|
$IFS is a wildcard).
|
|
|
|
|
|
2022-07-27 11:26:34 +00:00
|
|
|
2022-07-27:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug introduced on 2022-02-08 where $PPID was incorrect when a script
|
|
|
|
|
without a #! path was executed.
|
|
|
|
|
|
2022-07-27 21:33:44 +00:00
|
|
|
- Fixed a completion bug in the vi line editor. It now correctly completes
|
|
|
|
|
the sole member of a directory when tab is pressed immediately after /.
|
|
|
|
|
Thanks to K. Eugene Carlson for the report and the fix.
|
|
|
|
|
|
test/[/[[: Fix incorrect 0-skipping, float precision (re: c734568b)
The arguments to the binary numerical comparison operators (-eq,
-gt, etc.) in the [[ and test/[ commands are treated as arithmetic
expressions, even if $((...)) is not used. But there is some
seriously incorrect behaviour:
Reproducers (all should output 0/true):
$ [[ 0x0A -eq 10 ]]; echo $?
1
$ [[ 1+0x0A -eq 11 ]]; echo $?
0
$ (set --posix; [[ 010 -eq 8 ]]); echo $?
1
$ (set --posix; [[ +010 -eq 8 ]]); echo $?
0
$ [[ 0xA -eq 10 ]]; echo $?
1
$ xA=10; [[ 0xA -eq 10 ]]; echo $?
0
$ xA=WTF; [[ 0xA -eq 10 ]]; echo $?
ksh: WTF: parameter not set
(POSIX mode enables the recognition of the initial 0 as a prefix
indicating an octal number in arithmetic expressions.)
The cause is the two 'while' loops in this section in test_binop()
in src/cmd/ksh93/bltins/test.c:
502: if(op&TEST_ARITH)
503: {
504: while(*left=='0')
505: left++;
506: while(*right=='0')
507: right++;
508: lnum = sh_arith(left);
509: rnum = sh_arith(right);
510: }
So initial zeros are unconditionally skipped. Ostensibly this is to
disable the recognition of the initial zero as an octal prefix as
well as 0x as a hexadecimal prefix. This would be okay for
enforcing a decimal-only limitation for simple numbers, but to do
this for arithmetic expressions is flagrantly incorrect, to say the
least. (insert standard rant about AT&T quality standards here)
The fix for '[[' is simply to delete the two 'while' loops. But
that creates a problem for the deprecated-but-standard 'test'/'['
command, which also uses the test_binop() function. According to
POSIX, test/[ only parses simple decimal numbers, so octal, etc. is
not a thing. But, after that fix, 'test 08 -eq 10' in POSIX mode
yields true, which is unlike every other shell. (Note that this is
unlike [[ 08 -eq 10 ]], which yields true on bash because '[['
parses operands as arithmetic expressions.)
For test/[ in non-POSIX mode, we don't need to change anything. For
POSIX mode, we should only parse literal decimal numbers for these
operators in test/[, disallowing unexpanded arithmetic expressions.
This makes ksh's POSIX-mode test/[ act like every other shell and
like external .../bin/test commands shipped by OSs.
src/cmd/ksh93/bltins/text.c: test_binop():
- Correct a type mismatch. The left and right hand numbers should
be Sfdouble_t, the maximum double length on the current system,
and the type that sh_arith() returns. Instead, long double
(typeset -lF) values were reduced to double (typeset -F) before
comparing!
- For test/[ in POSIX, only accept simple decimal numbers: use
strtold() instead of sh_arith(). Do skip initial zeros here as
this disables the recognition of the hexadecimal prefix. Throw an
error on an invalid decimal number. Floating point constants are
still parsed, but that's fine as this does not cause any
incompatibility with POSIX.
- For code legibility, move handling of TEST_EQ, etc. to within the
if(op&TEST_ARITH) block. This also allows lnum and rnum to be
local to that block.
2022-07-26 17:04:33 +00:00
|
|
|
2022-07-26:
|
|
|
|
|
|
|
|
|
|
- Fixed incorrect handling of initial zeros in test/[ and [[ arithmetic
|
|
|
|
|
comparison operators. For example, [[ 0x0A -eq 10 ]] yielded false but
|
|
|
|
|
[[ 1+0x0A -eq 11 ]] yielded true.
|
|
|
|
|
|
|
|
|
|
- Fixed comparing long floating point (typeset -lF) values in test/[ and [[;
|
|
|
|
|
the values were reduced to standard floating point before comparing.
|
|
|
|
|
|
2022-07-25 01:38:23 +00:00
|
|
|
2022-07-25:
|
|
|
|
|
|
|
|
|
|
- Release candidate: ksh 93u+m/1.0.0-rc.1.
|
|
|
|
|
|
|
|
|
|
2722-07-24:
|
set -b/--notify: do not mess up the command line
This commit makes three interrelated changes.
First, the code for erasing the command line before redrawing it
upon a window size change is simplified and modernised. Instead of
erasing the line with lots of spaces, it now uses the sequence
obtained from 'tput ed' (usually ESC, '[', 'J') to "erase to the
end of screen". This avoids messing up the detection and automatic
redrawing of wrapped lines on terminals such as Apple Terminal.app.
Second, the -b/--notify option is made more usable. When it is on
and either the vi or emacs/gmacs line editor is in use, 'Done' and
similar notifications are now buffered and trigger a command line
redraw as if the window size changed, and the redraw routine prints
that notify buffer between erasing and redrawing the commmnd line.
The effect is that the notification appears to magically insert
itself directly above the line you're typing. (The notification
behaviour when not in the shell line editor, e.g. while running
commands such as external editors, is unchanged.)
Third, a bug is fixed that caused -b/--notify to only report on one
job when more than one terminated at the same time. The rest was
reported on the next command line as if -b were not on. Reproducer:
$ set -b; sleep 1 & sleep 1 & sleep 1 &
This commit also includes a fair number of other window size and
$COLUMNS/$LINES handling tweaks that made all this easier, not all
of which are mentioned below.
src/cmd/ksh93/include/fault.h,
src/cmd/ksh93/sh/fault.c:
- Replace sh_update_columns_lines with a new sh_winsize() function.
It calls astwinsize() and is to be used instead of it, and
instead of nv_getval(LINES) and nv_getval(COLUMNS) calls. It:
- Allows passing one or neither of lines or cols pointers.
- Updates COLUMNS and LINES, but only if they actually changed
from the last values. This makes .set discipline functions
defined for these variables more useful.
- Sets the sh.winch flag, but only if COLUMNS changes. If only
the height changes, the command line does not need redrawing.
src/cmd/ksh93/include/io.h:
- Add sh_editor_active() that allows checking whether one of vi,
emacs or gmacs is active without onerous #if SHOPT_* directives.
src/cmd/ksh93/sh/jobs.c: job_reap():
- Remove the fix backported in fc655f1a, which was really just a
workaround that papered over the real bug.
- Move a check for errno==ECHILD so it is only done when waitpid()
returns an error (pid < 0); the check was not correct because C
library functions that do not error out also do not change errno.
- Move the SH_NOTIFY && SH_TTYWAIT code section to within the
while(1) loop so it is run for each job, not only the
last-processed one. This fixes the bug where only one job was
notified when more than one ended at the same time.
- In that section, check if an editor is active; if so, set the
output file for job_list() to sh.notifybuffer instead of standard
error, list the jobs without the initial newline (JOB_NLFLAG),
and trigger a command line redraw by setting sh.winch.
src/cmd/ksh93/edit/edit.c:
- Obtain not just CURSOR_UP but also ERASE_EOS (renamed from
KILL_LINE) using 'tput'. The latter had the ANSI sequence
hardcoded. Define a couple of TPUT_* macros to make it easier to
deal with terminfo/termcap codes.
- Add get_tput() to make it easier to get several tput values
robustly (with SIGINT blocked, trace disabled, etc.)
- ed_crlf(): Removed. Going by those ancient #ifdefs, nothing that
93u+m will ever run on requires a '\r' before a '\n' to start a
new line on the terminal. Plus, as of 93u+, there were already
several spots in emacs.c and vi.c where it printed a sole '\n'.
- ed_read():
- Simplify/modernise command line erase using ERASE_EOS.
- Between erasing and redrawing, print the contents of the notify
buffer. This has the effect of inserting notifications above
the command line while the user is typing.
src/cmd/ksh93/features/cmds:
- To detect terminfo vs termcap codes, use all three codes we're
currently using. This matters on at least on my system (macOS
10.14.6) in which /usr/bin/tput has incomplete terminfo support
(no 'ed') but complete termcap support!
2022-07-24 02:06:46 +00:00
|
|
|
|
2022-07-24 08:17:17 +00:00
|
|
|
- Add new 'man' function in src/cmd/ksh93/fun/man. This integrates the --man
|
|
|
|
|
self-documentation of ksh built-in and external AST commands with your
|
|
|
|
|
system's 'man' command so you can conveniently use 'man' for all commands,
|
2022-07-24 12:01:15 +00:00
|
|
|
whether built-in or external. To use this, put the file in a directory in
|
|
|
|
|
your $FPATH and then issue 'autoload man' in your .kshrc to override the
|
|
|
|
|
system's man(1) command. See the file for details.
|
|
|
|
|
|
|
|
|
|
- Add new 'autocd' function in src/cmd/ksh93/fun/autocd. This activates a
|
|
|
|
|
feature like 'shopt -s autocd' in bash: type only a directory name to change.
|
|
|
|
|
See the file for details. Add this file to a directory in your $FPATH to use.
|
2022-07-24 08:17:17 +00:00
|
|
|
|
set -b/--notify: do not mess up the command line
This commit makes three interrelated changes.
First, the code for erasing the command line before redrawing it
upon a window size change is simplified and modernised. Instead of
erasing the line with lots of spaces, it now uses the sequence
obtained from 'tput ed' (usually ESC, '[', 'J') to "erase to the
end of screen". This avoids messing up the detection and automatic
redrawing of wrapped lines on terminals such as Apple Terminal.app.
Second, the -b/--notify option is made more usable. When it is on
and either the vi or emacs/gmacs line editor is in use, 'Done' and
similar notifications are now buffered and trigger a command line
redraw as if the window size changed, and the redraw routine prints
that notify buffer between erasing and redrawing the commmnd line.
The effect is that the notification appears to magically insert
itself directly above the line you're typing. (The notification
behaviour when not in the shell line editor, e.g. while running
commands such as external editors, is unchanged.)
Third, a bug is fixed that caused -b/--notify to only report on one
job when more than one terminated at the same time. The rest was
reported on the next command line as if -b were not on. Reproducer:
$ set -b; sleep 1 & sleep 1 & sleep 1 &
This commit also includes a fair number of other window size and
$COLUMNS/$LINES handling tweaks that made all this easier, not all
of which are mentioned below.
src/cmd/ksh93/include/fault.h,
src/cmd/ksh93/sh/fault.c:
- Replace sh_update_columns_lines with a new sh_winsize() function.
It calls astwinsize() and is to be used instead of it, and
instead of nv_getval(LINES) and nv_getval(COLUMNS) calls. It:
- Allows passing one or neither of lines or cols pointers.
- Updates COLUMNS and LINES, but only if they actually changed
from the last values. This makes .set discipline functions
defined for these variables more useful.
- Sets the sh.winch flag, but only if COLUMNS changes. If only
the height changes, the command line does not need redrawing.
src/cmd/ksh93/include/io.h:
- Add sh_editor_active() that allows checking whether one of vi,
emacs or gmacs is active without onerous #if SHOPT_* directives.
src/cmd/ksh93/sh/jobs.c: job_reap():
- Remove the fix backported in fc655f1a, which was really just a
workaround that papered over the real bug.
- Move a check for errno==ECHILD so it is only done when waitpid()
returns an error (pid < 0); the check was not correct because C
library functions that do not error out also do not change errno.
- Move the SH_NOTIFY && SH_TTYWAIT code section to within the
while(1) loop so it is run for each job, not only the
last-processed one. This fixes the bug where only one job was
notified when more than one ended at the same time.
- In that section, check if an editor is active; if so, set the
output file for job_list() to sh.notifybuffer instead of standard
error, list the jobs without the initial newline (JOB_NLFLAG),
and trigger a command line redraw by setting sh.winch.
src/cmd/ksh93/edit/edit.c:
- Obtain not just CURSOR_UP but also ERASE_EOS (renamed from
KILL_LINE) using 'tput'. The latter had the ANSI sequence
hardcoded. Define a couple of TPUT_* macros to make it easier to
deal with terminfo/termcap codes.
- Add get_tput() to make it easier to get several tput values
robustly (with SIGINT blocked, trace disabled, etc.)
- ed_crlf(): Removed. Going by those ancient #ifdefs, nothing that
93u+m will ever run on requires a '\r' before a '\n' to start a
new line on the terminal. Plus, as of 93u+, there were already
several spots in emacs.c and vi.c where it printed a sole '\n'.
- ed_read():
- Simplify/modernise command line erase using ERASE_EOS.
- Between erasing and redrawing, print the contents of the notify
buffer. This has the effect of inserting notifications above
the command line while the user is typing.
src/cmd/ksh93/features/cmds:
- To detect terminfo vs termcap codes, use all three codes we're
currently using. This matters on at least on my system (macOS
10.14.6) in which /usr/bin/tput has incomplete terminfo support
(no 'ed') but complete termcap support!
2022-07-24 02:06:46 +00:00
|
|
|
- New feature to make 'set -b'/'set -o notify' more usable. When that option
|
|
|
|
|
is on (and either the vi or emacs/gmacs line editor is in use), 'Done' and
|
|
|
|
|
similar notifications from completed background jobs no longer mess up the
|
|
|
|
|
command line you're typing. Instead, the notification is inserted directly
|
|
|
|
|
above the line you're typing, without affecting your command line display.
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused the -b/--notify option to only notify on one job if
|
|
|
|
|
more than one background job completed at the same time.
|
|
|
|
|
|
2022-07-24 05:26:43 +00:00
|
|
|
- Fixed a bug in the 'command -x' feature introduced on 2021-01-30; it is meant
|
|
|
|
|
to always run an external command, but failed to bypass path-bound builtins.
|
|
|
|
|
|
Fix: running external command influences $RANDOM sequence
The pseudorandom generator generates a reproducible sequnece of
values after seeding it with a known value. But:
$ (RANDOM=1; echo $RANDOM; echo $RANDOM)
2100
18270
$ (RANDOM=1; echo $RANDOM; ls >/dev/null; echo $RANDOM)
2100
30107
Since RANDOM was seeded with a specific value, the two command
lines should output the same pair of numbers. Running 'ls' in the
middle should make no difference.
The cause is a nv_getval(RANDNOD) call in xec.c, case TFORK, that
is run for all TFORK cases, in the parent process -- including
background jobs and external commands. What should happen instead
is that $RANDOM is reseeded in the child process.
This bug is in every version of ksh93 since before 1995.
There is also an opportunity for optimisation. As of 396b388e, the
RANDOM seed may be invalidated by setting rand_last to -2,
triggering a reseed the next time a $RANDOM value is obtained. This
was done to optimise the virtual subshell mechanism. But that can
also be used to eliminate unconditional reseeding elsewhere. So as
of this commit, RANDOM is never (re)seeded until it's used.
src/cmd/ksh93/include/variables.h,
src/cmd/ksh93/sh/subshell.c:
- Add RAND_SEED_INVALIDATED macro, a single source of truth for the
value that triggers a reseeding in sh_save_rand_seed().
- Add convenient sh_invalidate_rand_seed() function macro.
src/cmd/ksh93/sh/init.c,
src/cmd/ksh93/sh/xec.c:
- Optimisation: invalidate seed instead of reseeding directly.
- sh_exec(): case TFORK: Delete the nv_getval(RANDNOD) call. Add a
sh_invalidate_rand_seed() to the child part. This fixes the bug.
2022-07-20 22:22:30 +00:00
|
|
|
2022-07-21:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug where a reproducible $RANDOM sequence (after assigning a
|
|
|
|
|
specific value to $RANDOM) was influenced by running any external command.
|
|
|
|
|
|
2022-07-21 04:38:27 +00:00
|
|
|
- When an error occurs in a POSIX name() function or a dot script, the
|
|
|
|
|
function or script name is now reported in the error message.
|
|
|
|
|
|
Fix bad job control msg on trapped SIGINT and set -b (re: fc655f1)
When running an external command while trapping Ctrl+C via SIGINT,
and set -b is on, then a spurious Done job control message is
printed. No background job was executed.
$ trap 'ls' INT
$ set -b
$ <Ctrl+C>[file listing follows]
[1] + Done set -b
In jobs.c (487-493), job_reap() calls job_list() to list a running
or completed background job, passing the JOB_NFLAG bit to only
print jobs with the P_NOTIFY flag. But the 'ls' in the trap is not
a background job. So it is getting the P_NOTIFY flag by mistake.
In fact all processes get the P_NOTIFY flag by default when they
terminate. Somehow the shell normally does not follow a code path
that calls job_list() for foreground processes, but does when
running one from a trap. I have not yet figured out how that works.
What I do know is that there is no reason why non-background
processes should ever have the P_NOTIFY flag set on termination,
because those should never print any 'Done' messages. And we seem
to have a handy P_BG flag that is set for background processes; we
can check for this before setting P_NOTIFY. The only thing is that
flag is only compiled in if SHOPT_BGX is enabled, as it was added
to support that functionality.
For some reason I am unable to reproduce the bug in a pty session,
so there is no pty.sh regression test.
src/cmd/ksh93/sh/jobs.c:
- Rename misleadingly named P_FG flag to P_MOVED2FG; this flag is
not set for all foreground processes but only for processes moved
to the foreground by job_switch(), called by the fg command.
- Compile in the P_BG flag even when SHOPT_BGX is not enabled. We
need to set this flag to check for a background job.
- job_reap(): Do not set the P_NOTIFY flag for all terminated
processes, but only for those with P_BG set.
src/cmd/ksh93/sh/xec.c: sh_fork():
- Also pass special argument 1 for background job to job_post() if
SHOPT_BGX is not enabled. This is what gets it to set P_BG.
- Simplify 5 lines of convoluted code into 1.
Resolves: https://github.com/ksh93/ksh/issues/481
2022-07-14 03:51:17 +00:00
|
|
|
2022-07-14:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused a spurious "Done" message on the interactive shell
|
|
|
|
|
when an external command was run as a foreground job from a SIGINT trap.
|
|
|
|
|
|
Robustify ${.sh.level} scope switching (re: 69d37d5e, e1c41bb2)
Switching the function scope to a parent scope by assigning to
.sh.level (SH_LEVELNOD) leaves the shell in an inconsistent state,
causing invalid-free and/or use-after-free bugs. The intention of
.sh.level was always to temporarily switch scopes inside a DEBUG
trap, so this commit minimises the pitfalls and instability by
imposing some sensible limitations:
1. .sh.level is now a read-only variable except while executing a
DEBUG trap;
2. while it's writeable, attempts to unset .sh.level or to change
its attributes are ignored;
3. attempts to set a discipline function for .sh.level are ignored;
4. it is an error to set a level < 0 or > the current scope.
Even more crashing bugs are fixed by simplifiying the handling and
initialisation of .sh.level and by exempting it completely from
virtual subshell scoping (to which it's irrelevant).
TODO: one thing remains: scope corruption and use-after-free happen
when using the '.' command inside a DEBUG trap with ${.sh.level}
changed. Behaviour same as before this commit. To be investigated.
All changed files:
- Consistently use the int16_t type for level values as that is the
type of its non-pointer storage in SH_LEVELNOD.
- Update .sh.level by using an update_sh_level() macro that assigns
directly to the node value, then restores the scope if needed.
- To eliminate implicit typecasts, use the same int16_t type (the
type used by short ints such as SH_LEVELNOD) for all variables
containing a function and/or dot script level.
src/cmd/ksh93/include/variables.h:
- Add update_sh_level() macro.
src/cmd/ksh93/include/name.h,
src/cmd/ksh93/sh/macro.c:
- Add a nv_nonptr() macro that checks attributes for a non-pointer
value -- currently only signed or unsigned short integer value,
accessed via the 's' member of 'union Value' (e.g. np->nvalue.s).
- nv_isnull(): To avoid undefined behaviour, check for attributes
indicating a non-pointer value before accessing the nvalue.cp
pointer (re: 5aba0c72).
- varsub(): In the set/unset check, remove the now-redundant
exception for SH_LEVELNOD.
src/cmd/ksh93/data/variables.c,
src/cmd/ksh93/sh/init.c:
- shtab_variables[]: Make .sh.level a read-only short integer.
- sh_inittree(): To avoid undefined behaviour, do not assign to the
'union Value' char pointer if the attribute indicates a non-
pointer short integer value. Instead, the table value is ignored.
src/cmd/ksh93/sh/subshell.c: sh_assignok():
- Never create a subshell scope for SH_LEVELNOD.
src/cmd/ksh93/sh/xec.c:
- Get rid of 'struct Level' and its maxlevel member. This was only
used in put_level() to check for an out of range assignment, but
this can be trivially done by checking sh.fn_depth+sh.dot_depth.
- This in turn allows further simplification that reduces init for
.sh.level to a single nv_disc() call in sh_debug(), so get rid of
init_level().
- put_level(): Throw a "level out of range" error if assigned a
wrong level.
- sh_debug():
- Turn off the NV_RDONLY (read-only) attribute for SH_LEVELNOD
while executing the DEBUG trap.
- Restore the current scope when trap execution is finished.
- sh_funct(): Remove all .sh.level handling. POSIX functions (and
dot scripts) already handle it in b_dot_cmd(), so sh_funct(),
which is used by both, is the wrong place to do it.
- sh_funscope(): Update .sh.level for ksh syntax functions here
instead. Also, do not bother to initialise its discipline here,
as it can now only be changed in a DEBUG trap.
src/cmd/ksh93/bltins/typeset.c: setall():
- When it's not read-only, ignore all attribute changes for
.sh.level, as changing the attributes would crash the shell.
src/cmd/ksh93/sh/nvdisc.c: nv_setdisc():
- Ignore all attempts to set a discipline function for .sh.level,
as doing this would crash the shell.
src/cmd/ksh93/bltins/misc.c: b_dot_cmd():
- Bug fix: also update .sh.level when quitting a dot script.
src/cmd/ksh93/sh/name.c:
- _nv_unset():
- To avoid an inconsistent state, ignore all attempts to unset
.sh.level.
- To avoid undefined behaviour, do not zero np->nvalue.cp if
attributes for np indicate a non-pointer value (the actual bit
value of a null pointer is not defined by the standard, so
there is no guarantee that zeroing .cp will zero .s).
- sh_setscope(): For consistency, always set error_info.id (the
command name for error messages) to the new scope's cmdname.
Previously this was only done for two calls of this function.
- nv_name(): Fix a crashing bug by checking that np->nvname is a
non-null pointer before dereferencing it.
src/cmd/ksh93/include/nval.h:
- The NV_UINT16P macro (which is unsigned NV_INT16P) had a typo in
it, which went unnoticed for many years because it's not directly
used (though its bit flags are set and used indirectly). Let's
fix it anyway and keep it for completeness' sake.
2022-07-12 15:48:44 +00:00
|
|
|
2022-07-12:
|
|
|
|
|
|
|
|
|
|
- The .sh.level variable can now only be changed within a DEBUG trap. When
|
|
|
|
|
trap execution ends, the variable and the scope are now restored. These
|
|
|
|
|
changes disallow an inconsistent shell scoping state causing instability.
|
|
|
|
|
|
2022-07-10 18:12:10 +00:00
|
|
|
2022-07-10:
|
|
|
|
|
|
|
|
|
|
- Fixed a potential crash on retrieving an empty line from the command history.
|
|
|
|
|
|
2022-07-10 22:35:10 +00:00
|
|
|
- Fixed a potential crash in the lexical analyser on processing single-byte
|
|
|
|
|
characters with the highest bit set.
|
|
|
|
|
|
2022-07-09 20:40:25 +00:00
|
|
|
2022-07-09:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that broke '[[ ... ]]' test expressions for the command
|
|
|
|
|
following a syntax error in an associative array assignment.
|
|
|
|
|
|
2022-07-05 11:01:26 +00:00
|
|
|
2022-07-05:
|
|
|
|
|
|
|
|
|
|
- Fixed a spurious syntax error on encountering a process substitution
|
|
|
|
|
following a redirection without being an argument to a redirection.
|
|
|
|
|
For example, this now writes 'OK' to standard error: cat >&2 <(echo OK)
|
|
|
|
|
|
2022-07-05 20:09:29 +00:00
|
|
|
- Fixed the detection of a syntax error in compound assignments to
|
2022-07-30 22:11:20 +00:00
|
|
|
associative arrays.
|
2022-07-05 20:09:29 +00:00
|
|
|
|
Fix signal exit status of last command in subshell (re: b3050769)
Reproducer (on macOS/*BSD where SIGUSR1 has signal number 30):
$ ksh -c '(sh -c '\''kill -s USR1 $$'\''); echo $?'
ksh: 54220: User signal 1
30
Expected output for $?: 286, not 30. The signal is not reflected in
the 9th bit of the exit status.
This bug was introduced for virtual subshells in b3050769 but
exists in every ksh93 version for real (forked) subshells:
$ ksh -c '(ulimit -t unlimited; trap : EXIT; \
sh -c '\''kill -s USR1 $$'\''); echo $?'
ksh: 54267: User signal 1
30
(As of d6c9821c, a dummy trap is needed to trigger the bug, or it
will be masked by the exec optimization for the sh invocation.)
This is caused by the exit status being masked to 8 bits when a
subshell terminates. For a real subshell, this is inevitable as the
kernel does this. As of b3050769, virtual subshells behave in a
manner consistent with real subshells in this regard.
However, for both virtual and real subshells, if its last command
was terminated by a signal, then that should still be reflected in
the 9th bit of ksh's exit stauts.
The root of the problem is that ksh simply cannot rely internally
on the 9th bit of the exit status to determine if a command exited
due to a signal. The 9th bit may be trimmed by a subshell or may be
set by 'return' without a signal being involved. This commit fixes
it by introducing a separate flag which will be a reliable
indicator of this.
src/cmd/ksh93/include/shell.h:
- Add sh.chldexitsig flag (set if the last command was a child
process that exited due to a signal).
src/cmd/ksh93/sh/jobs.c: job_wait():
- When the last child process exited due to a signal, not only set
the 9th (SH_EXITSIG) bit of sh.exitval but also sh.chldexitsig.
src/cmd/ksh93/sh/subshell.c: sh_subshell():
- Fix the virtual subshell reproducer above. After trimming the
exit status to 8 bit, set the 9th bit if sh.chldexitsig is set.
This needs to be done in two places: one that runs in the parent
process after sh_subfork() and one for the regular virtual
subshell exit.
src/cmd/ksh93/sh/fault.c:
- sh_trap(): Save and restore sh.chldexitsig so that this fix does
not get deactivated if a trap is set.
- sh_done():
- Fix the real subshell reproducer above. When the last command
of a real subshell is a child process that exited due to a
signal (i.e., if (sh.chldexitsig && sh.realsubshell)), then
activate the code to pass down the signal to the parent
process. Since there is no way to pass a 9-bit exit status to a
parent process, this is the only way to ensure a correct exit
status in the parent shell environment.
- When exiting the main shell, use sh.chldexitsig and not the
unreliable SH_EXITSIG bit to determine if the 8th bit needs to
be set for a portable exit status indicating its last command
exited due to a signal.
2022-07-02 15:50:52 +00:00
|
|
|
2022-07-02:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug where, if the last command in a subshell was an external
|
|
|
|
|
command that was terminated by a signal, the exit status ($?) of the
|
|
|
|
|
subshell did not reflect this by adding 256 to the signal number.
|
|
|
|
|
|
2022-07-02 22:54:01 +00:00
|
|
|
- Fixed a bug that caused signal traps to be ignored if the shell was
|
|
|
|
|
signalled from a subshell that is the last command in the script.
|
|
|
|
|
|
2022-06-30 23:20:40 +00:00
|
|
|
2022-07-01:
|
|
|
|
|
|
|
|
|
|
- In scripts, $COLUMNS and $LINES are now kept up to date in scripts at
|
|
|
|
|
initialization and when the window size changes (previously, this
|
|
|
|
|
required setting a dummy trap for the SIGWINCH signal in the script).
|
|
|
|
|
|
Fix seeking on block devices with FD 0, 1 or 2
@stephane-chazelas reports:
> A very weird issue:
>
> To reproduce on GNU/Linux (here as superuser)
>
> # truncate -s10M file
> # export DEV="$(losetup -f --show file)"
> # ksh -c 'exec 3<> "$DEV" 3>#((0))' # fine
> # ksh -c 'exec 1<> file 1>#((0))' # fine
> # ksh -c 'exec 1<> "$DEV" 1>#((0))'
> ksh: 0: invalid seek offset
>
> Any seek offset is considered "invalid" as long as the file is a
> block device and the fd is 0, 1 or 2. It's fine for fds above 2
> and it's fine with any fd for regular files.
Apparently, block devices are not seekable with sfio. In io.c there
is specific code to avoid using sfio's sfseek(3) if there is no
sfio stream in sh.sftable[] for the file descriptor in question:
1398: Sfio_t *sp = sh.sftable[fn];
[...]
1420: if(sp)
1421: {
1422: off=sfseek(sp, off, SEEK_SET);
1423: sfsync(sp);
1424: }
1425: else
1426: off=lseek(fn, off, SEEK_SET);
For file descriptors 0, 1 or 2 (stdin/stdout/stderr), there is a
sh.sftable[] stream by default, and it is marked as not seekable.
This makes it return -1 in these lines in sfseek.c, even if the
system call called via SFSK() succeeds:
89: if(f->extent < 0)
90: { /* let system call set errno */
91: (void)SFSK(f,(Sfoff_t)0,SEEK_CUR,f->disc);
92: return (Sfoff_t)(-1);
93: }
...which explains the strange behaviour.
src/lib/libast/sfio/sfseek.c: sfseek():
- Allow for the possibility that the fallback system call might
succeed: let it handle both errno and the return value.
Resolves: https://github.com/ksh93/ksh/issues/318
2022-06-28 17:16:31 +00:00
|
|
|
2022-06-28:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused the <#((num)) or >#((num)) arithmetic seek
|
|
|
|
|
redirection operator to fail if used with file descriptor 0, 1 or 2
|
|
|
|
|
connected to a block device.
|
|
|
|
|
|
2022-06-22 16:02:25 +00:00
|
|
|
2022-06-22:
|
|
|
|
|
|
|
|
|
|
- Fixed: 'echo' failed when used inside a command substitution that
|
|
|
|
|
is inside a nested compound assignment.
|
|
|
|
|
|
2022-06-23 01:56:18 +00:00
|
|
|
- Fixed a crash in KEYBD trap handling introduced on 2021-11-29.
|
|
|
|
|
|
2022-06-20 17:41:28 +00:00
|
|
|
2022-06-20:
|
|
|
|
|
|
|
|
|
|
- Fixed a race condition that could cause redirections to fail with a
|
|
|
|
|
"cannot create" or "cannot open" error while processing a signal trap.
|
|
|
|
|
|
Fix incorrect exec optimisation with monitor/pipefail on
Reproducer script:
tempfile=/tmp/out2.$$.$RANDOM
bintrue=$(whence -p true)
for opt in monitor pipefail
do
(
set +x -o "$opt"
(
sleep .05
echo "ok $opt" >&2
) 2>$tempfile | "$bintrue"
) &
wait
cat "$tempfile"
rm -f "$tempfile"
done
Expected output:
ok monitor
ok pipefail
Actual output:
(none)
The 'monitor' and 'pipefail' options are supposed to make the shell
wait for the all commands in the pipeline to terminate and not only
the last component, regardless of whether the pipe between the
component commands is still open. In the failing reproducer, the
dummy external true command is subject to an exec optimization, so
it replaces the subshell instead of forking a new process. This is
incorrect, as the shell is no longer around to wait for the
left-hand part of the pipeline, so it continues in the background
without being waited for. Since it writes to standard error after
.05 seconds (after the pipe is closed), the 'cat' command reliably
finds the temp file empty. Without the sleep this would be a race
condition with unpredictable results.
Interestingly, this bug is only triggered for a (background
subshell)& and not for a forked (regular subshell). Which means the
exec optimization is not done for a forked regular subshell, though
there is no reason not to. That will be fixed in the next commit.
src/cmd/ksh93/sh/xec.c: sh_exec():
- case TFORK: Never allow an exec optimization if we're running a
command in a multi-command pipeline (pipejob is set) and the
shell needs to wait for all pipeline commands, i.e.: either the
time keyword is in use, the SH_MONITOR state is active, or the
SH_PIPEFAIL option is on.
- case TFIL: Fix the logic for setting job.waitall for the
non-SH_PIPEFAIL case. Do not 'or' in the boolean value but assign
it, and include the SH_TIMING (time keyword in use) state too.
- case TTIME: After that fix in case TFIL, we don't need to bother
setting job.waitall explicitly here.
src/cmd/ksh93/sh.1:
- Add missing documentation for the conditions where the shell
waits for all pipeline components (time, -o monitor/pipefail).
Resolves: https://github.com/ksh93/ksh/issues/449
2022-06-18 10:21:03 +00:00
|
|
|
2022-06-18:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug where, with the monitor or pipefail option on, the shell
|
|
|
|
|
failed to wait for all component commands in a pipeline to terminate if
|
|
|
|
|
the last component command was an external command and the pipeline was
|
|
|
|
|
the last command in a background subshell.
|
|
|
|
|
|
Fix the exec optimisation mess (re: 17ebfbf6, 6701bb30, d6c9821c)
This commit supersedes @lijog's Solaris patch 280-23332860 (see
17ebfbf6) as this is a more general fix that makes the patch
redundant. Of course its associated regression tests stay.
Reproducer script:
trap 'echo SIGUSR1 received' USR1
sh -c 'kill -s USR1 $PPID'
Run as a normal script.
Expected behaviour: prints "SIGUSR1 received"
Actual behaviour: the shell invoking the script terminates. Oops.
As of 6701bb30, ksh again allows an exec-without-fork optimisation
for the last command in a script. So the 'sh' command gets the same
PID as the script, therefore its parent PID ($PPID) is the invoking
script and not the script itself, which has been overwritten in
working memory. This shows that, if there are traps set, the exec
optimisation is incorrect as the expected process is not signalled.
While 6701bb30 reintroduced this problem for scripts, this has
always been an issue for certain other situations: forked command
substitutions, background subshells, and -c option argument
scripts. This commit fixes it in all those cases.
In sh_exec(), case TFORK, the optimisation (flagged in no_fork) was
only blocked for SIGINT and for the EXIT and ERR pseudosignals.
That is wrong. It should be blocked for all signal and pseudosignal
traps, except DEBUG (which is run before the command) and SIGKILL
and SIGSTOP (which cannot be trapped).
(I've also tested the behaviour of other shells. One shell, mksh,
never does an exec optimisation, even if no traps are set. I don't
know if that is intentional or not. I suppose it is possible that a
script might expect to receive a signal without trapping it first,
and they could conceivably be affected the same way by this exec
optimisation. But the ash variants (e.g. Busybox ash, dash, FreeBSD
sh), as well as bash, yash and zsh, all do act like this, so the
behaviour is very widespread. This commit makes ksh act like them.)
Multiple files:
- Remove the sh.errtrap, sh.exittrap and sh.end_fn flags and their
associated code from the superseded Solaris patch.
src/cmd/ksh93/include/shell.h:
- Add a scoped sh.st.trapdontexec flag for sh_exec() to disable
exec-without-fork optimisations. It should be in the sh.st scope
struct because it needs to be reset in subshell scopes.
src/cmd/ksh93/bltins/trap.c: b_trap():
- Set sh.st.trapdontexec if any trap is set and non-empty (an empty
trap means ignore the signal, which is inherited by an exec'd
process, so the optimisation is fine in that case).
- Only clear sh.st.trapdontexec if we're not in a ksh function
scope; unlike subshells, ksh functions fall back to parent traps
if they don't trap a signal themselves, so a ksh function's
parent traps also need to disable the exec optimisation.
src/cmd/ksh93/sh/fault.c: sh_sigreset():
- Introduce a new -1 mode for sh_funscope() to use, which acts like
mode 0 except it does not clear sh.st.trapdontexec. This avoids
clearing sh.st.trapdontexec for ksh functions scopes (see above).
- Otherwise, clear sh.st.trapdontexec whenever traps are reset.
src/cmd/ksh93/sh/xec.c: check_exec_optimization():
- Consolidate all the exec optimisation logic into this function,
including the logic from the no_fork flag in sh_exec()/TFORK.
- In the former no_fork flag logic, replace the three checks for
SIGINT, ERR and EXIT traps with a single check for the
sh.st.trapdontexec flag.
2022-06-18 15:44:30 +00:00
|
|
|
- When any trap except DEBUG, KILL or STOP is set to a non-empty command,
|
|
|
|
|
the last command in a script or forked subshell will no longer avoid forking
|
|
|
|
|
before executing; this optimization incorrectly bypassed the traps.
|
|
|
|
|
|
Forking workaround for converting to associative array in subshell
$ arch/*/bin/ksh -xc 'typeset -a a=(1 2 3); \
(typeset -A a; typeset -p a); typeset -p a'
typeset -A a=()
typeset -a a=(1 2 3)
The associative array in the subshell is empty, so the conversion
failed. So far, I have been unsuccessful at fixing this in the
array and/or virtual subshell code (a patch that fixes it there
would still be more than welcome).
As usual, real subshells work correctly, so this commit adds
another forking workaround. The use case is rare and specific
enough that I have no performance concerns.
src/cmd/ksh93/bltins/typeset.c: setall():
- Fork a virtual subshell if we're actually converting a variable
to an associative array, i.e.: the NV_ARRAY (-A, associative
array) attribute was passed, there are no assignments (sh.envlist
is NULL), and the variable is not unset.
src/cmd/ksh93/tests/arith.sh:
- Fix the "Array subscript quoting test" tests that should not have
been passing and that correctly failed after this fix; they used
'typeset -A' without an assignment in a subshell, assuming it was
unset in the parent shell, which it wasn't.
Resolves: https://github.com/ksh93/ksh/issues/409
2022-06-15 03:38:01 +00:00
|
|
|
2022-06-15:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug where converting an indexed array into an associative array in
|
|
|
|
|
a subshell failed, resulting in an empty associative array.
|
|
|
|
|
|
Fix subshell trap integrity, e.g. re-trapping a signal in subshell
Ksh handles local traps in virtual subshells the same way as local
traps in ksh-style shell functions, which can cause incorrect
operation.
Reproducer script:
trap 'echo "parent shell trap"; exit 0' USR1
(trap 'echo "subshell trap"' USR1; kill -USR1 $$)
echo wrong
Output on every ksh93 version: 'wrong'
Output on every other shell: 'parent shell trap'
The ksh93 output is wrong because $$ is the PID of the main shell,
therefore 'kill -USR1 $$' from a subshell needs to issue SIGUSR1 to
the main shell and trigger the 'echo SIGUSR1' trap.
This is an inevitable consequence of processing signals in a
virtual subshell. Signals are a process-wide property, but a
virtual subshell and the parent shell share the same process.
Therefore it is not possible to distinguish between the parent
shell and subshell trap.
This means virtual subshells are fundamentally incompatible with
receiving signals. No workaround can make this work properly.
Ksh could either assume the signal needs to be caught by the
subshell trap (wrong in this case, but right in others) or by the
parent shell trap. But it does neither and just gives up and does
nothing, which I suppose is the least bad way of doing it wrong.
As another example, consider a subshell that traps a signal, then
passes its own process ID (as of 9de65210, that's ${.sh.pid}) to
another process to say "here is where to signal me". A virtual
subshell will send it the PID that it shares with the the parent
shell. Even if a virtual subshell receives the signal correctly, it
may fork mid-execution afterwards, depending on the commands that
it runs (and this varies by implementation as we fix or work around
bugs). So its PID could be invalidated at any time.
Forking a virtual subshell at the time of trapping a signal is the
only way to ensure a persistent PID and correct operation.
src/cmd/ksh93/bltins/trap.c: b_trap():
- Fork when trapping (or ignoring) a signal in a virtual subshell.
(There's no need to fork when trapping a pseudosignal.)
src/cmd/ksh93/tests/signal.sh:
- Add tests. These are simplified versions of tests already there,
which issued 'kill' as a background job. Currently, running a
background job causes a virtual subshell to fork before forking
the 'kill' background job (or *any* background job, see e3d7bf1d)
-- an ugly partial workaround that I believe just became
redundant and which I will remove in the next commit.
2022-06-13 18:41:36 +00:00
|
|
|
2022-06-13:
|
|
|
|
|
|
|
|
|
|
- Trapping a signal that is not a pseudosignal will now cause a virtual
|
|
|
|
|
subshell to fork into a real one. This ensures a persistent PID where
|
|
|
|
|
other processes can signal the subshell (the PID of a virtual subshell may
|
|
|
|
|
change as other commands cause it to fork), and fixes a bug where a signal
|
|
|
|
|
could not be correctly or effectively trapped in a subshell if the same
|
|
|
|
|
signal was also trapped in the main shell environment.
|
|
|
|
|
|
2022-06-12 15:16:11 +00:00
|
|
|
2022-06-12:
|
|
|
|
|
|
|
|
|
|
- The POSIX mode now disables zero-padding of seconds in 'time'/'times' output.
|
|
|
|
|
|
posix: use real pipe(2) instead of socketpair(2)
The POSIX standard requires real UNIX pipes as in pipe(2). But on
systems supporting it (all modern ones), ksh uses socketpair(2)
instead to make it possible for certain commands to peek ahead
without consuming input from the pipe, which is not possible with
real pipes. See features/poll and sh/io.c.
But this can create undesired side effects: applications connected
to a pipe may test if they are connected to a pipe, which will fail
if they are connected to a socket. Also, on Linux:
$ cat /etc/passwd | head -1 /dev/stdin
head: cannot open '/dev/stdin' for reading: No such device or address
...which happens because, unlike most systems, Linux cannot open(2)
or openat(2) a socket (a limitation that is allowed by POSIX).
Unfortunately at least two things depend on the peekahead
capability of the _pipe_socketpair feature. One is the non-blocking
behaviour of the -n option of the 'read' built-in:
-n Causes at most n bytes to be read instead of a full
line, but will return when reading from a slow device as
soon as any characters have been read.
The other thing that breaks is the <#pattern and <##pattern
redirection operators that basically grep standard input, which
inherently requires peekahead.
Standard UNIX pipes always block on read and it is not possible to
peek ahead, so these features inevitably break. Which means we
cannot simply use standard pipes without breaking compatibility.
But we can at least fix it in the POSIX mode so that cross-platform
scripts work more correctly.
src/cmd/ksh93/sh/io.c: sh_pipe():
- If _pipe_socketpair is detected at compile time, then use a real
pipe via sh_rpipe() if the POSIX mode is active. (If
_pipe_socketpair is not detected, a real pipe is always used.)
src/cmd/ksh93/data/builtins.c:
- sh.1 documents the slow-device behaviour of -n but 'read --man'
did not. Add that, making it conditional on _pipe_socketpair.
Resolves: https://github.com/ksh93/ksh/issues/327
2022-06-09 14:51:44 +00:00
|
|
|
2022-06-09:
|
|
|
|
|
|
|
|
|
|
- The POSIX mode has been amended to use a UNIX pipe(2) instead of a
|
|
|
|
|
socketpair(2) to connect commands in a pipeline, as the standard requires.
|
|
|
|
|
(When reading directly from a pipeline in posix mode, the <#pattern and
|
|
|
|
|
<##pattern redirection operators will not work and the -n option to the
|
|
|
|
|
read built-in will not return early when reading from a slow device.)
|
|
|
|
|
|
2022-06-09 20:50:26 +00:00
|
|
|
- An interactive shell initialized in POSIX mode now once again has the
|
|
|
|
|
preset aliases defined. This amends a change made on 2020-09-11.
|
|
|
|
|
|
2022-06-09 21:22:24 +00:00
|
|
|
- If a long-form shell option name is abbreviated so it matches multiple
|
|
|
|
|
options, the error message from ksh or 'set' now reads "ambiguous option".
|
|
|
|
|
|
posix: block brace expansion of unquoted expansions (re: a14d17c0)
Historically, ksh (including ksh88 and mksh) allow brace expansion
not just on literal patterns but also on patterns resulting from
unquoted variable expansions or command substitutions:
$ a='{a,b}' ksh -c 'echo X{a,b} Y$a'
Xa Xb Ya Yb
Most people expect only the first (literal) pattern to be expanded,
as in bash and zsh:
$ a='{a,b}' bash -c 'echo X{a,b} Y$a'
Xa Xb Y{a,b}
The historic ksh behaviour is poorly documented and nearly unknown,
violates the principle of least astonishment, and makes unquoted
variable expansions even more unsafe. See discussion at:
https://www.austingroupbugs.net/view.php?id=1193
https://github.com/ksh93/ksh/issues/140
Unfortunately, we cannot change it in default ksh without breaking
backward compatibility. But we can at least fix it for the POSIX
mode (which disables brace expansion by default but allows turning
it back on), particularly as it looks like POSIX, if it decides to
specify brace expansion in a future version of the standard, will
disallow brace expansion on unquoted variable expansions.
src/cmd/ksh93/sh/macro.c: endfield():
- When deciding whether to do brace expansion + globbing or only
globbing, also check that we do not have POSIX mode and an
unquoted variable expansion (mp->pattern==1).
2022-06-08 19:56:18 +00:00
|
|
|
2022-06-08:
|
|
|
|
|
|
|
|
|
|
- If -B/--braceexpand is turned on in --posix mode, it now only allows brace
|
|
|
|
|
expansion on literal strings and not on the values of unquoted expansions.
|
|
|
|
|
This behaviour is like bash and zsh, as well as compliant with a future
|
|
|
|
|
version of the POSIX standard that may specify brace expansion:
|
|
|
|
|
https://www.austingroupbugs.net/view.php?id=1193
|
|
|
|
|
|
2022-06-07 03:16:48 +00:00
|
|
|
2022-06-07:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug in 'typeset' where compound variables could acquire a spurious
|
|
|
|
|
-x (export) attribute when printing their values, triggering other bugs.
|
|
|
|
|
|
2022-06-06 01:12:33 +00:00
|
|
|
2022-06-05:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug where tab completion would spuriously execute a command
|
|
|
|
|
substitution after entering: '`something<TAB> or generate a spurious
|
|
|
|
|
'end of file unexpected' syntax error after entering: '$(something<TAB>.
|
|
|
|
|
|
2022-06-06 01:46:57 +00:00
|
|
|
- Fixed bug where tab completion stopped working for arguments following a
|
|
|
|
|
quoted string with an escaped backslash in the form $'foo\'bar'.
|
|
|
|
|
|
Add --functrace shell option (re: 2a835a2d)
A side effect of the bug fixed in 2a835a2d caused the DEBUG trap
action to appear to be inherited by subshells, but in a buggy way
that could crash the shell. After the fix, the trap is reset in
subshells along with all the others, as it should be. Nonetheless,
as that bug was present for years, some have come to rely on it.
This commit implements that functionality properly. When the new
--functrace option is turned on, DEBUG trap actions are now
inherited by subshells as well as ksh function scopes. In addition,
since it makes logical sense, the new option also causes the
-x/--xtrace option's state to be inherited by ksh function scopes.
Note that changes made within the scope do not propagate upwards;
this is different from bash.
(I've opted against adding a -T short-form equivalent as on bash,
because -T was formerly a different option on 93u+ (see 63c55ad7)
and on mksh it has yet anohter a different meaning. To minimise
confusion, I think it's best to have the long-form name only.)
src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/data/options.c:
- Add new "functrace" (SH_FUNCTRACE) long-form shell option.
src/cmd/ksh93/sh/subshell.c: sh_subshell():
- When functrace is on, copy the parent's DEBUG trap action into
the virtual subshell scope after resetting the trap actions.
src/cmd/ksh93/sh/xec.c: sh_funscope():
- When functrace is on and xtrace is on in the parent scope, turn
it on in the child scope.
- Same DEBUG trap action handling as in sh_subshell().
Resolves: https://github.com/ksh93/ksh/issues/162
2022-06-04 16:02:05 +00:00
|
|
|
2022-06-04:
|
|
|
|
|
|
|
|
|
|
- Added a new --functrace long-form shell option which causes the -x/--xtrace
|
|
|
|
|
option's state and the DEBUG trap action to be inherited by function scopes
|
|
|
|
|
instead of being reset to default. Changes made to them within a function
|
|
|
|
|
scope still do not propagate back to the parent scope. Similarly, this
|
|
|
|
|
option also causes the DEBUG trap action to be inherited by subshells.
|
|
|
|
|
|
2022-06-03 10:34:34 +00:00
|
|
|
2022-06-03:
|
|
|
|
|
|
|
|
|
|
- Fixed a couple of bugs that caused floating point variables imported from
|
|
|
|
|
a parent ksh through the environment to be corrupted.
|
|
|
|
|
|
Honour attributes for local assignments preceding certain commands
Reproducer:
$ typeset -i NUM=123
$ (NUM=3+4 env; :)|grep ^NUM=
NUM=3+4
$ (NUM=3+4 env)|grep ^NUM=
NUM=7
The correct output is NUM=7. This is also the output if ksh is
compiled with SHOPT_SPAWN disabled, suggesting that the problem is
here in sh_ntfork(), where the invocation-local environment list is
set using a sh_scope() call:
src/cmd/ksh93/sh/xec.c:
3496: if(t->com.comset)
3497: {
3498: scope++;
3499: sh_scope(t->com.comset,0);
3500: }
Analysis:
When ksh is compiled with SHOPT_SPAWN disabled, external commands
are always run using the regular forking mechanism. First the shell
forks, then in the fork, the preceding assignments list (if any)
are executed and exported in the main scope. Replacing global
variables is not a problem as the variables are exported and the
forked shell is then replaced by the external command using
execve(2).
But when using SHOPT_SPAWN/sh_ntfork(), this cannot be done as the
fork(2) use is replaced by posix_spawn(2) which does not copy the
parent process into the child, therefore it's not possible to
execute anything in the child before calling execve(2). Which means
the preceding assignments list must be processed in the parent, not
the child. Which makes overwriting global variables a no-no.
To avoid overwriting global variables, sh_ntfork() treats preceding
assignments like local variables in functions, which means they do
not inherit any attributes from the parent scope. That is why the
integer attribute is not honoured in the buggy reproducers.
And this is not just an issue for external commands. sh_scope() is
also used for assignments preceding a built-in command. Which is
logical, as those don't create a process at all.
src/cmd/ksh93/sh/xec.c:
1325: if(argp)
1326: {
1327: scope++;
1328: sh_scope(argp,0);
1329: }
Which means this bug exists for them as well, regardless of whether
SHOPT_SPAWN is compiled in.
$ /bin/ksh -c 'typeset -i NUM; NUM=3+4 command eval '\''echo $NUM'\'
3+4
(expected: 7, as on mksh and zsh)
So, the attributes from the parent scope will need to be copied
into the child scope. This should be done in nv_setlist() which is
called from sh_scope() with both the NV_EXPORT and NV_NOSCOPE flags
passed. Those flag bits are how we can recognise the need to copy
attributes.
Commit f6bc5c0 fixed a similar inconsistency with the check for the
read-only attribute. In fact, the bug fixed there was simply a
specific instance of this bug. The extra check for readonly was
because the readonly attribute was not copied to the temporary
local scope. So that fix is now replaced by the more general fix
for this bug.
src/cmd/ksh93/sh/name.c: nv_setlist():
- Introduce a 'vartree' local variable to avoid repetitive
'sh.prefix_root ? sh.prefix_root : sh.var_tree' expressions.
- Use the NV_EXPORT|NV_NOSCOPE flag combination to check if parent
scope attributes need to be copied to the temporary local scope
of an assignment preceding a command.
- If so, copy everything but the value itself: attributes (nvflag),
size parameter (nvsize), discipline function pointer (nvfun) and
the name pointer (nvname). The latter is needed because some
code, at least put_lang() in init.c, compares names by comparing
the pointers instead of the strings; copying the nvname pointer
avoids a regression in tests/locale.sh.
src/cmd/ksh93/sh/xec.c: local_exports():
- Fix a separate bug exporting attributes to a new ksh function
scope, which was previously masked by the other bug. The
attributes (nvflag) were copied *after* nv_putval()ing the value,
which is incorrect as the behaviour of nv_putval() is influenced
by the attributes. But here, we're copying the value too, so we
can simplify the whole function by using nv_clone() instead. This
may also fix other corner cases. (re: c1994b87)
Resolves: https://github.com/ksh93/ksh/issues/465
2022-06-03 20:09:16 +00:00
|
|
|
- Fixed a bug where invocation-local assignments preceding a built-in or
|
Honour attribs for assignments preceding sp. builtins, POSIX functs
After the previous commit, one inconsistency was left. Assignments
preceding special built-ins and POSIX functions (which persist past
the command :-/) caused pre-existing attributes of the respective
variables to be cleared.
$ (f() { typeset -p n; }; typeset -i n; n=3+4 f)
n=3+4
(expected output: 'typeset -i n=7')
This problem was introduced shortly before the release of ksh 93u+,
in version 2012-05-04, by adding these lines of code to the code
for processing preceding assignments in sh_exec():
src/cmd/ksh93/sh/xec.c:
1055: if(np)
1056: flgs |= NV_UNJUST;
So, for special and declaration commands and POSIX functions, the
NV_UNJUST flag is passed to nv_open(). In older ksh versions, this
flag cleared justify attributes only, but in early 2012 it was
repurposed to clear *all* attributes -- without changing the name
or the relevant comment in name.h, which are now both misleading.
The reason for setting this flag in xec.c was to deal with some
bugs in 'typeset'. Removing those two lines causes regressions:
attributes.sh[316]: FAIL: typeset -L should not preserve old attributes
attributes.sh[322]: FAIL: typeset -R should not preserve old attributes
attributes.sh[483]: FAIL: typeset -F after typeset -L fails
attributes.sh[488]: FAIL: integer attribute not cleared for subsequent typeset
Those are all typeset regressions, which suggests this fix was
relevant to typeset only. This is corroborated by the relevant
AT&T ksh93/RELEASE entry:
12-04-27 A bug in which old attributes were not cleared when
assigning a value using typeset has been fixed.
So, to fix this 2012 regression without reintroducing the typeset
regressions, we need to set the NV_UNJUST flag for invocations of
the typeset family of commands only. This is changed in xec.c.
While we're at it, we might as well rename that little-used flag to
something that reflects its current purpose: NV_UNATTR.
2022-06-03 21:51:35 +00:00
|
|
|
external command or function call did not honour their pre-existing
|
Honour attributes for local assignments preceding certain commands
Reproducer:
$ typeset -i NUM=123
$ (NUM=3+4 env; :)|grep ^NUM=
NUM=3+4
$ (NUM=3+4 env)|grep ^NUM=
NUM=7
The correct output is NUM=7. This is also the output if ksh is
compiled with SHOPT_SPAWN disabled, suggesting that the problem is
here in sh_ntfork(), where the invocation-local environment list is
set using a sh_scope() call:
src/cmd/ksh93/sh/xec.c:
3496: if(t->com.comset)
3497: {
3498: scope++;
3499: sh_scope(t->com.comset,0);
3500: }
Analysis:
When ksh is compiled with SHOPT_SPAWN disabled, external commands
are always run using the regular forking mechanism. First the shell
forks, then in the fork, the preceding assignments list (if any)
are executed and exported in the main scope. Replacing global
variables is not a problem as the variables are exported and the
forked shell is then replaced by the external command using
execve(2).
But when using SHOPT_SPAWN/sh_ntfork(), this cannot be done as the
fork(2) use is replaced by posix_spawn(2) which does not copy the
parent process into the child, therefore it's not possible to
execute anything in the child before calling execve(2). Which means
the preceding assignments list must be processed in the parent, not
the child. Which makes overwriting global variables a no-no.
To avoid overwriting global variables, sh_ntfork() treats preceding
assignments like local variables in functions, which means they do
not inherit any attributes from the parent scope. That is why the
integer attribute is not honoured in the buggy reproducers.
And this is not just an issue for external commands. sh_scope() is
also used for assignments preceding a built-in command. Which is
logical, as those don't create a process at all.
src/cmd/ksh93/sh/xec.c:
1325: if(argp)
1326: {
1327: scope++;
1328: sh_scope(argp,0);
1329: }
Which means this bug exists for them as well, regardless of whether
SHOPT_SPAWN is compiled in.
$ /bin/ksh -c 'typeset -i NUM; NUM=3+4 command eval '\''echo $NUM'\'
3+4
(expected: 7, as on mksh and zsh)
So, the attributes from the parent scope will need to be copied
into the child scope. This should be done in nv_setlist() which is
called from sh_scope() with both the NV_EXPORT and NV_NOSCOPE flags
passed. Those flag bits are how we can recognise the need to copy
attributes.
Commit f6bc5c0 fixed a similar inconsistency with the check for the
read-only attribute. In fact, the bug fixed there was simply a
specific instance of this bug. The extra check for readonly was
because the readonly attribute was not copied to the temporary
local scope. So that fix is now replaced by the more general fix
for this bug.
src/cmd/ksh93/sh/name.c: nv_setlist():
- Introduce a 'vartree' local variable to avoid repetitive
'sh.prefix_root ? sh.prefix_root : sh.var_tree' expressions.
- Use the NV_EXPORT|NV_NOSCOPE flag combination to check if parent
scope attributes need to be copied to the temporary local scope
of an assignment preceding a command.
- If so, copy everything but the value itself: attributes (nvflag),
size parameter (nvsize), discipline function pointer (nvfun) and
the name pointer (nvname). The latter is needed because some
code, at least put_lang() in init.c, compares names by comparing
the pointers instead of the strings; copying the nvname pointer
avoids a regression in tests/locale.sh.
src/cmd/ksh93/sh/xec.c: local_exports():
- Fix a separate bug exporting attributes to a new ksh function
scope, which was previously masked by the other bug. The
attributes (nvflag) were copied *after* nv_putval()ing the value,
which is incorrect as the behaviour of nv_putval() is influenced
by the attributes. But here, we're copying the value too, so we
can simplify the whole function by using nv_clone() instead. This
may also fix other corner cases. (re: c1994b87)
Resolves: https://github.com/ksh93/ksh/issues/465
2022-06-03 20:09:16 +00:00
|
|
|
attributes set by typeset (such as -i or -F) in certain cases.
|
|
|
|
|
|
New feature: 'typeset -g' as in bash 4.2+
typeset -g allows directly manipulating the attributes of variables
at the global level from any context. This feature already exists
on bash 4.2 and later.
mksh (R50+), yash and zsh have this flag as well, but it's slightly
different: it ignores the current local scope, but a parent local
scope from a calling function may still be used -- whereas on bash,
'-g' always refers to the global scope. Since ksh93 uses static
scoping (see III.Q28 at <http://kornshell.com/doc/faq.html>), only
the bash behaviour makes sense here.
Note that the implementation needs to be done both in nv_setlist()
(name.c) and in b_typeset() (typeset.c) because assignments are
executed before the typeset built-in itself. Hence also the
pre-parsing of typeset options in sh_exec().
src/cmd/ksh93/include/nval.h:
- Add new NV_GLOBAL bit flag, using a previously unused bit that
still falls within the 32-bit integer range.
src/cmd/ksh93/sh/xec.c: sh_exec():
- When pre-parsing typeset flags, make -g pass the NV_GLOBAL flag
to the nv_setlist() call that processes shell assignments prior
to running the command.
src/cmd/ksh93/sh/name.c: nv_setlist():
- When the NV_GLOBAL bit flag is passed, save the current variable
tree pointer (sh.var_tree) as well as the current namespace
(sh.namespace) and temporarily set the former to the global
variable tree (sh.var_base) and the latter to NULL. This makes
assignments global and ignores namesapces.
src/cmd/ksh93/bltins/typeset.c:
- b_typeset():
- Use NV_GLOBAL bit flag for -g.
- Allow combining -n with -g, permitting 'typeset -gn var' or
'nameref -g var' to create a global nameref from a function.
- Do not allow a nonsensical use of -g when using nested typeset
to define member variables of 'typeset -T' types. (Type method
functions can still use it as normal.)
- setall():
- If NV_GLOBAL is passed, use sh.var_base and deactivate
sh.namespace as in nv_setlist(). This allows attributes
to be set correctly for global variables.
src/cmd/ksh93/tests/{functions,namespace}.sh:
- Add regression tests based on reproducers for problems found
by @hyenias in preliminary versions of this feature.
Resolves: https://github.com/ksh93/ksh/issues/479
2022-06-01 16:14:51 +00:00
|
|
|
2022-06-01:
|
|
|
|
|
|
|
|
|
|
- New 'typeset' feature inspired by bash (and zsh and mksh): the -g flag
|
|
|
|
|
causes the specified variable(s) to be created globally, ignoring the
|
|
|
|
|
local scope of ksh functions or name spaces. For example, this allows
|
|
|
|
|
creating a global array or numeric variable from a ksh function.
|
|
|
|
|
Thanks to @ormaaj for the feature suggestion and to @hyenias for testing.
|
|
|
|
|
|
2022-05-29 21:50:24 +00:00
|
|
|
2022-05-29:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug causing an inconsistent state after a special built-in command
|
|
|
|
|
threw an error within a namespace.
|
|
|
|
|
|
2022-05-27 23:02:46 +00:00
|
|
|
2022-05-27:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug introduced on 2022-03-05 where 'set --default', while turning
|
|
|
|
|
off the --posix option, did not re-enable the special handling of a repeated
|
|
|
|
|
$IFS whitespace character as non-whitespace.
|
|
|
|
|
|
2022-05-25 23:04:53 +00:00
|
|
|
2022-05-25:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug introduced on 2021-02-20 that caused incorrect output for
|
|
|
|
|
typeset -L/-R/-Z when the variable contained leading or trailing spaces.
|
|
|
|
|
Thanks to George Lijo for the report and the fix.
|
|
|
|
|
|
Fix line continuation within command substitutions
In command substitutions of the $(standard) and ${ shared state; }
form, backslash line continuation is broken.
Reproducer:
echo $(
echo one two\
three
)
Actual output (ksh93, all versions):
one two\ three
Expected output (every other shell, POSIX spec):
one twothree
src/cmd/ksh93/sh/lex.c: sh_lex(): case S_REG:
- Do not skip new-line joining if we're currently processing a
command substitution of one of these forms (i.e., if the
lp->lexd.dolparen level is > 0).
Background info/analysis:
comsub() is called from sh_lex() when S_PAR is the current state.
In src/cmd/ksh93/data/lexstates.c, we see that S_PAR is reached in
the ST_DOL state table at index 40. Decimal 40 is ( in ASCII. So,
the previous skipping of characters was done according to the
ST_DOL state table, and the character that stopped it was (. This
means we have $(.
Alternatively, comsub() may be called from sh_lex() by jumping to
the do_comsub label. In brief, that is the case when we have ${.
Regardless of which it is from the two, comsub() is now called from
sh_lex(). In comsub(), lp->lexd.dolparen is incremented at the
beginning and decremented at the end. Between them, we see that
sh_lex() is called. So, lp->lexd.dolparen in sh_lex() indicates the
depth of nesting $( or ${ statements we're in. Thus, it is also the
number of comsub() invocations seen in a backtrace taken in
sh_lex().
The codepath for `...` is different (and never had this bug).
Co-authored by: Martijn Dekker <martijn@inlv.org>
Resolves: https://github.com/ksh93/ksh/issues/367
2022-05-21 22:51:50 +00:00
|
|
|
2022-05-21:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug, present since the beginning of ksh93, that broke backslash line
|
|
|
|
|
continuation within $(standard) and ${ shared-state; } command substitutions.
|
|
|
|
|
Thanks to atheik for the analysis and the fix. (BUG_CSUBLNCONT)
|
|
|
|
|
|
2022-05-20 21:29:20 +00:00
|
|
|
2022-05-20:
|
|
|
|
|
|
|
|
|
|
- Fixed an ancient bug that caused a spurious syntax error when using double
|
|
|
|
|
quotes within a `backtick` command substitution within double quotes.
|
Fix line continuation within command substitutions
In command substitutions of the $(standard) and ${ shared state; }
form, backslash line continuation is broken.
Reproducer:
echo $(
echo one two\
three
)
Actual output (ksh93, all versions):
one two\ three
Expected output (every other shell, POSIX spec):
one twothree
src/cmd/ksh93/sh/lex.c: sh_lex(): case S_REG:
- Do not skip new-line joining if we're currently processing a
command substitution of one of these forms (i.e., if the
lp->lexd.dolparen level is > 0).
Background info/analysis:
comsub() is called from sh_lex() when S_PAR is the current state.
In src/cmd/ksh93/data/lexstates.c, we see that S_PAR is reached in
the ST_DOL state table at index 40. Decimal 40 is ( in ASCII. So,
the previous skipping of characters was done according to the
ST_DOL state table, and the character that stopped it was (. This
means we have $(.
Alternatively, comsub() may be called from sh_lex() by jumping to
the do_comsub label. In brief, that is the case when we have ${.
Regardless of which it is from the two, comsub() is now called from
sh_lex(). In comsub(), lp->lexd.dolparen is incremented at the
beginning and decremented at the end. Between them, we see that
sh_lex() is called. So, lp->lexd.dolparen in sh_lex() indicates the
depth of nesting $( or ${ statements we're in. Thus, it is also the
number of comsub() invocations seen in a backtrace taken in
sh_lex().
The codepath for `...` is different (and never had this bug).
Co-authored by: Martijn Dekker <martijn@inlv.org>
Resolves: https://github.com/ksh93/ksh/issues/367
2022-05-21 22:51:50 +00:00
|
|
|
Thanks to atheik for the analysis and the fix. (BUG_CSUBBTQUOT)
|
2022-05-20 21:29:20 +00:00
|
|
|
|
2022-03-11 01:15:41 +00:00
|
|
|
2022-03-10:
|
|
|
|
|
|
|
|
|
|
- Fixed another corner case bug in the 'test'/'[' command.
|
|
|
|
|
|
2022-03-11 19:24:35 +00:00
|
|
|
- A memory leak was fixed in the --man self-documentation options
|
|
|
|
|
of built-in commands.
|
|
|
|
|
|
2022-03-05 00:43:29 +00:00
|
|
|
2022-03-05:
|
|
|
|
|
|
|
|
|
|
- The 'enum' command can now create more than one type per invocation.
|
|
|
|
|
|
2022-03-08 19:20:57 +00:00
|
|
|
- The POSIX compatibility mode has been amended to disable the special handling
|
|
|
|
|
of a repeated $IFS whitespace character as non-whitespace.
|
|
|
|
|
|
2022-02-23 15:11:24 +00:00
|
|
|
2022-02-23:
|
2022-02-23 03:39:15 +00:00
|
|
|
|
|
|
|
|
- When reading input from the keyboard, ksh now turns off nonblocking I/O
|
|
|
|
|
mode for standard input if a previously ran program left it on, so that
|
|
|
|
|
interactive programs that expect it to be off work properly.
|
|
|
|
|
|
2022-02-23 15:11:24 +00:00
|
|
|
- Fixed a regression introduced on 2022-02-02 that caused interactive shells
|
|
|
|
|
to enter an infinite loop when a command failed to execute on Linux
|
|
|
|
|
systems with version 2.35 of glibc.
|
|
|
|
|
|
|
|
|
|
- Fixed a SIGTTOU lockup that could cause ksh to freeze under strace(1) after
|
|
|
|
|
a command failed to execute in an interactive shell.
|
|
|
|
|
|
2022-02-19 20:55:35 +00:00
|
|
|
2022-02-18:
|
|
|
|
|
|
|
|
|
|
- Fixed a regression introduced on 2021-04-11 that caused the += operator in
|
|
|
|
|
invocation-local assignments to crash the shell or modify variables outside
|
|
|
|
|
of the invocation-local scope.
|
|
|
|
|
|
|
|
|
|
2022-02-17:
|
2022-02-17 16:19:17 +00:00
|
|
|
|
|
|
|
|
- Fixed a crash, introduced on 2021-01-19, that occurred when using 'cd' in
|
|
|
|
|
a subshell with the PWD variable unset.
|
|
|
|
|
|
Fix crash when suspending a blocked write to a FIFO
Reproducer (symptoms on at least macOS and FreeBSD):
$ mkfifo f
$ echo foo > f
(press Ctrl+Z)
^Zksh: f: cannot create [Interrupted system call]
Abort
The shell either aborts (dev builds) or crashes with 'Illegal
instruction' (release builds). This is consistent with
UNREACHABLE() being reached.
Backtrace:
0 libsystem_kernel.dylib __kill + 10
1 ksh sh_done + 836 (fault.c:678)
2 ksh sh_fault + 1324
3 libsystem_platform.dylib _sigtramp + 29
4 dyld ImageLoaderMachOCompressed::resolve(ImageLoader::LinkContext const&, char const*, unsigned ch
5 libsystem_c.dylib abort + 127
6 ksh sh_redirect + 3576 (io.c:1356)
7 ksh sh_exec + 7231 (xec.c:1308)
8 ksh exfile + 3247 (main.c:607)
9 ksh sh_main + 3551 (main.c:368)
10 ksh main + 38 (pmain.c:45)
11 libdyld.dylib start + 1
This means that UNREACHABLE() is actually reached here:
ksh/src/cmd/ksh93/sh/io.c
1351: if((fd=sh_open(tname?tname:fname,o_mode,RW_ALL)) <0)
1352: {
1353: errormsg(SH_DICT,ERROR_system(1),((o_mode&O_CREAT)?e_create:e_open),fname);
1354: UNREACHABLE();
1355: }
The cause is that, in the following section of code in sh_fault():
ksh/src/cmd/ksh93/sh/fault.c
183: #ifdef SIGTSTP
184: if(sig==SIGTSTP)
185: {
186: sh.trapnote |= SH_SIGTSTP;
187: if(pp->mode==SH_JMPCMD && sh_isstate(SH_STOPOK))
188: {
189: sigrelease(sig);
190: sh_exit(SH_EXITSIG);
191: return;
192: }
193: }
194: #endif /* SIGTSTP */
...sh_exit() is not getting called and the function will not return
because the SH_STOPOK bit is not set while the shell is blocked
waiting to write to a FIFO.
Even if sh_exit() did get called, that would not fix it, because
that function also checks for the SH_STOPOK bit and returns without
doing a longjmp if the signal is SIGTSTP and the SH_STOPOK bit is
not set. That is direct the reason why UNREACHABLE() was raeched:
errormsg() does call sh_exit() but sh_exit() then does not longjmp.
src/cmd/ksh93/sh/fault.c: sh_fault():
- To avoid the crash, we simply need to return from sh_fault() if
SH_STOPOK is off, so that the code path does not continue, no
error message is given on Ctrl+Z, UNREACHABLE() is not reached,
and the shell resumes waiting on trying to write to the FIFO.
The sh.trapnote flag should not be set if we're not going to
process the signal. This makes ksh behave like all other shells.
Resolves: https://github.com/ksh93/ksh/issues/464
2022-02-17 20:07:56 +00:00
|
|
|
- Fixed a crash that could occur when or after entering the suspend character
|
|
|
|
|
(Ctrl+Z) while the shell was blocked trying to write to a FIFO special file.
|
|
|
|
|
|
2022-02-19 20:55:35 +00:00
|
|
|
2022-02-16:
|
2022-02-16 07:19:37 +00:00
|
|
|
|
|
|
|
|
- Backported minor additions to the 'read' built-in command from ksh 93v-:
|
|
|
|
|
'-a' is now the same as '-A' and '-u p' is the same as '-p'. This is for
|
|
|
|
|
compatibility with some 93v- or ksh2020 scripts. (Note that their change
|
|
|
|
|
to the '-p' option to support both prompts and coprocess reads was not
|
|
|
|
|
backported because we found it to be broken and unfixable.)
|
|
|
|
|
|
2022-02-15 17:54:45 +00:00
|
|
|
2022-02-15:
|
|
|
|
|
|
|
|
|
|
- A bug was fixed in fast filescan loops (like 'while <file; do ...; done',
|
|
|
|
|
enabled by default by the SHOPT_FILESCAN compile-time option) in which
|
|
|
|
|
"$@" incorrectly acted like "$*" within the do...done block.
|
|
|
|
|
|
|
|
|
|
- Improved sanity check for filescan loops: they are now only recognised if
|
|
|
|
|
'while' is followed by a single input redirection without any command
|
|
|
|
|
name/arguments or variable assignments.
|
|
|
|
|
|
|
|
|
|
- As SHOPT_FILESCAN has been enabled by default since ksh 93l 2001-06-01,
|
|
|
|
|
the fast filescan loop is now documented in the manual page and the
|
|
|
|
|
compile-time option is no longer considered experimental.
|
|
|
|
|
|
|
|
|
|
- Filescan loops are now disabled at runtime if the --posix option is active
|
|
|
|
|
as they break a portable use case: POSIXly, 'while <file; do stuff; done'
|
2022-03-01 19:38:15 +00:00
|
|
|
repeatedly executes 'stuff' while 'file' can successfully be opened for
|
2022-02-15 17:54:45 +00:00
|
|
|
reading, without actually reading from 'file'.
|
|
|
|
|
|
2022-02-12 21:27:36 +00:00
|
|
|
2022-02-12:
|
|
|
|
|
|
|
|
|
|
- In multibyte locales such as UTF-8, shell input is no longer corrupted when
|
|
|
|
|
a KEYBD trap is active. However, the KEYBD trap is not yet multibyte ready
|
|
|
|
|
and is now only triggered for ASCII characters (1-127) in a multibyte locale.
|
|
|
|
|
|
2022-02-11 02:21:47 +00:00
|
|
|
2022-02-11:
|
|
|
|
|
|
2022-02-11 12:02:27 +00:00
|
|
|
- On the interactive shell, tab/esc completion is no longer disabled as a side
|
|
|
|
|
effect of turning off pathname expansion with 'set -f' or 'set -o noglob'.
|
|
|
|
|
|
2022-02-11 02:21:47 +00:00
|
|
|
- Fixed a bug in which a >&- or <&- redirection could persist past a
|
|
|
|
|
subshell in certain corner cases involving 'exec' or 'redirect'.
|
|
|
|
|
|
2022-02-08 18:33:44 +00:00
|
|
|
2022-02-08:
|
|
|
|
|
|
Fix multiple bugs in executing scripts without a #! path
When executing a script without a hashbang path like #!/bin/ksh,
ksh forks itself, longjmps back to sh_main(), and then (among other
things) calling sh_reinit() which is the function that tries to
reinitialise as much of the shell as it can. This is its way of
ensuring the child script is run in ksh and not some other shell.
However, this appraoch is incredibly buggy. Among other things,
changes in built-in commands and custom type definitions survived
the reinitialisation, "exporting" variables didn't work properly,
and the hash table and ${.sh.stats} weren't reset. As a result,
depending on what the invoking script did, the invoked script could
easily fail or malfunction.
It is not actually possible to reinitialise the shell correctly,
because some of the shell state is in locally scoped static
variables that cannot simply be reinitialised. There are probably
huge memory leaks with this approach as well. At some point, all
this is going to need a total redesign. Clearly, the only reliable
way involves execve(2) and a start from scratch.
For now though, this seems to fix the known bugs at least. I'm sure
there are more to be discovered.
This commit makes another change: instead of the -h/trackall option
(which has been a no-op for decades), the posix option is now
inherited by the child script. Since there is no hashbang path from
which to decide whether the shell should run in POSIX mode nor not,
the best guess is probably the invoking script's setting.
src/cmd/ksh93/sh/init.c: sh_reinit():
- Keep the SH_INIT state on during the entire procedure.
- Delete remaining non-exported, non-default variables.
- Remove attributes from exported variables. In POSIX mode, remove
all attributes; otherwise, only remove readonly.
- Unset discipline function pointers for variables.
- Delete all custom types.
- Delete all functions and built-ins, then reinitialise the built-ins
table from scatch.
- Free the alias values before clearing the alias table.
- Same with hash table entries (tracked aliases).
- Reset statistics.
- Inherit SH_POSIX instead of SH_TRACKALL.
- Call user init function last, not somewhere in the middle.
src/cmd/ksh93/sh/name.c: sh_envnolocal():
- Be sure to preserve the export attribute of kept variables.
Resolves: https://github.com/ksh93/ksh/issues/350
2022-02-08 21:40:19 +00:00
|
|
|
- Multiple bugs were fixed in the method that ksh uses to execute a shell
|
|
|
|
|
script without a #! path. Among other things, custom types and other
|
|
|
|
|
changes in built-in commands will no longer affect the invoked script.
|
|
|
|
|
The state of the 'posix' option is now inherited by the script.
|
|
|
|
|
|
2022-02-08 18:33:44 +00:00
|
|
|
- To avoid an inconsistent state, the 'enum' and 'typeset -T' commands are
|
|
|
|
|
no longer allowed to replace special built-in commands. For instance,
|
|
|
|
|
'enum trap=(a b c)' is now an error because 'trap' is a special built-in.
|
|
|
|
|
The ability of 'typeset -T' to redefine a type is not affected.
|
|
|
|
|
|
|
|
|
|
- A command like 'typeset -T foo_t' without any assignment no longer creates
|
2022-03-01 19:38:15 +00:00
|
|
|
an incompletely defined 'foo_t' built-in command that will crash the shell
|
2022-02-08 18:33:44 +00:00
|
|
|
when used. Instead, it is now silently ignored for backwards compatibility.
|
|
|
|
|
|
2022-02-08 19:52:13 +00:00
|
|
|
- Compound arrays belonging to a type created with 'typeset -T' now have
|
|
|
|
|
their -C attribute preserved in the output of 'typeset -p'.
|
|
|
|
|
|
|
|
|
|
- A bug has been fixed in which the 'typeset -p' output of a two-dimensional
|
|
|
|
|
sparse indexed array would cause the second subscript to be treated as an
|
|
|
|
|
associative array when read back in by the shell. Elements that are sparse
|
|
|
|
|
indexed arrays are now prefixed with "typeset -a".
|
2022-02-08 19:40:17 +00:00
|
|
|
|
2022-02-09 00:01:40 +00:00
|
|
|
- The rewritten .sh.match code from ksh93v- has been backported to ksh93u+m,
|
|
|
|
|
fixing many bugs and improving performance by a considerable amount.
|
|
|
|
|
|
2022-02-05 15:04:12 +00:00
|
|
|
2022-02-05:
|
|
|
|
|
|
Fix conditional expansions ${array[i]=value}, ${array[i]?error}
$ unset foo
$ echo ${foo[42]=bar}
(empty line)
Instead of the empty line, 'bar' was expected. As foo[42] was
unset, the conditional assignment should have worked.
$ unset foo
$ : ${foo[42]?error: unset}
(no output)
The expansion should have thrown an error with the given message.
This bug was introduced in ksh 93t 2008-10-01. Thanks to @JohnoKing
for finding the breaking change.
Analysis: The problem was experimenally determined to be in in the
following lines of nv_putsub(). If the array member is unset (i.e.
null), the value is set to the empty string instead:
src/cmd/ksh93/sh/array.c
1250: else
1251: ap->val[size].cp = Empty;
It makes some sense: if there is a value (even an empty one), the
variable is set and these expansions should behave accordingly.
Sure enough, deleting these lines fixes the bug, but at the expense
of introducing a lot of other array-related regressions. So we need
a way to special-case the affected expansions.
Where to do this? If we replace line 1251 with an abort(3) call, we
get this stack trace:
0 libsystem_kernel.dylib __pthread_kill + 10
1 libsystem_pthread.dylib pthread_kill + 284
2 libsystem_c.dylib abort + 127
3 ksh nv_putsub + 1411 (array.c:1255)
4 ksh nv_endsubscript + 940 (array.c:1547)
5 ksh nv_create + 4732 (name.c:1066)
6 ksh nv_open + 1951 (name.c:1425)
7 ksh varsub + 4934 (macro.c:1322)
[rest omitted]
The special-casing needs to be done on line 1250 of array.c, but
flagged in varsub() which processes these expansions. So, varsub()
calls nv_open() calls nv_create() calls nv_endsubscript() calls
nv_putsub(). That's a fairly deep call stack, so passing an extra
flag argument does not seem doable. I did try an approach using a
couple of new bit flags passed via these functions' flags and mode
parameters, but the way this code base uses bit flags is so
intricate, it does not seem to be possible to add or change
anything without unwanted side effects in all sorts of places.
So the only fix I can think of adds yet another global flag
variable for a very special case. It's ugly, but it works.
An elegant fix would probably involve a fairly comprehensive
redesign, which is simply not going to happen.
src/cmd/ksh93/include/shell.h:
- Add global sh.cond_expan flag.
src/cmd/ksh93/sh/array.c: nv_putsub():
- Do not set value to empty string if sh.cond_expan is set.
src/cmd/ksh93/sh/macro.c: varsub():
- Set sh.cond_expan flag while calling nv_open() for one of the
affected expansions.
- Minor refactoring for legibility and to make the fix fit better.
- SSOT: Instead of repeating string "REPLY", use the node's nvname.
- Do not pointlessly add an extra 0 byte when saving id for error
message; sfstruse() already adds this.
Thanks to @oguz-ismail for the bug report.
Resolves: https://github.com/ksh93/ksh/issues/383
2022-02-05 22:52:47 +00:00
|
|
|
- Fixed: for indexed arrays, given an unset array member a[i] with i > 0,
|
|
|
|
|
${a[i]=value} failed to assign the value and ${v[i]?error} failed to throw
|
|
|
|
|
an error.
|
|
|
|
|
|
2022-02-05 15:04:12 +00:00
|
|
|
- Fixed: the -a/--allexport shell option incorrectly exported some variables
|
|
|
|
|
with names containing a dot to the environment. Note that the '-x' typeset
|
|
|
|
|
attribute may still be set for them, but this will now not have any
|
|
|
|
|
effect. Variables created in namespaces declared with the 'namespace'
|
|
|
|
|
keyword are only exported while their namespace is active.
|
|
|
|
|
|
2022-02-05 13:09:25 +00:00
|
|
|
2022-02-04:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug in 'typeset -p': for an indexed array variable that is set
|
|
|
|
|
but empty, it will no longer output a nonexistent '[0]=' element.
|
|
|
|
|
|
2022-02-04 05:52:53 +00:00
|
|
|
2022-02-02:
|
|
|
|
|
|
|
|
|
|
- Fixed the -a/allexport option to export all variables that are assigned
|
|
|
|
|
values. Previously, arithmetic $((var=1)) or conditional ${var:=value}
|
|
|
|
|
assignments were not exported even with allexport on.
|
|
|
|
|
|
2022-02-05 02:29:23 +00:00
|
|
|
- Fixed a memory leak that could occur when using traps.
|
|
|
|
|
|
|
|
|
|
- Fixed a use after free issue that could occur when modifying or setting
|
|
|
|
|
a SIGCHLD trap.
|
|
|
|
|
|
2022-02-01 23:38:14 +00:00
|
|
|
2022-02-01:
|
|
|
|
|
|
|
|
|
|
- Upon invocation, the interactive shell no longer leaves the user without
|
|
|
|
|
a line editor if the VISUAL or EDITOR variable does not indicate emacs,
|
|
|
|
|
gmacs or vi; instead, it now turns on the emacs option by default.
|
|
|
|
|
|
2022-01-31 21:09:50 +00:00
|
|
|
2022-01-31:
|
|
|
|
|
|
|
|
|
|
- Improved keyboard support for the vi and emacs built-in line editors:
|
|
|
|
|
- Added support for Home key sequences ^[[1~ and ^[[7~ as well as End key
|
|
|
|
|
sequences ^[[4~ and ^[[8~.
|
|
|
|
|
- Added support for arrow key sequences ^[OA, ^[OB, ^[OC and ^[OD.
|
|
|
|
|
- Added support for the following keyboard shortcuts (if the platform
|
|
|
|
|
supports the expected escape sequence):
|
|
|
|
|
- Ctrl-Left Arrow: Go back one word
|
|
|
|
|
- Alt-Left Arrow: Go back one word (Not supported on Haiku)
|
|
|
|
|
- Ctrl-Right Arrow: Go forward one word
|
|
|
|
|
- Alt-Right Arrow: Go forward one word (Not supported on Haiku)
|
|
|
|
|
- Ctrl-G: Cancel reverse search
|
|
|
|
|
- Ctrl-Delete: Delete next word (Not supported on Haiku)
|
|
|
|
|
- Added a keybind for the Insert key, which differs in the emacs and vi
|
|
|
|
|
editing modes:
|
|
|
|
|
- In emacs mode, Insert escapes the next character.
|
|
|
|
|
- In vi mode, Insert will switch the editor to insert mode (like in vim).
|
|
|
|
|
|
Fix SIGINT handling for external commands run from scripts
Reproducer:
$ ksh -c 'bash -c '\''kill -s INT $$'\''; echo "$?, continuing"'
Expected result: output "258, continuing"; exit status 0.
Actual result: no output; exit status 258. The child process sent
SIGINT only to itself and not to the process group, so the parent
script was wrongly interrupted.
Every shell except ksh93 produces the expected result. ksh93 also
gave the expected result before version 2008-01-31 93s+, which
introduced the code below.
Analysis: The problem is in these lines of code in xec.c,
sh_exec(), TFORK case, parent branch of fork:
1649: if(!sh_isstate(SH_MONITOR))
1650: {
1651: if(!(sh.sigflag[SIGINT]&(SH_SIGFAULT|SH_SIGOFF)))
1652: sh_sigtrap(SIGINT);
1653: sh.trapnote |= SH_SIGIGNORE;
1654: }
[...pipe and I/O handling, wait for command to finish...]
1667: if(!sh_isstate(SH_MONITOR))
1668: {
1669: sh.trapnote &= ~SH_SIGIGNORE;
1700: if(sh.exitval == (SH_EXITSIG|SIGINT))
1701: kill(sh.current_pid,SIGINT);
1702: }
When a user presses Ctrl+C, SIGINT is sent to the entire process
group. If job control is fully off (i.e., !sh_isstate(SH_MONITOR)),
then the process group includes the parent script. Therefore, in a
script such as
$ ksh -c 'bash -c '\''read x'\''; echo "$?, continuing"'
when the user presses Ctrl+C while bash waits for 'read x' input,
the parent ksh script should be interrupted as well.
Now, the code above ignores SIGINT while bash is running. (This is
done using special-casing in sh_fault() to handle that SH_SIGIGNORE
flag for SIGINT.) So, when Ctrl+C interrupts the process group, the
parent script is not getting interrupted as it should.
To compensate for that, the code then detects, using sh.exitval
(the child process' exit status), whether the child process was
killed by SIGINT. If so, it simply assumes that the signal was
meant for the process group including the parent script, so it
reissues SIGINT to itself after unignoring it.
But, as we can see from the broken reproducer above, that
assumption is not valid. Scripts are perfectly free to send SIGINT
to themselves only, and that must work as expected.
src/cmd/ksh93/sh/xec.c: sh_exec(): TFORK: parent branch:
- Instead of ignoring SIGINT, sigblock() it, which delays handling
the signal until sigrelease(). (Note that these are macros
defined in src/cmd/ksh93/features/sigfeatures according to OS
capabilities.)
- This makes reissuing SIGINT redundant, so delete that, which
fixes the bug.
src/cmd/ksh93/sh/fault.c:
- Nothing now sets the SH_SIGIGNORE flag in sh.trapnote, so remove
special-casing added in 2008-01-31 93s+.
2022-02-01 03:11:10 +00:00
|
|
|
- Fixed a bug in SIGINT handling: if a script ran an external command that
|
|
|
|
|
interrupted itself (and only itself, not the process group) with SIGINT,
|
|
|
|
|
the script that ran the command was also interrupted.
|
|
|
|
|
|
2022-01-28 04:43:16 +00:00
|
|
|
2022-01-28:
|
|
|
|
|
|
|
|
|
|
- Fixed longstanding job control breakage that occurred on the interactive
|
|
|
|
|
shell when suspending (Ctrl+Z) an external command invoked by a dot script
|
2022-01-28 05:28:15 +00:00
|
|
|
or POSIX shell function, or via 'eval'.
|
2022-01-28 04:43:16 +00:00
|
|
|
|
Fix memory leak in defpathinit() (#441)
Currently, running ksh under ASan without the ASAN_OPTIONS variable
set to 'detect_leaks=0' usually ends with ASan complaining about a
memory leak in defpathinit() (this leak doesn't grow in a loop, so
no regression test was added to leaks.sh). Reproducer:
$ ENV=/dev/null arch/*/bin/ksh
$ cp -?
cp: invalid option -- '?'
Try 'cp --help' for more information.
$ exit
=================================================================
==225132==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 85 byte(s) in 1 object(s) allocated from:
#0 0x7f6dab42d459 in __interceptor_calloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154
#1 0x5647b77fe144 in sh_calloc /home/johno/GitRepos/KornShell/ksh/src/cmd/ksh93/sh/init.c:265
#2 0x5647b788fea9 in path_addcomp /home/johno/GitRepos/KornShell/ksh/src/cmd/ksh93/sh/path.c:1567
#3 0x5647b78911ed in path_addpath /home/johno/GitRepos/KornShell/ksh/src/cmd/ksh93/sh/path.c:1705
#4 0x5647b7888e82 in defpathinit /home/johno/GitRepos/KornShell/ksh/src/cmd/ksh93/sh/path.c:442
#5 0x5647b78869f3 in ondefpath /home/johno/GitRepos/KornShell/ksh/src/cmd/ksh93/sh/path.c:67
--- cut ---
SUMMARY: AddressSanitizer: 174 byte(s) leaked in 2 allocation(s).
Analysis: The previous code was leaking memory because
defpathinit() returns a pointer from path_addpath(), which has
memory allocated to it in path_addcomp(). This is the code ASan
traced as having allocated memory:
442: return(path_addpath((Pathcomp_t*)0,(defpath),PATH_PATH));
In path_addpath():
1705: first = path_addcomp(first,old,cp,type);
[...]
1729: return(first);
In path_addcomp():
1567: pp = sh_newof((Pathcomp_t*)0,Pathcomp_t,1,len+1);
The ondefpath() function doesn't save a reference to the pointer
returned by defpathinit(), which causes the memory leak:
66: if(!defpath)
67: defpathinit();
The changes in this commit avoid this problem by setting the
defpath variable without also calling path_addpath().
src/cmd/ksh93/sh/path.c:
- Move the code for allocating defpath from defpathinit() into its
own dedicated function called std_path(). This function is called
by defpathinit() and ondefpath() to obtain the current string
stored in the defpath variable. This bugfix is adapted from a
fork of ksh2020: https://github.com/l0stman/ksh/commit/db5c83a6
2022-01-29 17:37:48 +00:00
|
|
|
- Fixed a memory leak that occurred when running a command that was found on
|
|
|
|
|
the PATH.
|
|
|
|
|
|
2022-01-26 04:20:56 +00:00
|
|
|
2022-01-26:
|
|
|
|
|
|
|
|
|
|
- On Cygwin, ksh now executes scripts that do not have a #! path itself,
|
|
|
|
|
like it does on other systems, instead of with /bin/sh.
|
|
|
|
|
|
2022-01-27 12:55:36 +00:00
|
|
|
- Fixed a spurious syntax error which occurred when attempting to use array
|
|
|
|
|
subscript expansion with unset variables (i.e., ${foo[${bar}..${baz}]}).
|
|
|
|
|
|
2022-01-24 08:22:39 +00:00
|
|
|
2022-01-24:
|
|
|
|
|
|
|
|
|
|
- Fixed a crashing bug in history expansion that could occur when using the "&"
|
|
|
|
|
modifier to repeat a substitution while there was no previous substitution.
|
|
|
|
|
|
2022-01-25 03:16:53 +00:00
|
|
|
- History expansion is now documented in the ksh(1) manual page.
|
|
|
|
|
|
|
|
|
|
- In history expansion, the history comment character '#' is now enabled by
|
|
|
|
|
default, as it is on bash.
|
|
|
|
|
|
|
|
|
|
- In history expansion, the 'a' modifier is now a synonym for 'g', as on bash.
|
|
|
|
|
|
|
|
|
|
- Fixed a bug in history expansion where the 'p' modifier had no effect if
|
|
|
|
|
the 'histverify' option is on.
|
|
|
|
|
|
2022-01-20 22:30:02 +00:00
|
|
|
2022-01-20:
|
|
|
|
|
|
2022-01-21 01:18:34 +00:00
|
|
|
- Disallow out-of-range event numbers in history expansion (-H/-o histexpand).
|
|
|
|
|
Previously these were accepted, resulting in corrupted expansions.
|
|
|
|
|
|
2022-01-20 22:30:02 +00:00
|
|
|
- Fixed a potential crash in history expansion due to a buffer overflow.
|
|
|
|
|
|
Expose --histreedit and --histverify options (re: 921bbcae)
This adds two long-form shell options that modify history expansion
(-H/--histexpand). If --histreedit is on and a history expansion
fails, the command line is reloaded into the next prompt's edit
buffer, allowing corrections. If --histverify is on, the results of
a history expansion are not immediately executed but instead loaded
into the next prompt's edit buffer, allowing further changes.
SH_HISTREEDIT and SH_HISTVERIFY were already handled all along in
slowread() in io.c and via 'reedit' arguments to functions called
there, but could not be turned on as they were only ever exposed as
shopt options in the removed bash compatibility mode (in theory
only, as it failed to compile). I had overlooked them until now.
So, since the code is there, let's expose these options through the
normal long options interface. They're working fine, and activating
them actually makes history expansion tolerable to use.
src/cmd/ksh93/data/options.c:
- Make these options available as "histreedit" and "histverify".
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/sh.1:
- Document the "new" options.
src/cmd/ksh93/include/defs.h:
- Remove unused SH_HISTAPPEND and SH_HISTORY2 macros which were
part of the removed bash compatibility code. Note that ksh does
not need a histappend option as it never overwrites the history
file (in the bash mode, this shopt option was a no-op).
2022-01-12 14:55:02 +00:00
|
|
|
2022-01-12:
|
|
|
|
|
|
|
|
|
|
- Added bash-inspired --histreedit and --histverify options that modify history
|
|
|
|
|
expansion (--histexpand), allowing editing on failure or before execution.
|
|
|
|
|
|
2022-01-07 15:44:05 +00:00
|
|
|
2022-01-04:
|
|
|
|
|
|
|
|
|
|
- Fixed an issue in vi mode that caused tab completion to fail if the
|
|
|
|
|
last character on the command line was a space.
|
|
|
|
|
|
2022-01-06 20:35:44 +00:00
|
|
|
2021-12-29:
|
|
|
|
|
|
|
|
|
|
- Fixed numerous build errors that prevented ksh from running on Haiku OS.
|
|
|
|
|
|
|
|
|
|
- Added support for the SIGKILLTHR signal on operating systems that
|
|
|
|
|
implement it.
|
|
|
|
|
|
2021-12-28 21:55:56 +00:00
|
|
|
2021-12-28:
|
|
|
|
|
|
2021-12-28 23:53:08 +00:00
|
|
|
- Fixed a bug that caused CDPATH to continue working after unsetting it.
|
|
|
|
|
|
2021-12-28 21:55:56 +00:00
|
|
|
- Added three options to the ulimit builtin with the same names and
|
|
|
|
|
functionality as in Bash:
|
|
|
|
|
- 'ulimit -k' sets the maximum number of kqueues.
|
|
|
|
|
- 'ulimit -P' sets the maximum number of pseudo-terminals.
|
|
|
|
|
- 'ulimit -R' sets the maximum time in microseconds a real-time process
|
|
|
|
|
can run before blocking.
|
|
|
|
|
Note that to use these options the operating system must support the
|
|
|
|
|
corresponding resource limit.
|
|
|
|
|
|
2021-12-28 22:22:38 +00:00
|
|
|
- Ported performance optimizations from illumos to improve the performance
|
|
|
|
|
of the cksum builtin. (Note that the cksum builtin is not enabled by
|
|
|
|
|
default.)
|
|
|
|
|
|
2021-12-27 03:00:15 +00:00
|
|
|
2021-12-27:
|
|
|
|
|
|
2021-12-27 06:37:01 +00:00
|
|
|
- Two bash-like flags for 'whence' were backported from ksh 93v-:
|
|
|
|
|
- 'whence -P/type -P' is an alias to the existing -p flag.
|
|
|
|
|
- 'whence -t/type -t' will print only the type of a command in a simple
|
|
|
|
|
format that is designed to be easy to use for scripts. Example:
|
|
|
|
|
$ type -t typeset; whence -t sh
|
|
|
|
|
builtin
|
|
|
|
|
file
|
|
|
|
|
|
2021-12-27 03:00:15 +00:00
|
|
|
- Fixed a crash or freeze that would occur on Linux when using Ctrl+C to
|
|
|
|
|
interrupt a command substitution containing a pipe in an interactive shell.
|
|
|
|
|
|
Fix crash in xtrace while processing here-document (re: d7cada7b)
Depending on the OS, the heredoc.sh regression tests, and possibly
others, still crashed with the -x option (xtrace) on.
Analysis: The lexer crashes in lex_advance(). Something has caused
an inconsistent lexer state, and it happened earlier on, so the
backtrace is useless for figuring out where that happened.
But I think I've found it. It's the sh_mactry() call here:
src/cmd/ksh93/sh/xec.c, lines 2800 to 2807 in f7213f03
2800: if(!(cp=nv_getval(sh_scoped(shp,PS4NOD))))
2801: cp = "+ ";
2802: else
2803: {
2804: sh_offoption(SH_XTRACE);
2805: cp = sh_mactry(shp,cp);
2806: sh_onoption(SH_XTRACE);
2807: }
sh_mactry() needs to parse the contents of $PS4 to perform
expansions and command substitutions in it, which involves the
lexer. If that happens in a here-document, the lexer is in the C
function call stack, in the middle of parsing the here-document.
Result: inconsistent lexer state. Solution: save and restore lexer
state in sh_mactry().
After this commit, all regression tests should pass with the
'-x'/'--xtrace' option in use, with no errors or crashes.
Note for backporters: this fix depends both on on d7cada7b and on
the consistency fix for the Lex_t type's size applied in a7ed5d9f.
src/cmd/ksh93/include/shlex.h:
- Cosmetic fix: remove a copied & pasted backslash. (re: a7ed5d9f)
src/cmd/ksh93/sh/macro.c: sh_mactry():
- Save and restore the lexer state before letting sh_mactrim()
indirectly parse and execute code.
src/cmd/ksh93/tests/*.sh:
- Turn off xtrace in various command substitutions that contain
2>&1 redirections, so that the xtrace output is not caught by
the command substitutions, causing tests to fail incorrectly.
- Turn off xtrace for a few code blocks with 2>&1 redirections,
stopping xtrace output from being written to standard output.
Resolves: https://github.com/ksh93/ksh/issues/306 (again)
2021-12-27 03:39:32 +00:00
|
|
|
- Fixed a crash that could occur while processing a here-document while
|
|
|
|
|
xtrace (set -x) is on and the $PS4 prompt contains parameter expansions or
|
|
|
|
|
command substitutions.
|
|
|
|
|
|
2021-12-28 16:53:30 +00:00
|
|
|
- The mkservice and eloop builtins can now be built by enabling the
|
|
|
|
|
new SHOPT_MKSERVICE setting in src/cmd/ksh93/SHOPT.sh.
|
|
|
|
|
|
2021-12-27 00:25:05 +00:00
|
|
|
2021-12-26:
|
|
|
|
|
|
|
|
|
|
- Listing aliases or tracked aliases in a script no longer corrupts
|
|
|
|
|
shcomp's generated bytecode.
|
|
|
|
|
|
|
|
|
|
- Listing specific aliases with 'alias -p' and specific tracked aliases
|
|
|
|
|
with 'alias -pt' now works as documented. This means that the following
|
|
|
|
|
string of commands now works as you would expect:
|
|
|
|
|
$ hash -r; unalias -a
|
|
|
|
|
$ alias foo=bar; hash cat
|
|
|
|
|
$ alias -p foo; alias -pt cat
|
|
|
|
|
alias foo=bar
|
|
|
|
|
alias -t cat
|
|
|
|
|
|
|
|
|
|
- As a result of the above fix, listing all tracked aliases with 'alias -pt'
|
|
|
|
|
now prints commands that can be reused to recreate the tracked aliases.
|
|
|
|
|
|
|
|
|
|
- Attempting to list a non-existent alias or tracked alias with the -p option
|
|
|
|
|
now causes an error and sets the exit status to the number of non-existent
|
|
|
|
|
aliases passed.
|
|
|
|
|
|
|
|
|
|
- Attempting to list 256 non-existent aliases now errors out with the exit
|
|
|
|
|
status set to one.
|
|
|
|
|
|
2021-12-22 13:25:26 +00:00
|
|
|
2021-12-22:
|
|
|
|
|
|
|
|
|
|
- Process substitutions run in a profile script no longer print their
|
|
|
|
|
process ID when run.
|
|
|
|
|
|
2021-12-22 03:10:11 +00:00
|
|
|
2021-12-21:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused subshells (such as code blocks in parentheses) to
|
|
|
|
|
partially behave like the interactive main shell after running anything
|
|
|
|
|
that invokes the parser. For example:
|
|
|
|
|
$ (eval :; sleep 1 & echo done)
|
|
|
|
|
[1] 30909 <--- incorrect job control output from subshell
|
|
|
|
|
done
|
|
|
|
|
|
Interactive: Avoid losing the job after suspending a subshell
Reproducer: run vi in a subshell:
$ (vi)
vi opens; now press Ctrl+Z to suspend. The output is as expected:
[2] + Stopped (vi)
…but the exit status is 18 (SIGTSTP's signal number) instead of 0.
Now do:
$ fg
(vi)
$
The exit status is 18 again, vi is not resumed, and the job is
lost. You have to find vi's pid manually using ps and kill it.
Forking all non-command substitution subshells invoked from the
interactive main shell is the only reliable and effective fix I've
found. I've tried to fork the subshell conditionally in every other
remotely plausible place I can think of in fault.c and xec.c, but I
can't get anything to work properly. If anyone can get this to work
without forking as much (or at all), please do submit a patch or PR
that supersedes this fix.
At least subshells of subshells don't need to fork, so the
performance impact can be limited. Plus, it's not as if most people
need maximum speed on the interactive command line. Scripts
(including login/profile scripts) are not affected at all.
Command substitutions can be handled differently. My testing shows
that all shells except ksh93 simply block SIGTSTP (the ^Z signal)
while they run. We should do the same, so they don't need to fork.
NOTE for any backporters: the subshell.c and fault.c changes depend
on commits 35b02626 and 48ba6964 to work correctly.
src/cmd/ksh93/sh/subshell.c: sh_subshell():
- If the interactive shell state bit is on, then before executing
the subshell's code:
- for command substitutions, block SIGTSTP;
- for other subshells, fork.
- For command substitutions, release SIGTSTP if the interactive
shell state bit was on upon invoking the subshell.
src/cmd/ksh93/sh/fault.c:
- Instead of checking for a virtual subshell, check the shell's
interactive state bit to decide whether to handle SIGTSTP, as
that is only turned on in the interactive main shell.
src/cmd/ksh93/sh/main.c: sh_main():
- To avoid bugs, ignore SIGTSTP while running profile scripts.
Blocking it doesn't work because delaying it until after
sigrelease() will cause a crash. Thanks to @JohnoKing for this.
- While we're here, prevent a possible overflow of the 'beenhere'
static char variable by only incrementing it once.
Co-authored-by: Johnothan King <johnothanking@protonmail.com>
Resolves: https://github.com/ksh93/ksh/issues/390
2021-12-22 04:43:42 +00:00
|
|
|
- Fixed: after suspending (Ctrl+Z) a subshell that is running an external
|
|
|
|
|
command, resuming the subshell with 'fg' failed and the job was lost.
|
|
|
|
|
$ (vi) <--- press Ctrl+Z
|
|
|
|
|
[2] + Stopped (vi)
|
|
|
|
|
$ fg
|
|
|
|
|
(vi) <--- vi failed to resume; immediate return to command line
|
|
|
|
|
$ fg
|
|
|
|
|
ksh: no such job
|
|
|
|
|
|
Re-fix defining types conditionally or in subshells (re: f508660d)
New version. I'm pretty sure the problems that forced me to revert
it earlier are fixed.
This commit mitigates the effects of the hack explained in the
referenced commit so that dummy built-in command nodes added by the
parser for declaration/assignment purposes do not leak out into the
execution level, except in a relatively harmless corner case.
Something like
if false; then
typeset -T Foo_t=(integer -i bar)
fi
will no longer leave a broken dummy Foo_t declaration command. The
same applies to declaration commands created with enum.
The corner case remaining is:
$ ksh -c 'false && enum E_t=(a b c); E_t -a x=(b b a c)'
ksh: E_t: not found
Since the 'enum' command is not executed, this should have thrown
a syntax error on the 'E_t -a' declaration:
ksh: syntax error at line 1: `(' unexpected
This is because the -c script is parsed entirely before being
executed, so E_t is recognised as a declaration built-in at parse
time. However, the 'not found' error shows that it was successfully
eliminated at execution time, so the inconsistent state will no
longer persist.
This fix now allows another fix to be effective as well: since
built-ins do not know about virtual subshells, fork a virtual
subshell into a real subshell before adding any built-ins.
src/cmd/ksh93/sh/parse.c:
- Add a pair of functions, dcl_hactivate() and dcl_dehacktivate(),
that (de)activate an internal declaration built-ins tree into
which check_typedef() can pre-add dummy type declaration command
nodes. A viewpath from the main built-ins tree to this internal
tree is added, unifying the two for search purposes and causing
new nodes to be added to the internal tree. When parsing is done,
we close that viewpath. This hides those pre-added nodes at
execution time. Since the parser is sometimes called recursively
(e.g. for command substitutions), keep track of this and only
activate and deactivate at the first level.
(Fixed compared to previous version of this commit: calling
dcl_dehacktivate() when the recursion level is already zero is
now a harmless no-op. Since this only occurs in error handling
conditions, who cares.)
- We also need to catch errors. This is done by setting libast's
error_info.exit variable to a dcl_exit() function that tidies up
and then passes control to the original (usually sh_exit()).
(Fixed compared to previous version of this commit: dcl_exit()
immediately deactivates the hack, no matter the recursion level,
and restores the regular sh_exit(). This is the right thing to
do when we're in the process of erroring out.)
- sh_cmd(): This is the most central function in the parser. You'd
think it was sh_parse(), but $(modern)-form command substitutions
use sh_dolparen() instead. Both call sh_cmd(). So let's simply
add a dcl_hacktivate() call at the beginning and a
dcl_deactivate() call at the end.
- assign(): This function calls path_search(), which among many
other things executes an FPATH search, which may execute
arbitrary code at parse time (!!!). So, regardless of recursion
level, forcibly dehacktivate() to avoid those ugly parser side
effects returning in that context.
src/cmd/ksh93/bltins/enum.c: b_enum():
- Fork a virtual subshell before adding a built-in.
src/cmd/ksh93/sh/xec.c: sh_exec():
- Fork a virtual subshell when detecting typeset's -T option.
Improves fix to https://github.com/ksh93/ksh/issues/256
2021-12-17 00:02:01 +00:00
|
|
|
2021-12-17:
|
|
|
|
|
|
Release 1.0.0-beta.2
Announcing: KornShell 93u+m 1.0.0-beta.2
https://github.com/ksh93/ksh
In May 2020, when every KornShell (ksh93) development project was
abandoned, development was rebooted in a new fork based on the last
stable AT&T version: ksh 93u+. This new fork is called ksh 93u+m as a
permanent nod to its origin. We're restarting it at version 1.0. Seven
months after the first beta, the second one is ready. Please test this
second beta and report any bugs you find, or help us fix known bugs.
We're now the default ksh93 in some OS distributions, at least Debian
and Slackware! Even though we don't think it's stable release quality
yet, the consensus seems to be that 93u+m is already much better than
the last AT&T release.
Main developers: Martijn Dekker, Johnothan King, hyenias
Contributors: Andy Fiddaman, Anuradha Weeraman, Chase, Gordon Woodhull,
Govind Kamat, Harald van Dijk, Lev Kujawski, Marc Wilson, Ryan Schmidt,
Sterling Jensen
HOW TO GET IT
Please download the source code tarball from our GitHub releases page:
https://github.com/ksh93/ksh/releases
To build, follow the instructions in README.md or src/cmd/ksh93/README.
HOW TO GET INVOLVED
To report a bug, please open an issue at our GitHub page (see above).
Alternatively, email me at martijn@inlv.org with your report.
To get involved in development, read the brief policy information in
README.md and then jump right in with a pull request or email a patch.
See the TODO file in the top-level directory for a to-do list.
*** MAIN CHANGES between 1.0.0-beta.1 and 1.0.0-beta.2 ***
New features in built-in commands:
- 'cd' now supports an -e option that, when combined with -P, verifies
that $PWD is correct after changing directories; this helps detect
access permission problems. See:
https://www.austingroupbugs.net/view.php?id=253
- 'printf' now supports a -v option as in bash. This assigns formatted
output directly to variables, which is very fast and will not strip
final newline (\n) characters.
- The 'return' command, when used to return from a function, can now
return any status value in the 32-bit signed integer range, like on
zsh. However, due to a traditional Unix kernel limitation, $? is
still trimmed to its least significant 8 bits whenever leaving a
(sub)shell environment.
- 'test'/'[' now supports all the same operators as [[ (including =~,
\<, \>) except for the different 'and'/'or' operators. Note that
'test'/'[' remains deprecated due to its unfixable pitfalls;
[[ ... ]] is recommended instead.
Shell language changes:
- Several improvements were made to the --noexec shell code linter.
- Arithmetic expressions in native ksh mode no longer interpret a
number with a leading zero as octal in any context. Use 8#octalnumber
instead (e.g. 8#400 == 256). Arithmetic expressions now also behave
identically within and outside ((...)) and $((...)).
- POSIX compatibility mode fixes (only applicable with the --posix shell
option on):
- A leading zero is now consistently recognised as introducing an octal
number in all arithmetic contexts.
- $((inf)) and $((nan)) are now interpreted as regular variables.
- The '.' built-in no longer runs ksh functions and now only runs
files.
Bugs fixed:
- '.' and '..' are now once again completed by tab completion.
- If SIGINT is set to ignore, the interactive shell no longer exits on
Ctrl+C.
- ksh now builds and runs on Apple's new M1 hardware.
- The 'return' and 'exit' commands no longer risk triggering actual
signals by returning or exiting with a status > 256.
- Ksh no longer behaves badly when parsing a type definition command
('typeset -T' or 'enum') without executing it or when executing it in
a subshell. Types can now safely be defined in subshells and defined
conditionally as in 'if condition; then enum ...; fi'.
- Discipline functions, especially those applied to PS2 or .sh.tilde,
will no longer crash your shell upon being interrupted or throwing an
error.
- Fixed a bug that could corrupt output if standard output is closed
upon initialising the shell.
- Fixed a bug in the [[ ... ]] compound command: the '!' logical
negation operator now correctly negates another '!', e.g.,
[[ ! ! 1 -eq 1 ]] now returns 0/true. Note that this has always been
the case for 'test'/'['.
- Fixed SHLVL so that replacing ksh by itself (exec ksh) will not
increase it.
- Arithmetic expressions are no longer allowed to assign out-of-range
values to variables of types declared with enum.
- The 'time' keyword no longer makes the --errexit shell option
ineffective.
- Various bugs in libcmd built-in commands (those bound to the
/opt/ast/bin path by default) have been fixed.
- Various other crashing bugs have been fixed.
Fixes for the shcomp byte code compiler:
- shcomp is now able to compile scripts that define types using enum.
- shcomp now refuses to mess up your terminal by writing bytecode
to it.
*** MAIN CHANGES between ksh 93u+ 2012-08-01 and 93u+m 1.0.0-beta.1 ***
Hundreds of bugs have been fixed, including many serious/critical bugs.
This includes upstreamed patches from OpenSUSE, Red Hat, and Solaris, fixes
backported from the abandoned 93v- beta and ksh2020 fork, as well as many
new fixes from the community. See the NEWS file for more information, and
the git commit log for complete documentation of every fix. Incompatible
changes have been minimised, but not at the expense of fixing bugs. For a
list of potentially incompatible changes, see src/cmd/ksh93/COMPATIBILITY.
Though there was a "no new features, bugfixes only" policy, some new
features were found necessary, either to fix serious design flaws or to
complete functionality that was evidently intended, but not finished.
Below is a summary of these new features.
New command line editor features:
- The forward-delete and End keys are now handled as expected in the
emacs and vi built-in line editors.
- In the vi and emacs line editors, repeat count parameters can now also
be used for the arrow keys and the forward-delete key. E.g., in emacs
mode, <ESC> 7 <left-arrow> will now move the cursor seven positions to
the left. In vi control mode, this would be entered as: 7 <left-arrow>.
New shell language features:
- The &>file redirection shorthand (for >file 2>&1) is now available for
all scripts and interactive sessions and not only for profile/login
scripts, bringing ksh 93u+m in line with mksh, bash, and zsh.
- File name generation (a.k.a. pathname expansion, a.k.a. globbing) now
never matches the special navigational names '.' (current directory)
and '..' (parent directory). This change makes a pattern like .*
useful; it now matches all hidden files (dotfiles) in the current
directory, without the harmful inclusion of '.' and '..'.
- Tilde expansion can now be extended or modified by defining a
.sh.tilde.get or .sh.tilde.set discipline function. This replaces a
2004 undocumented attempt to add this functionality via a .sh.tilde
command, which never worked and crashed the shell. See the manual for
details on the new method.
New features in built-in commands:
- Usage error messages now show the --help/--man self-documentation options.
- Path-bound built-ins (such as /opt/ast/bin/cat) can now be executed by
invoking the canonical path, so the following will now work as expected:
$ /opt/ast/bin/cat --version
version cat (AT&T Research) 2012-05-31
- 'command -x' now looks for external commands only, skipping built-ins.
In addition, its xargs-like functionality no longer freezes the shell on
Linux and macOS, making it effectively a new feature on these systems.
- 'redirect' now checks if all arguments are valid redirections before
performing them. If an error occurs, it issues an error message instead
of terminating the shell.
- 'suspend' now refuses to suspend a login shell, as there is probably no
parent shell to return to and the login session would freeze.
- 'times' now gives high precision output in a POSIX compliant format.
- 'typeset' now gives an informative error message if an incompatible
combination of options is given.
- 'whence -v/-a' now reports the location of autoloadable functions.
New features in shell options:
- A new --globcasedetect shell option is added on OSs where we can
check for a case-insensitive file system (currently Windows/Cygwin,
macOS, Linux and QNX 7.0+). When this option is turned on, file name
generation (globbing), as well as file name tab completion on
interactive shells, automatically become case-insensitive on file
systems where the difference between upper and lower case is ignored
for file names. This is transparently determined for each directory, so
a path pattern that spans multiple file systems can be part
case-sensitive and part case-insensitive.
- A new --nobackslashctrl shell option disables the special escaping
behaviour of the backslash character in the emacs and vi built-in
editors. Particularly in the emacs editor, this makes it much easier to
go backward, insert a forgotten backslash into a command, and then
continue editing without having your next cursor key replace your
backslash with garbage. Note that Ctrl+V (or whatever other character
was set using 'stty lnext') always escapes all control characters in
either editing mode.
- A new --posix shell option has been added to ksh 93u+m that makes the
ksh language more compatible with other shells by following the POSIX
standard more closely. See the manual page for details. It is enabled by
default if ksh is invoked as sh, otherwise it is disabled by default.
- Enhancement to -G/--globstar: symbolic links to directories are now
followed if they match a normal (non-**) glob pattern. For example, if
'/lnk' is a symlink to a directory, '/lnk/**' and '/l?k/**' now work as
you would expect.
2021-12-17 03:18:03 +00:00
|
|
|
- Release 1.0.0-beta.2.
|
|
|
|
|
|
Re-fix defining types conditionally or in subshells (re: f508660d)
New version. I'm pretty sure the problems that forced me to revert
it earlier are fixed.
This commit mitigates the effects of the hack explained in the
referenced commit so that dummy built-in command nodes added by the
parser for declaration/assignment purposes do not leak out into the
execution level, except in a relatively harmless corner case.
Something like
if false; then
typeset -T Foo_t=(integer -i bar)
fi
will no longer leave a broken dummy Foo_t declaration command. The
same applies to declaration commands created with enum.
The corner case remaining is:
$ ksh -c 'false && enum E_t=(a b c); E_t -a x=(b b a c)'
ksh: E_t: not found
Since the 'enum' command is not executed, this should have thrown
a syntax error on the 'E_t -a' declaration:
ksh: syntax error at line 1: `(' unexpected
This is because the -c script is parsed entirely before being
executed, so E_t is recognised as a declaration built-in at parse
time. However, the 'not found' error shows that it was successfully
eliminated at execution time, so the inconsistent state will no
longer persist.
This fix now allows another fix to be effective as well: since
built-ins do not know about virtual subshells, fork a virtual
subshell into a real subshell before adding any built-ins.
src/cmd/ksh93/sh/parse.c:
- Add a pair of functions, dcl_hactivate() and dcl_dehacktivate(),
that (de)activate an internal declaration built-ins tree into
which check_typedef() can pre-add dummy type declaration command
nodes. A viewpath from the main built-ins tree to this internal
tree is added, unifying the two for search purposes and causing
new nodes to be added to the internal tree. When parsing is done,
we close that viewpath. This hides those pre-added nodes at
execution time. Since the parser is sometimes called recursively
(e.g. for command substitutions), keep track of this and only
activate and deactivate at the first level.
(Fixed compared to previous version of this commit: calling
dcl_dehacktivate() when the recursion level is already zero is
now a harmless no-op. Since this only occurs in error handling
conditions, who cares.)
- We also need to catch errors. This is done by setting libast's
error_info.exit variable to a dcl_exit() function that tidies up
and then passes control to the original (usually sh_exit()).
(Fixed compared to previous version of this commit: dcl_exit()
immediately deactivates the hack, no matter the recursion level,
and restores the regular sh_exit(). This is the right thing to
do when we're in the process of erroring out.)
- sh_cmd(): This is the most central function in the parser. You'd
think it was sh_parse(), but $(modern)-form command substitutions
use sh_dolparen() instead. Both call sh_cmd(). So let's simply
add a dcl_hacktivate() call at the beginning and a
dcl_deactivate() call at the end.
- assign(): This function calls path_search(), which among many
other things executes an FPATH search, which may execute
arbitrary code at parse time (!!!). So, regardless of recursion
level, forcibly dehacktivate() to avoid those ugly parser side
effects returning in that context.
src/cmd/ksh93/bltins/enum.c: b_enum():
- Fork a virtual subshell before adding a built-in.
src/cmd/ksh93/sh/xec.c: sh_exec():
- Fork a virtual subshell when detecting typeset's -T option.
Improves fix to https://github.com/ksh93/ksh/issues/256
2021-12-17 00:02:01 +00:00
|
|
|
- Ksh no longer behaves badly when parsing a type definition command
|
|
|
|
|
('typeset -T' or 'enum') without executing it or when executing it in
|
|
|
|
|
a subshell. Types can now safely be defined in subshells and defined
|
|
|
|
|
conditionally as in 'if condition; then enum ...; fi'.
|
|
|
|
|
|
2021-12-17 02:24:56 +00:00
|
|
|
- Single digits can now be compared lexically in [[ ... ]] with the
|
|
|
|
|
< and > operators.
|
|
|
|
|
|
New selection of default libcmd /opt/ast/bin built-ins
Note that this is only about the /opt/ast/bin built-in commands,
not about the regular pathless builtins such as printf.
To use these, either add /opt/ast/bin to your $PATH or use a
command like 'builtin cp'. As usual, --man provides info.
Removed as defaults for lack of convincing advantages over the OS's
external commands:
- chmod, cmp, head, logname, mkdir, sync, uname, wc
Remain as useful defaults:
- basename, cat, cut, dirname. These are commonly used in
performance-sensitive code paths in scripts and having them as
built-ins can be good for performance.
- getconf: This is the only interface to some libast internals that
is available to ksh. It's also has better functionality than most
OS-shipped 'getconf' commands, e.g., it can list and query all
the configuration values.
Added as defaults:
- cp, ln, mv: Having these built in can speed up scripts that
manage files. Also the AST versions have extended functionality
(see cp --man, etc.).
- mktemp: External mktemp commands vary too widely and are
incompatible, but it's important that scripts can securely make
temporary files, so it's good to ship a known interface to this
functionality.
As a result, the statically linked ksh binary is very slightly
smaller than before.
Resolves: https://github.com/ksh93/ksh/issues/349
2021-12-16 07:48:06 +00:00
|
|
|
2021-12-16:
|
|
|
|
|
|
|
|
|
|
- Changed the default selection of compiled-in /opt/ast/bin built-in libcmd
|
2022-03-01 19:38:15 +00:00
|
|
|
commands to: basename, cat, cp, cut, dirname, getconf, ln, mktemp, mv.
|
New selection of default libcmd /opt/ast/bin built-ins
Note that this is only about the /opt/ast/bin built-in commands,
not about the regular pathless builtins such as printf.
To use these, either add /opt/ast/bin to your $PATH or use a
command like 'builtin cp'. As usual, --man provides info.
Removed as defaults for lack of convincing advantages over the OS's
external commands:
- chmod, cmp, head, logname, mkdir, sync, uname, wc
Remain as useful defaults:
- basename, cat, cut, dirname. These are commonly used in
performance-sensitive code paths in scripts and having them as
built-ins can be good for performance.
- getconf: This is the only interface to some libast internals that
is available to ksh. It's also has better functionality than most
OS-shipped 'getconf' commands, e.g., it can list and query all
the configuration values.
Added as defaults:
- cp, ln, mv: Having these built in can speed up scripts that
manage files. Also the AST versions have extended functionality
(see cp --man, etc.).
- mktemp: External mktemp commands vary too widely and are
incompatible, but it's important that scripts can securely make
temporary files, so it's good to ship a known interface to this
functionality.
As a result, the statically linked ksh binary is very slightly
smaller than before.
Resolves: https://github.com/ksh93/ksh/issues/349
2021-12-16 07:48:06 +00:00
|
|
|
Add /opt/ast/bin to your $PATH to use these. Type 'cp --man', etc. for info.
|
|
|
|
|
|
2021-12-16 15:40:42 +00:00
|
|
|
- A bug introduced on 2020-09-17 was fixed that caused interactive ksh to exit
|
|
|
|
|
if Ctrl+C was pressed while SIGINT was being ignored (as in "trap '' INT").
|
|
|
|
|
|
2021-12-13 00:33:26 +00:00
|
|
|
2021-12-13:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug introduced on 2020-08-09 that prevented '.' and '..' from
|
|
|
|
|
being completed when using file name tab completion.
|
|
|
|
|
|
2021-12-14 09:22:30 +00:00
|
|
|
- Fixed a bug on illumos that caused the chown builtin to fail with 'Invalid
|
|
|
|
|
argument' after failing to change the ownership of a file twice using an ID
|
|
|
|
|
that doesn't exist in /etc/passwd. (Note that the chown builtin is not
|
|
|
|
|
enabled by default.)
|
|
|
|
|
|
Fix two more PS2/SIGINT crashing bugs (re: 3023d53b)
*** Crash 1: ***
ksh crashed if the PS1 prompt contains one or more command
substitutions and you enter a multi-line command substitution
on the command line, then interrupt while on the PS2 prompt.
$ ENV=/./dev/null /usr/local/bin/ksh -o emacs
$ PS1='$(echo foo) $(echo bar) $(echo baz) ! % '
foo bar baz 16999 % echo $(
> true <-- here, press Ctrl+C instead of Return
Memory fault
The crash occurred due to a corrupted lexer state while trying to
display the PS1 prompt.
Analysis: My fix for the crashing bug with Ctrl+C in commit
3023d53b is incorrect and only worked accidentally. sh_fault() is
not the right place to reset the lexer state because, when we press
Ctrl+C on a PS2 prompt, ksh had been waiting for input to finish
lexing a multi-line command, so sh_lex() and other lexer functions
are on the function call stack and will be returned to.
src/cmd/ksh93/sh/fault.c: sh_fault():
- Remove incorrect SIGINT fix.
src/cmd/ksh93/sh/io.c: io_prompt():
- Reset the lexer state immediately before printing every PS1
prompt. Even in situations where this is redundant it should be
perfectly safe, the overhead is negligible, and it resolves this
crash. It may pre-empt other problems as well.
*** Crash 2: ***
If an INT trap is set, and you start entering a multi-line command
substitution, then press Ctrl+C on the PS2 prompt to trigger the
crash, the lexer state is corrupted because the lexer is invoked to
eval the trap action. A crash then occurs on entering the final ')'
of the command substitution.
$ trap 'echo TRAPPED' INT
$ echo $(
> trueTRAPPED <-- press Ctrl+C to output "TRAPPED"
> )
Memory fault
Technically, as SIGINT is trapped, it should not interrupt, so ksh
should execute the trap, then continue with the PS2 prompt to let
the user finish inputting the command. But I have been unsuccessful
in many different attempts to make this work properly. I managed to
get multi-line command substitutions to lex correctly by saving and
restoring the lexer state, but command substitutions were still
corrupted at the parser and/or execution level and I have not
managed to trace the cause of that.
My testing showed that all other shells interrupt the PS2 prompt
and return to PS1 when the user presses Ctrl+C, even if SIGINT is
trapped. I think that is a reasonable alternative, and it is
something I managed to make work.
src/cmd/ksh93/sh/fault.c: sh_chktrap():
- Immediately after invoking sh_trap() to run a trap action, check
if we're in a PS2 prompt (sh.nextprompt == 2). If so, assume the
lexer state is now overwritten. Closing the fcin stream with
fcclose() seems to reliably force the lexer to stop doing
anything else. Then we can just reset the prompt to PS1 and
invoke sh_exit() to start new command line, which will now reset
the lexer state as per above.
2021-12-11 03:27:54 +00:00
|
|
|
2021-12-11:
|
|
|
|
|
|
|
|
|
|
- Fixed two more crashing bugs that occurred if ksh received a signal (such
|
|
|
|
|
as SIGINT due to Ctrl+C) while the user is entering a multi-line command
|
|
|
|
|
substitution in an interactive shell.
|
|
|
|
|
|
2021-12-12 21:25:07 +00:00
|
|
|
- The shell linter's warning for variable expansion in ((...)) now tells the
|
|
|
|
|
user which variable is causing performance degradation.
|
|
|
|
|
|
|
|
|
|
- The shell linter now warns the user x=$((expr)) is slower than ((x=expr))
|
|
|
|
|
when assigning a value to a variable.
|
|
|
|
|
|
2021-12-09 06:05:36 +00:00
|
|
|
2021-12-09:
|
|
|
|
|
|
|
|
|
|
- Increased the general robustness of discipline function handling, fixing
|
|
|
|
|
crashing bugs with PS2.get() and .sh.tilde.get() disciplines, among others.
|
|
|
|
|
|
|
|
|
|
- Fixed a crash that occurred on the interactive shell if the PS1 prompt
|
|
|
|
|
contains multiple command substitutions and the user interrupts input
|
|
|
|
|
while the shell is on a PS2 prompt waiting for the user to complete a
|
|
|
|
|
command substitution of the form $( ... ).
|
|
|
|
|
|
Fix 'return' emitting signals; allow arbitrary return values
When a global EXIT trap is set, and a ksh-style function exits with
a status > 256 that could have been the result of a signal, then
the shell incorrectly issues that signal to itself. Depending on
the signal, this causes ksh to terminate itself ungracefully:
$ cat /tmp/exit267
trap 'echo OK' EXIT # This trap triggers the crash
function foo { return 267; }
foo
$ bash /tmp/exit267
OK
$ ksh-3aee10d7 /tmp/exit267
OK
$ ksh /tmp/exit267
Memory fault(coredump)
On most systems, status 267 corresponds to SIGSEGV. The reported
memory fault is not real; it results from ksh incorrectly killing
itself with that signal.
The problem is caused by two factors:
1. As of 93u+ 2012-08-01, ksh explicitly allows 'return' to use an
exit status corresponding to a signal (from 257 to end of signal
range). The rest of the integer range is trunctated to 8 bits.
This is contrary to both 'man ksh' and 'return --man' which both
say it's always truncated to 8 bits. Plus, combined with point 2
below, this new behaviour is nonsensical, as 'return' has no
business actually generating signals. However, a couple of
regression tests now depend on this, as may some scripts.
2. When a ksh-style function does not handle a signal, the signal
is passed down to the parent environment and ksh does this by
reissuing the signal to its own process after leaving the
function scope. However, it does this by checking the exit
status, which is very bad practice as there is no guarantee
that an exit status corresponding to a signal was in fact
produced by a signal, particularly after they changed the
behaviour of 'return' per 1 above.
This commit fixes both issues. It also takes a proper decision on
allowable 'return' exit status arguments. Since 93u+ was released
nearly a decade ago and some scripts may now rely on being able to
pass certain exit statuses out of the 8-bit range, we should not
disallow this now. But neither should we be half-hearted in
allowing only some arbitrary selection of 9-bit statuses; 'return'
values categorically should have nothing to do with signals, so
this is no basis for limiting them. We're now allowing the full
unsigned integer range, which is usually 32 bits. This is like zsh,
and may create some interesting possibilities for scripts.
Just don't forget that $? will still lose all but its 8 least
significant bits when leaving the current (sub)shell environment.
src/cmd/ksh93/sh/xec.c: sh_funscope():
- Fix passing down unhandled signals from interrupted ksh functions
(jumpval==SH_JMPFUN) to the parent environment. Do not pay any
attention to the exit status. Instead, use sh.lastsig (a.k.a.
shp->lastsig). It is set by sh_fault() in fault.c for just this
purpose and contains the last signal handled for the current
command. It is reset in sh_exec() before running any new command.
So if it contains a signal, that is the one that interrupted the
ksh function, so it's the correct one to pass down. (Further
evidence: sh_subshell() was already using this in the same way.)
src/cmd/ksh93/bltins/cflow.c: b_return():
- Allow any signed int return value when invoked as and behaving
like 'return'.
- Add warning if a passed value is out of int range. Set the exit
status to 128 in that case; int overflow is undefined behaviour
in C and we want consistent behaviour across platforms. It should
be safe enough to check if the long and int values are equal.
- Refactor for clarity.
src/cmd/ksh93/sh/subshell.c: sh_subshell():
- If a function returns with a status out of the 8 bit range in a
virtual subshell, this status could be passed down to the parent
shell in full. However, if the subshell forks, then the kernel
will enforce an 8-bit exit status. That is inconsistent. Scripts
should not be able to tell the difference between forked and
non-forked subshells, so artificially enforce that limit here.
Other changed files:
- Documentation updates and copy-edits.
- Update an AT&T functions.sh regress test to allow arbitrary
integer return values for functions.
- Add regression tests based in part on @JohnoKing's reproducers.
- Rework some vaguely related regression tests to fail gracefully.
Thanks to Johnothan King for the report and the testing.
Fixes: https://github.com/ksh93/ksh/issues/364
2021-12-08 03:14:15 +00:00
|
|
|
2021-12-08:
|
|
|
|
|
|
|
|
|
|
- Fixed: if a function returned with a status > 256 using the 'return' command
|
|
|
|
|
and the return value corresponded to a value that could have resulted from a
|
|
|
|
|
signal, and an EXIT trap was active, then the shell mistakenly issued that
|
|
|
|
|
signal to itself. Depending on the signal, this could cause the shell to
|
|
|
|
|
terminate ungracefully, e.g. 'return 267' caused SIGSEGV ("memory fault").
|
|
|
|
|
|
|
|
|
|
- For the 'return' built-in command, you can now freely specify any
|
|
|
|
|
return value that fits in a signed integer, typically a 32-bit value.
|
|
|
|
|
Note that $? is truncated to 8 bits when the current (sub)shell exits.
|
|
|
|
|
|
2021-12-08 06:56:52 +00:00
|
|
|
- The head and tail builtins now correctly handle files that do not have an
|
|
|
|
|
ending newline. (Note that the tail builtin is not compiled in by default.)
|
|
|
|
|
|
2021-12-05 18:55:15 +00:00
|
|
|
2021-12-05:
|
|
|
|
|
|
|
|
|
|
- Fixed an issue on illumos that caused some parameters in the getconf
|
|
|
|
|
builtin to fail.
|
|
|
|
|
|
2021-12-06 05:48:32 +00:00
|
|
|
- The cd built-in command now supports a -e option (as specified in
|
|
|
|
|
https://www.austingroupbugs.net/view.php?id=253). Passing -e alongside -P
|
|
|
|
|
is used to guarantee the cd built-in returns with exit status 1 if the
|
|
|
|
|
current working directory couldn't be determined after successfully changing
|
|
|
|
|
the directory.
|
|
|
|
|
|
2021-12-02 04:43:41 +00:00
|
|
|
2021-12-01:
|
|
|
|
|
|
|
|
|
|
- Fixed a memory fault that occurred when a discipline function exited
|
|
|
|
|
with an error from a special builtin or when a discipline function exited
|
|
|
|
|
because of a signal.
|
|
|
|
|
|
2021-11-29 08:16:23 +00:00
|
|
|
2021-11-29:
|
|
|
|
|
|
2021-11-29 12:38:17 +00:00
|
|
|
- Fixed a memory fault that prevented ksh from functioning on ARM-based Macs.
|
|
|
|
|
|
2021-11-29 08:16:23 +00:00
|
|
|
- A bug that caused the time keyword to override the errexit shell option has
|
|
|
|
|
been fixed.
|
|
|
|
|
|
2021-11-30 03:26:53 +00:00
|
|
|
- Fixed a crash that could occur when a KEYBD trap was set and a multi-line
|
|
|
|
|
command substitution was input in an interactive shell.
|
|
|
|
|
|
2021-11-30 20:26:17 +00:00
|
|
|
- The shell linter's warnings for obsolete arithmetic operators in [[ ... ]]
|
|
|
|
|
and unnecessary variable expansion in ((...)) have been improved.
|
|
|
|
|
|
2021-11-24 08:10:25 +00:00
|
|
|
2021-11-24:
|
|
|
|
|
|
|
|
|
|
- The --posix mode was amended to stop the '.' command (but not 'source') from
|
|
|
|
|
looking up functions defined with the 'function' keyword. In the POSIX
|
|
|
|
|
standard and on other shells, the '.' command finds only script files.
|
|
|
|
|
|
2021-11-25 02:45:21 +00:00
|
|
|
- The rm built-in's -d/--directory option has been fixed. It now properly
|
|
|
|
|
removes empty directories and refuses to remove non-empty directories
|
|
|
|
|
(as specified in https://www.austingroupbugs.net/view.php?id=802). Note
|
|
|
|
|
that the rm built-in command isn't compiled in by default.
|
|
|
|
|
|
arith: implement range checking for enum types
Within arithmetic expressions, enumeration values of variables of a
type created with the 'enum' command translate to index numbers
from 0 to the number of elements minus 1. However, there was no
range checking on this in the arithmetic subsystem, allowing the
assignment of out-of-range values that did not correspond to any
enumeration value.
Variables of an enum type are internally unsigned short integers
(NV_UINT16), like those created with 'integer -su', except with an
additional discipline function (ENUM_disc).
src/cmd/ksh93/bltins/enum.c,
src/cmd/ksh93/include/builtins.h:
- To implement range checking, the arithmetic system needs access
to the 'nelem' (number of elements) member of 'struct Enum'. This
is only defined locally in enum.c. We could move that to name.h
so arith.c can access it, but enum.c has code that supports
compiling as standalone. So, instead, define a quick extern
function, b_enum_elem(), that does the necessary type conversion
and returns a type's number of elements.
- Add --man documentation for the arithmetic subsystem behaviour
for enum types. Tell the enuminfo() function, which dynamically
inserts values into the documentation, how to process new \f tags
'lastv' (the last-defined value) and 'lastn' (the number of the
last element).
src/cmd/ksh93/sh/arith.c: arith():
- For NV_UINT16 variables with an ENUM_disc discipline, check the
range using b_enum_elem() and error out if necessary.
Resolves: https://github.com/ksh93/ksh/issues/335
2021-11-23 20:35:30 +00:00
|
|
|
2021-11-23:
|
|
|
|
|
|
|
|
|
|
- A bug was fixed that allowed arithmetic expressions to assign out-of-range
|
|
|
|
|
values to variables of an enumeration type defined with the 'enum' command,
|
|
|
|
|
causing undefined behavior. Within arithmetic expressions, enumeration
|
|
|
|
|
values translate to index numbers from 0 to the number of elements minus 1.
|
|
|
|
|
That range is now checked for. Decimal fractions are ignored.
|
|
|
|
|
|
Fix enum type definition pre-parsing for shcomp and dot/source
Parser limitations prevent shcomp or source from handling enum
types correctly:
$ cat /tmp/colors.sh
enum Color_t=(red green blue orange yellow)
Color_t -A Colors=([foo]=red)
$ shcomp /tmp/colors.sh > /dev/null
/tmp/colors.sh: syntax error at line 2: `(' unexpected
$ source /tmp/colors.sh
/bin/ksh: source: syntax error: `(' unexpected
Yet, for types created using 'typeset -T', this works. This is done
via a check_typedef() function that preliminarily adds the special
declaration builtin at parse time, with details to be filled in
later at execution time.
This hack will produce ugly undefined behaviour if the definition
command creating that type built-in is then not actually run at
execution time before the type built-in is accessed.
But the hack is necessary because we're dealing with a fundamental
design flaw in the ksh language. Dynamically addable built-ins that
change the syntactic parsing of the shell language on the fly are
an absurdity that violates the separation between parsing and
execution, which muddies the waters and creates the need for some
kind of ugly hack to keep things like shcomp more or less working.
This commit extends that hack to support enum.
src/cmd/ksh93/sh/parse.c:
- check_typedef():
- Add 'intypeset' parameter that should be set to 1 for typeset
and friends, 2 for enum.
- When processing enum arguments, use AST getopt(3) to skip over
enum's options to find the name of the type to be defined.
(getopt failed if we were running a -c script; deal with this
by zeroing opt_info.index first.)
- item(): Update check_typedef() call, passing lexp->intypeset.
- simple(): Set lexp->intypeset to 2 when processing enum.
The rest of the changes are all to support the above and should be
fairly obvious, except:
src/cmd/ksh93/bltins/enum.c:
- enuminfo(): Return on null pointer, avoiding a crash upon
executing 'Type_t --man' if Type_t has not been fully defined due
to the definition being pre-added at parse time but not executed.
It's all still wrong, but a crash is worse.
Resolves: https://github.com/ksh93/ksh/issues/256
2021-11-21 06:12:27 +00:00
|
|
|
2021-11-21:
|
|
|
|
|
|
|
|
|
|
- It is now possible to use types defined by 'enum' in contexts where the
|
|
|
|
|
script is entirely parsed before (or without) being executed, such as
|
|
|
|
|
dotted/sourced scripts and scripts compiled by shcomp.
|
|
|
|
|
|
2021-11-23 00:15:24 +00:00
|
|
|
- Added support for the size mode to the stty(1) built-in. This mode is used
|
|
|
|
|
to display the terminal's number of rows and columns. Note that the stty
|
|
|
|
|
built-in is not compiled in by default. This can be changed by adding
|
|
|
|
|
stty to the table of built-ins in src/cmd/ksh93/data/builtins.c.
|
|
|
|
|
|
2021-11-20 08:41:49 +00:00
|
|
|
2021-11-20:
|
|
|
|
|
|
|
|
|
|
- Listing types with 'typeset -T' no longer displays incomplete versions of
|
|
|
|
|
types created by the enum built-in.
|
|
|
|
|
|
2021-11-18 22:36:28 +00:00
|
|
|
2021-11-18:
|
|
|
|
|
|
|
|
|
|
- The printf built-in command now supports a -v option as on bash and zsh.
|
|
|
|
|
This allows you to assign formatted output directly to a variable.
|
|
|
|
|
|
Fix a few issues with $RANDOM seeding in subshells (#339)
This commit fixes an issue I found in the subshell $RANDOM
reseeding code.
The main issue is a performance regression in the shbench fibonacci
benchmark, introduced in commit af6a32d1. Performance dropped in
this benchmark because $RANDOM is always reseeded and restored,
even when it's never used in a subshell. Performance results from
before and after this performance fix (results are on Linux with
CC=gcc and CCFLAGS='-O2 -D_std_malloc'):
$ ./shbench -b bench/fibonacci.ksh -l 100 ./ksh-0f06a2e ./ksh-af6a32d ./ksh-f31e368 ./ksh-randfix
benchmarking ./ksh-0f06a2e, ./ksh-af6a32d, ./ksh-f31e368, ./ksh-randfix ...
*** fibonacci.ksh ***
# ./ksh-0f06a2e # Recent version of ksh93u+m
# ./ksh-af6a32d # Commit that introduced the regression
# ./ksh-f31e368 # Commit without the regression
# ./ksh-randfix # Ksh93u+m with this patch applied
-------------------------------------------------------------------------------------------------
name ./ksh-0f06a2e ./ksh-af6a32d ./ksh-f31e368 ./ksh-randfix
-------------------------------------------------------------------------------------------------
fibonacci.ksh 0.481 [0.459-0.515] 0.472 [0.455-0.504] 0.396 [0.380-0.442] 0.407 [0.385-0.439]
-------------------------------------------------------------------------------------------------
src/cmd/ksh93/include/variables.h,
src/cmd/ksh93/sh/{init,subshell}.c:
- Rather than reseed $RANDOM every time a subshell is created, add
a sh_save_rand_seed() function that does this only when the
$RANDOM variable is used in a subshell. This function is called
by the $RANDOM discipline functions nget_rand() and put_rand().
As a minor optimization, sh_save_rand_seed doesn't reseed if it's
called from put_rand().
- Because $RANDOM may have a seed of zero (i.e., RANDOM=0),
sp->rand_seed isn't enough to tell if $RANDOM has been reseeded.
Add sp->rand_state for this purpose.
- sh_subshell(): Only restore the former $RANDOM seed and state if
it is necessary to prevent a subshell leak.
src/cmd/ksh93/tests/variables.sh:
- Add two regression tests for bugs I ran into while making this
patch.
2021-11-19 07:17:25 +00:00
|
|
|
- Fixed a performance regression introduced on 2021-05-03 that caused
|
|
|
|
|
the shbench[*] fibonacci benchmark to run slower.
|
|
|
|
|
[*]: https://github.com/ksh-community/shbench
|
|
|
|
|
|
arithmetic: Fix the octal leading zero mess (#337)
In C/POSIX arithmetic, a leading 0 denotes an octal number, e.g.
010 == 8. But this is not a desirable feature as it can cause
problems with processing things like dates with a leading zero.
In ksh, you should use 8#10 instead ("10" with base 8).
It would be tolerable if ksh at least implemented it consistently.
But AT&T made an incredible mess of it. For anyone who is not
intimately familiar with ksh internals, it is inscrutable where
arithmetic evaluation special-cases a leading 0 and where it
doesn't. Here are just some of the surprises/inconsistencies:
1. The AT&T maintainers tried to honour a leading 0 inside of
((...)) and $((...)) and not for arithmetic contexts outside it,
but even that inconsistency was never quite consistent.
2. Since 2010-12-12, $((x)) and $(($x)) are different:
$ /bin/ksh -c 'x=010; echo $((x)) $(($x))'
10 8
That's a clear violation of both POSIX and the principle of
least astonishment. $((x)) and $(($x)) should be the same in
all cases.
3. 'let' with '-o letoctal' acts in this bizarre way:
$ set -o letoctal; x=010; let "y1=$x" "y2=010"; echo $y1 $y2
10 8
That's right, 'let y=$x' is different from 'let y=010' even
when $x contains the same string value '010'! This violates
established shell grammar on the most basic level.
This commit introduces consistency. By default, ksh now acts like
mksh and zsh: the octal leading zero is disabled in all arithmetic
contexts equally. In POSIX mode, it is enabled equally.
The one exception is the 'let' built-in, where this can still be
controlled independently with the letoctal option as before (but,
because letoctal is synched with posix when switching that on/off,
it's consistent by default).
We're also removing the hackery that causes variable expansions for
the 'let' builtin to be quietly altered, so that 'x=010; let y=$x'
now does the same as 'let y=010' even with letoctal on.
Various files:
- Get rid of now-redundant sh.inarith (shp->inarith) flag, as we're
no longer distinguishing between being inside or outside ((...)).
src/cmd/ksh93/sh/arith.c:
- arith(): Let disabling POSIX octal constants by skipping leading
zeros depend on either the letoctal option being off (if we're
running the "let" built-in") or the posix option being off.
- sh_strnum(): Preset a base of 10 for strtonll(3) depending on the
posix or letoctal option being off, not on the sh.inarith flag.
src/cmd/ksh93/include/argnod.h,
src/cmd/ksh93/sh/args.c,
src/cmd/ksh93/sh/macro.c:
- Remove astonishing hackery that violated shell grammar for 'let'.
src/cmd/ksh93/sh/name.c (nv_getnum()),
src/cmd/ksh93/sh/nvdisc.c (nv_getn()):
- Remove loops for skipping leading zeroes that included a broken
check for justify/zerofill attributes, thereby fixing this bug:
$ typeset -Z x=0x15; echo $((x))
-ksh: x15: parameter not set
Even if this code wasn't redundant before, it is now: sh_arith()
is called immediately after the removed code and it ignores
leading zeroes via sh_strnum() and strtonll(3).
Resolves: https://github.com/ksh93/ksh/issues/334
2021-11-17 03:24:18 +00:00
|
|
|
2021-11-16:
|
|
|
|
|
|
|
|
|
|
- By default, arithmetic expressions in ksh no longer interpret a number
|
|
|
|
|
with a leading zero as octal in any context. Use 8#octalnumber instead.
|
|
|
|
|
Before, ksh would arbitrarily recognize the leading octal zero in some
|
|
|
|
|
contexts but not others, e.g., both of:
|
|
|
|
|
$ x=010; echo "$((x)), $(($x))"
|
|
|
|
|
$ set -o letoctal; x=010; let y=$x z=010; echo "$y, $z"
|
|
|
|
|
would output '10, 8'. These now output '10, 10' and '8, 8', respectively.
|
|
|
|
|
Arithmetic expressions now also behave identically within and outside
|
|
|
|
|
((...)) and $((...)). Setting the --posix compliance option turns on the
|
|
|
|
|
recognition of the leading octal zero for all arithmetic contexts.
|
|
|
|
|
|
2021-11-15 17:49:19 +00:00
|
|
|
2021-11-15:
|
|
|
|
|
|
|
|
|
|
- In arithmetic evaluation, the --posix compliance option now disables the
|
|
|
|
|
special floating point constants Inf and NaN so that $((inf)) and $((nan))
|
|
|
|
|
refer to the variables by those names as the standard requires. (BUG_ARITHNAN)
|
|
|
|
|
|
2021-11-16 00:46:47 +00:00
|
|
|
- Fixed two file descriptor leaks in the hist builtin that occurred when
|
|
|
|
|
the -s flag ran a command or encountered an error.
|
|
|
|
|
|
2021-11-14 17:27:05 +00:00
|
|
|
2021-11-14:
|
|
|
|
|
|
2021-11-15 02:18:18 +00:00
|
|
|
- Fixed: ksh crashed after unsetting .sh.match and then matching a pattern.
|
|
|
|
|
|
2021-11-14 17:27:05 +00:00
|
|
|
- Another test/[ fix: "test \( string1 -a string2 \)" and "test \( string1 -o
|
|
|
|
|
string2 \)" no longer give an incorrect "argument expected" error message.
|
|
|
|
|
|
2021-11-13 02:17:54 +00:00
|
|
|
2021-11-13:
|
|
|
|
|
|
test/[: binary operators: fix '<' and add '=~'; some more cleanups
In ksh88, the test/[ built-in supported both the '<' and '>'
lexical sorting comparison operators, same as in [[. However, in
every version of ksh93, '<' does not work though '>' still does!
Still, the code for both is present in test_binop():
src/cmd/ksh93/bltins/test.c
548: case TEST_SGT:
549: return(strcoll(left, right)>0);
550: case TEST_SLT:
551: return(strcoll(left, right)<0);
Analysis: The binary operators are looked up in shtab_testops[] in
data/testops.c using a macro called sh_lookup, which expands to a
sh_locate() call. If we examine that function in sh/string.c, it's
easy to see that on systems using ASCII (i.e. all except IBM
mainframes), it assumes the table is sorted in ASCII order.
src/cmd/ksh93/sh/string.c
64: while((c= *tp->sh_name) && (CC_NATIVE!=CC_ASCII || c <= first))
The problem was that the '<' operator was not correctly sorted in
shtab_testops[]; it was sorted immediately before '>', but after
'='. The ASCII order is: < (60), = (61), > (62). This caused '<' to
never be found in the table.
The test_binop() function is also used by [[, yet '<' always worked
in that. This is because the parser has code that directly checks
for '<' and '>' within [[ (in sh/parse.c, lines 1949-1952).
This commit also adds '=~' to 'test', which took three lines of
code and allowed eliminating error handling in test_binop() as
test/[ and [[ now support the same binary ops. (re: fc2d5a60)
src/cmd/ksh93/*/*.[ch]:
- Rename a couple of very misleadingly named macros in test.h:
. For == and !=, the TEST_PATTERN bit is off for pattern compares
and on for literal string compares! Rename to TEST_STRCMP.
. The TEST_BINOP bit does not denote all binary operators, but
only the logical -a/-o ops in test/[. Rename to TEST_ANDOR.
src/cmd/ksh93/bltins/test.c: test_binop():
- Add support for =~. This is only used by test/[. The method is
implemented in two lines that convert the ERE to a shell pattern
by prefixing it with ~(E), then call test_strmatch with that
temporary string to match the ERE and update ${.sh.match}.
- Since all binary ops from shtab_testops[] are now accounted for,
remove unknown op error handling from this function.
src/cmd/ksh93/data/testops.c:
- shtab_testops[]:
. Correctly sort the '<' (TEST_SLT) entry.
. Remove ']]' (TEST_END). It's not an op and doesn't belong here.
- Update sh_opttest[] documentation with =~, \<, \>.
- Remove now-unused e_unsupported_op[] error message.
src/cmd/ksh93/sh/lex.c: sh_lex():
- Check for ']]' directly instead of relying on the removed
TEST_END entry from shtab_testops[].
src/cmd/ksh93/tests/bracket.sh:
- Add relevant tests.
src/cmd/ksh93/tests/builtins.sh:
- Fix an old test that globally deleted the 'test' builtin. Delete
it within the command substitution subshell only.
- Remove the test for non-support of =~ in test/[.
- Update the test for invalid test/[ op to use test directly.
2021-11-13 22:57:15 +00:00
|
|
|
- The test/[ built-in command now supports the '<' and '=~' operators from [[.
|
|
|
|
|
As of now, test/[ supports the same operators as [[ except for the different
|
|
|
|
|
and/or operators. Note: test/[ remains deprecated due to its many pitfalls.
|
|
|
|
|
|
2021-11-13 02:17:54 +00:00
|
|
|
- The test/[ built-in command is fixed so that the binary -a (and) and -o (or)
|
|
|
|
|
operators, as in [ "$a" -a "$b" ] or [ "$a" -o "$b" ], work even if "$a" is
|
|
|
|
|
'!' or '('. To avoid breaking backwards compatibility with the nonstandard
|
|
|
|
|
unary [ -a "$file" ] and [ -o "$option" ] operators in combination with '!'
|
|
|
|
|
or parentheses, this fix is only activated if the posix option is on.
|
|
|
|
|
|
Fix bug on closed stdout; improve BUG_PUTIOERR fix (re: 93e15a30)
Stéphane Chazelas reported:
> As noted in this austin-group-l discussion[*] (relevant to this
> issue):
>
> $ ksh93u+m -c 'pwd; echo "$?" >&2; echo test; echo "$?" >&2' >&-
> 0
> 1
> /home/chazelas
>
> when stdout is closed, pwd does claim it succeeds (by returning a
> 0 exit status), while echo doesn't (not really relevant to the
> problem here, only to show it doesn't affect all builtins), and
> the output that pwd failed to write earlier ends up being written
> on stderr here instead of stdout upon exit (presumably) because
> of that >&2 redirection.
>
> strace shows ksh93 attempting write(1, "/home/chazelas\n", 15) 6
> times (1, the last one, successful).
>
> It gets even weirder when redirecting to a file:
>
> $ ksh93u+m -c 'pwd; echo "$?" >&2; echo test; echo "$?" > file' >&-
> 0
> $ cat file
> 1
> 1
> ome/chazelas
In my testing, the problem does not occur when closing stdout at
the start of the -c script itself (using redirect >&- or exec >&-);
it only occurs if stdout was closed before initialising the shell.
That made me suspect that the problem had to do with an
inconsistent file descriptor state in the shell. ksh uses internal
sh_open() and sh_close() functions, among others, to maintain that
state.
src/cmd/ksh93/sh/main.c: sh_main():
- If the shell is initialised with stdin, stdout or stderr closed,
then make the shell's file descriptor state tables reflect that
fact by calling sh_close() for the closed file descriptors.
This commit also improves the BUG_PUTIOERR fix from 93e15a30. Error
checking after sfsync() is not sufficient. For instance, on
FreeBSD, the following did not produce a non-zero exit status:
ksh -c 'echo hi' >/dev/full
even though this did:
ksh -c 'echo hi >/dev/full'
Reliable error checking requires not only checking the result of
every SFIO command that writes output, but also synching the buffer
at the end of the operation and checking the result of that.
src/cmd/ksh93/bltins/print.c:
- Make exitval variable global to allow functions called by
b_print() to set a nonzero exit status.
- Check the result of all SFIO output commands that write output.
- b_print(): Always sfsync() at the end, except if the s (history)
flag was given. This allows getting rid of the sfsync() call that
required the workaround introduced in 846ad932.
[*] https://www.mail-archive.com/austin-group-l@opengroup.org/msg08056.html
Resolves: https://github.com/ksh93/ksh/issues/314
2021-11-07 10:14:54 +00:00
|
|
|
2021-11-07:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that could corrupt output if standard output is closed upon
|
|
|
|
|
initializing the shell.
|
|
|
|
|
|
|
|
|
|
- Improved BUG_PUTIOERR fix (2020-05-14) with more error checking. On
|
|
|
|
|
systems with the "disk full" error testing device /dev/full, an
|
|
|
|
|
echo/print/printf to /dev/full now always yields a non-zero exit status.
|
|
|
|
|
|
Fix octal number arguments in printf integer arithmetic
Bug 1: POSIX requires numbers used as arguments for all the %d,
%u... in printf to be interpreted as in the C language, so
printf '%d\n' 010
should output 8 when the posix option is on. However, it outputs 10.
This bug was introduced as a side effect of a change introduced in
the 2012-02-07 version of ksh 93u+m, which caused the recognition
of leading-zero numbers as octal in arithmetic expressions to be
disabled outside ((...)) and $((...)). However, POSIX requires
leading-zero octal numbers to be recognised for printf, too.
The change in question introduced a sh.arith flag that is set while
we're processing a POSIX arithmetic expression, i.e., one that
recognises leading-zero octal numbers.
Bug 2: Said flag is not reset in a command substitution used within
an arithmetic expression. A command substitution should be a
completely new context, so the following should both output 10:
$ ksh -c 'integer x; x=010; echo $x'
10 # ok; it's outside ((…)) so octals are not recognised
$ ksh -c 'echo $(( $(integer x; x=010; echo $x) ))'
8 # bad; $(comsub) should create new non-((…)) context
src/cmd/ksh93/bltins/print.c: extend():
- For the u, d, i, o, x, and X conversion modifiers, set the POSIX
arithmetic context flag before calling sh_strnum() to convert the
argument. This fixes bug 1.
src/cmd/ksh93/sh/subshell.c: sh_subshell():
- When invoking a command substitution, save and unset the POSIX
arithmetic context flag. Restore it at the end. This fixes bug 2.
Reported-by: @stephane-chazelas
Resolves: https://github.com/ksh93/ksh/issues/326
2021-09-13 02:23:50 +00:00
|
|
|
2021-09-13:
|
|
|
|
|
|
|
|
|
|
- Disable the POSIX arithmetic context while running a command substitution
|
|
|
|
|
invoked from within an arithmetic expression. This fixes a bug that caused
|
|
|
|
|
integer arguments with a leading zero to be incorrectly interpreted as octal
|
|
|
|
|
numbers in non-POSIX arithmetic contexts within such command substitutions.
|
|
|
|
|
|
Fix 'ps' output for hashbangless scripts on Linux/macOS
When invoking a script without an interpreter (#!hashbang) path,
ksh forks, but there is no exec syscall in the child. The existing
command line is overwritten in fixargs() with the name of the new
script and associated arguments. In the generic/fallback version of
fixargs() which is used on Linux and macOS, if the new command line
is longer than the existing one, it is truncated. This works well
when calling a script with a shorter name.
However, it generates a misleading name in the common scenario
where a script is invoked from an interactive shell, which
typically has a short command line. For instance, if "/tmp/script"
is invoked, "ksh" gets replaced with "/tm" in "ps" output.
A solution is found in the fact that, on these systems, the
environment is stored immediately after the command line arguments.
This space can be made available for use by a longer command line
by moving the environment strings out of the way.
src/cmd/ksh93/sh/main.c: fixargs():
- Refactor BSD setproctitle(3) version to be more self-contained.
- In the generic (Linux/macOS) version, on init (i.e. mode==0), if
the command line is smaller than 128 bytes and the environment
strings have not yet been moved (i.e. if they still immediately
follow the command line arguments in memory), then strdup the
environment strings, pointing the *environment[] members to the
new strings and adding the length of the strings to the maximum
command line buffer size.
Reported-by: @gkamat
Resolves: https://github.com/ksh93/ksh/pull/300
2021-09-12 03:16:51 +00:00
|
|
|
2021-09-12:
|
|
|
|
|
|
|
|
|
|
- When invoking a script without an interpreter/hashbang path on Linux and
|
|
|
|
|
macOS, ksh can now update 'ps' output to show longer command lines.
|
|
|
|
|
|
Fix `backtick` comsubs by making them act like $(modern) ones
ksh93 currently has three command substitution mechanisms:
- type 1: old-style backtick comsubs that use a pipe;
- type 3: $(modern) comsubs that use a temp file, currently with
fallback to a pipe if a temp file cannot be created;
- type 2: ${ shared-state; } comsubs; same as type 3, but shares
state with parent environment.
Type 1 is buggy. There are at least two reproducers that make it
hang. The Red Hat patch applied in 4ce486a7 fixed a hang in
backtick comsubs but reintroduced another hang that was fixed in
ksh 93v-. So far, no one has succeeded in making pipe-based comsubs
work properly.
But, modern (type 3) comsubs use temp files. How does it make any
sense to have two different command substitution mechanisms at the
execution level? The specified functionality between backtick and
modern command substitutions is exactly the same; the difference
*should* be purely syntactic.
So this commit removes the type 1 comsub code at the execution
level, treating them all like type 3 (or 2). As a result, the
related bugs vanish while the regression tests all pass.
The only side effect that I can find is that the behaviour of bug
https://github.com/ksh93/ksh/issues/124 changes for backtick
comsubs. But it's broken either way, so that's neutral.
So this commit can now be added to my growing list of ksh93 issues
fixed by simply removing code.
src/cmd/ksh93/sh/xec.c:
- Remove special code for type 1 comsubs from iousepipe(),
sh_iounpipe(), sh_exec() and _sh_fork().
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/subshell.c:
- Remove pipe support from sh_subtmpfile(). This also removes the
use of a pipe as a fallback for $(modern) comsubs. Instead, panic
and error out if temp file creation fails. If the shell cannot
create a temporary file, there are fatal system problems anyway
and a script should not continue.
- No longer pass comsub type to sh_subtmpfile().
All other changes:
- Update sh_subtmpfile() calls.
src/cmd/ksh93/tests/subshell.sh:
- Add two regression tests based on reproducers from bug reports.
Resolves: https://github.com/ksh93/ksh/issues/305
Resolves: https://github.com/ksh93/ksh/issues/316
2021-08-13 06:01:46 +00:00
|
|
|
2021-08-13:
|
|
|
|
|
|
|
|
|
|
- An issue was fixed that could cause old-style `backtick` command
|
|
|
|
|
substitutions to hang in certain cases.
|
|
|
|
|
|
[[ ... ]]: fix '!' to negate another '!'
Bug: [[ ! ! 1 -eq 1 ]] returns false, but should return true.
This bug was reported for bash, but ksh has it too:
https://lists.gnu.org/archive/html/bug-bash/2021-06/msg00006.html
Op 24-05-21 om 17:47 schreef Chet Ramey:
> On 5/22/21 2:45 PM, Vincent Menegaux wrote:
>> Previously, these commands:
>>
>> [[ ! 1 -eq 1 ]]; echo $?
>> [[ ! ! 1 -eq 1 ]]; echo $?
>>
>> would both result in `1', since parsing `!' set CMD_INVERT_RETURN
>> instead of toggling it.
>
> Interestingly, ksh93 produces the same result as bash. I agree
> that it's more intuitive to toggle it.
Also interesting is that '!' as an argument to the simple
'test'/'[' command does work as expected (on both bash and ksh93):
'test ! ! 1 -eq 1' and '[ ! ! 1 -eq 1 ]' return 0/true.
Even the man page for [[ is identical for bash and ksh93:
| ! expression
| True if expression is false.
This suggests it's supposed to be a logical negation operator, i.e.
'!' is implicitly documented to negate another '!'. Bolsky & Korn's
1995 ksh book, p. 167, is slightly more explicit about it:
"! test-expression. Logical negation of test-expression."
I also note that multiple '!' negators in '[[' work as expected on
mksh, yash and zsh.
src/cmd/ksh93/sh/parse.c: test_primary():
- Fix bitwise logic for '!': xor the TNEGATE bit into tretyp
instead of or'ing it, which has the effect of toggling it.
2021-06-03 13:47:25 +00:00
|
|
|
2021-06-03:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug in the [[ compound command: the '!' logical negation operator
|
|
|
|
|
now correctly negates another '!', e.g., [[ ! ! 1 -eq 1 ]] now returns
|
|
|
|
|
0/true. Note that this has always been the case for 'test'/'['.
|
|
|
|
|
|
2021-05-18 22:02:49 +00:00
|
|
|
2021-05-18:
|
|
|
|
|
|
|
|
|
|
- Fixed SHLVL so that replacing ksh by itself (exec ksh) will not increase it.
|
|
|
|
|
|
2021-05-19 04:53:56 +00:00
|
|
|
- Fixed a regression introduced on 2020-08-05 that caused a non-interactive
|
|
|
|
|
shell to exit if an I/O redirection of a function call encountered an error.
|
|
|
|
|
|
2021-05-13 07:30:28 +00:00
|
|
|
2021-05-13:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug with 'test -t 1' that was introduced on 2021-04-26:
|
|
|
|
|
v=$(test -t 1 >/dev/tty && echo ok) did not assign 'ok' to v.
|
|
|
|
|
|
2021-05-10 16:40:44 +00:00
|
|
|
2021-05-10:
|
|
|
|
|
|
|
|
|
|
- Release 1.0.0-beta.1.
|
|
|
|
|
|
2021-05-08 03:43:37 +00:00
|
|
|
2021-05-07:
|
|
|
|
|
|
|
|
|
|
- Backported three ksh 93v- math.tab changes, allowing for an exp10()
|
|
|
|
|
arithmetic function if one exists in the C library, a new float()
|
|
|
|
|
function, and lastly an updated int() function that rounds to zero
|
|
|
|
|
instead of being an alias to floor().
|
|
|
|
|
|
Fix two more 'command' bugs
BUG 1: Though 'command' is specified/documented as a regular
builtin, preceding assignments survive the invocation (as with
special or declaration builtins) if 'command' has no command
arguments in these cases:
$ foo=wrong1 command; echo $foo
wrong1
$ foo=wrong2 command -p; echo $foo
wrong2
$ foo=wrong3 command -x; echo $foo
wrong3
Analysis: sh_exec(), case TCOM (simple command), contains the
following loop that skips over 'command' prefixes, preparsing any
options and remembering the offset in the 'command' variable:
src/cmd/ksh93/sh/xec.c
1059 while(np==SYSCOMMAND || !np && com0
&& nv_search(com0,shp->fun_tree,0)==SYSCOMMAND)
1060 {
1061 register int n = b_command(0,com,&shp->bltindata);
1062 if(n==0)
1063 break;
1064 command += n;
1065 np = 0;
1066 if(!(com0= *(com+=n)))
1067 break;
1068 np = nv_bfsearch(com0, shp->bltin_tree, &nq, &cp);
1069 }
This skipping is not done if the preliminary b_command() call on
line 1061 (with argc==0) returns zero. This is currently the case
for command -v/-V, so that 'command' is treated as a plain and
regular builtin for those options.
The cause of the bug is that this skipping is even done if
'command' has no arguments. So something like 'foo=bar command' is
treated as simply 'foo=bar', which of course survives.
So the fix is for b_command() to return zero if there are no
arguments. Then b_command() itself needs changing to not error out
on the second/main b_command() call if there are no arguments.
src/cmd/ksh93/bltins/whence.c: b_command():
- When called with argc==0, return a zero offset not just for -v
(X_FLAG) or -V (V_FLAG), but also if there are no arguments left
(!*argv) after parsing options.
- When called with argc>0, do not issue a usage error if there are
no arguments, but instead return status 0 (or, if -v/-V was given,
status 2 which was the status of the previous usage message).
This way, 'command -v $emptyvar' now also works as you'd expect.
BUG 2: 'command -p' sometimes failed after executing certain loops.
src/cmd/ksh93/sh/path.c: defpath_init():
- astconf() returns a pointer to memory that may be overwritten
later, so duplicate the string returned. Backported from ksh2020.
(re: f485fe0f, aa4669ad, <https://github.com/att/ast/issues/959>)
src/cmd/ksh93/tests/builtins.sh:
- Update the test for BUG_CMDSPASGN to check every variant of
'command' (all options and none; invoking/querying all kinds of
command and none) with a preceding assignment. (re: fae8862c)
This also covers bug 2 as 'command -p' was failing on macOS prior
to the fix due to a loop executed earlier in another test.
2021-05-05 01:42:45 +00:00
|
|
|
2021-05-05:
|
|
|
|
|
|
|
|
|
|
- Fixed: a preceding variable assignment like foo=bar in 'foo=bar command'
|
|
|
|
|
(with no command arguments after 'command') incorrectly survived the
|
|
|
|
|
'command' regular built-in command invocation.
|
|
|
|
|
|
|
|
|
|
- Fixed: 'command -p some_utility' intermittently failed to find the utility
|
|
|
|
|
under certain conditions due to a memory corruption issue.
|
|
|
|
|
|
2021-05-03 01:59:09 +00:00
|
|
|
2021-05-03:
|
|
|
|
|
|
Fix $RANDOM to act consistently in subshells (#294)
This fixes the following:
1. Using $RANDOM in a virtual/non-forked subshell no longer
influences the reproducible $RANDOM sequence in the parent
environment.
2. When invoking a subshell $RANDOM is now re-seeded (as mksh and
bash do) so that invocations in repeated subshells (including
forked subshells) longer produce identical sequences by default.
3. Program flow corruption that occurred in scripts on executing
( ( simple_command & ) ).
src/cmd/ksh93/include/variables.h:
- Move 'struct rand' here as it will be needed in subshell.c. Add
rand_seed member to save the pseudorandom generator seed. Remove
the pointer to the shell state as it's redundant.
src/cmd/ksh93/sh/init.c:
- put_rand(): Store given seed in rand_seed while calling srand().
No longer pointlessly limit the number of possible seeds with the
RANDMASK bitmask (that mask is to limit the values to 0-32767,
it should not limit the number of possible sequences to 32768).
- nget_rand(): Instead of using rand(), use rand_r() to update the
random_seed value. This makes it possible to save/restore the
current seed of the pseudorandom generator.
- Add sh_reseed_rand() function that reseeds the pseudorandom
generator by calling srand() with a bitwise-xor combination of
the current PID, the current time with a granularity of 1/10000
seconds, and a sequence number that is increased on each
invocation.
- nv_init(): Set the initial seed using sh_reseed_rand() here
instead of in sh_main(), as this is where the other struct rand
members are initialised.
src/cmd/ksh93/sh/main.c: sh_main():
- Remove the srand() call that was replaced by the sh_reseed_rand()
call in init.c.
src/cmd/ksh93/sh/subshell.c: sh_subshell():
- Upon entering a virtual subshell, save the current $RANDOM seed
and state, then reseed $RANDOM for the subshell.
- Upon exiting a virtual subshell, restore $RANDOM seed and state
and reseed the generator using srand() with the restored seed.
src/cmd/ksh93/sh/xec.c: sh_exec():
- When optimizing out a subshell that is the last command, still
act like a subshell: reseed $RANDOM and increase ${.sh.subshell}.
- Fix a separate bug discovered while implementing this. Do not
optimize '( simple_command & )' when in a virtual subshell; doing
this causes program flow corruption.
- When optimizing '( simple_command & )', also reseed $RANDOM and
increment ${.sh.subshell}.
src/cmd/ksh93/tests/subshell.sh,
src/cmd/ksh93/tests/variables.sh:
- Add various tests for all of the above.
Co-authored-by: Johnothan King <johnothanking@protonmail.com>
Resolves: https://github.com/ksh93/ksh/issues/285
2021-05-03 03:03:46 +00:00
|
|
|
- Subshells (even if non-forked) now keep a properly separated state of the
|
|
|
|
|
pseudorandom generator used for $RANDOM, so that using $RANDOM in a
|
|
|
|
|
non-forked subshell no longer influences a reproducible $RANDOM sequence in
|
|
|
|
|
the parent environment. In addition, upon invoking a subshell, $RANDOM is now
|
|
|
|
|
reseeded (as mksh and bash do).
|
|
|
|
|
|
|
|
|
|
- Fixed program flow corruption that occurred in scripts on executing a
|
|
|
|
|
background job in a nested subshell, as in ( ( simple_command & ) ).
|
|
|
|
|
|
2021-05-03 01:59:09 +00:00
|
|
|
- Completed the 2021-04-30 fix for ${var<OP>'{}'} where <OP> is '-', '+',
|
|
|
|
|
':-' or ':+' by fixing a bug that caused an extra '}' to be output.
|
|
|
|
|
|
2021-05-03 23:34:06 +00:00
|
|
|
- Following the resolution of Austin Group bug 1393[*] that is set to be
|
|
|
|
|
included in the next version of the POSIX standard, the 'command' prefix
|
|
|
|
|
in POSIX mode (set -o posix) no longer disables the declaration properties
|
|
|
|
|
of declaration built-ins. This reverts a change introduced on 2020-09-11.
|
|
|
|
|
[*] https://austingroupbugs.net/view.php?id=1393
|
|
|
|
|
|
2021-12-12 21:40:38 +00:00
|
|
|
- Fixed arithmetic assignment operations for multidimensional indexed arrays.
|
2021-05-04 02:13:14 +00:00
|
|
|
|
2021-04-30 04:10:04 +00:00
|
|
|
2021-04-30:
|
|
|
|
|
|
2021-04-30 18:45:50 +00:00
|
|
|
- The emacs 'ESC .' (M-.) and vi '_' commands now take shell quoting into
|
|
|
|
|
account when repeating a word from the previous command line. For example, if
|
|
|
|
|
the previous command is 'ls Stairway\ To\ Heaven.mp3', then they now insert
|
|
|
|
|
'Stairway\ To\ Heaven.mp3' instead of 'Heaven.mp3'. Thanks to Govind Kamat.
|
|
|
|
|
|
2021-04-30 04:10:04 +00:00
|
|
|
- Fixed a bug introduced on 2020-09-05 that caused "echo ${var:+'{}'}"
|
|
|
|
|
to be misparsed.
|
|
|
|
|
|
2021-04-30 23:47:39 +00:00
|
|
|
- Fixed: the effects of 'builtin', 'exec' and 'ulimit' leaked out of a parent
|
|
|
|
|
virtual subshell if run from a ${ shared-state; } command substitution.
|
|
|
|
|
|
2021-04-26 17:09:36 +00:00
|
|
|
2021-04-26:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug introduced on 2021-02-20 in which a shared-state command
|
|
|
|
|
substitution stopped sharing its state with the calling shell environment
|
2021-12-12 21:40:38 +00:00
|
|
|
if it executed a command that locally redirected standard output.
|
2021-04-26 17:09:36 +00:00
|
|
|
|
2021-04-22 02:25:24 +00:00
|
|
|
2021-04-22:
|
|
|
|
|
|
|
|
|
|
- shcomp (the shell bytecode compiler) was fixed to correctly compile process
|
|
|
|
|
substitutions used as the file name to a redirection, as in 'cmd < <(cmd)'.
|
|
|
|
|
|
Fix LINENO after unsetting it a virtual subshell (#283)
There is a TODO note in variables.sh that notes the value of LINENO
is wrong after a virtual subshell. The following script should
print '6', but the bug causes it to print '1' instead:
$ cat /tmp/lineno
#!/bin/ksh
(
unset LINENO
:
)
echo $LINENO
This bug started to occur after the bugfix applied in 7b994b6a.
However, that commit is not where the cause of bug was (when that
bugfix is applied to ksh versions 2008-07-25 through 2012-01-01,
$LINENO works fine). Rather, the cause of this bug was introduced
in 93u+ 2012-02-29. In that version, the mp->nvfun pointer was only
copied from np->nvfun if the variable can be freed from memory.
This is what caused 7b994b6a to break $LINENO in subshells, so to
fix this bug the mp->nvfun and np->nvfun must point to the same
object, even when the variable isn't freed from memory.
src/cmd/ksh93/sh/subshell.c: nv_restore():
- Always copy the np->nvfun pointer to mp->nvfun. To prevent
crashes, the value of np->nvfun->nofree is set to the value given
by the nofree variable, which is set before _nv_unset. See also
commit 7e7f1372, which fixed a crash that happened because
_nv_unset discards the NV_NOFREE flag.
src/cmd/ksh93/tests/variables.sh:
- Remove the workaround for LINENO after a virtual subshell.
- Add a regression test for the value of LINENO when unset in a
virtual subshell, then used after the subshell. Note that before
commit 997ad43b LINENO's value was corrupted after being unset in
a subshell, so the test checks for corruption of the LINENO
variable (in prior commits LINENO was set to '49' because of the
previous bug).
2021-04-22 18:16:25 +00:00
|
|
|
- Fixed a bug introduced on 2020-07-13 that set LINENO to the wrong line
|
|
|
|
|
number after leaving a virtual subshell in which LINENO had been unset.
|
|
|
|
|
|
2021-04-21 18:28:46 +00:00
|
|
|
2021-04-21:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug introduced on 2020-09-28 that caused an interactive ksh to exit
|
|
|
|
|
if a profile script (such as ~/.kshrc) contains a syntax error.
|
|
|
|
|
|
2021-04-21 02:34:54 +00:00
|
|
|
2021-04-20:
|
|
|
|
|
|
|
|
|
|
- Fixed three problems with the /opt/ast/bin/getconf built-in command:
|
|
|
|
|
1. The -l/--lowercase option did not change all variable names to lower case.
|
|
|
|
|
2. The -q/--quote option now quotes all string values. Previously, it only
|
|
|
|
|
quoted string values that had a space or other non-shellsafe character.
|
|
|
|
|
3. The -c/--call, -n/--name and -s/--standard options matched all variable
|
|
|
|
|
names provided by 'getconf -a', even if none were actual matches.
|
|
|
|
|
|
2021-04-21 02:56:03 +00:00
|
|
|
- The readonly attribute of ksh variables is no longer imported from
|
|
|
|
|
or exported to other ksh shell instances through the environment.
|
|
|
|
|
|
2021-04-16 13:45:01 +00:00
|
|
|
2021-04-16:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug in emacs mode: after using tab completion to complete the name
|
|
|
|
|
of a directory, it was not possible to type numbers after the slash.
|
|
|
|
|
|
Fix <>; redirection for final command exec optimization (#277)
The <>; operator doesn't work correctly if it's used as the last
command of a -c script. Reproducer:
$ echo test > a; ksh -c 'echo x 1<>; a'; cat a
x
st
This bug is caused by ksh running the last command of -c scripts
with execve(2) instead of posix_spawn(3) or fork(2). The <>;
operator is noted by the man page as being incompatible with the
exec builtin (see also the ksh93u+ man page), so it's not
surprising this bug occurs when ksh runs a command using execve:
> <>;word cannot be used with the exec and redirect built-ins.
The ksh2020 fix simply removed the code required for ksh to use
this optimization at all. It's not a performance friendly fix and
only papers over the bug, so this commit provides a better fix.
This bug was first reported at:
https://github.com/att/ast/issues/9
In addition, this commit re-enables the execve(2) optimization for
the last command for scripts loaded from a file. It was enabled in
in older ksh versions, and was only disabled in interactive shells:
https://github.com/ksh93/ast-open-history/blob/2011-06-30/src/cmd/ksh93/sh/main.c#L593-L599
It was changed on 2011-12-24 to only be used for -c scripts:
https://github.com/ksh93/ast-open-history/blob/2011-12-24/src/cmd/ksh93/sh/main.c#L593-L599
We think there is no good reason why scripts loaded from a file
should be optimised less than scripts loaded from a -c argument.
They're both scripts; there's no essential difference between them.
So this commit reverts that change. If there is a bug left in the
optimization after this fix, this revert increases the chance of
exposing it so that it can be fixed.
src/cmd/ksh93/sh/xec.c:
- The IOREWRITE flag is set when handling the <>; operator, so to
fix this bug, avoid exec'ing the last command if it uses <>;. See
also commit 17ebfbf6, which fixed another issue related to the
execve optimization.
src/cmd/ksh93/tests/io.sh:
- Enable a regression test that was failing because of this bug.
- Add the reproducer from https://github.com/att/ast/issues/9 as a
regression test.
src/cmd/ksh93/sh/main.c:
- Only avoid the non-forking optimization in interactive shells.
src/cmd/ksh93/tests/signal.sh:
- Add an extra comment to avoid the non-forking optimization in the
regression test for rhbz#1469624.
- If the regression test for rhbz#1469624 fails, show the incorrect
exit status in the error message.
src/cmd/ksh93/tests/builtins.sh,
src/cmd/ksh93/tests/options.sh:
- This bugfix was causing the options regression test to segfault
when run under shcomp. The cause is the same as
<https://github.com/ksh93/ksh/issues/165>, so as a workaround,
avoid parsing process substitutions with shcomp until that is
fixed. This workaround should also avoid the other problem
detailed in <https://github.com/ksh93/ksh/issues/274>.
Resolves: https://github.com/ksh93/ksh/issues/274
2021-04-15 17:29:50 +00:00
|
|
|
- Fixed an optimization bug that caused the <>; redirection operator to fail
|
2021-04-17 20:31:38 +00:00
|
|
|
when used with the last command in a -c script.
|
Fix <>; redirection for final command exec optimization (#277)
The <>; operator doesn't work correctly if it's used as the last
command of a -c script. Reproducer:
$ echo test > a; ksh -c 'echo x 1<>; a'; cat a
x
st
This bug is caused by ksh running the last command of -c scripts
with execve(2) instead of posix_spawn(3) or fork(2). The <>;
operator is noted by the man page as being incompatible with the
exec builtin (see also the ksh93u+ man page), so it's not
surprising this bug occurs when ksh runs a command using execve:
> <>;word cannot be used with the exec and redirect built-ins.
The ksh2020 fix simply removed the code required for ksh to use
this optimization at all. It's not a performance friendly fix and
only papers over the bug, so this commit provides a better fix.
This bug was first reported at:
https://github.com/att/ast/issues/9
In addition, this commit re-enables the execve(2) optimization for
the last command for scripts loaded from a file. It was enabled in
in older ksh versions, and was only disabled in interactive shells:
https://github.com/ksh93/ast-open-history/blob/2011-06-30/src/cmd/ksh93/sh/main.c#L593-L599
It was changed on 2011-12-24 to only be used for -c scripts:
https://github.com/ksh93/ast-open-history/blob/2011-12-24/src/cmd/ksh93/sh/main.c#L593-L599
We think there is no good reason why scripts loaded from a file
should be optimised less than scripts loaded from a -c argument.
They're both scripts; there's no essential difference between them.
So this commit reverts that change. If there is a bug left in the
optimization after this fix, this revert increases the chance of
exposing it so that it can be fixed.
src/cmd/ksh93/sh/xec.c:
- The IOREWRITE flag is set when handling the <>; operator, so to
fix this bug, avoid exec'ing the last command if it uses <>;. See
also commit 17ebfbf6, which fixed another issue related to the
execve optimization.
src/cmd/ksh93/tests/io.sh:
- Enable a regression test that was failing because of this bug.
- Add the reproducer from https://github.com/att/ast/issues/9 as a
regression test.
src/cmd/ksh93/sh/main.c:
- Only avoid the non-forking optimization in interactive shells.
src/cmd/ksh93/tests/signal.sh:
- Add an extra comment to avoid the non-forking optimization in the
regression test for rhbz#1469624.
- If the regression test for rhbz#1469624 fails, show the incorrect
exit status in the error message.
src/cmd/ksh93/tests/builtins.sh,
src/cmd/ksh93/tests/options.sh:
- This bugfix was causing the options regression test to segfault
when run under shcomp. The cause is the same as
<https://github.com/ksh93/ksh/issues/165>, so as a workaround,
avoid parsing process substitutions with shcomp until that is
fixed. This workaround should also avoid the other problem
detailed in <https://github.com/ksh93/ksh/issues/274>.
Resolves: https://github.com/ksh93/ksh/issues/274
2021-04-15 17:29:50 +00:00
|
|
|
|
Allow invoking path-bound built-in commands by direct path or preceding `PATH` assignment (#275)
Path-bound builtins on ksh (such as /opt/ast/bin/cat) break some
basic assumptions about paths in the shell that should hold true,
e.g., that a path output by whence -p or command -v should actually
point to an executable command. This commit should fix the
following:
1. Path-bound built-ins (such as /opt/ast/bin/cat) can now be
executed by invoking the canonical path (independently of the
value of $PATH), so the following will now work as expected:
$ /opt/ast/bin/cat --version
version cat (AT&T Research) 2012-05-31
$ (PATH=/opt/ast/bin:$PATH; "$(whence -p cat)" --version)
version cat (AT&T Research) 2012-05-31
In the event an external command by that path exists, the
path-bound builtin will now override it when invoked using the
canonical path. To invoke a possible external command at that
path, you can still use a non-canonical path, e.g.:
/opt//ast/bin/cat or /opt/ast/./bin/cat
2. Path-bound built-ins will now also be found on a PATH set
locally using an assignment preceding the command, so something
like the following will now work as expected:
$ PATH=/opt/ast/bin cat --version
version cat (AT&T Research) 2012-05-31
The builtin is not found by sh_exec() because the search for
builtins happens long before invocation-local preceding
assignments are processsed. This only happens in sh_ntfork(),
before forking, or in sh_fork(), after forking. Both sh_ntfork()
and sh_fork() call path_spawn() to do the actual path search, so
a check there will cover both cases.
This does mean the builtin will be run in the forked child if
sh_fork() is used (which is the case on interactive shells with
job.jobcontrol set, or always after compiling with SHOPT_SPAWN
disabled). Searching for it before forking would mean
fundamentally redesigning that function to be basically like
sh_ntfork(), so this is hard to avoid.
src/cmd/ksh93/sh/path.c: path_spawn():
- Before doing anything else, check if the passed path appears in
the builtins tree as a pathbound builtin. If so, run it. Since a
builtin will only be found if a preceding PATH assignment
temporarily changed the PATH, and that assignment is currently in
effect, we can just sh_run() the builtin so a nested sh_exec()
invocation will find and run it.
- If 'spawn' is not set (i.e. we must return), set errno to 0 and
return -2. See the change to sh_ntfork() below.
src/cmd/ksh93/sh/xec.c:
- sh_exec(): When searching for built-ins and the restricted option
isn't active, also search bltin_tree for names beginning with a
slash.
- sh_ntfork(): Only throw an error if the PID value returned is
exactly -1. This allows path_spawn() to return -2 after running a
built-in to tell sh_ntfork() to do the right things to restore
state.
src/cmd/ksh93/sh/parse.c: simple():
- When searching for built-ins at parse time, only exclude names
containing a slash if the restricted option is active. This
allows finding pointers to built-ins invoked by literal path like
/opt/ast/bin/cat, as long as that does not result from an
expansion. This is not actually necessary as sh_exec() will also
cover this case, but it is an optimisation.
src/lib/libcmd/getconf.c:
- Replace convoluted deferral to external command by a simple
invocation of the path to the native getconf command determined
at compile time (by src/lib/libast/comp/conf.sh). Based on:
https://github.com/ksh93/ksh/issues/138#issuecomment-816384871
If there is ever a system that has /opt/ast/bin/getconf as its
default native external 'getconf', then there would still be an
infinite recursion crash, but this seems extremely unlikely.
Resolves: https://github.com/ksh93/ksh/issues/138
2021-04-15 03:08:12 +00:00
|
|
|
2021-04-14:
|
|
|
|
|
|
|
|
|
|
- Path-bound built-ins (such as /opt/ast/bin/cat) can now be executed by
|
|
|
|
|
invoking the canonical path, so the following will now work as expected:
|
|
|
|
|
$ /opt/ast/bin/cat --version
|
|
|
|
|
version cat (AT&T Research) 2012-05-31
|
|
|
|
|
$ (PATH=/opt/ast/bin:$PATH; "$(whence -p cat)" --version)
|
|
|
|
|
version cat (AT&T Research) 2012-05-31
|
|
|
|
|
Non-canonical paths such as /opt/ast/./bin/cat will not find the built-ins.
|
|
|
|
|
|
|
|
|
|
- Path-bound built-ins will now also be found on a PATH set locally using an
|
|
|
|
|
assignment preceding the command, so the following will now work as expected:
|
|
|
|
|
$ PATH=/opt/ast/bin cat --version
|
|
|
|
|
version cat (AT&T Research) 2012-05-31
|
|
|
|
|
|
Fix the exit status returned when a command isn't executable (#273)
Previous discussion: https://github.com/att/ast/issues/485
If ksh attempts to execute a non-executable command found in the
PATH, in some instances the error message and return status are
incorrect. In the example below, ksh returns with exit status 126
when using the -c execve(2) optimization or when using fork(2) in
an interactive shell. However, using posix_spawn(3) causes the exit
status to change:
$ echo 'print cannot execute' > /tmp/x
# Runs command with spawnveg (i.e., posix_spawn or vfork)
$ ksh -c 'PATH=/tmp; x; echo $?'
ksh: x: not found
127
# Runs command with execve
$ ksh -c 'PATH=/tmp; x'; echo $?
ksh: x: cannot execute [Permission denied]
126
# Runs command with fork
$ ksh -ic 'PATH=/tmp; x; echo $?'
ksh: x: cannot execute [Permission denied]
126
Since 'x' is in the PATH but can't be executed, the correct exit
status is 126, not 127. It's worth noting this bug doesn't cause
the regression tests to fail with ksh93u+m, but it does cause one
test to fail when run under dtksh:
path.sh[706]: Long nonexistent command name: got status 126, ''
This commit backports various fixes for this bug from ksh2020, with
additional fixes applied (since there were still some additional
issues the ksh2020 patch didn't fix). The lacking regression test
for exit status 126 in path.sh has been rewritten to test for more
scenarios where ksh failed to return the correct error message
and/or exit status. I can also confirm with this patch applied the
path.sh regression tests now pass when run under dtksh.
src/cmd/ksh93/sh/path.c:
- Add a comment to path_absolute() describing 'oldpp' is the
current pointer in the while loop and 'pp' is the next pointer.
Backported from:
https://github.com/att/ast/commit/a6cad450
- The patch from ksh2020 didn't fix this bug in the SHOPT_SPAWN
code (because ksh2020 prefers fork(2)), so issues with the exit
status could still occur when using spawnveg. To fix this, always
set 'noexec' to the value of errno if can_execute fails. Before
this fix, errno was discarded if 'pp' was a null pointer and
can_execute failed.
- If a command couldn't be executed and the error wasn't ENOENT,
save errno in a 'not_executable' variable. If an executable
command couldn't be found in the PATH, exit with status 126 and
set errno to the saved value. This was based on a ksh2020 bugfix,
but it has been reworked a little bit to fix a bug that caused a
mismatch between the error message shown and errno. Example with
a non-executable file in PATH:
$ nonexec
ksh2020: nonexec: cannot execute [No such file or directory]
The ksh2020 patch: <https://github.com/att/ast/pull/493>
- Backport a ksh2020 bugfix for directories in the PATH when
running one of the added regression tests on OpenBSD:
https://github.com/att/ast/pull/767
src/cmd/ksh93/data/msg.c,
src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/sh/{path,xec}.c:
- If a command name is too long (ENAMETOOLONG), then it wasn't
found in the PATH. For that case return exit status 127, like
for ENOENT.
src/cmd/ksh93/tests/path.sh:
- Replace the old test with a new set of more extensive tests.
These tests check the error message and exit status when ksh
attempts to run a command using any of the following:
- execve(2), used with the last command run with -c (*A tests).
- posix_spawn(3)/vfork(2), used in noninteractive scripts (*B tests).
- fork(2), used in interactive shells with job control (*C tests).
- command -x (*D tests).
- exec(1) (*E tests).
- Add a regression test from ksh2020 for attempting to execute a
directory:
https://github.com/att/ast/pull/758
src/lib/libast/include/ast.h,
src/lib/libast/include/wait.h:
- Avoid bitshifts in macros for static error codes. The return
values of command not found and exec related errors are static
values and should not require any macro magic for calculation.
Backported from: https://github.com/att/ast/commit/c073b102
- Simplify EXIT_* and W* macros to use 8 bits.
2021-04-15 02:37:57 +00:00
|
|
|
2021-04-13:
|
|
|
|
|
|
|
|
|
|
- Fixed a few bugs that could cause ksh to show the wrong error message and/or
|
|
|
|
|
return the wrong exit status if a command couldn't be executed. In
|
|
|
|
|
scenarios where the command was found in the PATH but it was not executable,
|
|
|
|
|
ksh now returns with exit status 126. Otherwise, ksh will return with exit
|
|
|
|
|
status 127 (such as if the command isn't found or if the command name is
|
|
|
|
|
too long).
|
|
|
|
|
|
2021-04-13 02:15:34 +00:00
|
|
|
2021-04-12:
|
|
|
|
|
|
|
|
|
|
- Corrected a memory fault when an attempt was made to unset the default
|
|
|
|
|
nameref KSH_VERSION from the shell environment prior to any other name
|
2021-04-23 21:02:30 +00:00
|
|
|
reference variable creation or modification.
|
2021-04-13 02:15:34 +00:00
|
|
|
|
2021-04-12 00:24:33 +00:00
|
|
|
2021-04-11:
|
|
|
|
|
|
|
|
|
|
- Fixed two related regressions introduced on 2020-06-16:
|
|
|
|
|
1. The += assignment failed to append the value of variables when used
|
|
|
|
|
in an invocation-local scope. The following should print '5', but
|
|
|
|
|
the regression resulted in '3' being printed instead:
|
|
|
|
|
$ integer foo=2; foo+=3 command eval 'echo $foo'
|
|
|
|
|
3
|
|
|
|
|
2. Any += assignment used in an invocation-local scope could modify
|
|
|
|
|
readonly variables.
|
|
|
|
|
|
Reset arithmetic recursion level on all errors (re: 264ba48b)
The recursion level for arithmetic expressions is kept track of in
a static 'level' variable in streval.c. It is reset when arithmetic
expressions throw an error.
But an error for an arithmetic expression may also occur elsewhere
-- at least in one case: when an arithmetic expression attempts to
change a read-only variable. In that case, the recursion level is
never reset because that code does not have access to the static
'level' variable.
If many such conditions occur (as in the new readonly.sh regression
tests), an arithmetic command like 'i++' may eventually fail with a
'recursion too deep' error.
To mitigate the problem, MAXLEVEL in streval.c was changed from 9
to 1024 in 264ba48b (as in the ksh 93v- beta). This commit leaves
that increase, but adds a proper fix.
src/cmd/ksh93/include/defs.h:
- Add global sh.arithrecursion (a.k.a. shp->arithrecursion)
variable to keep track of the arithmetic recursion level,
replacing the static 'level' variable in streval.c.
src/cmd/ksh93/sh/xec.c: sh_exec():
- Reset sh.arithrecursion before starting a new simple command
(TCOM), a new subshell with parentheses (TPAR), a new pipe
(TFIL), or a new [[ ... ]] command (TTST). These are the same
places where 'echeck' is set to 1 for --errexit and ERR trap
checks, so it should cover everything.
src/cmd/ksh93/sh/streval.c:
- Change all uses of 'level' to sh.arithrecursion.
- _seterror, aritherror(): No longer bother to reset the level
to zero here; xec.c should have this covered for all cases now.
src/cmd/ksh93/tests/arith.sh:
- Add tests for main shell and subshell.
2021-04-10 23:08:26 +00:00
|
|
|
2021-04-10:
|
|
|
|
|
|
|
|
|
|
- Fixed: the internal count of the recursion level for arithmetic expressions
|
|
|
|
|
was not reset when certain errors occurred in a virtual subshell. This could
|
|
|
|
|
cause an erroneous "recursion to deep" error when a loop executed many
|
|
|
|
|
subshells containing arithmetic expressions with errors, e.g. for testing.
|
|
|
|
|
|
2021-04-09 22:26:07 +00:00
|
|
|
2021-04-09:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused ksh to enable -c during the shell's initialization
|
|
|
|
|
if the only argument passed was --posix.
|
|
|
|
|
|
|
|
|
|
- Fixed a related bug that caused 'set --posix' to leave the braceexpand and
|
|
|
|
|
letoctal shell options unchanged.
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused 'set --default' to unset the restricted option
|
|
|
|
|
in restricted shells.
|
|
|
|
|
|
2021-04-08 13:39:30 +00:00
|
|
|
2021-04-08:
|
|
|
|
|
|
|
|
|
|
- Path-bound builtins will now be used by restricted shells if /opt/ast/bin
|
|
|
|
|
is in the $PATH upon invoking the shell or before setting it to restricted.
|
|
|
|
|
|
2021-04-09 02:49:48 +00:00
|
|
|
- Fixed a bug that caused "printf '%T\n' now" to ignore $LC_ALL and $LC_TIME
|
|
|
|
|
if the current locale was previously set, unset then set again.
|
|
|
|
|
|
2021-04-08 12:06:22 +00:00
|
|
|
2021-04-07:
|
|
|
|
|
|
|
|
|
|
- The $LC_TIME variable is now recognized by ksh and if set to an invalid
|
|
|
|
|
locale will show an error.
|
|
|
|
|
|
2021-04-08 12:24:17 +00:00
|
|
|
- Fixed BUG_CSUBSTDO: If standard output is closed before running a command
|
|
|
|
|
substitution, redirecting any other file descriptor no longer closes standard
|
|
|
|
|
output inside of the command substitution.
|
|
|
|
|
|
Fix unsetting array element after expanding array subscript range
Simple reproducer:
set -A arr a b c d; : ${arr[1..2]}; unset arr[1]; echo ${arr[@]}
Output:
a
Expected output:
a c d
The ${arr[1..2]} expansion broke the subsequent 'unset' command
so that it unsets element 1 and on, instead of only 1.
This regression was introduced in nv_endsubscript() on 2009-07-31:
https://github.com/ksh93/ast-open-history/commit/c47896b4/src/cmd/ksh93/sh/array.c
That change checks for the ARRAY_SCAN attribute which enables
processing ranges of array elements instead of single array
elements, and restores it after. That restore is evidently not
correct as it causes the subsequent unset command to malfunction.
If we revert that change, the bug disappears and the regression
tests show no failures. However, I don't know what this was meant
to accomplish and what other bug we might introduce by reverting
this. However, no corresponding regression test was added along
with the 2009-07-31 change, nor is there any corresponding message
in the changelog. So this looks to be one of those mystery changes
that we'll never know the reason for.
Since we currently have proof that this change causes breakage and
no evidence that it fixes anything, I'll go ahead and revert it
(and add a regression test, of course). If that causes another
regression, hopefully someone will find it at some point.
src/cmd/ksh93/sh/array.c: nv_endsubscript():
- Revert the 2009-07-31 change that saves/restores the ARRAY_SCAN
attribute.
- Keep the 'ap' pointer as it is now used by newer code. Move the
declaration up to the beginning of the block, as is customary.
src/cmd/ksh93/sh/init.c:
- Cosmetic change: remove an unused array_scan() macro that I found
when grepping the code for ARRAY_SCAN. The macro was introduced
in version 2001-06-01 but the code that used it was replaced in
version 2001-07-04, without removing the macro itself.
Resolves: https://github.com/ksh93/ksh/issues/254
2021-04-05 21:11:23 +00:00
|
|
|
2021-04-05:
|
|
|
|
|
|
|
|
|
|
- Fixed a regression, introduced in ksh 93t+ 2009-07-31, that caused a command
|
|
|
|
|
like 'unset arr[3]' to unset not just element 3 of the array but all elements
|
|
|
|
|
starting from 3, if a range expansion like ${arr[5..10]} was previously used.
|
|
|
|
|
|
Backport bugfixes for arrays of 'enum' types from ksh 93v- beta
These fixes are applied rather blindly as no one has yet managed to
understand the almost entirely uncommented arrays and variables
handling code (arrays.c, name.c, nvdisc.c, nvtree.c, nvtype.c).
Hopefully we'll figure all that out at some point. In the meantime
these backported fixes appear to work fine, and these bugs impact
the usability of 'enum', so I'm just going to have to violate my
own policy and backport these fixes without understanding them.
Thanks to @JohnoKing for putting in a lot of work tracing these.
Further discussion at: https://github.com/ksh93/ksh/issues/87
src/cmd/ksh93/sh/array.c:
- nv_arraysettype():
* Further simplify the function. After my initial simplification
of it (re: 5491fe97), I don't believe there's actually a need
to save a duplicate copy of the value. Use the pointer returned
by nv_getval() directly to restore the value.
* Cope with a null value (nv_getval() returning a NULL pointer).
This is needed for compatibility with the backported fix in
nvtype.c (below).
- array_putval(): If the array's value pointer (up->cp) is a
pointer to the empty string, it is set to NULL before calling
nv_putv() to prevent an empty string from being deleted. Backport
a fix from 93v- that restores the pointer to the empty string if
the NV_NOFREE attribute is set. Removing it somehow causes these
regressions:
enum.sh[86]: ${array[@]} doesn't yield all values for
associative enum arrays (expected 'green blue blue red
yellow green red orange'; got 'green blue blue yellow
green orange')
enum.sh[94]: unsetting associative enum array does not work
(got 'Color_t -A Colors=([foo]=red [rood]=red)')
enum.sh[116]: assigning first enum element to indexed array
failed (expected 'red red'; got 'BUG BUG')
- nv_associative(): Do not increase the 'nelem' (number of
elements) value of the array's 'header' struct if the array is
associative and of an enum type. The original 93v- fix only
checked for the NV_INTEGER attribute, but backporting that caused
several regressions. Using a debug output command I've determined
that the exact value of 'type' is somehow consistently set to
0x26 if the array is associative and of an enum type, which is
NV_INTEGER | NV_LTOU | NV_RJUST as defined in include/nval.h. I
cannot find where/how that value is determined. In any case this
fix, based on but more specific than the 93v- one, appears to
work fine. Removing it somehow causes this regression:
enum.sh[94]: unsetting associative enum array does not work
(got 'Color_t -A Colors=()')
src/cmd/ksh93/sh/nvtype.c: nv_settype():
- Another fix backported from 93v-. If the variable is an array,
also set the type of element 0 of that array using a call to
nv_arraysettype(). The value may be null. Removing this somehow
causes this regression:
enum.sh[94]: unsetting associative enum array does not work
(got 'Color_t -A Colors=()')
src/cmd/ksh93/tests/enum.sh:
- Add tests for all the bugs fixed here, plus some hypothetical
bugs (e.g., do the same tests for indexed enum type arrays as for
associative enum type arrays, even though indexed enum type
arrays didn't have all the same problems).
Co-authored-by: Johnothan King <johnothanking@protonmail.com>
Resolves: https://github.com/ksh93/ksh/issues/87
2021-04-06 04:45:46 +00:00
|
|
|
- Several fixes for arrays of a type created by 'enum' were backported from ksh
|
|
|
|
|
93v-, further to the two enum array fixes already applied on 2021-02-01:
|
|
|
|
|
1. The array[@]} expansion was fixed for associative arrays of an enum type.
|
|
|
|
|
2. Assignments now work correctly for all enum values for both indexed and
|
|
|
|
|
associative arrays.
|
|
|
|
|
3. 'unset' will now completely unset an associative array of an enum type.
|
|
|
|
|
|
2021-04-05 05:43:19 +00:00
|
|
|
2021-04-04:
|
|
|
|
|
|
Fix bell character handling when redrawing command line (#250)
To set a window title in bash and zsh, the $PS1 prompt can be set
with the title placed between $'\E]0;' and $'\a':
set -o emacs # Or vi mode
typeset -A fmt=(
[start_title]=$'\E]0;'
[end_title]=$'\a'
)
PS1="${fmt[start_title]}$(hostname): $(uname)${fmt[end_title]}\$ "
This also works in ksh unless the shell receives SIGWINCH. With a
$PS1 that sets a window title, the prompt breaks until two
interrupts are received. This is caused by ed_setup() skipping
$'\a' (the bell character) when setting up the e_prompt buffer
which is an edited version of the final line of the PS1 prompt for
use when redrawing the command line.
One fix would be to avoid cutting out the bell character. But if
the prompt contains a bell, we only want the terminal to beep when
a new prompt is printed, and not upon refreshing the command line,
e.g. when receiving SIGWINCH or pressing Ctrl+L.
To avoid the problem, this commit adds code that cuts out sequences
of the form ESC ] <number> ; <text> BELL from the prompt redraw
buffer altogether. They are not needed there because these
sequences will already have taken effect when the full prompt was
printed by io_prompt().
This commit also adds a tweak that should improve the recognition
of other escape sequences to count their length.
src/cmd/ksh93/edit/edit.c: ed_setup():
- When preparing the e_prompt buffer, cut out dtterm/xterm
Operating System Commands that set window/icon title, etc.
See: https://invisible-island.net/xterm/ctlseqs/ctlseqs.html
- When counting the length of escape sequences in that part of PS1,
try to recognize some more types of sequences. These changes are
part of a ksh2020 patch: https://github.com/att/ast/issues/399
src/cmd/ksh93/sh.1:
- Document that any '!' in escape sequences in the PS1 prompt needs
to be changed to '!!'. To avoid breaking compatibility, this
requirement is documented instead of backporting the changes to
io_prompt() from https://github.com/att/ast/issues/399 which try
to remove that requirement for specific escape sequences.
Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-04-05 07:06:53 +00:00
|
|
|
- A bug was fixed that caused a broken prompt display upon redrawing the
|
|
|
|
|
command line if the last line of the prompt includes an xterm escape
|
|
|
|
|
sequence that is terminated by $'\a' (the bell character).
|
|
|
|
|
|
2021-04-05 05:43:19 +00:00
|
|
|
- Harden readonly variables. Readonly variables or arrays no longer allow
|
|
|
|
|
attribute changes which would otherwise allow their value to be altered.
|
|
|
|
|
Expanded support for readonly variables within multidimensional arrays.
|
|
|
|
|
|
2021-04-04 21:18:43 +00:00
|
|
|
2021-04-03:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused the uname builtin's -d option to change the output
|
|
|
|
|
of the -o option.
|
|
|
|
|
|
|
|
|
|
- Fixed a possible crash that could occur when showing the domain name
|
|
|
|
|
with the uname builtin's -d option.
|
|
|
|
|
|
2021-04-02 00:19:19 +00:00
|
|
|
2021-03-31:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused 'cd -' to ignore the current value of $OLDPWD
|
|
|
|
|
when it's set to a different directory in a new scope.
|
|
|
|
|
|
|
|
|
|
- Fixed a related bug that caused ksh to use the wrong value for $PWD
|
|
|
|
|
when in a new scope.
|
|
|
|
|
|
2021-03-30 10:25:20 +00:00
|
|
|
2021-03-29:
|
|
|
|
|
|
|
|
|
|
- Fixed an intermittent crash that could occur in vi mode when using the 'b'
|
|
|
|
|
or 'B' commands to go back one word.
|
|
|
|
|
|
2021-03-27 21:51:16 +00:00
|
|
|
2021-03-27:
|
|
|
|
|
|
|
|
|
|
- The 'test' builtin will now show an error message when given the invalid ']]'
|
|
|
|
|
or '=~' operators; it also properly returns with exit status 2 now (instead
|
|
|
|
|
of exit status 1). If the invalid operator is supported by [[ ... ]] (such
|
|
|
|
|
as '=~'), test will now suggest the usage of [[ ... ]] instead.
|
|
|
|
|
|
Add --globcasedetect shell option for globbing and completion
One of the best-kept secrets of libast/ksh93 is that the code
includes support for case-insensitive file name generation (a.k.a.
pathname expansion, a.k.a. globbing) as well as case-insensitive
file name completion on interactive shells, depending on whether
the file system is case-insensitive or not. This is transparently
determined for each directory, so a path pattern that spans
multiple file systems can be part case-sensitive and part case-
insensitive. In more precise terms, each slash-separated path name
component pattern P is treated as ~(i:P) if its parent directory
exists on a case-insensitive file system. I recently discovered
this while dealing with <https://github.com/ksh93/ksh/issues/223>.
However, that support is dead code on almost all current systems.
It depends on pathconf(2) having a _PC_PATH_ATTRIBUTES selector.
The 'c' attribute is supposedly returned if the given directory is
on a case insensitive file system. There are other attributes as
well (at least 'l', see src/lib/libcmd/rm.c). However, I have been
unable to find any system, current or otherwise, that has
_PC_PATH_ATTRIBUTES. Google and mailing list searches yield no
relevant results at all. If anyone knows of such a system, please
add a comment to this commit on GitHub, or email me.
An exception is Cygwin/Windows, on which the "c" attribute was
simply hardcoded, so globbing/completion is always case-
insensitive. As of Windows 10, that is wrong, as it added the
possibility to mount case-sensitive file systems.
On the other hand, this was never activated on the Mac, even
though macOS has always used a case-insensitive file like Windows.
But, being UNIX, it can also mount case-sensitive file systems.
Finally, Linux added the possibility to create individual case-
insensitive ext4 directories fairly recently, in version 5.2.
https://www.collabora.com/news-and-blog/blog/2020/08/27/using-the-linux-kernel-case-insensitive-feature-in-ext4/
So, since this functionality latently exists in the code base, and
three popular OSs now have relevant file system support, we might
as well make it usable on those systems. It's a nice idea, as it
intuitively makes sense for globbing and completion behaviour to
auto-adapt to file system case insensitivity on a per-directory
basis. No other shell does this, so it's a nice selling point, too.
However, the way it is coded, this is activated unconditionally on
supported systems. That is not a good idea. It will surprise users.
Since globbing is used with commands like 'rm', we do not want
surprises. So this commit makes it conditional upon a new shell
option called 'globcasedetect'. This option is only compiled into
ksh on systems where we can actually detect FS case insensitivity.
To implement this, libast needs some public API additions first.
*** libast changes ***
src/lib/libast/features/lib:
- Add probes for the linux/fs.h and sys/ioctl.h headers.
Linux needs these to use ioctl(2) in pathicase(3) (see below).
src/lib/libast/path/pathicase.c,
src/lib/libast/include/ast.h,
src/lib/libast/man/path.3,
src/lib/libast/Mamfile:
- Add new pathicase(3) public API function. This uses whatever
OS-specific method it can detect at compile time to determine if
a particular path is on a case-insensitive file system. If no
method is available, it only sets errno to ENOSYS and returns -1.
Currently known to work on: macOS, Cygwin, Linux 5.2+, QNX 7.0+.
- On systems (if any) that have the mysterious _PC_PATH_ATTRIBUTES
selector for pathconf(2), call astconf(3) and check for the 'c'
attribute to determine case insensitivity. This should preserve
compatibility with any such system.
src/lib/libast/port/astconf.c:
- dynamic[]: As case-insensitive globbing is now optional on all
systems, do not set the 'c' attribute by default on _WINIX
(Cygwin/Windows) systems.
- format(): On systems that do not have _PC_PATH_ATTRIBUTES, call
pathicase(3) to determine the value for the "c" (case
insensitive) attribute only. This is for compatibility as it is
more efficient to call pathicase(3) directly.
src/lib/libast/misc/glob.c,
src/lib/libast/include/glob.h:
- Add new GLOB_DCASE public API flag to glob(3). This is like
GLOB_ICASE (case-insensitive matching) except it only makes the
match case-insensitive if the file system for the current
pathname component is determined to be case-insensitive.
- gl_attr(): For efficiency, call pathicase(3) directly instead of
via astconf(3).
- glob_dir(): Only call gl_attr() to determine file system case
insensitivity if the GLOB_DCASE flag was passed. This makes case
insensitive globbing optional on all systems.
- glob(): The options bitmask needs to be widened to fit the new
GLOB_DCASE option. Define this centrally in a new GLOB_FLAGMASK
macro so it is easy to change it along with GLOB_MAGIC (which
uses the remaining bits for a sanity check bit pattern).
src/lib/libast/path/pathexists.c:
- For efficiency, call pathicase(3) directly instead of via
astconf(3).
*** ksh changes ***
src/cmd/ksh93/features/options,
src/cmd/ksh93/SHOPT.sh:
- Add new SHOPT_GLOBCASEDET compile-time option. Set it to probe
(empty) by default so that the shell option is compiled in on
supported systems only, which is determined by new iffe feature
test that checks if pathicase(3) returns an ENOSYS error.
src/cmd/ksh93/data/options.c,
src/cmd/ksh93/include/shell.h:
- Add -o globcasedetect shell option if compiling with
SHOPT_GLOBCASEDET.
src/cmd/ksh93/sh/expand.c: path_expand():
- Pass the new GLOB_DCASE flag to glob(3) if the
globcasedetect/SH_GLOBCASEDET shell option is set.
src/cmd/ksh93/edit/completion.c:
- While file listing/completion is based on globbing and
automatically becomes case-insensitive when globbing does, it
needs some additional handling to make a string comparison
case-insensitive in corresponding cases. Otherwise, partial
completions may be deleted from the command line upon pressing
tab. This code was already in ksh 93u+ and just needs to be
made conditional upon SHOPT_GLOBCASEDET and globcasedetect.
- For efficiency, call pathicase(3) directly instead of via
astconf(3).
src/cmd/ksh93/sh.1:
- Document the new globcasedetect shell option.
2021-03-22 15:01:03 +00:00
|
|
|
2021-03-22:
|
|
|
|
|
|
|
|
|
|
- A new --globcasedetect shell option is added to ksh on OSs where we can check
|
|
|
|
|
for a case-insensitive file system (currently macOS, Windows/Cygwin, Linux
|
Fix more compiler warnings, typos and other minor issues (#260)
Many of these changes are minor typo fixes. The other changes
(which are mostly compiler warning fixes) are:
NEWS:
- The --globcasedetect shell option works on older Linux kernels
when used with FAT32/VFAT file systems, so remove the note about
it only working with 5.2+ kernels.
src/cmd/ksh93/COMPATIBILITY:
- Update the documentation on function scoping with an addition
from ksh93v- (this does apply to ksh93u+).
src/cmd/ksh93/edit/emacs.c:
- Check for '_AST_ksh_release', not 'AST_ksh_release'.
src/cmd/INIT/mamake.c,
src/cmd/INIT/ratz.c,
src/cmd/INIT/release.c,
src/cmd/builtin/pty.c:
- Add more uses of UNREACHABLE() and noreturn, this time for the
build system and pty.
src/cmd/builtin/pty.c,
src/cmd/builtin/array.c,
src/cmd/ksh93/sh/name.c,
src/cmd/ksh93/sh/nvtype.c,
src/cmd/ksh93/sh/suid_exec.c:
- Fix six -Wunused-variable warnings (the name.c nv_arrayptr()
fixes are also in ksh93v-).
- Remove the unused 'tableval' function to fix a -Wunused-function
warning.
src/cmd/ksh93/sh/lex.c:
- Remove unused 'SHOPT_DOS' code, which isn't enabled anywhere.
https://github.com/att/ast/issues/272#issuecomment-354363112
src/cmd/ksh93/bltins/misc.c,
src/cmd/ksh93/bltins/trap.c,
src/cmd/ksh93/bltins/typeset.c:
- Add dictionary generator function declarations for former
aliases that are now builtins (re: 1fbbeaa1, ef1621c1, 3ba4900e).
- For consistency with the rest of the codebase, use '(void)'
instead of '()' for print_cpu_times.
src/cmd/ksh93/sh/init.c,
src/lib/libast/path/pathshell.c:
- Move the otherwise unused EXE macro to pathshell() and only
search for 'sh.exe' on Windows.
src/cmd/ksh93/sh/xec.c,
src/lib/libast/include/ast.h:
- Add an empty definition for inline when compiling with C89.
This allows the timeval_to_double() function to be inlined.
src/cmd/ksh93/include/shlex.h:
- Remove the unused 'PIPESYM2' macro.
src/cmd/ksh93/tests/pty.sh:
- Add '# err_exit #' to count the regression test added in
commit 113a9392.
src/lib/libast/disc/sfdcdio.c:
- Move diordwr, dioread, diowrite and dioexcept behind
'#ifdef F_DIOINFO' to fix one -Wunused-variable warning and
multiple -Wunused-function warnings (sfdcdio() only uses these
functions when F_DIOINFO is defined).
src/lib/libast/string/fmtdev.c:
- Fix two -Wimplicit-function-declaration warnings on Linux by
including sys/sysmacros.h in fmtdev().
2021-04-08 18:58:07 +00:00
|
|
|
and QNX 7.0+). When this option is turned on, file name generation
|
Add --globcasedetect shell option for globbing and completion
One of the best-kept secrets of libast/ksh93 is that the code
includes support for case-insensitive file name generation (a.k.a.
pathname expansion, a.k.a. globbing) as well as case-insensitive
file name completion on interactive shells, depending on whether
the file system is case-insensitive or not. This is transparently
determined for each directory, so a path pattern that spans
multiple file systems can be part case-sensitive and part case-
insensitive. In more precise terms, each slash-separated path name
component pattern P is treated as ~(i:P) if its parent directory
exists on a case-insensitive file system. I recently discovered
this while dealing with <https://github.com/ksh93/ksh/issues/223>.
However, that support is dead code on almost all current systems.
It depends on pathconf(2) having a _PC_PATH_ATTRIBUTES selector.
The 'c' attribute is supposedly returned if the given directory is
on a case insensitive file system. There are other attributes as
well (at least 'l', see src/lib/libcmd/rm.c). However, I have been
unable to find any system, current or otherwise, that has
_PC_PATH_ATTRIBUTES. Google and mailing list searches yield no
relevant results at all. If anyone knows of such a system, please
add a comment to this commit on GitHub, or email me.
An exception is Cygwin/Windows, on which the "c" attribute was
simply hardcoded, so globbing/completion is always case-
insensitive. As of Windows 10, that is wrong, as it added the
possibility to mount case-sensitive file systems.
On the other hand, this was never activated on the Mac, even
though macOS has always used a case-insensitive file like Windows.
But, being UNIX, it can also mount case-sensitive file systems.
Finally, Linux added the possibility to create individual case-
insensitive ext4 directories fairly recently, in version 5.2.
https://www.collabora.com/news-and-blog/blog/2020/08/27/using-the-linux-kernel-case-insensitive-feature-in-ext4/
So, since this functionality latently exists in the code base, and
three popular OSs now have relevant file system support, we might
as well make it usable on those systems. It's a nice idea, as it
intuitively makes sense for globbing and completion behaviour to
auto-adapt to file system case insensitivity on a per-directory
basis. No other shell does this, so it's a nice selling point, too.
However, the way it is coded, this is activated unconditionally on
supported systems. That is not a good idea. It will surprise users.
Since globbing is used with commands like 'rm', we do not want
surprises. So this commit makes it conditional upon a new shell
option called 'globcasedetect'. This option is only compiled into
ksh on systems where we can actually detect FS case insensitivity.
To implement this, libast needs some public API additions first.
*** libast changes ***
src/lib/libast/features/lib:
- Add probes for the linux/fs.h and sys/ioctl.h headers.
Linux needs these to use ioctl(2) in pathicase(3) (see below).
src/lib/libast/path/pathicase.c,
src/lib/libast/include/ast.h,
src/lib/libast/man/path.3,
src/lib/libast/Mamfile:
- Add new pathicase(3) public API function. This uses whatever
OS-specific method it can detect at compile time to determine if
a particular path is on a case-insensitive file system. If no
method is available, it only sets errno to ENOSYS and returns -1.
Currently known to work on: macOS, Cygwin, Linux 5.2+, QNX 7.0+.
- On systems (if any) that have the mysterious _PC_PATH_ATTRIBUTES
selector for pathconf(2), call astconf(3) and check for the 'c'
attribute to determine case insensitivity. This should preserve
compatibility with any such system.
src/lib/libast/port/astconf.c:
- dynamic[]: As case-insensitive globbing is now optional on all
systems, do not set the 'c' attribute by default on _WINIX
(Cygwin/Windows) systems.
- format(): On systems that do not have _PC_PATH_ATTRIBUTES, call
pathicase(3) to determine the value for the "c" (case
insensitive) attribute only. This is for compatibility as it is
more efficient to call pathicase(3) directly.
src/lib/libast/misc/glob.c,
src/lib/libast/include/glob.h:
- Add new GLOB_DCASE public API flag to glob(3). This is like
GLOB_ICASE (case-insensitive matching) except it only makes the
match case-insensitive if the file system for the current
pathname component is determined to be case-insensitive.
- gl_attr(): For efficiency, call pathicase(3) directly instead of
via astconf(3).
- glob_dir(): Only call gl_attr() to determine file system case
insensitivity if the GLOB_DCASE flag was passed. This makes case
insensitive globbing optional on all systems.
- glob(): The options bitmask needs to be widened to fit the new
GLOB_DCASE option. Define this centrally in a new GLOB_FLAGMASK
macro so it is easy to change it along with GLOB_MAGIC (which
uses the remaining bits for a sanity check bit pattern).
src/lib/libast/path/pathexists.c:
- For efficiency, call pathicase(3) directly instead of via
astconf(3).
*** ksh changes ***
src/cmd/ksh93/features/options,
src/cmd/ksh93/SHOPT.sh:
- Add new SHOPT_GLOBCASEDET compile-time option. Set it to probe
(empty) by default so that the shell option is compiled in on
supported systems only, which is determined by new iffe feature
test that checks if pathicase(3) returns an ENOSYS error.
src/cmd/ksh93/data/options.c,
src/cmd/ksh93/include/shell.h:
- Add -o globcasedetect shell option if compiling with
SHOPT_GLOBCASEDET.
src/cmd/ksh93/sh/expand.c: path_expand():
- Pass the new GLOB_DCASE flag to glob(3) if the
globcasedetect/SH_GLOBCASEDET shell option is set.
src/cmd/ksh93/edit/completion.c:
- While file listing/completion is based on globbing and
automatically becomes case-insensitive when globbing does, it
needs some additional handling to make a string comparison
case-insensitive in corresponding cases. Otherwise, partial
completions may be deleted from the command line upon pressing
tab. This code was already in ksh 93u+ and just needs to be
made conditional upon SHOPT_GLOBCASEDET and globcasedetect.
- For efficiency, call pathicase(3) directly instead of via
astconf(3).
src/cmd/ksh93/sh.1:
- Document the new globcasedetect shell option.
2021-03-22 15:01:03 +00:00
|
|
|
(globbing), as well as file name tab completion on interactive shells,
|
|
|
|
|
automatically become case-insensitive on file systems where the difference
|
|
|
|
|
between upper- and lowercase is ignored for file names. This is transparently
|
|
|
|
|
determined for each directory, so a path pattern that spans multiple file
|
|
|
|
|
systems can be part case-sensitive and part case-insensitive.
|
|
|
|
|
The option is not compiled into ksh on systems where we do not know of a
|
|
|
|
|
method to check for file system case insensitivity. The shell option can be
|
|
|
|
|
force-compiled by setting SHOPT_GLOBCASEDET to 1 in src/cmd/ksh93/SHOPT.sh,
|
|
|
|
|
but it won't have any effect on non-supported systems, so this is not
|
|
|
|
|
recommended. It can be removed from ksh by setting SHOPT_GLOBCASEDET to 0.
|
|
|
|
|
|
2021-03-17 22:34:45 +00:00
|
|
|
2021-03-17:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug with file name completion on the interactive shell in multibyte
|
|
|
|
|
locales. Upon encountering two filenames with multibyte characters starting
|
|
|
|
|
with the same byte, a partial multibyte character was autocompleted.
|
|
|
|
|
|
2021-03-16 16:13:13 +00:00
|
|
|
2021-03-16:
|
|
|
|
|
|
2021-12-09 06:31:37 +00:00
|
|
|
- Tilde expansion can now be extended or modified by defining a .sh.tilde.get
|
|
|
|
|
or .sh.tilde.set discipline function. This replaces a 2004 undocumented
|
|
|
|
|
attempt to add this functionality via a .sh.tilde built-in, which never
|
|
|
|
|
worked and crashed the shell. See the manual for details on the new method.
|
|
|
|
|
|
2021-03-16 16:13:13 +00:00
|
|
|
- Fixed a bug in interactive shells: if a variable used by the shell called
|
|
|
|
|
a discipline function (such as PS1.get() or COLUMNS.set()), the value of $?
|
|
|
|
|
was set to the exit status of the discipline function instead of the last
|
|
|
|
|
command run.
|
|
|
|
|
|
2021-03-15 21:49:02 +00:00
|
|
|
2021-03-15:
|
|
|
|
|
|
|
|
|
|
- If the HOME variable is unset, the bare tilde ~ now expands to the current
|
|
|
|
|
user's system-configured home directory instead of merely the username.
|
|
|
|
|
|
2021-03-16 10:19:00 +00:00
|
|
|
- Tighten up potential invalid typeset attribute combos when more than
|
|
|
|
|
one numeric type has been requested. In particular, -F and -i are no
|
|
|
|
|
longer masked over by previously given float types.
|
|
|
|
|
|
Re-enable SHOPT_DEVFD, fixing process substitution fd leaks (#218)
This commit fixes a long-standing bug (present since at least
ksh93r) that caused a file descriptor leak when passing a process
substitution to a function, or (if compiled with SHOPT_SPAWN) to a
nonexistent command.
The leaks only occurred when ksh was compiled with SHOPT_DEVFD; the
FIFO method was unaffected.
src/cmd/ksh93/sh/xec.c: sh_exec():
- When a process substitution is passed to a built-in, the
remaining file descriptor is closed with sh_iorestore. Do the
same thing when passing a process substitution to a function.
This is done by delaying the sh_iorestore() call to 'setexit:'
where both built-ins and functions terminate and set the exit
status ($?).
This means that call now will not be executed if a longjmp is
done, e.g. due to an error in a special built-in. However, there
is already another sh_iorestore() call in main.c, exfile(), line
418, that handles that scenario.
- sh_ntfork() can fail, so rather than assume it will succeed,
handle a failure by closing extra file descriptors with
sh_iorestore(). This fixes the leak on command not found with
SHOPT_SPAWN.
src/cmd/ksh93/include/defs.h:
- Since the file descriptor leaks are now fixed, remove the
workaround that forced ksh to use the FIFO method.
src/cmd/ksh93/SHOPT.sh:
- Add SHOPT_DEVFD as a configurable option (default: probe).
src/cmd/ksh93/tests/io.sh:
- Add a regression test for the 'not found' file descriptor leak.
- Add a test to ensure it keeps working with 'command'.
Fixes: https://github.com/ksh93/ksh/issues/67
2021-03-13 13:46:42 +00:00
|
|
|
2021-03-13:
|
|
|
|
|
|
|
|
|
|
- Fixed a file descriptor leak that occurred when ksh used /dev/fd for
|
|
|
|
|
process substitutions passed to functions.
|
|
|
|
|
|
|
|
|
|
- Fixed a separate file descriptor leak that happened when a process
|
|
|
|
|
substitution was passed to a nonexistent command.
|
|
|
|
|
|
Fix unused process substitutions hanging (#214)
On systems where ksh needs to use the older and less secure FIFO
method for process substitutions (which is currently all of them as
the more modern and solid /dev/fd method is still broken, see #67),
process substitutions could leave background processes hanging in
these two scenarios:
1. If the parent process exits without opening a pipe to the child
process forked by the process substitution. The fifo_check()
function in xec.c, which is periodically called to check if the
parent process still exists while waiting for it to open the
FIFO, verified the parent process's existence by checking if the
PPID had reverted to 1, the traditional PID of init. However,
POSIX specifies that the PPID can revert to any implementation-
defined system process in that case. So this breaks on certain
systems, causing unused process substitutions to hang around
forever as they never detect that the parent disappeared.
The fix is to save the current PID before forking and having the
child check if the PPID has changed from that saved PID.
2. If command invoked from the main shell is passed a process
substitution, but terminates without opening the pipe to the
process substitution. In that case, the parent process never
disappears in the first place, because the parent process is the
main shell. So the same infinite wait occurs in unused process
substitutions, even after correcting problem 1.
The fix is to remember all FIFOs created for any number of
process substitutions passed to a single command, and unlink any
remaining FIFOs as they represent unused command substitutions.
Unlinking them FIFOs causes sh_open() in the child to fail with
ENOENT on the next periodic check, which can easily be handled.
Fixing these problems causes the FIFO method to act identically to
the /dev/fd method, which is good for compatibility. Even when #67
is fixed this will still be important, as ksh also runs on systems
that do not have /dev/fd (such as AIX, HP-UX, and QNX), so will
fall back to using FIFOs.
--- Fix problem 1 ---
src/cmd/ksh93/sh/xec.c:
- Add new static fifo_save_ppid variable.
- sh_exec(): If a FIFO is defined, save the current PID in
fifo_save_ppid for the forked child to use.
- fifo_check(): Compare PPID against the saved value instead of 1.
--- Fix problem 2 ---
To keep things simple I'm abusing the name-value pair routines used
for variables for this purpose. The overhead is negligible. A more
elegant solution is possible but would involve adding more code.
src/cmd/ksh93/include/defs.h: _SH_PRIVATE:
- Define new sh.fifo_tree pointer to a new FIFO cleanup tree.
src/cmd/ksh93/sh/args.c: sh_argprocsubs():
- After launching a process substitution in the background,
add the FIFO to the cleanup list before freeing it.
src/cmd/ksh93/sh/xec.c:
- Add fifo_cleanup() that unlinks all FIFOs in the cleanup list and
clears/closes the list. They should only still exist if the
command never used them, however, just run 'unlink' and don't
check for existence first as that would only add overhead.
- sh_exec():
* Call fifo_cleanup() on finishing all simple commands (when
setting $?) or when a special builtin fails.
* When forking, clear/close the cleanup list; we do not want
children doing duplicate cleanup, particularly as this can
interfere when using multiple process substitutions in one
command.
* Process substitution handling:
> Change FIFO check frequency from 500ms to 50ms.
Note that each check sends a signal that interrupts open(2),
causing sh_open() to reinvoke it. This causes sh_open() to
fail with ENOENT on the next check when the FIFO no longer
exists, so we do not need to add an additional check for
existence to fifo_check(). Unused process substitutions now
linger for a maximum of 50ms.
> Do not issue an error message if errno == ENOENT.
- sh_funct(): Process substitutions can be passed to functions as
well, and we do not want commands within the function to clean up
the FIFOs for the process substitutions passed to it from the
outside. The problem is solved by simply saving fifo_tree in a
local variable, setting it to null before running the function,
and cleaning it up before restoring the parent one at the end.
Since sh_funct() is called recursively for multiple-level
function calls, this correctly gives each function a locally
scoped fifo_tree.
--- Tests ---
src/cmd/ksh93/tests/io.sh:
- Add tests covering the failing scenarios.
Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-03-12 11:43:23 +00:00
|
|
|
2021-03-11:
|
|
|
|
|
|
|
|
|
|
- Fixed an intermittent bug that caused process substitutions to infinitely
|
|
|
|
|
loop in Linux virtual machines that use systemd.
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused process substitutions to leave lingering processes
|
|
|
|
|
if the command invoking them never reads from them.
|
|
|
|
|
|
2021-03-09 04:35:45 +00:00
|
|
|
2021-03-09:
|
|
|
|
|
|
|
|
|
|
- The ${!foo@} and ${!foo*} expansions yield variable names beginning with foo,
|
|
|
|
|
but excluded 'foo' itself. The fix for this is now backported from 93v- beta.
|
|
|
|
|
|
Fix test -v for numeric types & set/unset state for short int
This commit fixes two interrelated problems.
1. The -v unary test/[/[[ operator is documented to test if a
variable is set. However, it always returns true for variable
names with a numeric attribute, even if the variable has not
been given a value. Reproducer:
$ ksh -o nounset -c 'typeset -i n; [[ -v n ]] && echo $n'
ksh: n: parameter not set
That is clearly wrong; 'echo $n' should never be reached and the
error should not occur, and does not occur on mksh or bash.
2. Fixing the previous problem revealed serious breakage in short
integer type variables that was being masked. After applying
that fix and then executing 'typeset -si var=0':
- The conditional assignment expansions ${var=123} and
${var:=123} assigned 123 to var, even though it was set to 0.
- The expansions ${var+s} and ${var:+n} incorrectly acted as if
the variable was unset and empty, respectively.
- '[[ -v var ]]' and 'test -v var' incorrectly returned false.
The problems were caused by a different storage method for short
ints. Their values were stored directly in the 'union Value'
member of the Namval_t struct, instead of allocated on the stack
and referred to by a pointer, as regular integers and all other
types do. This inherently broke nv_isnull() as this leaves no
way to distinguish between a zero value and no value at all.
(I'm also pretty sure it's undefined behaviour in C to check for
a null pointer at the address where a short int is stored.)
The fix is to store short ints like other variables and refer
to them by pointers. The NV_INT16P combined bit mask already
existed for this, but nv_putval() did not yet support it.
src/cmd/ksh93/bltins/test.c: test_unop():
- Fix problem 1. For -v, only check nv_isnull() and do not check
for the NV_INTEGER attribute (which, by the way, is also used
for float variables by combining it with other bits).
See also 5aba0c72 where we recently fixed nv_isnull() to
work properly for all variable types including short ints.
src/cmd/ksh93/sh/name.c: nv_putval():
- Fix problem 2, part 1. Add support for NV_INT16P. The code is
simply copied and adapted from the code for regular integers, a
few lines further on. The regular NV_SHORT code is kept as this
is still used for some special variables like ${.sh.level}.
src/cmd/ksh93/bltins/typeset.c: b_typeset():
- Fix problem 2, part 2. Use NV_INT16P instead of NV_SHORT.
src/cmd/ksh93/tests/attributes.sh:
- Add set/unset/empty/nonempty tests for all numeric types.
src/cmd/ksh93/tests/bracket.sh,
src/cmd/ksh93/tests/comvar.sh:
- Update a couple of existing tests.
- Add test for [[ -v var ]] and [[ -n ${var+s} ]] on unset
and empty variables with many attributes.
src/cmd/ksh93/COMPATIBILITY:
- Add a note detailing the change to test -v.
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/sh.1:
- Correct 'typeset -C' documentation. Variables declared as
compound are *not* initially unset, but initially have the empty
compound value. 'typeset' outputs them as:
typeset -C foo=()
and not:
typeset -C foo
and nv_isnull() is never true for them. This may or may not
technically be a bug. I don't think it's worth changing, but
it should at least be documented correctly.
2021-03-10 00:38:00 +00:00
|
|
|
- test -v var, [ -v var ], and [[ -v var ]] did not correctly test if a
|
|
|
|
|
variable is set or unset after it has been given a numeric attribute with
|
|
|
|
|
'typeset' but not yet assigned a value. This has been fixed so that
|
|
|
|
|
[[ -v var ]] is now equivalent to [[ -n ${var+set} ]] as documented.
|
|
|
|
|
|
Fixes for -G/--globstar (re: 5312a59d)
The fix for '.' and '..' in regular globbing broke '.' and '..' in
globstar. No globstar pattern that contains '.' or '..' as any
pathname component still matched. This commit fixes that.
This commit also makes symlink/** mostly work, which it never has
done in any ksh93 version. It is correct and expected that symlinks
found by patterns are not resolved, but symlinks were not resolved
even when specified as explicit non-pattern pathname components.
For example, /tmp/** breaks if /tmp is a symlink (e.g. on macOS),
which looks like a bug.
src/lib/libast/include/glob.h,
src/lib/libast/misc/glob.c: glob_dir():
- Make symlink/** work. we can check if the string pointed to by
pat is exactly equal to *. If so, we are doing regular globbing
for that particular pathname element, and it's okay to resolve
symlinks. If not (if it's **), we're doing globstar and we should
not be matching symlinks.
- Let's also introduce proper identification of symlinks (GLOB_SYM)
and not lump them in with other special files (GLOB_DEV).
- Fix the bug with literal '.' and '..' components in globstar
patterns. In preceding code, the matchdir pointer gets set to the
complete glob pattern if we're doing globstar for the current
pathname element, null if not. The pat pointer gets set to the
elements of the pattern that are still left to be processed;
already-done elements are trimmed from it by increasing the
pointer. So, to do the right thing, we need to make sure that '.'
or '..' is skipped if, and only if, it is the final element in
the pattern (i.e., if pat does not contain a slash) and is not
specified literally as '.' or '..', i.e., only if '.' or '..' was
actually resolved from a glob pattern. After this change,
'**/.*', '**/../.*', etc. do the right thing, showing all your
hidden files and directories without undesirable '.' and '..'
results; '.' and '..' are skipped as final elements, unless you
literally specify '**/.', '**/..', '**/foo/bar/..', etc.
src/cmd/ksh93/COMPATIBILITY:
- Note the symlink/** globstar change.
src/cmd/ksh93/sh.1:
- Try to document the current globstar behaviour more exhausively.
src/cmd/ksh93/tests/glob.sh:
- Add tests. Try to cover all the corner cases.
src/cmd/ksh93/tests/shtests:
- Since tests in glob.sh do not use err_exit, they were not
counted. Special-case glob.sh for counting the tests: count the
lines starting with a test_* function call.
Resolves: https://github.com/ksh93/ksh/issues/146
2021-03-06 18:45:42 +00:00
|
|
|
2021-03-07:
|
|
|
|
|
|
Fix set/unset state for short integer (typeset -si) (#211)
This commit fixes at least three bugs:
1. When issuing 'typeset -p' for unset variables typeset as short
integer, a value of 0 was incorrectly diplayed.
2. ${x=y} and ${x:=y} were still broken for short integer types
(re: 9f2389ed). ${x+set} and ${x:+nonempty} were also broken.
3. A memory fault could occur if typeset -l followed a -s option
with integers. Additonally, now the last -s/-l wins out as the
option to utilize instead of it always being short.
src/cmd/ksh93/include/name.h:
- Fix the nv_isnull() macro by removing the direct exclusion of
short integers from this set/unset test. This breaks few things
(only ${.sh.subshell} and ${.sh.level}, as far as we can tell)
while potentially correcting many aspects of short integer use
(at least bugs 1 and 2 above), as this macro is widely used.
- union Value: add new pid_t *pidp pointer member for PID values
(see further below).
src/cmd/ksh93/bltins/typeset.c: b_typeset():
- To fix bug 3 above, unset the 'shortint' flag and NV_SHORT
attribute bit upon encountering the -l optiobn.
*** To fix ${.sh.subshell} to work with the new nv_isnull():
src/cmd/ksh93/sh/defs.h:
- Add new 'realsubshell' member to the shgd (aka shp->gd) struct
which will be the integer value for ${.sh.subshell}.
src/cmd/ksh93/sh/init.c,
src/cmd/ksh93/data/variables.c:
- Initialize SH_SUBSHELLNOD as a pointer to shgd->realsubshell
instead of using a short value (.s) directly. Using a pointer
allows nv_isnull() to return a positive for ${.sh.subshell} as
a non-null pointer is what it checks for.
- While we're at it, initialize PPIDNOD ($PPID) and SH_PIDNOD
(${.sh.pid}) using the new pdip union member, which is more
correct as they are values of type pid_t.
src/cmd/ksh93/sh/subshell.c,
src/cmd/ksh93/sh/xec.c:
- Update the ${.sh.subshell} increases/decreases to refer to
shgd->realsubshell (a.k.a. shp->gd->realsubshell).
*** To fix ${.sh.level} after changing nv_isnull():
src/cmd/ksh93/sh/macro.c: varsub():
- Add a specific exception for SH_LEVLNOD to the nv_isnull() test,
so that ${.sh.level} is always considered to be set. Its handling
throughout the code is too complex/special for a simple fix, so
we have to special-case it, at least for now.
*** Regression test additions:
src/cmd/ksh93/tests/attributes.sh:
- Add in missing short integer tests and correct the one that
existed. The -si test now yields 'typeset -x -r -s -i foo'
instead of 'typeset -x -r -s -i foo=0' which brings it in line
with all the others.
- Add in some other -l attribute tests for floats. Note, -lX test
was not added as the size of long double is platform dependent.
src/cmd/ksh93/tests/variables.sh:
- Add tests for ${x=y} and ${x:=y} used on short int variables.
Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-03-08 04:19:36 +00:00
|
|
|
- Fixed the typeset -p display of short integers without an assigned value.
|
|
|
|
|
Also, the last -s or -l attribute option supplied for an integer is used.
|
|
|
|
|
|
Fixes for -G/--globstar (re: 5312a59d)
The fix for '.' and '..' in regular globbing broke '.' and '..' in
globstar. No globstar pattern that contains '.' or '..' as any
pathname component still matched. This commit fixes that.
This commit also makes symlink/** mostly work, which it never has
done in any ksh93 version. It is correct and expected that symlinks
found by patterns are not resolved, but symlinks were not resolved
even when specified as explicit non-pattern pathname components.
For example, /tmp/** breaks if /tmp is a symlink (e.g. on macOS),
which looks like a bug.
src/lib/libast/include/glob.h,
src/lib/libast/misc/glob.c: glob_dir():
- Make symlink/** work. we can check if the string pointed to by
pat is exactly equal to *. If so, we are doing regular globbing
for that particular pathname element, and it's okay to resolve
symlinks. If not (if it's **), we're doing globstar and we should
not be matching symlinks.
- Let's also introduce proper identification of symlinks (GLOB_SYM)
and not lump them in with other special files (GLOB_DEV).
- Fix the bug with literal '.' and '..' components in globstar
patterns. In preceding code, the matchdir pointer gets set to the
complete glob pattern if we're doing globstar for the current
pathname element, null if not. The pat pointer gets set to the
elements of the pattern that are still left to be processed;
already-done elements are trimmed from it by increasing the
pointer. So, to do the right thing, we need to make sure that '.'
or '..' is skipped if, and only if, it is the final element in
the pattern (i.e., if pat does not contain a slash) and is not
specified literally as '.' or '..', i.e., only if '.' or '..' was
actually resolved from a glob pattern. After this change,
'**/.*', '**/../.*', etc. do the right thing, showing all your
hidden files and directories without undesirable '.' and '..'
results; '.' and '..' are skipped as final elements, unless you
literally specify '**/.', '**/..', '**/foo/bar/..', etc.
src/cmd/ksh93/COMPATIBILITY:
- Note the symlink/** globstar change.
src/cmd/ksh93/sh.1:
- Try to document the current globstar behaviour more exhausively.
src/cmd/ksh93/tests/glob.sh:
- Add tests. Try to cover all the corner cases.
src/cmd/ksh93/tests/shtests:
- Since tests in glob.sh do not use err_exit, they were not
counted. Special-case glob.sh for counting the tests: count the
lines starting with a test_* function call.
Resolves: https://github.com/ksh93/ksh/issues/146
2021-03-06 18:45:42 +00:00
|
|
|
- Fixed a bug with -G/--globstar introduced on 2020-08-09: patterns did not
|
|
|
|
|
match anything if any pathname component was '.' or '..', e.g. '**/./glob.c'
|
|
|
|
|
never matched. The 2020-08-09 fix does still apply to patterns like '.*'.
|
|
|
|
|
|
|
|
|
|
- Enhancement to -G/--globstar: symbolic links to directories are now followed
|
|
|
|
|
if they match a normal (non-**) glob pattern. For example, if '/lnk' is a
|
|
|
|
|
symlink to a directory, '/lnk/**' and '/l?k/**' now work as you would expect.
|
|
|
|
|
|
2021-03-07 17:01:17 +00:00
|
|
|
- Fixed a bug introduced on 2021-02-11 that caused job control on interactive
|
|
|
|
|
ksh sessions to misbehave if the login shell was replaced by ksh using 'exec'.
|
|
|
|
|
|
Fix ${x=y} and ${x:=y} for numeric types of x
These POSIX expansions first assign y to x if x is unset or empty,
respectively, and then they yield the value of x. This was not
working on any ksh93 version if x was typeset as numeric (integer
or float) but still unset, as in not assigned a value.
$ unset a; typeset -i a; printf '%q\n' "${a:=42}" "$a"
0
''
Expected output:
42
42
src/cmd/ksh93/sh/macro.c:
- Fix the test for set/unset variable. It was broken because it
only checked for the existence of the node, which exists after
'typeset', but did not check if a value had been assigned. This
additional check needs to be done with the nv_isnull() macro, but
only for expansions of the regular M_BRACE type. Special
expansions cannot have an unset state.
- As of commit 95294419, we know that an nv_optimize() call may be
needed before using nv_isnull() if the shell is compiled with
SHOPT_OPTIMIZE. Move the nv_optimize() call from that commit
forward to before the new check that calls nv_isnull(), and only
bother with it if the type is M_BRACE.
src/cmd/ksh93/tests/variables.sh:
- Add tests for this bug. Test float and integer, and also check
that ${a=b} and ${a:=b} correctly treat the value of 'b' as an
arithmetic expression of which the result is assigned to 'a' if
'a' was typeset as numeric.
src/cmd/ksh93/tests/attributes.sh,
src/cmd/ksh93/tests/comvar.sh,
src/cmd/ksh93/tests/nameref.sh,
src/cmd/ksh93/tests/types.sh:
- Fix a number of tests to report failures correctly.
Resolves: https://github.com/ksh93/ksh/issues/157
2021-03-06 03:56:52 +00:00
|
|
|
2021-03-06:
|
|
|
|
|
|
|
|
|
|
- Fixed an old expansion bug: expansions of type ${var=value} and ${var:=value}
|
|
|
|
|
did not perform an assignment and yielded the value 0 if 'var' was typeset as
|
|
|
|
|
numeric (integer or float) but had not yet been assigned a value.
|
|
|
|
|
|
2021-03-06 06:43:38 +00:00
|
|
|
- Fixed a bug introduced on 2020-08-19: Ctrl+D would break after an
|
|
|
|
|
interactive shell received SIGWINCH.
|
|
|
|
|
|
Fix command history corruption on syntax error (re: e999f6b1)
Analysis: When a syntax error occurs, the shell performs a
longjmp(3) back to exfile() in main.c on line 417:
415| if(jmpval)
416| {
417| Sfio_t *top;
418| sh_iorestore((void*)shp,0,jmpval);
419| hist_flush(shp->gd->hist_ptr);
420| sfsync(shp->outpool);
The first thing it does is restore the file descriptor state
(sh_iorestore), then it flushes the history file (hist_flush), then
it synchronises sfio's logical stream state with the physical
stream state using (sfsync).
However, the fix applied in e999f6b1 caused sh_iorestore() to sync
all sfio streams unconditionally. So this was done before
hist_flush(), which caused unpredictable behaviour, including
temporary and/or permanent history corruption, as this also synched
shp->outpool before hist_flush() had a chance to do its thing.
The fix is to only call sfsync() in sh_iorestore() if we're
actually about to call ftruncate(2), and not otherwise.
Moral of the story: bug fixes should be as specific as possible to
minimise the risk of side effects.
src/cmd/ksh93/sh/io.c: sh_iorestore():
- Only call sfsync() if we're about to truncate a file.
src/cmd/ksh93/tests/pty.sh:
- Add test.
Thanks to Marc Wilson for reporting the bug and to Johnothan King
for finding the commit that introduced it.
Resolves: https://github.com/ksh93/ksh/issues/209
Relevant: https://github.com/att/ast/issues/61
2021-03-07 00:27:33 +00:00
|
|
|
- Fixed a bug introduced on 2020-05-21: on an interactive shell, command lines
|
|
|
|
|
containing a syntax error were not added to the command history file and
|
|
|
|
|
sometimes corrupted the command history.
|
|
|
|
|
|
Remove obsolete quote balancing hack
The old Bourne shell failed to check for closing quotes and command
substitution backticks when encountering end-of-file in a parser
context (such as a script). ksh93 implemented a hack for partial
compatibility with this bug, tolerating unbalanced quotes and
backticks in backtick command subsitutions, 'eval', and command
line invocation '-c' scripts only.
This hack became broken for backtick command substitutions in
fe20311f/350b52ea as a memory leak was fixed by adding a newline to
the stack at the end of the command substitution. That extra
newline becomes part of any string whose quotes are not properly
terminated, causing problems such as the one detailed here:
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01889.html
$ touch abc
$ echo `ls "abc`
ls: abc
: not found
No other fix for the memory leak is known that doesn't cause other
problems. (The alternative fix detailed in the referenced mailing
list post causes a different corner-case regression.)
Besides, the hack has always caused other corner case bugs as well:
$ ksh -c '((i++'
Actual: ksh: i++(: not found
(If an external command 'i++(' existed, it would be run)
Expect: ksh: syntax error at line 1: `(' unmatched
$ ksh -c 'i=0; echo $((++i'
Actual: (empty line; the arithmetic expansion is ignored)
Expect: ksh: syntax error at line 1: `(' unmatched
$ ksh -c 'echo $(echo "hi)'
Actual: ksh: syntax error at line 1: `(' unmatched
Expect: ksh: syntax error at line 1: `"' unmatched
So, it's time to get rid of this hack. The old Bourne shell is
dead and buried. No other shell tries to support this breakage.
Tolerating syntax errors is just asking for strange side effects,
inconsistent states, and corner case bugs. We should not want to do
that. Old scripts that rely on this will just need to be fixed.
src/cmd/ksh93/sh/lex.c:
- struct lexdata: Remove 'char balance' member for remembering an
unbalanced quote or backtick.
- sh_lex(): Remove the back to remember and compensate for
unbalanced quotes/backticks that was executed only if we were
executing a script from a string, as opposed to a file.
src/cmd/ksh93/COMPATIBILITY:
- Note the change.
Resolves: https://github.com/ksh93/ksh/issues/199
2021-03-05 21:55:25 +00:00
|
|
|
2021-03-05:
|
|
|
|
|
|
|
|
|
|
- Unbalanced quotes and backticks now correctly produce a syntax error
|
|
|
|
|
in -c scripts, 'eval', and backtick-style command substitutions.
|
|
|
|
|
|
Fix arbitrary command execution vuln in array subscripts in arith
This commit fixes an arbitrary command execution vulnerability in
array subscripts used within the arithmetic subsystem.
One of the possible reproducers is:
var='1$(echo INJECTION >&2)' ksh -c \
'typeset -A a; ((a[$var]++)); typeset -p a'
Output before this commit:
INJECTION
typeset -A a=([1]=1)
The 'echo' command has been surreptitiously executed from an
external environment variable.
Output after this commit:
typeset -A a=(['1$(echo INJECTION >&2)']=1)
The value is correctly used as an array subscript and nothing in it
is parsed or executed. This is as it should be, as ksh93 supports
arbitrary subscripts for associative arrays.
If we think about it logically, the C-style arithmetic subsystem
simply has no business messing around with shell expansions or
quoting at all, because those don't belong to it. Shell expansions
and quotes are properly resolved by the main shell language before
the arithmetic subsystem is even invoked. It is particularly
important to maintain that separation because the shell expansion
mechanism also executes command substitutions.
Yet, the arithmetic subsystem subjected array subscripts that
contain `$` (and only array subscripts -- how oddly specific) to
an additional level of expansion and quote resolution. For some
unfathomable reason, there are two lines of code doing specifically
this. The vulnerability is fixed by simply removing those.
Incredibly, variants of this vulnerability are shared by bash, mksh
and zsh. Instead of fixing it, it got listed in Bash Pitfalls!
http://mywiki.wooledge.org/BashPitfalls#y.3D.24.28.28_array.5B.24x.5D_.29.29
src/cmd/ksh93/sh/arith.c:
- scope(): Remove these two lines that implement the vulnerability.
if(strchr(sub,'$'))
sub = sh_mactrim(shp,sub,0);
- scope(), arith(): Remove the NV_SUBQUOTE flag from two
nv_endsubscript() calls. That flag causes the array subscript to
retain the current level of shell quoting. The shell quotes
everything as in "double quotes" before invoking the arithmetic
subsystem, and the bad sh_mactrim() call removed one level of
quoting. Since we're no longer doing that, this flag should no
longer be passed, or subscripts may get extra backslash escapes.
src/cmd/ksh93/include/name.h,
src/cmd/ksh93/sh/array.c:
- nv_endsubscript(): The NV_SUBQUOTE flag was only passed from
arith.c. Since it is now unused, remove it.
src/cmd/ksh93/tests/arith.sh:
- Tweak some tests: fix typos, report wrong values.
- Add 21 tests. Most are based on reproducers contributed by
@stephane-chazelas and @hyenias. They verify that this
vulnerability is gone and that no quoting bugs were introduced.
Resolves: https://github.com/ksh93/ksh/issues/152
2021-03-04 13:37:13 +00:00
|
|
|
2021-03-04:
|
|
|
|
|
|
|
|
|
|
- Fixed an arbitrary command execution vulnerability that occurred when
|
|
|
|
|
parsing the subscripts of arrays within arithmetic commands and expansion.
|
|
|
|
|
|
2021-03-03 03:26:39 +00:00
|
|
|
2021-03-01:
|
|
|
|
|
|
|
|
|
|
- Fixed the retention of size attributes when 'readonly' or 'typeset -r'
|
|
|
|
|
was applied to an existing variable.
|
|
|
|
|
|
emacs: Fix three tab completion bugs
1. The editor accepted literal tabs without escaping in certain
cases, causing buggy and inconsistent completion behaviour.
https://github.com/ksh93/ksh/issues/71#issuecomment-656970959
https://github.com/ksh93/ksh/issues/71#issuecomment-657216472
2. After completing a filename by choosing from a file completion
menu, the terminal cursor was placed one position too far to the
right, corrupting command line display. This happened with
multiline active.
https://github.com/ksh93/ksh/issues/71#issue-655093805
3. A completion menu was displayed if the file name to be completed
was at the point where the rest of it started with a number,
even if that part uniquely identified it so the menu had 1 item.
https://www.mail-archive.com/ast-users@lists.research.att.com/msg00436.html
src/cmd/ksh93/edit/emacs.c:
- Cosmetic consistency: change two instances of cntl('[') to ESC.
- ed_emacsread(): Fix number 1 by refusing to continue into default
processing if a tab character was not used for tab completion.
Instead, beep and continue to the next read loop iteration. This
behaviour is consistent with most other shells, so I doubt there
will be objections. To enter a literal tab it's simple enough to
escape it with ^V (the 'stty lnext' character) or \.
- draw(): Fix number 2 by correcting an off-by-one error in the
ed_setcursor() call that updates the terminal's cursor display
in multiline mode. The 'old' and 'new' parameters need to have
identical values in this particular call to avoid the cursor
position being off by one to the right. This change makes it
match the corresponding ed_setcursor() call in vi.c. See below*
for details. Thanks to Lev Kujawski for the help in analysing.
src/cmd/ksh93/edit/completion.c: ed_expand():
- Fix number 3 by changing from '=' mode (menu-based completion) to
'\' mode (ordinary filename completion) if the menu would only
show one option, which was pointless and annoying. This never
happened in vi mode, so possibly the ed_expand() call in emacs.c
could have been improved instead. But I'm comfortable with fixing
it here and not in emacs.c, because this fixes it at a more
fundamental level, plus it's straightforward and obvious here.
Resolves: https://github.com/ksh93/ksh/issues/71
____
* Further details on bug number 2:
At https://github.com/ksh93/ksh/issues/71#issuecomment-786391565
Martijn Dekker wrote:
> I'm back to my original hypothesis that there is somehow an
> off-by-one error related to the ed_setcursor() call that gets
> executed when in multiline mode. I cannot confirm whether that
> off-by-one error is actually in the call itself, or occurs
> sometime earlier on one of the many possible occasions where
> ep->cursor is changed. But everything else appears to work
> correctly, so it's not unlikely that the problem is in the call
> itself.
>
> For reference, this is the original version of that call in
> emacs.c:
>
> ksh/src/cmd/ksh93/edit/emacs.c
> Lines 1556 to 1557 in df2b9bf
> if(ep->ed->e_multiline && option == REFRESH)
> ed_setcursor(ep->ed, ep->screen, ep->cursor-ep->screen, ep->ed->e_peol, -1);
>
> There is a corresponding call in the vi.c refresh() function
> (which does the same thing as draw() in emacs.c), where the third
> (old) and fourth (new) arguments are actually identical:
>
> ksh/src/cmd/ksh93/edit/vi.c
>
> Lines 2086 to 2087 in df2b9bf
> if(vp->ed->e_multiline && vp->ofirst_wind==INVALID)
> ed_setcursor(vp->ed, physical, last_phys+1, last_phys+1, -1);
>
> The expectation for this particular call is in fact that they
> should be identical, so that a delta of zero is calculated in
> that function. Delta not being zero is what causes the cursor to
> be positioned wrong.
>
> In vi.c, last_phys is a macro that is defined as editb.e_peol,
> and editb is a macro that is defined as (*vp->ed). Which means
> last_phys means vp->ed->e_peol, which is the same as
> ep->ed->e_peol in emacs.c. (These editors were originally
> separate programs by different authors, and I suppose this is how
> it shows. Korn didn't want to change all the variable names to
> integrate them, so made macros instead.)
>
> That leaves the question of why vi.c adds 1 to both last_phys
> a.k.a. e_peol arguments, and emacs.c uses e_peol for new without
> adding anything. Analysing the ed_setcursor() code could answer
> that question.
>
> So, this patch makes emacs.c do it the same way vi.c does. Let's
> make the third argument identical to the fourth. My brief testing
> shows the bug is fixed, and the regression tests yield no
> failures. This fix is also the most specific change possible, so
> there are few opportunities for side effects (I hope).
At https://github.com/ksh93/ksh/issues/71#issuecomment-786466652
Lev Kujawski wrote:
> I did a bit of research on this, and I think the fix to have the
> Emacs editing mode do the same as Vi is correct.
>
> From RELEASE:
> 08-05-01 In multiline edit mode, the refresh operation will now clear
> the remaining portion of the last line.
>
> Here's a fragment from the completion.c of the venerable but
> dated CDE DtKsh:
>
> else
> while (*com)
> {
> *out++ = ' ';
> out = strcopy(out,*com++);
> }
> *cur = (out-outbuff);
> /* restore rest of buffer */
> out = strcopy(out,stakptr(0));
> *eol = (out-outbuff);
>
> Noticeably missing is the code to add a space after file name
> completions. So, it seems plausible that if multiline editing
> mode was added beforehand,the ep->ed->p_eol !=
> ep->cursor-ep->screen case might never have occurred during
> testing.
>
> Setting the 'first' parameter to -1 seems to be a pretty explicit
> indicator that the author(s) intended the line clearing code to
> run, hence the entry in RELASE.
>
> The real issue is that if we update the cursor by calling
> ed_setcursor on line 1554 with old != new, the later call to
> setcursor on line 1583, here:
>
> I = (ncursor-nscreen) - ep->offset;
> setcursor(ep,i,0);
>
> will use outdated screen information to call setcursor, which,
> coincidentally, calls ed_setcursor.
2021-02-26 11:20:58 +00:00
|
|
|
2021-02-26:
|
|
|
|
|
|
|
|
|
|
- Fixed three long-standing bugs with tab completion in the emacs editor:
|
|
|
|
|
|
|
|
|
|
1. The editor accepted literal tabs without escaping in certain cases,
|
|
|
|
|
causing buggy and inconsistent completion behaviour. Details:
|
|
|
|
|
https://github.com/ksh93/ksh/issues/71#issuecomment-656970959
|
|
|
|
|
https://github.com/ksh93/ksh/issues/71#issuecomment-657216472
|
|
|
|
|
To enter a literal tab in emacs, you need to escape it with ^V or \.
|
|
|
|
|
|
|
|
|
|
2. After completing a filename by choosing from a file completion menu,
|
|
|
|
|
the terminal cursor was placed one position too far to the right,
|
|
|
|
|
corrupting command line display. This happened with multiline active.
|
|
|
|
|
Details: https://github.com/ksh93/ksh/issues/71#issue-655093805
|
|
|
|
|
|
|
|
|
|
3. A completion menu was displayed if the file name to be completed was
|
|
|
|
|
at the point where the rest of it started with a number, even if that
|
|
|
|
|
part uniquely identified it so the menu only showed one item. Details:
|
|
|
|
|
https://www.mail-archive.com/ast-users@lists.research.att.com/msg00436.html
|
|
|
|
|
|
Fix ${.sh.fun} leaking out of DEBUG trap
The value of the ${.sh.fun} variable, which is supposed to contain
the name of the function currently being executed, leaks out of the
DEBUG trap if it executes a function. Reproducer:
$ fn() { echo "executing the function"; }
$ trap fn DEBUG
$ trap - DEBUG
executing the function
$ echo ${.sh.fun}
fn
${.sh.fun} should be empty outside the function.
Annalysis:
The sh_debug() function in xec.c, which executes the DEBUG trap
action, contains these lines, which are part of restoring the state
after running the trap action with sh_trap():
nv_putval(SH_PATHNAMENOD,shp->st.filename,NV_NOFREE);
nv_putval(SH_FUNNAMENOD,shp->st.funname,NV_NOFREE);
shp->st = savst;
First the SH_PATHNAMENOD (${.sh.file}) and SH_FUNNAMENOD
(${.sh.fun}) variables get restored from the values in the shell's
scoped information struct (shp->st), but that is done *before*
restoring the parent scope with 'shp->st = savst;'. It should be
done after. Fixing the order is sufficient to fix the bug.
However, I am not convinced that these nv_putval() calls are good
for anything at all. Setting, unsetting, restoring, etc. the
${.sh.fun} and ${.sh.file} variables is already being handled
perfectly well elsewhere in the code for executing functions and
sourcing dot scripts. The DEBUG trap is neither here nor there.
There's no reason for it to get involved with these variables.
I was unable to break anything after simply removing those two
lines. So I strongly suspect this is another case, out of many now,
where a bug in ksh93 is properly fixed by removing some code.
I couldn't get ${.sh.file} to leak similarly -- I think this is
because SH_PATHNAMENOD (and not SH_FUNNOD) is set explicitly in
exfile() in main.c, masking this incorrect restore. It is the only
place where SH_PATHNAMENOD and SH_FUNNOD are not both set.
src/cmd/ksh93/sh/xec.c:
- Remove these two spurious nv_putval() calls.
src/cmd/ksh93/tests/variables.sh:
- Add regression test for leaking ${.sh.fun}.
2021-02-27 01:01:30 +00:00
|
|
|
- A bug with ${.sh.fun} in combination with the DEBUG trap has been fixed.
|
|
|
|
|
The ${.sh.fun} variable wrongly continued to contain the name of the last
|
|
|
|
|
function executed by the DEBUG trap after the trap action completed.
|
|
|
|
|
|
editors: fix broken SIGWINCH handling
In the emacs editor:
1. press ESC
2. change the size of your terminal window
and your screen is mysteriously cleared. (Until recent fixes, the
shell probably also crashed somewhere in the job control code.)
The cause is the way SIGWINCH is handled by ed_read() in edit.c.
For the emacs editor, it sends a Ctrl+L character to the input
buffer. The Ctrl+L command refreshes the command line. And it so
happens that ESC plus Ctrl+L is a command to clear the screen in
the emacs editor.
With the exeption of vi insert/command mode for which it uses a
shared flag, edit.c does not know the state of the editor, because
their data are internal to emacs.c and vi.c. So it doesn't know
whether you're in some mode that treats keyboard input specially.
Which means this way of dealing with SIGWINCH is fundamentally
misdesigned and is not worth fixing.
It gets sillier: in addition to sending keyboard commands, edit.c
was also communicating directly with emacs.c and vi.c via a flag,
e_nocrnl, which means 'please don't make Ctrl+L emit a linefeed'
(it normally refreshes on a new line but that is undesirable for
SIGWINCH). So there is already a hack that breaks the barrier
between edit.c and emacs.c/vi.c. Let's do that properly instead.
As of this commit, ed_read() does not send any fake keystrokes.
Instead, two extern functions, emacs_redraw() and vi_redraw(), are
defined for redrawing the command line. These are put in emacs.c
and vi.c so they have access to relevant static data and functions.
Then, instead of sending keyboard commands to the editor and
returning, ed_read() simply calls the redraw function for the
active editor, then continues and waits for input. Much cleaner.
src/cmd/ksh93/include/edit.h:
- Remove e_nocrnl flag from Edit_t struct.
- Define externs emacs_redraw() and vi_redraw(). Since Emacs_t and
Vi_t types are not known here, we have to declare void* pointers
and the functions will have to use typecasts.
src/cmd/ksh93/edit/edit.c:
- ed_read(): Call emacs_redraw() or vi_redraw() as per above.
- ed_getchar(): Remove comment about a nonexistent while loop.
src/cmd/ksh93/edit/emacs.c:
- Updates corresponding to removal of e_nocrnl flag.
- Add emacs_redraw(). This one is pretty simple. Refresh the
command line, then ed_flush() to update the cursor display.
src/cmd/ksh93/edit/vi.c:
- Updates corresponding to removal of e_nocrnl flag. Also remove a
similar internal 'nonewline' flag which is now equally redundant.
- Move the Ctrl+L handling code (minus writing the newline) into
the vi_redraw() function.
- Change two cases where vi set nonewline and sent Ctrl+L to itself
into simple vi_redraw() calls.
- Add vi_redraw(). This is more complicated as it incorporates the
previous Ctrl+L code. It needs an added refresh() call with a
check whether we're currently in command or insert mode, as those
use different refresh methods. Luckily edit.c already maintains
an *e_vi_insert flag in ed_getchar() that we can use. Since vi's
refresh() already calls ed_flush(), we don't need to add that.
2021-02-21 23:07:30 +00:00
|
|
|
2021-02-21:
|
|
|
|
|
|
|
|
|
|
- Fixed: The way that SIGWINCH was handled (i.e. the signal emitted when the
|
|
|
|
|
terminal window size changes) could cause strange emacs/vi editor behaviour.
|
|
|
|
|
|
DEBUG trap: restore status 2 trigger to skip command (re: d00b4b39)
So now we know what that faulty check for shp->indebug in sh_trap()
was meant to do: it was meant to pass down the trap handler's exit
status, via sh_debug(), down to sh_exec() (xec.c) so that it could
then skip the execution of the next command if the trap's exit
status is 2, as documented in the manual page. As of d00b4b39, exit
status 2 was not passed down, so this stopped working.
This commit reinstates that functionality, but without the exit
status bug in command substitutions caused by the old way.
src/cmd/ksh93/sh/fault.c: sh_trap():
- Save the trap's exit status before restoring the parent
envionment's exit status. Make this saved exit status the return
value of the function. (This does not break anything, AFAICT; the
majority of sh_trap() calls ignore the return value, and the few
that don't ignore it seem to expect it to return exactly this.)
src/cmd/ksh93/sh/xec.c: sh_exec():
- The sh_trap() fix has one side effect: whereas the exit status of
a skipped command was always 2 (as per the trap handler), now it
is always 0, because it gets reset in sh_exec() but no command is
executed. That is probably not a desirable change in behaviour,
so let's fix that here instead: set sh.exitval to 2 when skipping
commands.
src/cmd/ksh93/sh.1:
- Document that ${.sh.command} shell-quotes its arguments for use
by 'eval' and such. This fact was not documented anywhere, AFAIK.
src/cmd/ksh93/shell.3:
- Document that $? (exit status) is made local to trap handlers.
- Document that sh_trap() returns the trap handler's exit status.
src/cmd/ksh93/tests/basic.sh:
- Add test for this bug.
- Add a missing test for the exit status 255 functionality (if a
DEBUG trap handler yields this exit status and we're executing a
function or dot script, a return is triggered).
Fixes: https://github.com/ksh93/ksh/issues/187
2021-02-20 05:13:51 +00:00
|
|
|
2021-02-20:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug introduced on 2021-01-20: if a DEBUG trap action yielded exit
|
|
|
|
|
status 2, the execution of the next command was not skipped as documented.
|
|
|
|
|
|
Fix multiple buffer overflows with justified strings (-L/-R/-Z)
ksh crashed in various different and operating system-dependent
ways when attempting to create or apply justification strings
using typeset -L/-R/-Z, especially if large sizes are used.
The crashes had two immediate causes:
- In nv_newattr(), when applying justification attributes, a buffer
was allocated for the justified string that was exactly 8 bytes
longer than the original string. Any larger justification string
caused a buffer overflow (!!!).
- In nv_putval(), when applying existing attributes to a new value,
the corresponding memmove() either did not zero-terminate the
justified string (if the original string was longer than the
justified string) or could read memory past the original string
(if the original string was shorter than the justified string).
Both scenarios can cause a crash.
This commit fixes other minor issues as well, such as a mysterious
8 extra bytes allocated by several malloc/realloc calls. This may
have been some naive attempt to paper over the above bugs. It seems
no one can make any other kind of sense of it.
A readjustment bug with zero-filling was also fixed.
src/cmd/ksh93/sh/name.c:
- nv_putval():
. Get rid of the magical +8 bytes for malloc and realloc. Just
allocate one extra byte for the terminating zero.
. Fix the memmove operation to use strncpy instead, so that
buffer overflows are avoided in both scenarios described above.
Also make it conditional upon a size adjustment actually
happening (i.e. if 'dot' is nonzero).
. Mild refactoring: combine two 'if(sp)' blocks into one;
declare variables only used there locally for legibility.
- nv_newattr():
* Replace the fatally broken "let's allocate string length + 8
bytes no matter the size of the adjustment" routine with a new
one based on work by @hyenias (see comments in #142). It is
efficient with memory, taking into account numeric types,
growing strings, and shrinking strings.
* Fix zero-filling in readjustment after changing the initial
size of a -Z attribute. If the number was zero, all zeros were
still skipped, leaving an empty string.
Thanks to @hyenias for originally identifying this breakage and
laying the groundwork for fixing nv_newattr(), and to @lijog for
the crash analysis that revealed the key to the nv_putval() fix.
Resolves: https://github.com/ksh93/ksh/issues/142
Resolves: https://github.com/ksh93/ksh/issues/181
2021-02-20 12:27:53 +00:00
|
|
|
- Fixed multiple buffer overflows causing crashes in typeset -L/-R-/-Z.
|
|
|
|
|
|
|
|
|
|
- Fixed typeset -Z zero-filling: if the number was zero, all zeros
|
|
|
|
|
were skipped when changing the initial size value of the -Z attribute,
|
|
|
|
|
leaving an empty string.
|
|
|
|
|
|
2021-02-18 14:46:10 +00:00
|
|
|
2021-02-18:
|
|
|
|
|
|
|
|
|
|
- A bug was fixed in the 'read' builtin that caused it to fail to process
|
|
|
|
|
multibyte characters properly in Shift-JIS locales.
|
|
|
|
|
|
2021-02-17 14:29:12 +00:00
|
|
|
2021-02-17:
|
|
|
|
|
|
|
|
|
|
- Emacs mode fixes:
|
|
|
|
|
1. Erasing a backslash while doing a reverse search (^R) no longer deletes
|
|
|
|
|
extra characters.
|
|
|
|
|
2. The backslash now escapes a subsequent interrupt (^C) as documented.
|
|
|
|
|
|
Fix subshell scoping of changes in shared command substitution
A ${ shared-state command substitution; } (internally called
subshare) is documented to share its state with the parent shell
environment, so all changes made within the command substitution
survive outside of it. However, when it is run within a
virtual/non-forked subshell, variables that are not already local
to that subshell will leak out of it into the grandparent state.
Reproducer:
$ ksh -c '( v=${ bug=BAD; } ); echo "$bug"'
BAD
If the variable pre-exists in the subshell, the bug does not occur:
$ ksh -c '( bug=BAD1; v=${ bug=BAD2; } ); echo "$bug"'
(empty line, as expected)
The problem is that the sh_assignok() function, which is
responsible for variable scoping in virtual subshells, does not
ever bother to create a virtual subshell scope for a subshare.
That is an error if a subshare's parent (or higher-up ancestor)
environment is a virtual subshell, because a scope needs to be
created in that parent environment if none exists.
To make this bugfix possible, first we need to get something out of
the way. nv_restore() temporarily sets the subshell's pointer to
the preesnt working directory, shpwd, to null. This causes
sh_assignok() to assume that the subshell is a subshare (because
subshares don't store their own PWD) and refuse to create a scope.
However, nv_restore() sets it to null for a different purpose: to
temporarily disable scoping for *all* virtual subshells, making
restoring possible. This is a good illustration of why it's often
not a good idea to use the same variable for unrelated purposes.
src/cmd/ksh93/sh/subshell.c:
- Add a global static subshell_noscope flag variable to replace the
misuse of sh.shpwd described above.
- sh_assignok():
. Check subshell_noscope instead of shpwd to see if scope
creation is disabled. This makes it possible to distinguish
between restoring scope and handling subshares.
. If the current environment is a subshare that is in a virtual
subshell, create a scope in the parent subshell. This is done
by temporarily making the parent virtual subshell the current
subshell (by setting the global subshell_data pointer to it)
and calling sh_assignok() again, recursively.
- nv_restore(): To disable subshell scope creation while restoring,
set subshell_noscope instead of saving and unsetting sh.shpwd.
src/cmd/ksh93/tests/subshell.sh:
- Add tests. I like tests. Tests are good.
Fixes: https://github.com/ksh93/ksh/issues/143
2021-02-17 15:33:48 +00:00
|
|
|
- Fixed a longstanding bug with shared-state command substitutions of the form
|
|
|
|
|
${ command; }. If these were executed in a subshell, changes made within
|
|
|
|
|
could survive not only the command substitution but also the parent subshell.
|
|
|
|
|
|
Add new 'nobackslashctrl' shell option; other minor editor tweaks
The following emacs editor 'feature' kept making me want to go back
to bash. I forget a backslash escape in a command somewhere. So I
go back to insert it. I type the \, then want to go forward. My
right arrow key, instead of moving the cursor, then replaces my
backslash with garbage. Why? The backslash escapes the following
control character at the editor level and inserts it literally.
The vi editor has a variant of this which is much less harmful. It
only works in insert mode and the backslash only escapes the next
kill or erase character.
In both editors, this feature is completely redundant with the
'stty lnext' character which is ^V by default -- and works better
as well because it also escapes ^C, ^J (linefeed) and ^M (Return).
[In fact, you could even issue 'stty lnext \\' and get a much more
consistent version of this feature on any shell. You have to type
two backslashes to enter one, but it won't kill your cursor keys.]
If it were up to me alone, I'd simply remove this misfeature from
both editors. However, it is long-standing documented behaviour.
It's in the 1995 book. Plus, POSIX specifies the vi variant of it.
So, this adds a shell option instead. It was quite trivial to do.
Now I can 'set --nobackslashctrl' in my ~/.kshrc. What a relief!
Note: To keep .kshrc compatibile with older ksh versions, use:
command set --nobackslashctrl 2>/dev/null
src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/data/options.c:
- Add new SH_NOBACKSLCTRL/"nobackslashctrl" long-form option. The
"no" prefix shows it to the user as "backslashctrl" which is on
by default. This avoids unexpectedly changing historic behaviour.
src/cmd/ksh93/edit/emacs.c: ed_emacsread(),
src/cmd/ksh93/edit/vi.c: getline():
- Only set the flag for special backslash handling if
SH_NOBACKSLCTRL is off.
src/cmd/ksh93/sh.1,
src/cmd/ksh93/data/builtins.c:
- Document the new option (as "backslashctrl", on by default).
Other minor tweaks:
src/cmd/ksh93/edit/edit.c:
- ed_setup(): Add fallback #error if no tput method is set. This
should never be triggered; it's to catch future editing mistakes.
- escape(): cntl('\t') is nonsense as '\t' is already a control
character, so change this to just '\t'.
- xcommands(): Let's enable the ^X^D command for debugging
information on non-release builds.
src/cmd/ksh93/features/cmds:
- The tput feature tests assumed a functioning terminal in $TERM.
However, for all we know we might be compiling with no tty and
TERM=dumb. The tput commands won't work then. So set TERM=ansi
to use a standard default.
2021-02-16 00:07:03 +00:00
|
|
|
2021-02-15:
|
|
|
|
|
|
2021-02-16 06:50:12 +00:00
|
|
|
- Fixed a regression introduced by ksh93 (was not in ksh88): an empty 'case'
|
|
|
|
|
list on a single line ('case x in esac') was a syntax error.
|
|
|
|
|
|
2021-02-16 01:47:15 +00:00
|
|
|
- Fixed a bug in the emacs built-in editor, introduced on 2020-09-17, that
|
|
|
|
|
made the Meta-D and Meta-H keys delete single characters instead of words.
|
|
|
|
|
|
Add new 'nobackslashctrl' shell option; other minor editor tweaks
The following emacs editor 'feature' kept making me want to go back
to bash. I forget a backslash escape in a command somewhere. So I
go back to insert it. I type the \, then want to go forward. My
right arrow key, instead of moving the cursor, then replaces my
backslash with garbage. Why? The backslash escapes the following
control character at the editor level and inserts it literally.
The vi editor has a variant of this which is much less harmful. It
only works in insert mode and the backslash only escapes the next
kill or erase character.
In both editors, this feature is completely redundant with the
'stty lnext' character which is ^V by default -- and works better
as well because it also escapes ^C, ^J (linefeed) and ^M (Return).
[In fact, you could even issue 'stty lnext \\' and get a much more
consistent version of this feature on any shell. You have to type
two backslashes to enter one, but it won't kill your cursor keys.]
If it were up to me alone, I'd simply remove this misfeature from
both editors. However, it is long-standing documented behaviour.
It's in the 1995 book. Plus, POSIX specifies the vi variant of it.
So, this adds a shell option instead. It was quite trivial to do.
Now I can 'set --nobackslashctrl' in my ~/.kshrc. What a relief!
Note: To keep .kshrc compatibile with older ksh versions, use:
command set --nobackslashctrl 2>/dev/null
src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/data/options.c:
- Add new SH_NOBACKSLCTRL/"nobackslashctrl" long-form option. The
"no" prefix shows it to the user as "backslashctrl" which is on
by default. This avoids unexpectedly changing historic behaviour.
src/cmd/ksh93/edit/emacs.c: ed_emacsread(),
src/cmd/ksh93/edit/vi.c: getline():
- Only set the flag for special backslash handling if
SH_NOBACKSLCTRL is off.
src/cmd/ksh93/sh.1,
src/cmd/ksh93/data/builtins.c:
- Document the new option (as "backslashctrl", on by default).
Other minor tweaks:
src/cmd/ksh93/edit/edit.c:
- ed_setup(): Add fallback #error if no tput method is set. This
should never be triggered; it's to catch future editing mistakes.
- escape(): cntl('\t') is nonsense as '\t' is already a control
character, so change this to just '\t'.
- xcommands(): Let's enable the ^X^D command for debugging
information on non-release builds.
src/cmd/ksh93/features/cmds:
- The tput feature tests assumed a functioning terminal in $TERM.
However, for all we know we might be compiling with no tty and
TERM=dumb. The tput commands won't work then. So set TERM=ansi
to use a standard default.
2021-02-16 00:07:03 +00:00
|
|
|
- A new 'backslashctrl' shell option has been added. It is on by default.
|
|
|
|
|
Turning it off (set +o backslashctrl or set --nobackslashctrl) disables the
|
|
|
|
|
special escaping behaviour of the backslash character in the emacs and vi
|
|
|
|
|
built-in editors. Particularly in the emacs editor, this makes it much easier
|
|
|
|
|
to go back, insert a forgotten backslash into a command, and then continue
|
|
|
|
|
editing without having your next cursor key replace your backslash with
|
|
|
|
|
garbage. Note that Ctrl+V (or whatever other character was set using
|
|
|
|
|
'stty lnext') always escapes all control characters in either editing mode.
|
|
|
|
|
|
2021-02-14 19:39:01 +00:00
|
|
|
2021-02-14:
|
|
|
|
|
|
2021-12-12 21:40:38 +00:00
|
|
|
- Due to a deficiency in some UNIX variants, the 'sleep' built-in command could
|
2021-02-14 19:39:01 +00:00
|
|
|
occasionally sleep for slightly less than the time specified. It now performs
|
|
|
|
|
an additional check against the system clock to make sure it sleeps at least
|
|
|
|
|
the given amount of time. Thanks to Lev Kujawski for adding this feature.
|
|
|
|
|
|
Fix bugs related to --posix shell option (re: 921bbcae, f45a0f16)
This fixes the following:
1. 'set --posix' now works as an equivalent of 'set -o posix'.
2. The posix option turns off braceexpand and turns on letoctal.
Any attempt to override that in a single command such as 'set -o
posix +o letoctal' was quietly ignored. This now works as long
as the overriding option follows the posix option in the command.
3. The --default option to 'set' now stops the 'posix' option, if
set or unset in the same 'set' command, from changing other
options. This allows the command output by 'set +o' to correctly
restore the current options.
src/cmd/ksh93/data/builtins.c:
- To make 'set --posix' work, we must explicitly list it in
sh_set[] as a supported option so that AST optget(3) recognises
it and won't override it with its own default --posix option,
which converts the optget(3) string to at POSIX getopt(3) string.
This means it will appear as a separate entry in --man output,
whether we want it to or not. So we might as well use it as an
example to document how --optionname == -o optionname, replacing
the original documentation that was part of the '-o' description.
src/cmd/ksh93/sh/args.c: sh_argopts():
- Add handling for explitit --posix option in data/builtins.c.
- Move SH_POSIX syncing SH_BRACEEXPAND and SH_LETOCTAL from
sh_applyopts() into the option parsing loop here. This fixes
the bug that letoctal was ignored in 'set -o posix +o letoctal'.
- Remember if --default was used in a flag, and do not sync options
with SH_POSIX if the flag is set. This makes 'set +o' work.
src/cmd/ksh93/include/argnod.h,
src/cmd/ksh93/data/msg.c,
src/cmd/ksh93/sh/args.c: sh_printopts():
- Do not potentially translate the 'on' and 'off' labels in 'set
-o' output. No other shell does, and some scripts parse these.
src/cmd/ksh93/sh/init.c: sh_init():
- Turn on SH_LETOCTAL early along with SH_POSIX if the shell was
invoked as sh; this makes 'sh -o' and 'sh +o' show expected
options (not that anyone does this, but correctness is good).
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/include/shell.h:
- The state flags were in defs.h and most (but not all) of the
shell options were in shell.h. Gather all the shell state and
option flag definitions into one place in shell.h for clarity.
- Remove unused SH_NOPROFILE and SH_XARGS option flags.
src/cmd/ksh93/tests/options.sh:
- Add tests for these bugs.
src/lib/libast/misc/optget.c: styles[]:
- Edit default optget(3) option self-documentation for clarity.
Several changed files:
- Some SHOPT_PFSH fixes to avoid compiling dead code.
2021-02-14 23:51:19 +00:00
|
|
|
- A few bugs were fixed that 93u+m introduced along with the new '-o posix'
|
|
|
|
|
shell option on 2020-09-01:
|
|
|
|
|
1. 'set --posix' now works as the expected equivalent of 'set -o posix'.
|
|
|
|
|
2. As of 2020-09-18, the posix option turns off braceexpand and turns on
|
|
|
|
|
letoctal. Any attempt to override that in a single command such as
|
|
|
|
|
'set -o posix +o letoctal' was quietly ignored. This now works as long
|
|
|
|
|
as the overriding option follows the posix option on the command line.
|
|
|
|
|
3. The --default option to 'set' now stops the 'posix' option, if set or
|
|
|
|
|
unset in the same 'set' command, from changing other options. This allows
|
|
|
|
|
the command output by 'set +o' to correctly restore the current options.
|
|
|
|
|
|
2021-02-11 13:41:10 +00:00
|
|
|
2021-02-11:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused ksh to lose track of all running background jobs if
|
|
|
|
|
a shared-state command substitution of the form v=${ cmd; } was used twice.
|
|
|
|
|
|
Fix most of job control (-m/-o monitor) in scripts
If I haven't missed anything, this should make the non-interactive
aspects of job control in scripts work as expected, except for the
"<command unknown>" issue in the output of 'bg', 'fg' and 'jobs'
(which is not such a high priority as those commands are really
designed for interactive use).
Plus, I believe I now finally understand what these three are for:
* The job.jobcontrol variable is set to nonzero by job_init() in
jobs.c if, and only if, the shell is interactive *and* managed to
get control of the terminal. Therefore, any changing of terminal
settings (tcsetpgrp(3), tty_set()) should only be done if
job.jobcontrol is nonzero. This commit changes several checks for
sh_isoption(SH_INTERACTIVE) to checks for job.jobcontrol for
better consistency with this.
* The state flag, sh_isstate(SH_MONITOR), determines whether the
bits of job control that are relevant for both scripts and
interactive shells are active, which is mostly making sure that a
background job gets its own process group (setpgid(3)).
* The shell option, sh_isoption(SH_MONITOR), is just that. When the
user turns it on or off, the state flag is synched with it. It
should usually not be directly checked for, as the state may be
temporarily turned off without turning off the option.
Prior discussion:
https://www.mail-archive.com/austin-group-l@opengroup.org/msg06456.html
src/cmd/ksh93/bltins/typeset.c, src/cmd/ksh93/sh/args.c:
- Move synching the SH_MONITOR state flag with the SH_MONITOR
shell option from b_set() (the 'set' builtin) to sh_applyopts()
which is indirectly called from b_set() and is also used when
parsing the shell invocation command line. This ensures -m is
properly enabled in both scenarios.
src/cmd/ksh93/sh/jobs.c:
- job_init(): Do not refuse to initialise job control on
non-interactive shells. Instead, skip everything that should only
be done on interactive shells (i.e., everything to do with the
terminal). This function is now even more of a mess than it was
before, so refactoring may be desirabe at some point.
- job_close(), job_set(), job_reset(), job_wait(): Do not reset the
terminal process group (tcsetpgrp()) if job.jobcontrol isn't on.
src/cmd/ksh93/sh/xec.c:
- sh_exec(): TFORK: For SIGINT handling, check the SH_MONITOR
state flag, not the shell option.
- sh_exec(): TFORK: Do not turn off the SH_MONITOR state flag in
forked children. The non-interactive part of job control should
stay active. Instead, turn off the SH_INTERACTIVE state flag so
we don't get interactive shell behaviour (i.e. job control noise
on the terminal) in forked subshells.
- _sh_fork(), sh_ntfork(): Do not reset the terminal process group
(tcsetpgrp()) if job.jobcontrol isn't on. Do not turn off the
SH_MONITOR state flag in forked children.
src/cmd/ksh93/sh/subshell.c: sh_subfork():
- Do not turn off the monitor option and state in forked subshells.
The non-interactive part of job control should stay active.
src/cmd/ksh93/bltins/misc.c: b_bg():
- Check isstate(SH_MONITOR) instead of sh_isoption(SH_MONITOR) &&
job.jobcontrol before throwing a 'no job control' error.
This fixes a minor bug: fg, bg and disown could quietly fail.
src/cmd/ksh93/tests/jobs.sh:
- Add tests for 'fg' with job control IDs (%%, %1) in scripts.
- Add test checking that a background job launched from a subsell
with job control enabled correctly becomes the leader of its own
process group.
Makes progress on: https://github.com/ksh93/ksh/issues/119
2021-02-11 22:05:00 +00:00
|
|
|
- Job control (the -m/-o monitor option) has been fixed for scripts. Background
|
|
|
|
|
jobs are now correctly assigned their own process group when run from
|
|
|
|
|
subshells (except command substitutions). The 'fg' command now also works for
|
|
|
|
|
scripts as it does on other shells, though 'wait' should be preferred.
|
|
|
|
|
|
2021-02-05 03:42:10 +00:00
|
|
|
2021-02-05:
|
|
|
|
|
|
|
|
|
|
- Fixed a longstanding bug that caused redirections that store a file
|
|
|
|
|
descriptor > 10 in a variable, such as {var}>file, to stop working if
|
|
|
|
|
brace expansion (the -B or -o braceexpand option) was turned off. (Note
|
|
|
|
|
that '{var}' is not a brace expansion as it does not contain ',' or '..'.)
|
|
|
|
|
|
2021-02-04 17:31:22 +00:00
|
|
|
2021-02-04:
|
|
|
|
|
|
|
|
|
|
- Fixed ksh crashing if an autoloaded function tried to autoload itself.
|
|
|
|
|
ksh now errors out gracefully with an "autoload loop" error message.
|
|
|
|
|
|
|
|
|
|
- Fixed crash on trying a very long nonexistent command.
|
|
|
|
|
|
2021-02-01 16:19:04 +00:00
|
|
|
2021-02-01:
|
|
|
|
|
|
2021-02-01 23:35:18 +00:00
|
|
|
- Fixed a bug in 'typeset': the '-s' modifier option for short integer will
|
|
|
|
|
now only be applied if the integer option '-i' is also present, avoiding
|
|
|
|
|
inconsistent results and a crash.
|
|
|
|
|
|
2021-02-01 16:19:04 +00:00
|
|
|
- Fixed: scalar arrays (-a) and associative arrays (-A) of a type created by
|
|
|
|
|
'enum' allowed values not specified by the enum type, corrupting results.
|
|
|
|
|
|
|
|
|
|
- Fixed: the "${array[@]}" expansion for associative arrays of a type created
|
|
|
|
|
by 'enum' expanded to random numbers instead of the array's values.
|
|
|
|
|
|
command -x: fix efficiency; always run external cmd (re: acf84e96)
This commit fixes 'command -x' to adapt to OS limitations with
regards to data alignment in the arguments list. A feature test is
added that detects if the OS aligns the argument on 32-bit or
64-bit boundaries or not at all, allowing 'command -x' to avoid
E2BIG errors while maximising efficiency.
Also, as of now, 'command -x' is a way to bypass built-ins and
run/query an external command. Built-ins do not limit the length of
their argument list, so '-x' never made sense to use for them. And
because '-x' hangs on Linux and macOS on every ksh93 release
version to date (see acf84e96), few use it, so there is little
reason not to make this change.
Finally, this fixes a longstanding bug that caused the minimum exit
status of 'command -x' to be 1 if a command with many arguments was
divided into several command invocations. This is done by replacing
broken flaggery with a new SH_XARG state flag bit.
src/cmd/ksh93/features/externs:
- Add new C feature test detecting byte alignment in args list.
The test writes a #define ARG_ALIGN_BYTES with the amount of
bytes the OS aligns arguments to, or zero for no alignment.
src/cmd/ksh93/include/defs.h:
- Add new SH_XARG state bit indicating 'command -x' is active.
src/cmd/ksh93/sh/path.c: path_xargs():
- Leave extra 2k in the args buffer instead of 1k, just to be sure;
some commands add large environment variables these days.
- Fix a bug in subtracting the length of existing arguments and
environment variables. 'size -= strlen(cp)-1;' subtracts one less
than the size of cp, which makes no sense; what is necessary is
to substract the length plus one to account for the terminating
zero byte, i.e.: 'size -= strlen(cp)+1'.
- Use the ARG_ALIGN_BYTES feature test result to match the OS's
data alignment requirements.
- path_spawn(): E2BIG: Change to checking SH_XARG state bit.
src/cmd/ksh93/bltins/whence.c: b_command():
- Allow combining -x with -p, -v and -V with the expected results
by setting P_FLAG to act like 'whence -p'. E.g., as of now,
command -xv printf
is equivalent to
whence -p printf
but note that 'whence' has no equivalent of 'command -pvx printf'
which searches $(getconf PATH) for a command.
- When -x will run a command, now set the new SH_XARG state flag.
src/cmd/ksh93/sh/xec.c: sh_exec():
- Change to using the new SH_XARG state bit.
- Skip the check for built-ins if SH_XARG is active, so that
'command -x' now always runs an external command.
src/lib/libcmd/date.c, src/lib/libcmd/uname.c:
- These path-bound builtins sometimes need to run the external
system command by the same name, but they did that by hardcoding
an unportable direct path. Now that 'command -x' runs an external
command, change this to using 'command -px' to guarantee using
the known-good external system utility in the default PATH.
- In date.c, fix the format string passed to 'command -px date'
when setting the date; it was only compatible with BSD systems.
Use the POSIX variant on non-BSD systems.
2021-01-30 05:51:22 +00:00
|
|
|
2021-01-30:
|
|
|
|
|
|
|
|
|
|
- The -x option to the 'command' built-in now causes it to bypass built-ins
|
|
|
|
|
so that it always runs/queries an external command. See 'command --man'.
|
|
|
|
|
|
|
|
|
|
- Fixed a bug in 'command -x' that caused the minimum exit status to be 1 if
|
|
|
|
|
a command with many arguments was divided into several command invocations.
|
|
|
|
|
|
|
|
|
|
- The 2020-08-16 fix is improved with a compile-time feature test that
|
2021-02-01 16:19:04 +00:00
|
|
|
detects if the OS requires extra bytes per argument in the arguments list,
|
command -x: fix efficiency; always run external cmd (re: acf84e96)
This commit fixes 'command -x' to adapt to OS limitations with
regards to data alignment in the arguments list. A feature test is
added that detects if the OS aligns the argument on 32-bit or
64-bit boundaries or not at all, allowing 'command -x' to avoid
E2BIG errors while maximising efficiency.
Also, as of now, 'command -x' is a way to bypass built-ins and
run/query an external command. Built-ins do not limit the length of
their argument list, so '-x' never made sense to use for them. And
because '-x' hangs on Linux and macOS on every ksh93 release
version to date (see acf84e96), few use it, so there is little
reason not to make this change.
Finally, this fixes a longstanding bug that caused the minimum exit
status of 'command -x' to be 1 if a command with many arguments was
divided into several command invocations. This is done by replacing
broken flaggery with a new SH_XARG state flag bit.
src/cmd/ksh93/features/externs:
- Add new C feature test detecting byte alignment in args list.
The test writes a #define ARG_ALIGN_BYTES with the amount of
bytes the OS aligns arguments to, or zero for no alignment.
src/cmd/ksh93/include/defs.h:
- Add new SH_XARG state bit indicating 'command -x' is active.
src/cmd/ksh93/sh/path.c: path_xargs():
- Leave extra 2k in the args buffer instead of 1k, just to be sure;
some commands add large environment variables these days.
- Fix a bug in subtracting the length of existing arguments and
environment variables. 'size -= strlen(cp)-1;' subtracts one less
than the size of cp, which makes no sense; what is necessary is
to substract the length plus one to account for the terminating
zero byte, i.e.: 'size -= strlen(cp)+1'.
- Use the ARG_ALIGN_BYTES feature test result to match the OS's
data alignment requirements.
- path_spawn(): E2BIG: Change to checking SH_XARG state bit.
src/cmd/ksh93/bltins/whence.c: b_command():
- Allow combining -x with -p, -v and -V with the expected results
by setting P_FLAG to act like 'whence -p'. E.g., as of now,
command -xv printf
is equivalent to
whence -p printf
but note that 'whence' has no equivalent of 'command -pvx printf'
which searches $(getconf PATH) for a command.
- When -x will run a command, now set the new SH_XARG state flag.
src/cmd/ksh93/sh/xec.c: sh_exec():
- Change to using the new SH_XARG state bit.
- Skip the check for built-ins if SH_XARG is active, so that
'command -x' now always runs an external command.
src/lib/libcmd/date.c, src/lib/libcmd/uname.c:
- These path-bound builtins sometimes need to run the external
system command by the same name, but they did that by hardcoding
an unportable direct path. Now that 'command -x' runs an external
command, change this to using 'command -px' to guarantee using
the known-good external system utility in the default PATH.
- In date.c, fix the format string passed to 'command -px date'
when setting the date; it was only compatible with BSD systems.
Use the POSIX variant on non-BSD systems.
2021-01-30 05:51:22 +00:00
|
|
|
maximising the efficiency of 'command -x' for the system it runs on.
|
|
|
|
|
|
Fix field splitting bug triggered by DEBUG trap
An unquoted variable expansion evaluated in a DEBUG trap action
caused IFS field splitting to be deactivated in code executed after
the trap action. Thanks to Koichi Nakashima for the reproducer:
| v=''
| trap ': $v' DEBUG
| A="a b c"
| set -- $A
| printf '%s\n' "$@"
|
| Expected
|
| a
| b
| c
|
| Actual
|
| a b c
src/cmd/ksh93/sh/fault.c: sh_trap():
- Remove incorrect save/restore of sh.ifstable, the internal state
table for field splitting. This reverts three lines added in ksh
93t+ 2009-11-30. Analysis: As an expansion is split into fields
(macro.c, lines 2367-2471), sh.ifstable is modified. If that
happens within a DEBUG trap, any modifications in ifstable are
undone by the restoring memccpy, leaving an inconsistent state.
src/cmd/ksh93/COMPATIBILITY:
- Document the DEBUG trap fixes, particularly the incorrect
inheritance by subshells and functions that some scripts may now
rely on because this bug is so longstanding. (re: 2a835a2d)
src/cmd/ksh93/tests/basic.sh:
- Add relevant tests.
Resolves: https://github.com/ksh93/ksh/issues/155
TODO: add a -T (-o functrace) option as in bash, which should allow
subshells and ksh-style functions to inherit DEBUG traps.
P.S.: The very handy multishell repo allows us to use 'git blame'
to trace the origin of the recently fixed DEBUG trap bugs.
The off-by-one error causing various bugs, reverted in 2a835a2d,
was introduced in ksh 93t 2008-07-25:
https://github.com/multishell/ksh93/commit/8e947ccf
(fault.c, line 321)
The incorrect check causing the exit status bug, reverted in
d00b4b39, was introduced in ksh 93t 2008-11-04:
https://github.com/multishell/ksh93/commit/b1ade268
(fault.c, line 459)
The ifstable save/restore causing the field splitting bug, reverted
in this commit, was introduced in ksh 93t+ 2009-11-30:
https://github.com/multishell/ksh93/commit/53d9f009
(fault.c, lines 440, 444, 482)
So all the bugs reported in #155 were fixed by simply reverting
these specific changes. I think that they are some experiments that
the developers simply forgot to remove. I've suspected such a thing
multiple times before. ksh93 was developed by researchers who were
genius innovators, but incredibly sloppy maintainers.
2021-01-24 15:46:46 +00:00
|
|
|
2021-01-24:
|
|
|
|
|
|
2021-01-24 22:45:08 +00:00
|
|
|
- Fixed a bug in 'typeset': combining the -u option with -F or -E caused the
|
|
|
|
|
variable to become a hexadecimal floating point in error.
|
|
|
|
|
|
Fix field splitting bug triggered by DEBUG trap
An unquoted variable expansion evaluated in a DEBUG trap action
caused IFS field splitting to be deactivated in code executed after
the trap action. Thanks to Koichi Nakashima for the reproducer:
| v=''
| trap ': $v' DEBUG
| A="a b c"
| set -- $A
| printf '%s\n' "$@"
|
| Expected
|
| a
| b
| c
|
| Actual
|
| a b c
src/cmd/ksh93/sh/fault.c: sh_trap():
- Remove incorrect save/restore of sh.ifstable, the internal state
table for field splitting. This reverts three lines added in ksh
93t+ 2009-11-30. Analysis: As an expansion is split into fields
(macro.c, lines 2367-2471), sh.ifstable is modified. If that
happens within a DEBUG trap, any modifications in ifstable are
undone by the restoring memccpy, leaving an inconsistent state.
src/cmd/ksh93/COMPATIBILITY:
- Document the DEBUG trap fixes, particularly the incorrect
inheritance by subshells and functions that some scripts may now
rely on because this bug is so longstanding. (re: 2a835a2d)
src/cmd/ksh93/tests/basic.sh:
- Add relevant tests.
Resolves: https://github.com/ksh93/ksh/issues/155
TODO: add a -T (-o functrace) option as in bash, which should allow
subshells and ksh-style functions to inherit DEBUG traps.
P.S.: The very handy multishell repo allows us to use 'git blame'
to trace the origin of the recently fixed DEBUG trap bugs.
The off-by-one error causing various bugs, reverted in 2a835a2d,
was introduced in ksh 93t 2008-07-25:
https://github.com/multishell/ksh93/commit/8e947ccf
(fault.c, line 321)
The incorrect check causing the exit status bug, reverted in
d00b4b39, was introduced in ksh 93t 2008-11-04:
https://github.com/multishell/ksh93/commit/b1ade268
(fault.c, line 459)
The ifstable save/restore causing the field splitting bug, reverted
in this commit, was introduced in ksh 93t+ 2009-11-30:
https://github.com/multishell/ksh93/commit/53d9f009
(fault.c, lines 440, 444, 482)
So all the bugs reported in #155 were fixed by simply reverting
these specific changes. I think that they are some experiments that
the developers simply forgot to remove. I've suspected such a thing
multiple times before. ksh93 was developed by researchers who were
genius innovators, but incredibly sloppy maintainers.
2021-01-24 15:46:46 +00:00
|
|
|
- Fixed: an unquoted variable expansion evaluated in a DEBUG trap action caused
|
|
|
|
|
IFS field splitting to be deactivated in code executed after the trap action.
|
|
|
|
|
This bug was introduced in ksh 93t+ 2009-11-30.
|
|
|
|
|
|
2021-01-24 00:49:36 +00:00
|
|
|
2021-01-23:
|
|
|
|
|
|
|
|
|
|
- Fixed: when the DEBUG trap was redefined in a subshell, the DEBUG trap in
|
|
|
|
|
the parent environment was corrupted or the shell crashed.
|
2021-01-24 03:51:00 +00:00
|
|
|
When a redirection was used in a DEBUG trap action, the trap was disabled.
|
Fix field splitting bug triggered by DEBUG trap
An unquoted variable expansion evaluated in a DEBUG trap action
caused IFS field splitting to be deactivated in code executed after
the trap action. Thanks to Koichi Nakashima for the reproducer:
| v=''
| trap ': $v' DEBUG
| A="a b c"
| set -- $A
| printf '%s\n' "$@"
|
| Expected
|
| a
| b
| c
|
| Actual
|
| a b c
src/cmd/ksh93/sh/fault.c: sh_trap():
- Remove incorrect save/restore of sh.ifstable, the internal state
table for field splitting. This reverts three lines added in ksh
93t+ 2009-11-30. Analysis: As an expansion is split into fields
(macro.c, lines 2367-2471), sh.ifstable is modified. If that
happens within a DEBUG trap, any modifications in ifstable are
undone by the restoring memccpy, leaving an inconsistent state.
src/cmd/ksh93/COMPATIBILITY:
- Document the DEBUG trap fixes, particularly the incorrect
inheritance by subshells and functions that some scripts may now
rely on because this bug is so longstanding. (re: 2a835a2d)
src/cmd/ksh93/tests/basic.sh:
- Add relevant tests.
Resolves: https://github.com/ksh93/ksh/issues/155
TODO: add a -T (-o functrace) option as in bash, which should allow
subshells and ksh-style functions to inherit DEBUG traps.
P.S.: The very handy multishell repo allows us to use 'git blame'
to trace the origin of the recently fixed DEBUG trap bugs.
The off-by-one error causing various bugs, reverted in 2a835a2d,
was introduced in ksh 93t 2008-07-25:
https://github.com/multishell/ksh93/commit/8e947ccf
(fault.c, line 321)
The incorrect check causing the exit status bug, reverted in
d00b4b39, was introduced in ksh 93t 2008-11-04:
https://github.com/multishell/ksh93/commit/b1ade268
(fault.c, line 459)
The ifstable save/restore causing the field splitting bug, reverted
in this commit, was introduced in ksh 93t+ 2009-11-30:
https://github.com/multishell/ksh93/commit/53d9f009
(fault.c, lines 440, 444, 482)
So all the bugs reported in #155 were fixed by simply reverting
these specific changes. I think that they are some experiments that
the developers simply forgot to remove. I've suspected such a thing
multiple times before. ksh93 was developed by researchers who were
genius innovators, but incredibly sloppy maintainers.
2021-01-24 15:46:46 +00:00
|
|
|
DEBUG traps were also incorrectly inherited by subshells and ksh functions.
|
2022-01-30 20:09:14 +00:00
|
|
|
All this was caused by a bug introduced in ksh 93t 2008-06-24.
|
2021-01-24 00:49:36 +00:00
|
|
|
|
Build system: make SHOPT_* editable again; allow indenting Mamfiles
The build system is adapted to make SHOPT_* compile-time options
editable without nmake. We can now easily change ksh's compile-time
options by editing src/cmd/ksh93/SHOPT.sh. The bin/package script
is adapted to turn these into compile flags. This resolves the most
important drawback of not using nmake.
Also, mamake now has support for indented Mam (Make Abstract
Machine) code. Only one type of block (make...done) is supported in
Mamfiles, so they are easy to indent automatically. A script to
(re)do this is included.
Since nmake is not going to be restored (it has too many problems
that no one is interested in fixing), this at least makes mamake
significantly easier to work with.
The Makefiles are deleted. They may still be handy for reference to
understand the Mamfiles, but they haven't actually matched the
Mamfiles for a while -- and you can still look in the git history.
Deleting them requires some adaptations to bin/package and mamake.c
because, even though they do not use those files, they still looked
for them to decide whether to build code in a directory.
Finally, this commit incorporates some #pragmas for clang to
suppress annoying warnings about the coding style used in this
historic code base. (gcc does not complain so much.)
src/cmd/ksh93/SHOPT.sh:
- Added.
bin/package, src/cmd/INIT/package.sh:
- cd into our own directory in case we were run from another dir.
- $makefiles: only look for Mamfiles.
- Add ksh compile-options via KSH_SHOPTFLAGS. Include SHOPT.sh.
- make_recurse(): Do not write a missing Makefile.
- finalize environment: Look for Mamfiles instead of Makefiles.
src/cmd/INIT/mamake.c:
- Tell clang to suppress annoying warnings about coding style.
- Update version string and self-documentation.
- input(): Add support for indented Mam code by skipping initial
whitespace on each input line.
- files[]: Instead of looking for various of Makefiles to decide
where to build, only look for Mamfiles.
src/Makefile, src/cmd/INIT/Makefile, src/cmd/Makefile,
src/cmd/builtin/Makefile, src/cmd/ksh93/Makefile, src/lib/Makefile,
src/lib/libast/Makefile, src/lib/libcmd/Makefile,
src/lib/libdll/Makefile, src/lib/libsum/Makefile:
- Removed.
src/Mamfile, src/cmd/INIT/Mamfile, src/cmd/Mamfile,
src/cmd/builtin/Mamfile, src/cmd/ksh93/Mamfile, src/lib/Mamfile,
src/lib/libast/Mamfile, src/lib/libcmd/Mamfile,
src/lib/libdll/Mamfile, src/lib/libsum/Mamfile:
- Indent the code with tabs.
- In ksh93/Mamfile, add ${KSH_SHOPT_FLAGS} to every $CC command.
- In ksh93/Mamfile, add "prev SHOPT.sh" for every *.o file
so they are rebuilt whenever SHOPT.sh changes.
bin/Mamfile_indent:
- Added, in case someone wants to re-indent a Mamfile.
src/cmd/INIT/proto.c, src/cmd/INIT/ratz.c, src/cmd/INIT/release.c,
src/lib/libast/features/common, src/lib/libast/include/ast.h:
- Tell clang to suppress annoying warnings about coding style that
it disapproves of (mainly concerning the use of parentheses).
src/cmd/INIT/cc.darwin, src/cmd/INIT/cc.freebsd,
src/cmd/INIT/cc.openbsd:
- Remove now-redundant clang warning suppression flags.
Resolves: https://github.com/ksh93/ksh/issues/60
2021-01-22 23:23:14 +00:00
|
|
|
2021-01-22:
|
|
|
|
|
|
|
|
|
|
- Compile-time shell options can now be edited in src/cmd/ksh93/SHOPT.sh
|
|
|
|
|
before building.
|
|
|
|
|
|
2021-01-20 17:40:09 +00:00
|
|
|
2021-01-20:
|
|
|
|
|
|
|
|
|
|
- Fixed: executing a DEBUG trap in a command substitution had side effects
|
|
|
|
|
on the exit status ($?) of non-trap commands.
|
2022-01-30 20:09:14 +00:00
|
|
|
This bug was introduced in ksh 93t 2008-09-21.
|
2021-01-20 17:40:09 +00:00
|
|
|
|
typeset: add error msgs for incompatible options; improve usage msg
This adds informative error messages if incompatible options are
given. It also documents the exclusive -m, -n and -T options on
separate usage lines, as was already done with -f. The usage
message for incompatible options now looks something like this:
| $ ksh -c 'typeset -L10 -F -f -i foo'
| ksh: typeset: -i/-F/-E/-X cannot be used with -L/-R/-Z
| ksh: typeset: -f cannot be used with other options
| Usage: typeset [-bflmnprstuxACHS] [-a[type]] [-i[base]] [-E[n]]
| [-F[n]] [-L[n]] [-M[mapping]] [-R[n]] [-X[n]]
| [-h string] [-T[tname]] [-Z[n]] [name[=value]...]
| Or: typeset -f [name...]
| Or: typeset -m [name=name...]
| Or: typeset -n [name=name...]
| Or: typeset -T [tname[=(type definition)]...]
| Help: typeset [ --help | --man ] 2>&1
(see also the previous commit, e21a053e)
Unfortunately the first "Usage" line has some redundancies with the
"Or:" lines showing separate usages. It doesn't seem to be possible
to avoid this; it's a flaw in how libast generates everything
(usage, help, manual) from one huge getopt(3) string. I still think
the three added "Or:" lines are an improvement as it wasn't
previously shown that these options need to be used on their own.
src/cmd/ksh93/bltins/typeset.c: b_typeset():
- Instead of only showing a generic usage message, add an
informative error message if incompatible options were given.
- Conflicting options detection was failing because NV_LJUST and
NV_EXPNOTE have the same bitmask value. Use a new 'isadjust'
flag for -L/-R/-Z to remember if one of these was set.
- Detect conflict between -L/-R/-Z and a float option, not just -i.
src/cmd/ksh93/include/name.h, src/cmd/ksh93/data/msg.c:
- Add the two new error messages for incompatible options.
src/cmd/ksh93/data/builtins.c: sh_opttypeset[]:
- Add a space after 'float' in in "[+float?\btypeset -lE\b]" as
this makes 'float' appear on its own line, improving formatting.
- Show -m, -n, -T on separate usage lines like -f, as none of these
can be combined with other options.
- Remove "cannot be combined with other options" from -m and -n
descriptions, as that should now be clear from the separate usage
lines -- and even if not, the error message is now informative.
src/cmd/ksh93/sh.1, src/cmd/ksh93/COMPATIBILITY:
- Update.
src/cmd/ksh93/tests/types.sh:
- Remove obsolete test: 'typeset -RF' is no longer accepted.
(It crashed in 93u+, so this is not an incompatibility...)
Resolves: https://github.com/ksh93/ksh/issues/48
2021-01-21 00:24:13 +00:00
|
|
|
- The typeset builtin command now gives an informative error message if an
|
|
|
|
|
incompatible combination of options is given.
|
|
|
|
|
|
2021-01-19 18:47:41 +00:00
|
|
|
2021-01-19:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash when using 'cd' in a virtual/non-forking subshell in a
|
|
|
|
|
situation where the current working directory cannot be determined.
|
|
|
|
|
|
2021-01-08 22:15:48 +00:00
|
|
|
2021-01-08:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash on exceeding the maximum size of the $PS1 prompt.
|
|
|
|
|
The maximum size is also increased from 160 to 256 bytes.
|
|
|
|
|
|
2021-01-07 17:34:47 +00:00
|
|
|
2021-01-07:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash that could occur while ksh updated ${.sh.match}.
|
|
|
|
|
|
Implement hash tables for virtual subshells (re: 102868f8, 9d428f8f)
The forking fix implemented in 102868f8 and 9d428f8f, which stops
the main shell's hash table from being cleared if PATH is changed
in a subshell, can cause a significant performance penalty for
certain scripts that do something like
( PATH=... command foo )
in a subshell, especially if done repeatedly. This is because the
hash table is cleared (and hence a subshell forks) even for
temporary PATH assignments preceding commands.
It also just plain doesn't work. For instance:
$ hash -r; (ls) >/dev/null; hash
ls=/bin/ls
Simply running an external command in a subshell caches the path in
the hash table that is shared with a main shell. To remedy this, we
would have to fork the subshell before forking any external
command. And that would be an unacceptable performance regression.
Virtual subshells do not need to fork when changing PATH if they
get their own hash tables. This commit adds these. The code for
alias subshell trees (which was removed in ec888867 because they
were broken and unneeded) provided the beginning of a template for
their implementation.
src/cmd/ksh93/sh/subshell.c:
- struct subshell: Add strack pointer to subshell hash table.
- Add sh_subtracktree(): return pointer to subshell hash table.
- sh_subfuntree(): Refactor a bit for legibility.
- sh_subshell(): Add code for cleaning up subshell hash table.
src/cmd/ksh93/sh/name.c:
- nv_putval(): Remove code to fork a subshell upon resetting PATH.
- nv_rehash(): When in a subshell, invalidate a hash table entry
for a subshell by creating the subshell scope if needed, then
giving that entry the NV_NOALIAS attribute to invalidate it.
src/cmd/ksh93/sh/path.c: path_search():
- To set a tracked alias/hash table entry, use sh_subtracktree()
and pass the HASH_NOSCOPE flag to nv_search() so that any new
entries are added to the current subshell table (if any) and do
not influence any parent scopes.
src/cmd/ksh93/bltins/typeset.c: b_alias():
- b_alias(): For hash table entries, use sh_subtracktree() instead
of forking a subshell. Keep forking for normal aliases.
- setall(): To set a tracked alias/hash table entry, pass the
HASH_NOSCOPE flag to nv_search() so that any new entries are
added to the current subshell table (if any) and do not influence
any parent scopes.
src/cmd/ksh93/sh/init.c: put_restricted():
- Update code for clearing the hash table (when changing $PATH) to
use sh_subtracktree().
src/cmd/ksh93/bltins/cd_pwd.c:
- When invalidating path name bindings to relative paths, use the
subshell hash tree if applicable by calling sh_subtracktree().
- rehash(): Call nv_rehash() instead of _nv_unset()ting the hash
table entry; this is needed to work correctly in subshells.
src/cmd/ksh93/tests/leaks.sh:
- Add leak tests for various PATH-related operations in the main
shell and in a virtual subshell.
- Several pre-existing memory leaks are exposed by the new tests
(I've confirmed these in 93u+). The tests are disabled and marked
TODO for now, as these bugs have not yet been fixed.
src/cmd/ksh93/tests/subshell.sh:
- Update.
Resolves: https://github.com/ksh93/ksh/issues/66
2021-01-07 21:18:33 +00:00
|
|
|
- Any changes to the hash table (a.k.a. "tracked aliases", i.e. cached $PATH
|
|
|
|
|
searches) in a subshell now no longer affect the parent shell's hash table.
|
|
|
|
|
|
2021-01-05 04:55:07 +00:00
|
|
|
2021-01-05:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug in 'cd' that caused 'cd ./foo' to search for 'foo' in $CDPATH.
|
|
|
|
|
|
2021-01-03 23:54:36 +00:00
|
|
|
2021-01-03:
|
|
|
|
|
|
|
|
|
|
- The invocation
|
|
|
|
|
$ ksh +s
|
|
|
|
|
caused an infinite loop and corrupted ~/.sh_history. This is now fixed so
|
|
|
|
|
that the '-s' option is automatically turned on if there are no non-option
|
|
|
|
|
command arguments, as documented in Bolsky & Korn (1995), p. 261.
|
|
|
|
|
|
2020-11-26 13:50:30 +00:00
|
|
|
2020-10-22:
|
|
|
|
|
|
|
|
|
|
- Fixed: 'typeset -F0', 'typeset -E0', and 'typeset -X0' floating point
|
|
|
|
|
numerics having a precision of 0 with variable assignment.
|
|
|
|
|
'typeset -F0 x; x=4.56' worked but not 'typeset -F0 x=4.56'.
|
|
|
|
|
|
2020-11-26 13:30:24 +00:00
|
|
|
2020-10-21:
|
|
|
|
|
|
|
|
|
|
- Fixed: More concisely correct the exporting of uppercase and lowercase
|
|
|
|
|
variables when only the export and change case attributes were applied.
|
|
|
|
|
This fix improves upon the previous 2020-09-30 modifications.
|
|
|
|
|
|
Mitigate PWD race condition in non-forking subshells
Virtual/non-forking subshells that change the present working
directory (PWD) with 'cd' suffer from a serious race condition. The
PWD is changed within the same process. This means it may not be
possible to change back to the original PWD when exiting the
subshell, as some other process may destroy the PWD or modify its
permissions in the meantime. ksh did not handle this error
condition at all, so, after exiting a subshell that invoked 'cd',
it could silently end up running the script's following command(s)
in the wrong directory. Which might be 'rm -rf *'. So, ouch.
The proper and obvious fix is never to allow a virtual subshell to
change the PWD, as it can never be guaranteed you can return to a
previous directory. If the PWD is changed in a child process, there
is no need to restore it in the parent process, and this whole
problem is avoided. So subshells really should always fork on
encountering a 'cd' command.
But forking is slow. It is not uncommon for scripts to 'cd' in a
subshell that is run repeatedly in a loop.
There is also the issue of custom builtins that can be added to ksh
via shared libraries. In the standard shell language, 'cd' is the
only command that changes the PWD, so we could just make that
command fork the subshell it is run from. But there's no telling
what a custom builtin might do.
So this commit implements a compromise that will not affect
performance unless there is the pathological condition of a PWD
that has been rendered inaccessible in some way:
1. When entering a virtual subshell, if the parent shell's PWD
proves inaccessible upon saving it, the subshell will now fork into
a separate process, avoiding the unrestorable PWD problem.
2. If some attack renders the parent shell's PWD unrestorable
*after* ksh enters a virtual subshell, ksh will now error out when
exiting it. There is nothing else left to do then. Continuing would
mean running arbitrary commands in the wrong PWD.
src/cmd/ksh93/sh/subshell.c:
- Put all the code/variables only needed for fchdir() behind '#if
_lib_fchdir'. This makes it clearer what's what.
(I don't know if there is still any system out there without
fchdir(3); I haven't found any. The chdir(3) fallback version may
be removed later as there is no way to make it remotely secure.)
- Fix the attempt to use the O_PATH mode for open(2) as a fallback
for nonexistent O_SEARCH on Linux. Define _GNU_SOURCE on Linux,
or <fcntl.h> (which is included indirectly) won't define O_PATH.
- Fix use of O_SEARCH. The code was simply wrong, repeating an
open(".",O_RDONLY) instead. Since a nonexistent O_SEARCH is now
redefined as either O_PATH or O_RDONLY, we can simply
open(".",O_SEARCH) and be done with it.
- Fix fatal error handling. Introduce fatal error condition for
failure to fchdir(3) back to the parent's PWD; rename 'duped' to
'fatalerror' and use it for error numbers; save and restore errno
on fatal error so the message will report the cause. (We must
call errormsg() near the end of sh_subshell() to avoid crashes.)
- If open(".",O_SEARCH) was not able get a file descriptor to our
PWD on entry, then call sh_subfork() immediately before running
the subshell commands. (Forking earlier causes a crash.)
- When restoring the PWD, if fchdir(3) fails, do *not* fall back to
chdir(3). We already know the PWD is inaccessible, so if chdir(3)
"succeeds" then, it's very likely to be a substitute injected by
an attacker.
src/cmd/ksh93/bltins/cd_pwd.c:
- If we don't have fchdir(3), then sh_subshell() must fall back to
chdir(2) to restore the PWD. That is highly vulnerable, as a
well-timed rename would allow an attacker to usurp the PWD. We
can't do anything about that if some custom builtin changes the
PWD, but we can at least make 'cd' always fork a subshell, which
slows down ksh but removes the need for the parent shell ever to
restore the PWD. (There is certainly no popular system where this
is relevant and there might not be any such current system.)
This commit adds no regression test because a portable regression
test is not really doable. Different kernels, external /bin/pwd
utilities, etc. all have quite different behaviour under the
pathological condition of an inaccessible PWD, so both the
before-fix and the after-fix behaviour differs. See link below.
Resolves: https://github.com/ksh93/ksh/issues/141
Thanks to Stéphane Chazelas for the bug report.
2020-10-06 21:13:41 +00:00
|
|
|
2020-10-06:
|
|
|
|
|
|
|
|
|
|
- The security of virtual/non-forking subshells that locally change the present
|
|
|
|
|
working directory (PWD) using 'cd' has been improved in two ways.
|
|
|
|
|
1. On entering a subshell, if the parent shell's PWD proves inaccessible upon
|
|
|
|
|
saving it, the subshell will now fork into a separate process so the
|
|
|
|
|
parent process never changes its PWD, avoiding the need to restore it.
|
|
|
|
|
2. If some attack renders the parent shell's PWD unrestorable *after* ksh
|
|
|
|
|
enters a virtual subshell, ksh will now error out on exiting it, as
|
|
|
|
|
continuing would mean running arbitrary commands in the wrong PWD.
|
|
|
|
|
Hopefully this is an acceptable compromise between performance and security.
|
|
|
|
|
The proper fix would be to always fork a subshell when changing the working
|
|
|
|
|
directory within it, but the resulting slowdown would likely be unpopular.
|
|
|
|
|
|
2020-09-29 22:34:02 +00:00
|
|
|
2020-09-30:
|
|
|
|
|
|
|
|
|
|
- Fixed: 'typeset -xu' and 'typeset -xl' (export + change case) failed to
|
|
|
|
|
change the case of a variable's value in certain conditions.
|
|
|
|
|
|
Fix pipefail with (errexit or ERR trap) regression
ksh 93u+ introduced a regression in the combination of the
'set -o pipefail' and 'set -e'/'set -o errexit' options:
$ ksh93 -o errexit -o pipefail -c \
'(exit 3) | true; echo "still here despite $? status"'
still here despite 3 status
The bug is in how the the huge sh_exec() function in xec.c handles
the 'echeck' flag. Near the end of sh_exec(), this flag triggers a
sh_chktrap() call to check whether to trigger any traps, including
the ERR trap -- and that same function also handles the errexit
option, which is basically the same as 'trap "exit" ERR'.
We can learn more easily how sh_exec() works by inserting debug
warnings in all its 'switch(type&COMMSK)' cases, like:
case TCOM:
errormsg(SH_DICT,ERROR_warn(0),"[DEBUG] TCOM");
... and same for all the others. With that done, the output
of a very simple dummy pipeline looks as follows:
$ arch/*/bin/ksh -c 'true | true | true'
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TFIL
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TFORK
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TFORK
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TSETIO
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TCOM
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TCOM
arch/darwin.i386-64/bin/ksh: warning: [DEBUG] TCOM
So, it looks like sh_exec() handles this pipeline as follows:
TFIL
|_____TFORK
| |_____TCOM
|_____TFORK
| |_____TCOM
|_____TSETIO
|_____TCOM
Each time a pipeline like command1 | command2 | ... is executed,
sh_exec() is invoked with type TFIL; this then recursively invokes
sh_exec() to handle the individual elements. The last element of
the pipe triggers a sh_exec() run with type TSETIO; since it is run
in the current shell environment, it is effectively treated as a
command with an input redirection. All the previous elements are of
type TFORK instead, because they are executed asynchronously in
separate, forked subshell processes. Finally, the TFORK or TSETIO
code then recursively calls sh_exec() again with type TCOM to
actually execute the commands.
When reading the code, we find that the 'echeck' flag is set as
part of the TSETIO code. This makes sense of why only an error in
the last element of the pipe triggers the errexit/ERR trap action.
So that's the bug: the flag is set in the wrong place.
This can be fixed by setting that flag in the TFIL handling code
instead, as this is what calls everything else and collects all the
exit statuses. So the sh_chktrap() call is now executed after
handling the entire pipeline, at the TFIL recursion level.
This also allows getting rid of the special-casing in the buggy
TSETIO version. The SH_ERREXIT state is restored at the end of each
sh_exec() call, so since we're now doing this at a lower recursion
level, it will already have been restored.
src/cmd/ksh93/sh/xec.c: sh_exec():
- Fix the bug as per the above.
src/cmd/ksh93/tests/options.sh:
- Add tests for errexit and ERR trap combined with pipefail.
src/cmd/ksh93/tests/basic.sh:
- Tweak a couple of tests that reported a trap wasn't triggered
even if it was actually triggered more than once.
Fixes: https://github.com/ksh93/ksh/issues/121
Thanks to Stéphane Chazelas for the bug report.
2020-09-30 14:13:34 +00:00
|
|
|
- A ksh 93u+ regression was fixed in the combination of ERR trap handling and
|
|
|
|
|
the 'pipefail' option. A pipeline now triggers the ERR trap correctly again
|
|
|
|
|
if the 'pipefail' option is active and any of the pipeline elements return a
|
|
|
|
|
nonzero exit status. Similarly, if both the 'errexit' and 'pipefail' options
|
|
|
|
|
are active, ksh now correctly exits if any pipeline element returns nonzero.
|
|
|
|
|
|
2020-10-01 04:13:00 +00:00
|
|
|
- Autoloading a function no longer causes the calling script's $LINENO to be
|
|
|
|
|
off by the number of lines in the function definition file that was loaded.
|
|
|
|
|
This also corrects line numbers in warnings and error messages.
|
|
|
|
|
|
Fix signal/trap behaviour in ksh functions (rhbz#1454804)
Prior discussion:
https://bugzilla.redhat.com/1454804
On 2017-05-23 13:33:25 UTC, Paulo Andrade wrote:
> In previous ksh versions, when exiting the scope of a ksh
> (not posix) function, it would restore the trap table of
> the "calling context" and if the reason the function exited
> was a signal, it would call sh_fault() passing as argument
> the signal value.
> Newer ksh checks it, but calls kill(getpid(), signal_number)
> after restoring the trap table, but only calls for SIGINT and
> SIGQUIT.
[...]
> The old way appears to have been more appropriate, but there
> must be a reason to only pass SIGINT and SIGQUIT as it is an
> explicit patch.
The last paragraph is where I differ. This would not be the first
example of outright breakage that appeared to be added deliberately
and that 93u+m has fixed or removed, see e.g. 8477d2ce ('printf %H'
had code that deleted all multibyte characters), cefe087d, or
781f0a39. Sometimes it seems the developers added a little
experiment and then forgot all about it, so it became a misfeature.
In this instance, the correct pre-2012 ksh behaviour is still
explicitly documented in (k)sh.1: "A trap condition that is not
caught or ignored by the function causes the function to terminate
and the condition to be passed on to the caller". Meaning, if there
is no function-local trap, the signal defaults to the parent scope.
There is no language that limits this to SIGINT and SIGQUIT only.
It also makes no sense at all to do so -- signals such as SIGPIPE,
SIGTERM, or SIGSEGV need to be caught by default and to do
otherwise results in misbehaviour by default.
src/cmd/ksh93/sh/xec.c: sh_funscope():
- When resending a signal after restoring the global traps state,
remove the spurious check that limits this to SIGINT and SIGQUIT.
- Replace it with a check for nsig!=0, as that means there were
parent trap states to restore. Otherwise 'kill' may be called
with an invalid signal argument, causing a crash on macOS.
src/cmd/ksh93/tests/signal.sh:
- Update a test to check that a function-local SIGTERM trap is
triggered correctly when signalled from another process.
- Complete the tests for 3aee10d7; this bug needed fixing before
we could test that previous fix in a ksh function scope.
- Add a test for triggering global traps from ksh functions,
testing multiple POSIX-standard signals.
2020-09-29 00:28:08 +00:00
|
|
|
2020-09-28:
|
|
|
|
|
|
|
|
|
|
- While executing a ksh-style function, ksh 93u+ ignored all signals for which
|
|
|
|
|
the function had not set a local trap, except for SIGINT and SIGQUIT. This
|
|
|
|
|
was contrary to the manual, which states that a "trap condition that is not
|
|
|
|
|
caught or ignored by the function causes the function to terminate and the
|
|
|
|
|
condition to be passed on to the caller". This has now been fixed in 93u+m to
|
|
|
|
|
match the documentation, so that e.g. global traps work as expected again.
|
|
|
|
|
|
2020-09-27 19:26:09 +00:00
|
|
|
2020-09-27:
|
|
|
|
|
|
2020-09-28 02:47:53 +00:00
|
|
|
- The shell's lexical analysis of a 'case' statement within a do...done block
|
2020-09-27 19:26:09 +00:00
|
|
|
within a command substitution of the form $(...) has been fixed so that code
|
|
|
|
|
like the following no longer throws a spurious syntax error:
|
|
|
|
|
x=$(for i in 1; do case $i in word) true;; esac; done)
|
|
|
|
|
Previously, this required a leading parenthesis before 'word', although the
|
|
|
|
|
syntax error claimed that the ';;' was unexpected.
|
|
|
|
|
|
2020-09-26 18:26:18 +00:00
|
|
|
2020-09-26:
|
|
|
|
|
|
|
|
|
|
- 'whence -f' now completely ignores the existence of functions, as documented.
|
|
|
|
|
|
2020-09-26 18:28:55 +00:00
|
|
|
- ksh now does not import environment variables whose names are not valid in
|
|
|
|
|
the shell language, as it would be impossible to change or unset them.
|
|
|
|
|
However, they stay in the environment to be passed to child processes.
|
|
|
|
|
|
Fix argv rewrite on invoking hashbangless script (rhbz#1047506)
The fixargs() function is invoked when ksh needs to run a script
without a #!/hashbang/path. Instead of letting the kernel invoke a
shell, ksh exfile()s the script itself from sh_main(). In the
forked child, it calls fixargs() to set the argument list in the
environment to the args of the new script, so that 'ps' and
/proc/PID/cmdline show the expected output.
But fixargs() is broken because, on systems other than HP-UX (on
which ksh uses pstat(2)), ksh simply inserts a terminating zero.
The arguments list is not a zero-terminated C string. Unix systems
expect the entire arguments buffer to be zeroed out, otherwise 'ps'
and /proc/*/cmdline will have fragments of previous command lines
in the output.
The Red Hat patch for this bug is:
https://src.fedoraproject.org/rpms/ksh/blob/642af4d6/f/ksh-20120801-argvfix.patch
However, that fix is incomplete because 'command_len' was also
hardcoded to be limited to 64 characters (!), which still gave
invalid 'ps' output if the erased command line was longer.
src/cmd/ksh93/sh/main.c: fixargs():
- Remove CMD_LENGTH macro which was defined as 64.
- Remove code that limited the erasure of the arguments buffer to
CMD_LENGTH characters. That code also had quite a dodgy strdup()
call -- it copies arguments to the heap, but they are never freed
(or even used), so it's a memory leak. Also, none of this is
ever done if the length is calculated using pstat(2) on HP-UX,
which is a clear indication that it's unnecessary.
(I think this code block must have been some experiment they
forgot to remove. One reason why I think so is that a 64 byte
arguments limit never made sense, even in the 1980s when they
wrote ksh on 80-column CRT displays. Another indication of this
is that fixing it didn't require adding anything; the code to do
the right thing was already there, it was just being overridden.)
- Zero out the full arguments length as in the Red Hat patch.
src/cmd/ksh93/tests/basic.sh:
- Add test. It's sort of involved because 'ps' is one of the least
portable commands in practice, in spite of standardisation.
2020-09-25 06:19:43 +00:00
|
|
|
2020-09-25:
|
|
|
|
|
|
whence -v/-a: report path to autoloadable functions
Since at least 1999, whence -v on pdksh (and its successor mksh)
reports the path where an autoloadable function may be found:
$ mkdir ~/fun; FPATH=~/fun
$ echo 'myfn() { echo hi; }' >~/fun/myfn
$ whence -v myfn
myfn is a undefined (autoload from /home/user/fun/myfn) function
Whereas ksh93 only reports, rather uselessly:
myfn is an undefined function
As of this commit, whence -v/-a on ksh 93u+m does the same as
pdksh, but with correct grammar:
myfn is an undefined function (autoload from /home/user/fun/myfn)
This may be a small violation of my own "no new features" policy
for 93u+m, but I couldn't resist. This omission has been annoying
me, and it's just embarrassing to lack a pdksh feature :)
src/cmd/ksh93/include/path.h,
src/cmd/ksh93/data/msg.c:
- Add e_autoloadfrom[] = " (autoload from %s)" message.
src/cmd/ksh93/bltins/whence.c: whence():
- Report the path (if any) when reporting an undefined function.
This needs to be done in two places:
1. When a function has been explicitly marked undefined with
'autoload', we need to do a quick path_search() loop to find
the path. (These undefined functions take precedence over
regular commands, so are reported first.)
2. When a function is not explicitly autoloaded but merely
available in $FPATH, that path search was already done, so all
we need to do is report it. (These are reported last.)
Note that the output remains as on 93u+ if no function definition
file is found on $FPATH. This is also like pdksh/mksh.
src/cmd/ksh93/data/builtins.c:
- Bump 'whence' version date. The inline docs never detailed very
exactly what 'whence -v' reports, so no need for further edits.
src/cmd/ksh93/tests/path.sh:
- Regress-test the new whence behaviour plus actual autoloading,
including the command override behaviour of autoloaded functions.
2020-09-25 13:39:08 +00:00
|
|
|
- whence -v/-a now reports the path to the file that an "undefined" (i.e.
|
|
|
|
|
autoloadable) function will be loaded from when invoked, if found in $FPATH.
|
|
|
|
|
|
Fix argv rewrite on invoking hashbangless script (rhbz#1047506)
The fixargs() function is invoked when ksh needs to run a script
without a #!/hashbang/path. Instead of letting the kernel invoke a
shell, ksh exfile()s the script itself from sh_main(). In the
forked child, it calls fixargs() to set the argument list in the
environment to the args of the new script, so that 'ps' and
/proc/PID/cmdline show the expected output.
But fixargs() is broken because, on systems other than HP-UX (on
which ksh uses pstat(2)), ksh simply inserts a terminating zero.
The arguments list is not a zero-terminated C string. Unix systems
expect the entire arguments buffer to be zeroed out, otherwise 'ps'
and /proc/*/cmdline will have fragments of previous command lines
in the output.
The Red Hat patch for this bug is:
https://src.fedoraproject.org/rpms/ksh/blob/642af4d6/f/ksh-20120801-argvfix.patch
However, that fix is incomplete because 'command_len' was also
hardcoded to be limited to 64 characters (!), which still gave
invalid 'ps' output if the erased command line was longer.
src/cmd/ksh93/sh/main.c: fixargs():
- Remove CMD_LENGTH macro which was defined as 64.
- Remove code that limited the erasure of the arguments buffer to
CMD_LENGTH characters. That code also had quite a dodgy strdup()
call -- it copies arguments to the heap, but they are never freed
(or even used), so it's a memory leak. Also, none of this is
ever done if the length is calculated using pstat(2) on HP-UX,
which is a clear indication that it's unnecessary.
(I think this code block must have been some experiment they
forgot to remove. One reason why I think so is that a 64 byte
arguments limit never made sense, even in the 1980s when they
wrote ksh on 80-column CRT displays. Another indication of this
is that fixing it didn't require adding anything; the code to do
the right thing was already there, it was just being overridden.)
- Zero out the full arguments length as in the Red Hat patch.
src/cmd/ksh93/tests/basic.sh:
- Add test. It's sort of involved because 'ps' is one of the least
portable commands in practice, in spite of standardisation.
2020-09-25 06:19:43 +00:00
|
|
|
- When ksh invoked a shell script that does not have a leading
|
|
|
|
|
#!/hashbang/path, 'ps' and /proc/<PID>/cmdline showed corrupted output if
|
|
|
|
|
the new script's command line was shorter than that of the invoking script.
|
|
|
|
|
This has been fixed by wiping the arguments buffer correctly.
|
|
|
|
|
|
2020-09-24 05:36:44 +00:00
|
|
|
2020-09-24:
|
|
|
|
|
|
|
|
|
|
- An omission made it impossible to turn off brace expansion within command
|
|
|
|
|
substitutions (`...`, $(...) or ${ ...; }) as the code for parsing these
|
|
|
|
|
did not check the -B/braceexpand option. This check has now been added.
|
|
|
|
|
|
2020-09-22 23:56:09 +00:00
|
|
|
2020-09-23:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash that could occur when running a pipeline containing
|
|
|
|
|
backtick-style command substitutions with job control enabled.
|
|
|
|
|
|
Fix typeset -l/-u crash on special vars (rhbz#1083713)
When using typeset -l or -u on a variable that cannot be changed
when the shell is in restricted mode, ksh crashed.
This fixed is inspired by this Red Hat fix, which is incomplete:
https://src.fedoraproject.org/rpms/ksh/blob/642af4d6/f/ksh-20120801-tpstl.patch
The crash was caused by the nv_shell() function. It walks though a
discipline function tree to get the pointer to the interpreter
associated with it. Evidently, the problem is that some pointer in
that walk is not set correctly for all special variables.
Thing is, ksh only has one shell language interpreter, and only one
global data structure (called 'sh') to keep its main state[*]. Yet,
the code is full of 'shp' pointers to that structure. Most (not
all) functions pass that pointer around to each other, accessing
that struct indirectly, ostensibly to account for the non-existent
possibility that there might be more than one interpreter state.
The "why" of that is an interesting cause for speculation that I
may get to sometime. For now, it is enough to know that, in the
code as it is, it matters not one iota what pointer to the shell
interpreter state is used; they all point to the same thing (unless
it's broken, as in this bug).
So, rather than fixing nv_shell() and/or associated pointer
assignments, this commit simply removes it, and replaces it with
calls to sh_getinterp(), which always returns a pointer to sh (see
init.c, where that function is defined as literally 'return &sh').
[*] Defined in shell.h, with the _SH_PRIVATE part in defs.h
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/name.c:
- Remove nv_shell().
src/cmd/ksh93/sh/init.c:
- In all the discipline functions for special variables, initialise
shp using sh_getinterp() instead of nv_shell().
src/cmd/ksh93/tests/variables.sh:
- Add regression test for typeset -l/-u on all special variables.
2020-09-23 17:15:45 +00:00
|
|
|
- Fixed a crash that occurred when using 'typeset -u' or 'typeset -l' on a
|
|
|
|
|
special variable such as PATH, ENV or SHELL.
|
|
|
|
|
|
2020-09-21 23:42:15 +00:00
|
|
|
2020-09-21:
|
|
|
|
|
|
|
|
|
|
- A bug was fixed that caused command substitutions embedded in here-documents
|
|
|
|
|
to lose the output of the commands they ran. This bug occurred when ksh was
|
|
|
|
|
compiled with the SHOPT_SPAWN compile-time option.
|
|
|
|
|
|
2022-03-01 19:38:15 +00:00
|
|
|
- Bugfix: var=$(< file) now reads the file even if the standard input, standard
|
2020-09-22 00:45:59 +00:00
|
|
|
output and/or standard error file descriptors are closed.
|
|
|
|
|
|
Multiple 'whence' and path search fixes
Hopefully this doesn't introduce new bugs, but it does fix at
least the following:
1. When whence -v/-a found an "undefined" (i.e. autoloadable)
function in $FPATH, it actually loaded the function as a side
effect of reporting on its existence (!). Now it only reports.
2. 'whence' will now canonicalise paths properly. Examples:
$ whence ///usr/lib/../bin//./env
/usr/bin/env
$ (cd /; whence -v dev/../usr/bin//./env)
dev/../usr/bin//./env is /usr/bin/env
3. 'whence' no longer prefixes a spurious double slash when doing
something like 'cd / && whence bin/echo'. On Cygwin, an initial
double slash denotes a network server, so this was not just a
cosmetic problem.
4. 'whence -a' now reports a "tracked alias" (a.k.a. hash table
entry, i.e. cached $PATH search) even if an actual alias by the
same name exists. This needed fixing because in fact the hash
table entry continues to be used when bypassing the alias.
Aliases and "tracked aliases" are not remotely the same thing;
confusing nomenclature is not a reason to report wrong results.
5. When using 'hash' or 'alias -t' on a command that is also a
builtin to force caching a $PATH search for the external
command, 'whence -a' double-reported the path:
$ hash printf; whence -a printf
printf is a shell builtin
printf is /usr/bin/printf
printf is a tracked alias for /usr/bin/printf
This is now fixed so that the second output line is gone.
Plus, if there were multiple versions of the command on $PATH,
the tracked alias was reported at the end, which is the wrong
order. This is also fixed.
src/cmd/ksh93/bltins/whence.c: whence():
- Refactor the do...while loop that handles whence -v/-a for path
searches in such a way that the code actually makes sense and
stops looking like higher esotericism. Just doing this fixed #2,
#4 and #5 above (the latter two before I even noticed them). For
instance, the path_fullname() call to canonicalise paths was
already there; it was just never used.
- Remove broken 'notrack' flaggery for deciding whether to report a
hash table entry a.k.a. "tracked alias"; instead, check the hash
table (shp->track_tree).
src/cmd/ksh93/sh/path.c:
- path_search(): Re #3: When prefixing the PWD, first check if
we're in '/' and if so, don't prefix it; otherwise, adding the
next slash causes an initial double slash. (Since '/' is the only
valid single-character absolute path, all we need to do is check
if the second character pwd[1] is non-null.)
- path_search(): Re #1: Stop autoloading when called by 'whence':
* The 'flag==2' check to avoid autoloading a function was
broken. The flag value is 2 on the first whence() loop
iteration, but 3 on subsequent ones. Change to 'flag >= 2'.
* However, this only fixes it if the function file does not have
the x permission bit, as executable files are handled by
path_absolute() which unconditionally autoloads functions!
So, pass on our flag parameter when callling path_absolute().
- path_absolute(): Re #1: Add flag parameter. Do not autoload
functions if flag >= 2.
src/cmd/ksh93/include/path.h,
src/cmd/ksh93/bltins/typeset.c,
src/cmd/ksh93/sh/main.c,
src/cmd/ksh93/sh/xec.c:
- Re #1: Update path_absolute() calls, adding a 0 flag parameter.
src/cmd/ksh93/include/name.h:
- Remove now-unused pathcomp member from union Value. It was
introduced in 99065353 to allow examining the value of a tracked
alias. This commit uses nv_getval() instead.
src/cmd/ksh93/tests/builtins.sh,
src/cmd/ksh93/tests/path.sh:
- Add and tweak various related tests.
Fixes: https://github.com/ksh93/ksh/issues/84
2020-09-20 05:56:09 +00:00
|
|
|
2020-09-20:
|
|
|
|
|
|
|
|
|
|
- Bugfix: when whence -v/-a found an "undefined" (i.e. autoloadable) function
|
|
|
|
|
in $FPATH, it actually loaded the function as a side effect of reporting on
|
|
|
|
|
its existence. Now it only reports, as documented.
|
|
|
|
|
|
|
|
|
|
- 'whence' will now canonicalise paths properly, resolving '.' and '..'
|
|
|
|
|
elements in paths given to it. It also no longer prefixes a spurious
|
|
|
|
|
double slash when doing something like 'cd / && whence bin/echo'.
|
|
|
|
|
|
2020-09-18 18:32:34 +00:00
|
|
|
2020-09-18:
|
|
|
|
|
|
|
|
|
|
- Setting the 'posix' option now turns off the 'braceexpand' option, as brace
|
|
|
|
|
expansion is not specified by POSIX and potentially incompatible with sh
|
|
|
|
|
scripts. In addition, 'set -o posix' now turns on the 'letoctal' option
|
|
|
|
|
instead of controlling that behaviour directly. 'set +o posix' does the
|
|
|
|
|
reverse of these.
|
|
|
|
|
|
emacs, vi: Support repeat parameters to VT220 keys (re: f2a3f4e3)
In the vi and emacs line editors, repeat count parameters can now
also be used for the arrow keys and the forward-delete key. E.g.,
in emacs mode, <ESC> 7 <left-arrow> will now move the cursor seven
positions to the left. In vi control mode, this would be entered
as: 7 <left-arrow>.
src/cmd/ksh93/edit/emacs.c:
- ed_emacsread(): Upon getting ^[ (ESC), save current repeat count
in a new variable; restore and reset it upon the next character.
- escape(): Minor bugfix: when processing a ^[[x sequence where 'x'
is a character other than '~' (which would be DEL), also reinsert
the final character into the buffer so scripts can detect them.
src/cmd/ksh93/edit/vi.c:
- cntlmode(): Do not reset the repeat count if the command is '[',
the character following ESC in VT220 escape sequences.
- mvcursor():
* Do not use getcount() to get the character following '[', as
that was parsing repetition parameters in the wrong place.
There wouldn't be any, so this would reset the repeat count.
* After that, no more need for the special-casing of ^[[3~ (DEL)
introduced in f2a3f4e3. Move it to within the 'switch' block.
* When handling left and right arrows and Home and End keys, do
not modify cursor directly but ed_ungetchar() the corresponding
traditional command keys as with the rest. Otherwise a repeat
count parameter would now wrongly survive those keys.
src/cmd/ksh93/sh.1:
- Document control character notation used for vi mode docs.
- Since vi control mode beeps and aborts on ESC except if a
subsequent [ is already in the input buffer upon receiving ESC,
document that VT220 escape sequences only preserve repeat counts
when entered into the input buffer all at once.
- Don't skip the initial ESC in the documentation of the VT220
escape sequences. In control mode, skipping the initial ESC still
works as before, but that is now undocumented, as it's really
nothing more than an artefact of VT220 escape processing.
- Move the two long paragraphs on '-o viraw' and canonical (i.e.
line-based) input processing from the vi editor introduction to
the options section under 'viraw'. It is much too arcane for the
intro, and besides, ksh 93u+ (and hence also 93u+m) has
SHOPT_VIRAW enabled by default, so the shell is compiled to force
this option on at all times, making it even less relevant for
most users.
2020-09-17 15:28:03 +00:00
|
|
|
2020-09-17:
|
|
|
|
|
|
|
|
|
|
- In the vi and emacs line editors, repeat count parameters can now also be
|
|
|
|
|
used for the arrow keys and the forward-delete key. E.g., in emacs mode,
|
|
|
|
|
<ESC> 7 <left-arrow> will now move the cursor seven positions to the left.
|
|
|
|
|
In vi control mode, this would be entered as: 7 <left-arrow>.
|
|
|
|
|
|
2020-09-18 01:03:39 +00:00
|
|
|
- When a background job on an interactive shell received SIGINT or SIGPIPE, the
|
|
|
|
|
job termination message was empty. It now shows "Interrupt" or "Broken Pipe".
|
|
|
|
|
|
A few job control (-m, -o monitor) fixes (rhbz#960034)
This patch from Red Hat fixes the following:
1. ksh was ignoring the -m (-o monitor) option when specified on
the invocation command line.
2. Scripts did not properly terminate their background processes
on Ctrl+C if the -m option was turned off. Reproducer:
xterm &
read junk
When run as a script without turning on -m, pressing Ctrl+C
should terminate the xterm, and now does.
3. Scripts no longer attempt to set the terminal foreground process
group ID, as only interactive shells should be doing that.
This makes some progress on https://github.com/ksh93/ksh/issues/119
but we're a long way from fixing all of that.
src/cmd/ksh93/sh/main.c: exfile():
- On non-interactive shells, do not turn off the monitor option.
Instead, if it was turned on, turn on the SH_MONITOR state flag.
src/cmd/ksh93/edit/edit.c: ed_getchar():
- On Ctrl+C, issue SIGINT to the current process group using
killpg(2) instead of going via sh_fault(), which handles a
signal only for the current shell process.
src/cmd/ksh93/sh/jobs.c: job_reap(), job_reset(),
src/cmd/ksh93/sh/xec.c: sh_exec():
- Only attempt to set the terminal foreground process group ID
using tcsetpgrp(3) if the shell is interactive.
Original patch: https://src.fedoraproject.org/rpms/ksh/blob/642af4d6/f/ksh-20120801-kshmfix.patch
This was applied to Red Hat's ksh 93u+ on 8 July 2013.
2020-09-18 02:42:27 +00:00
|
|
|
- The -m (-o monitor) option is no longer ignored when specified on the shell
|
|
|
|
|
invocation command line.
|
|
|
|
|
|
|
|
|
|
- A script that is interrupted with Ctrl+C now terminates its background jobs
|
|
|
|
|
as expected, unless the -m (-o monitor) option was turned on.
|
|
|
|
|
|
2020-09-14 10:15:05 +00:00
|
|
|
2020-09-14:
|
|
|
|
|
|
|
|
|
|
- Corrected rounding of floating point values by ksh's printf %f formatting
|
|
|
|
|
operator. Fix contributed by @hyenias.
|
|
|
|
|
|
Handle forward-delete key in emacs and vi editors
On every modern system, the forward-delete key on PC/Mac keyboards
generates the VT220 sequence ESC [ 3 ~. Every other shell with an
editor handles this now, ksh93 seems to be the last not to.
src/cmd/ksh93/edit/emacs.c: escape():
- Handle the ^[[3 as part of normal escape processing, then read an
extra character to check for the final '~'. If detected, insert
an ERASECHAR key event.
src/cmd/ksh93/edit/vi.c: mvcursor():
- Replace the ^[[3~ sequence by an 'x' command. We have to
special-case its processing, because vi mode parses numbers as
repetition operators. The escape sequence contains a number,
making it incompatible with normal command handling. This means
number repetitions don't work with the forward-delete key. If
that annoys anyone enough to fix it, a patch would be welcome.
For now, it will do to make the forward-delete key stop
exhibiting bizarre behaviour (beep + change case + move forward).
src/cmd/ksh93/sh.1
- Copy-edit emacs documentation for VT220-style sequences; map them
to their actual key, otherwise it's meaningless to the reader.
- Document the new forward-delete key behaviour for emacs mode.
- Leave the forward-delete key for vi mode undocumented for now, as
repetitions don't work, so it doesn't really match the vi canon.
(OTOH, it doesn't work in vim, either...)
2020-09-14 22:49:05 +00:00
|
|
|
- The forward-delete key now works as expected in emacs and vi editing modes.
|
|
|
|
|
|
Fix 'command' expansion bug and POSIX compliance
The 'command' name can now result from an expansion, e.g.:
c=command; "$c" ls
set -- command ls; "$@"
both work now. This fixes BUG_CMDEXPAN.
If -o posix is on, 'command' now disables not only the "special"
but also the "declaration" properties of builtin commands that it
invokes. This is because POSIX specifies 'command' as a simple
regular builtin, and any command name following 'command' is just
an argument to the 'command' command, so there is nothing that
allows any further arguments (such as assignment-arguments) to be
treated specially by the parser. So, if and only if -o posix is on:
a. Arguments that start with a variable name followed by '=' are
always treated as regular words subject to normal shell syntax.
b. Since assignment-arguments are not processed as assignments
before the command itself, 'command' can now stop the shell from
exiting (as required by the standard) if a command that it
invokes (such as 'export') tries to modify a readonly variable.
This fixes BUG_CMDSPEXIT.
Most of 'command' is integrated in the parser and parse tree
executer, so that is where it needed fixing.
src/cmd/ksh93/sh/parse.c: simple():
- If the posix option is on, do not skip past SYSCOMMAND so that
any declaration builtin commands that are arguments to 'command'
are not detected and thus not treated specially at parsetime.
src/cmd/ksh93/sh/xec.c: sh_exec():
- When detecting SYSCOMMAND in order to skip past it, not only
compare the Namval_t pointer 'np' to SYSCOMMAND, but also handle
the case where that pointer is NULL, as when the command name
results from an expansion. In that case, search the function tree
shp->fun_tree for the name and see if that yields the SYSCOMMAND
pointer. fun_tree is initialised with a dtview to bltin_tree, so
searching fun_tree instead allows for overriding 'command' with a
shell function (which the POSIX standard requires us to allow).
src/cmd/ksh93/sh.1,
src/cmd/ksh93/data/builtins.c:
- Update documentation to match these changes.
- Various related edits and improvements.
src/cmd/ksh93/tests/builtins.sh:
- Check that 'command' works if resulting from an expansion.
- Check that 'command' can be overridden by a shell function.
2020-09-11 07:33:29 +00:00
|
|
|
2020-09-11:
|
|
|
|
|
|
|
|
|
|
- The 'command' regular builtin utility (which runs a simple command, removing
|
|
|
|
|
special properties) has been made fully POSIX compliant.
|
|
|
|
|
1. The 'command' name can now result from an expansion (fixing BUG_CMDEXPAN),
|
|
|
|
|
e.g. 'c=command; "$c" ls' and 'set -- command ls; "$@"' now work.
|
|
|
|
|
2. If and only if the POSIX mode (the new -o posix shell option) is active,
|
|
|
|
|
then the 'command' utility now disables not only "special" but also
|
|
|
|
|
"declaration" properties of builtin commands that it invokes, meaning:
|
|
|
|
|
a. arguments that start with a variable name followed by '=' are
|
|
|
|
|
always treated as regular words subject to normal shell syntax;
|
|
|
|
|
b. 'command' can now stop the shell from exiting if a command that it
|
|
|
|
|
invokes tries to modify a readonly variable (fixing BUG_CMDSPEXIT).
|
|
|
|
|
|
Reinstate 'r' and 'history' as preset aliases for interactive ksh
Following a community discussion, it became clear that 'r' is
particularly problematic as a regular builtin, as the name can and
does conflict with at least one legit external command by that
name. There was a consensus against removing it altogether and
letting users set the alias in their login scripts. However,
aliases are easier to bypass, remove or rename than builtins are.
My compromise is to reinstate 'r' as a preset alias on interactive
shells only, along with 'history', as was done in 17f81ebe before
they were converted to builtins in 03224ae3. So this reintroduces
the notion of predefined aliases to ksh 93u+m, but only for
interactive shells that are not initialised in POSIX mode.
src/cmd/ksh93/Makefile,
src/cmd/ksh93/Mamfile,
src/cmd/ksh93/include/shtable.h,
src/cmd/ksh93/data/aliases.c:
- Restore aliases.c containing shtab_aliases[], a table specifying
the preset aliases.
src/cmd/ksh93/include/shtable.h,
src/cmd/ksh93/sh/init.c:
- Rename inittree() to sh_inittree() and make it extern, because we
need to use it in main.c (sh_main()).
src/cmd/ksh93/sh/main.c: sh_main():
- Init preset aliases from shtab_aliases[] only if the shell is
interactive and not in POSIX mode.
src/cmd/ksh93/bltins/typeset.c,
src/cmd/ksh93/tests/alias.sh:
- unall(): When unsetting an alias, pass on the NV_NOFREE attribute
to nv_delete() to avoid an erroneous attempt to free a preset
alias from read-only memory. See: 5d50f825
src/cmd/ksh93/data/builtins.c:
- Remove "history" and "r" entries from shtab_builtins[].
- Revert changes to inline fc/hist docs in sh_opthist[].
src/cmd/ksh93/bltins/hist.c: b_hist():
- Remove handling for 'history' and 'r' as builtins.
src/cmd/ksh93/sh.1:
- Update accordingly.
Resolves: https://github.com/ksh93/ksh/issues/125
2020-09-11 18:51:35 +00:00
|
|
|
- The 'history' (== 'hist -l') and 'r' (== 'hist -s') interactive shell
|
|
|
|
|
history commands have reverted to preset aliases and are now only loaded if
|
|
|
|
|
the shell is interactive and not initialised in POSIX mode. This avoids
|
|
|
|
|
unneeded conflicts with external commands by these names, particularly 'r'.
|
|
|
|
|
|
Fix BUG_LOOPRET2 and related return/exit misbehaviour
The 'exit' and 'return' commands without an argument failed to pass
down the exit status of the last-run command when incorporated in a
block with redirection, &&/|| list, 'case' statement, or 'while',
'until' or 'for' loop.
src/cmd/ksh93/bltins/cflow.c:
- Use $?, which is sh.savexit a.k.a. shp->savexit, as the default
exit status value if there is no argument, instead of
shp->oldexit. This fixes the default exit status behaviour to
match POSIX and other shells.
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/include/shell.h:
- Remove now-unused sh.oldexit (a.k.a. shp->oldexit) private struct
member. It appeared to fulfill the same function as sh.savexit,
but in a slightly broken way.
- Move the savexit/$? declaration from the _SH_PRIVATE part of the
struct definition to the public API part. Since $? uses this,
it's clearly a publicly exposed value already, and this is
generally the one to use. (If anything, it's exitval that should
have been private.) This declares savexit right next to exitval,
rewriting the comments to clarify the difference between them.
src/cmd/ksh93/sh/fault.c,
src/cmd/ksh93/sh/subshell.c,
src/cmd/ksh93/sh/xec.c:
- Remove assignments to shp->oldexit.
src/cmd/ksh93/tests/basic.sh:
- Add thorough regression tests for the default exit status
behaviour of 'return' and 'exit' in various lexical contexts.
- Verify that 'for' and 'case' without any command, as well as a
lone redirection, still correctly reset the exit status to 0.
Fixes: #117
2020-09-09 18:02:20 +00:00
|
|
|
2020-09-09:
|
|
|
|
|
|
|
|
|
|
- Fixed BUG_LOOPRET2 and related bugs. The 'exit' and 'return' commands without
|
|
|
|
|
an argument now correctly default to passing down the exit status of the
|
|
|
|
|
last-run command. Tests like the following, in which the last-run command is
|
|
|
|
|
'false', now correctly output 1 instead of 0:
|
|
|
|
|
fn() { return || true; }; false; fn; echo "$?"
|
|
|
|
|
fn() { while return; do true; done; }; false; fn; echo "$?"
|
|
|
|
|
fn() { for i in 1; do return; done; }; false; fn; echo "$?"
|
|
|
|
|
fn() { case 1 in 1) return ;; esac; }; false; fn; echo "$?"
|
|
|
|
|
fn() { { return; } 2>&1; }; false; fn; echo "$?"
|
|
|
|
|
|
2020-09-05 14:20:22 +00:00
|
|
|
2020-09-05:
|
|
|
|
|
|
|
|
|
|
- Fixed erroneous syntax errors in parameter expansions such as ${var:-wor)d}
|
|
|
|
|
or ${var+w(ord}. The parentheses now correctly lose their normal grammatical
|
|
|
|
|
meaning within the braces. Fix by Eric Scrivner backported from ksh2020.
|
|
|
|
|
|
2020-09-04 03:29:52 +00:00
|
|
|
2020-09-04:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused a syntax error to be thrown if the special parameter
|
|
|
|
|
expansions ${!} and ${$} (including braces) were used within a here-document.
|
|
|
|
|
Bug reported by @Saikiran-m on GitHub.
|
|
|
|
|
|
Remove SHOPT_BASH; keep &> redir operator, '-o posix' option
On 16 June there was a call for volunteers to fix the bash
compatibility mode; it has never successfully compiled in 93u+.
Since no one showed up, it is now removed due to lack of interest.
A couple of things are kept, which are now globally enabled:
1. The &>file redirection shorthand (for >file 2>&1). As a matter
of fact, ksh93 already supported this natively, but only while
running rc/profile/login scripts, and it issued a warning. This
makse it globally available and removes the warning, bringing
ksh93 in line with mksh, bash and zsh.
2. The '-o posix' standard compliance option. It is now enabled on
startup if ksh is invoked as 'sh' or if the POSIXLY_CORRECT
variable exists in the environment. To begin with, it disables
the aforementioned &> redirection shorthand. Further compliance
tweaks will be added in subsequent commits. The differences will
be fairly minimal as ksh93 is mostly compliant already.
In all changed files, code was removed that was compiled (more
precisely, failed to compile/link) if the SHOPT_BASH preprocessor
identifier was defined. Below are other changes worth mentioning:
src/cmd/ksh93/sh/bash.c,
src/cmd/ksh93/data/bash_pre_rc.sh:
- Removed.
src/cmd/ksh93/data/lexstates.c,
src/cmd/ksh93/include/shlex.h,
src/cmd/ksh93/sh/lex.c:
- Globally enable &> redirection operator if SH_POSIX not active.
- Remove warning that was issued when &> was used in rc scripts.
src/cmd/ksh93/data/options.c,
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/args.c:
- Keep SH_POSIX option (-o posix).
- Replace SH_TYPE_BASH shell type by SH_TYPE_POSIX.
src/cmd/ksh93/sh/init.c:
- sh_type(): Return SH_TYPE_POSIX shell type if ksh was invoked
as sh (or rsh, restricted sh).
- sh_init(): Enable posix option if the SH_TYPE_POSIX shell type
was detected, or if the CONFORMANCE ast config variable was set
to "standard" (which libast sets on init if POSIXLY_CORRECT
exists in the environment).
src/cmd/ksh93/tests/options.sh,
src/cmd/ksh93/tests/io.sh:
- Replace regression tests for &> and move to io.sh. Since &> is
now for general use, no longer test in an rc script, and don't
check that a warning is issued.
Closes: #9
Progresses: #20
2020-09-01 05:19:19 +00:00
|
|
|
2020-09-01:
|
|
|
|
|
|
|
|
|
|
- The bash-style '&>file' redirection shorthand (for '>file 2>&1') is now
|
|
|
|
|
always recognised and not only when running rc/profile init scripts. It no
|
|
|
|
|
longer issues a warning. This brings ksh93 in line with mksh, bash and zsh.
|
|
|
|
|
|
|
|
|
|
- A long-form shell option '-o posix' has been added, which implements a
|
|
|
|
|
mode for better compatibility with the POSIX standard. It is automatically
|
|
|
|
|
turned on if ksh is invoked under the name 'sh'.
|
|
|
|
|
For now, it:
|
|
|
|
|
* disables the &> redirection shorthand
|
2020-09-01 06:16:51 +00:00
|
|
|
* causes the 'let' arithmetic command to recognise octal numbers by
|
|
|
|
|
leading zeros regardless of the setting of the 'letoctal' option
|
2020-09-01 07:11:27 +00:00
|
|
|
* causes file descriptors > 2 to be left open when invoking another program
|
2020-09-01 07:36:28 +00:00
|
|
|
* makes the <> redirection operator default to stdin instead of stdout
|
|
|
|
|
(this keeps the 2020-05-13 BUG_REDIRIO fix for the POSIX mode while
|
|
|
|
|
restoring traditional ksh93 behaviour for backwards compatibility)
|
-o posix: disable '[ -t ]' == '[ -t 1 ]' hack
On ksh93, 'test -t' is equivalent to 'test -t 1' (and of course
"[ -t ]" is equivalent to "[ -t 1 ]").
This is purely for compatibility with ancient Bourne shell
breakage. No other shell supports this. ksh93 should probably keep
it for backwards compatibility, but it should definitely be
disabled in POSIX mode as it is a violation of the standard; 'test
-t' is an instance of 'test "$string"', which tests if the string
is empty, so it should test if the string '-t' is empty (quod non).
This also replaces the fix for 'test -t 1' in a command
substitution with a better one that avoids forking (re: cafe33f0).
src/cmd/ksh93/sh/parse.c:
- qscan(): If the posix option is active, disable the parser-based
hack that converts a simple "[ -t ]" to "[ -t 1 ]".
src/cmd/ksh93/bltins/test.c:
- e3(): If the posix option is active, disable the part of the
compatibility hack that was used for compound expressions
that end in '-t', e.g. "[ -t 2 -o -t ]".
- test_unop(): Remove the forking fix for "[ -t 1 ]".
src/cmd/ksh93/edit/edit.c:
- tty_check(): This function is used by "[ -t 1 ]" and in other
contexts as well, so a fix here is more comprehensive. Forking
here would cause a segfault, but we don't actually need to. This
adds a fix that simply returns false if we're in a virtual
subshell that is also a command substitution. Since command
substitutions always fork upon redirecting standard output within
them (making them no longer virtual), it is safe to do this.
src/cmd/ksh93/tests/bracket.sh
- Add comprehensive regression tests for test/[/[[ -t variants in
command substitutions, in simple and compound expressions, with
and without redirecting stdout to /dev/tty within the comsub.
- Add tests verifying that -o posix disables the old hack.
- Tweak other tests, including one that globally disabled xtrace.
2020-09-01 19:24:44 +00:00
|
|
|
* disables a noncompliant 'test -t' == 'test -t 1' compatibility hack
|
-o posix: don't import/export variable attributes thru environment
When exporting variables, ksh exports their attributes (such as
'integer' or 'readonly') in a magic environment variable called
"A__z" (string defined in e_envmarker[] in data/msg.c). Child
shells recognise that variable and restore the attributes.
This little-known feature is risky; the environment cannot
necessarily be trusted and that A__z variable is easy to manipulate
before or between ksh invocations, so you can cause a script's
variables to be of the wrong type, or readonly. Backwards
compatibility requires keeping it, at least for now. But it should
be disabled in the posix mode, as it violates POSIX.
To do this, we have to solve a catch-22 in init.c. We must parse
options to know whether to turn on posix mode; it may be specified
as '-o posix' on the command line. The option parsing loop depends
on an initialised environment[*], while environment initialisation
(i.e., importing attributes) should depend on the posix option.
The catch-22 can be solved because initialising just the values
before option parsing is enough to avoid regressions. Importing the
attributes can be delayed until after option parsing. That involves
basically splitting env_init() into two parts while keeping a local
static state variable between them.
src/cmd/ksh93/sh/init.c:
- env_init():
* Split the function in two stages based on a new
'import_attributes' parameter. Import values in the first
stage; import attributes from A__z in the second (if ever).
Make the 'next' variable static as it keeps a state needed for
the attributes import stage.
* Single point of truth, greppability: don't hardcode "A__z" in
separate character comparisons, but use e_envmarker[].
* Fix an indentation error.
- sh_init(): When initialising the environment (env_init), don't
import the attributes from A__z yet; parse options first, then
import attributes only if posix option is not set.
src/cmd/ksh93/sh/name.c:
- sh_envgen(): Don't export variable attributes to A__z if the
posix option is set.
src/cmd/ksh93/tests/attributes.sh:
- Check that variable attributes aren't imported or exported
if the POSIX option is set.
src/cmd/ksh93/sh.1:
- Update.
This was the last item on the TODO list for -o posix for now.
Closes: #20
[*] If environment initialisation is delayed until after option
parsing, bin/shtests shows various regressions, including:
restricted mode breaks; the locale is not initialised properly
so that multibyte variable names break; $SHLVL breaks.
2020-09-05 08:33:50 +00:00
|
|
|
* disables passing an exported variable's attributes (such as integer or
|
|
|
|
|
readonly) to a new ksh process through the environment
|
Remove SHOPT_BASH; keep &> redir operator, '-o posix' option
On 16 June there was a call for volunteers to fix the bash
compatibility mode; it has never successfully compiled in 93u+.
Since no one showed up, it is now removed due to lack of interest.
A couple of things are kept, which are now globally enabled:
1. The &>file redirection shorthand (for >file 2>&1). As a matter
of fact, ksh93 already supported this natively, but only while
running rc/profile/login scripts, and it issued a warning. This
makse it globally available and removes the warning, bringing
ksh93 in line with mksh, bash and zsh.
2. The '-o posix' standard compliance option. It is now enabled on
startup if ksh is invoked as 'sh' or if the POSIXLY_CORRECT
variable exists in the environment. To begin with, it disables
the aforementioned &> redirection shorthand. Further compliance
tweaks will be added in subsequent commits. The differences will
be fairly minimal as ksh93 is mostly compliant already.
In all changed files, code was removed that was compiled (more
precisely, failed to compile/link) if the SHOPT_BASH preprocessor
identifier was defined. Below are other changes worth mentioning:
src/cmd/ksh93/sh/bash.c,
src/cmd/ksh93/data/bash_pre_rc.sh:
- Removed.
src/cmd/ksh93/data/lexstates.c,
src/cmd/ksh93/include/shlex.h,
src/cmd/ksh93/sh/lex.c:
- Globally enable &> redirection operator if SH_POSIX not active.
- Remove warning that was issued when &> was used in rc scripts.
src/cmd/ksh93/data/options.c,
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/args.c:
- Keep SH_POSIX option (-o posix).
- Replace SH_TYPE_BASH shell type by SH_TYPE_POSIX.
src/cmd/ksh93/sh/init.c:
- sh_type(): Return SH_TYPE_POSIX shell type if ksh was invoked
as sh (or rsh, restricted sh).
- sh_init(): Enable posix option if the SH_TYPE_POSIX shell type
was detected, or if the CONFORMANCE ast config variable was set
to "standard" (which libast sets on init if POSIXLY_CORRECT
exists in the environment).
src/cmd/ksh93/tests/options.sh,
src/cmd/ksh93/tests/io.sh:
- Replace regression tests for &> and move to io.sh. Since &> is
now for general use, no longer test in an rc script, and don't
check that a warning is issued.
Closes: #9
Progresses: #20
2020-09-01 05:19:19 +00:00
|
|
|
|
Speed up 'read', fixing macOS hang (take 2)
This fixes a hanging bug that could occur on macOS when using the
'read' command to read from a FIFO and encountering end-of-file
without a final newline character. It also makes the 'read' command
perform 15-25% faster on macOS and Linux.
The previous version (ff385e5a) failed on SunOS/Solaris/Illumos
because those systems apparently don't (fully) support the POSIX
standard recv(2) syscall with MSG_PEEK[*], which is the feature
that iffe detects under the 'socket_peek' identifier. On Illumos,
using that methods causes a compilation failure (unknown identifier
MSG_PEEK); on Solaris 11.4, that method causes multiple regressions
in tests/io.sh, suggesting the method compiles but doesn't work at
all. Instead, SunOS/Solaris/Illumos requires the method using
ioctl(2)+I_PEEK and select(2). No other system that ksh currently
builds on requires this method, so it is now only used on
SunOS/Solaris/Illumos.
So far, this version of sfpkrd() has been tested to work correctly
on Linux, macOS, FreeBSD, NetBSD, OpenBSD, HP-UX, Solaris, and
OmniOS (an Illumos distribution).
It still fails to peek on Cygwin, but in the exact same way it
failed before, so that's no loss.
To test, run the 'io' test set: bin/shtests -p io
src/lib/libast/sfio/sfpkrd.c: sfpkrd():
- Remove long-obsolete Mac OS X and Solaris bug workarounds.
- Remove methods that are no longer needed.
On systems with a POSIX compliant recv(2), the only thing that
is required to avoid regressions is the code that was conditional
upon the socket_peek feature test, which tests for the correct
functioning of the recv(2) syscall. This has now been made
mandatory for non-SunOS/Solaris/Illumos systems (using an #error
directive if it is not detected), with the other methods removed.
The result performs 15-25% faster on macOS and Linux while
passing all the regression tests.
On macOS, avoiding the select(2) method fixes the hanging bug.
On SunOS/Solaris/Illumos (the '__sun' identifier), the method
using ioctl(2)+I_PEEK and select(2) (iffe feature IDs:
stream_peek and lib_select) is preserved.
Resolves: https://github.com/ksh93/ksh/issues/118 (again)
[*] https://pubs.opengroup.org/onlinepubs/9699919799/functions/recv.html
2020-08-19 20:58:46 +00:00
|
|
|
2020-08-19:
|
|
|
|
|
|
|
|
|
|
- Sped up the 'read' command on most systems by 15-25%. Fixed a hanging bug
|
|
|
|
|
on reading from a FIFO that could occur on macOS.
|
|
|
|
|
|
2020-08-17 19:23:39 +00:00
|
|
|
2020-08-17:
|
|
|
|
|
|
|
|
|
|
- 'command -p' incorrectly used the hash table entry (a.k.a. tracked alias)
|
|
|
|
|
for a command if its path was previously hashed. It has now been fixed so
|
|
|
|
|
it never consults the hash table.
|
|
|
|
|
|
Fix 'command -x' on macOS, Linux, Solaris
'command -x' (basically builtin xargs for 'command') worked for
long argument lists on *BSD and HP-UX, but not on macOS and Linux,
where it reliably entered into an infinite loop.
The problem was that it assumed that every byte of the environment
space can be used for arguments, without accounting for alignment
that some OSs do. MacOS seems to be the most wasteful one: it
aligns on 16-byte boundaries and requires some extra bytes per
argument as well.
src/cmd/ksh93/sh/path.c:
- path_xargs(): When calculating how much space to subtract per
argument, add 16 extra bytes to the length of each argument, then
align the result on 16-byte boundaries. The extra 16 bytes is
more than even macOS needs, but hopefully it is future-proof.
- path_spawn(): If path_xargs() does fail, do not enter a retry
loop (which always becomes an infinite loop if the argument list
exceeds OS limitations), but abort with an error message.
2020-08-16 08:08:07 +00:00
|
|
|
2020-08-16:
|
|
|
|
|
|
|
|
|
|
- Fixed 'command -x' on macOS, Linux and Solaris by accounting for a 16-byte
|
|
|
|
|
argument alignment. If execution does fail, it now aborts with an internal
|
|
|
|
|
error message instead of entering an infinite retry loop.
|
|
|
|
|
|
Fix leak and crash upon defining functions in subshells
A memory leak occurred upon leaving a virtual subshell if a
function was defined within it. If this was done more than 32766
(= 2^15-2 = the 'short' max value - 1) times, the shell crashed.
Discussion and reproducer: https://github.com/ksh93/ksh/issues/114
src/cmd/ksh93/sh/subshell.c: table_unset():
- A subshell-defined function was never freed because a broken
check for autoloaded functions (which must not be freed[*]). It
looked for an initial '/' in the canonical path of the script
file that defined the function, but that path is also stored for
regular functions. Now use a check that executes nv_search() in
fpathdict, the same method used in _nv_unset() in name.c for a
regular function unset.
src/cmd/ksh93/bltins/misc.c: b_dot_cmd():
- Fix an additional memory leak introduced in bd88cc7f, that caused
POSIX functions (which are run with b_dot_cmd() like dot scripts)
to leak extra. This fix avoids both the crash fixed there and the
memory leak by introducing a 'tofree' variable remembering the
filename to free. Thanks to Johnothan King for the patch.
src/lib/libast/include/stk.h,
src/lib/libast/misc/stk.c,
src/lib/libast/man/stk.3,
src/lib/libast/man/stak.3:
- Make the stack more resilient by extending the stack reference
counter 'stkref' from (signed) short to unsigned int. On modern
systems with 32-bit ints, this extends the maximum number of
elements on a stack from 2^15-1==32767 to 2^32-1==4294967295.
The ref counter can never be negative, so there is no reason for
signedness. sizeof(int) is defined as the size of a single CPU
word, so this should not affect performance at all.
On a 16-bit system (not that ksh still compiles there), this
doubles the max number of entries to 2^16-1=65535.
src/cmd/ksh93/tests/leaks.sh:
- Add leak regression tests for ksh functions, POSIX functions, dot
scripts run with '.', and dot scripts run with 'source'.
src/cmd/ksh93/tests/path.sh:
- Add an output builtin with a redirect to an autoloaded function
so that a crash[*] is triggered if the check for an autoloaded
function is ever removed from table_unset(), as was done in ksh
93v- (which crashed).
[*] Freeing autoloaded functions after leaving a virtual subshell
causes a crashing bug: https://github.com/att/ast/issues/803
Co-authored-by: Johnothan King <johnothanking@protonmail.com>
Fixes: https://github.com/ksh93/ksh/issues/114
2020-08-13 22:29:27 +00:00
|
|
|
2020-08-13:
|
|
|
|
|
|
|
|
|
|
- Fixed memory leaks and a crashing bug that occurred when defining and
|
|
|
|
|
running functions in subshells.
|
|
|
|
|
|
Fix crash upon running many subshells (#113)
Co-authored-by: Martijn Dekker <martijn@inlv.org>
An intermittent crash occurred after running many thousands of
virtual/non-forked subshells. One reproducer is a crash in the
shbench fibonacci.ksh test, as documented here:
https://github.com/ksh-community/shbench/blob/f3d9e134/bench/fibonacci.ksh#L4-L10
The apparent cause was the signed and insufficiently large 'short'
data type of 'curenv' and related variables which wrapped around to
a negative number when overflowing. These IDs are necessary for the
'wait' builtin to obtain the exit status from a background job.
This fix is inspired by a patch based on ksh 93v-:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-longenv.dif?expand=1
https://src.fedoraproject.org/rpms/ksh/blob/f24/f/ksh-20130628-longer.patch
However, we change the type to 'unsigned int' instead of 'long'. On
all remotely modern systems, ints are 32-bit values, and using this
type avoids a performance degradation on 32-bit sytems. Making them
unsigned prevents an overflow to negative values.
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/include/jobs.h,
src/cmd/ksh93/include/nval.h,
src/cmd/ksh93/include/shell.h:
- Change the types of the static global 'subenv' and the subshell
structure members 'curenv', 'jobenv', 'subenv', 'p_env' and
'subshell' to one consistent type, unsigned int.
src/cmd/ksh93/sh/jobs.c,
src/cmd/ksh93/sh/macro.c:
src/cmd/ksh93/sh/name.c:
src/cmd/ksh93/sh/nvtype.c,
src/cmd/ksh93/sh/subshell.c:
- Updates to match new variable types.
src/cmd/ksh93/tests/subshell.sh:
- Show wrong exit status in message on failure of 'wait' builtin.
2020-08-12 17:50:59 +00:00
|
|
|
2020-08-11:
|
|
|
|
|
|
|
|
|
|
- Fixed an intermittent crash upon running a large number of subshells.
|
|
|
|
|
|
2020-08-10 21:15:53 +00:00
|
|
|
2020-08-10:
|
|
|
|
|
|
|
|
|
|
- A number of fixes have been applied to the printf formatting directives
|
|
|
|
|
%H and %#H (as well as the undocumented equivalents %(html)q and %(url)q):
|
|
|
|
|
1. Both formatters have been made multibyte/UTF-8 aware, and no longer
|
|
|
|
|
delete multibyte characters. Invalid UTF-8 byte sequences are rendered
|
|
|
|
|
as ASCII question marks.
|
|
|
|
|
2. %H no longer wrongly changes spaces to non-breaking spaces ( ).
|
|
|
|
|
3. %H now converts the single quote (') to '%#39;' instead of '''
|
|
|
|
|
which is not a valid entity in all HTML versions.
|
|
|
|
|
4. %#H failed to encode some reserved characters (e.g. '?') while encoding
|
|
|
|
|
some unreserved ones (e.g. '~'). It now percent-encodes all characters
|
|
|
|
|
except those 'unreserved' as per RFC3986 (ASCII alphanumeric plus -._~).
|
|
|
|
|
|
2020-08-11 00:02:31 +00:00
|
|
|
- Fixed a crash that occurred intermittently after running an external
|
|
|
|
|
command from a command substitution expanded from the $PS1 shell prompt.
|
|
|
|
|
|
2020-08-09 23:02:23 +00:00
|
|
|
2020-08-09:
|
|
|
|
|
|
|
|
|
|
- File name generation (a.k.a. pathname expansion, a.k.a. globbing) now
|
|
|
|
|
never matches the special navigational names '.' (current directory) and
|
|
|
|
|
'..' (parent directory). This change makes a pattern like .* useful; it
|
|
|
|
|
now matches all hidden files (dotfiles) in the current directory, without
|
|
|
|
|
the harmful inclusion of '.' and '..'.
|
|
|
|
|
|
redirect: check args before executing redirections (re: 7b82c338)
The 'redirect' builtin command did not error out before executing
any valid redirections. For example, 'redirect ls >foo.txt' issued
an "incorrect syntax" error, but still created 'foo.txt' and left
standard output permanently redirected to it.
src/cmd/ksh93/sh/xec.c: sh_exec():
- If we have redirections (io != NULL), and the command is
SYSREDIR, then check for arguments and error out if there are
any, before calling sh_redirect() to execute redirections.
(Note, the other check for arguments in b_exec() in bltins/misc.c
must be kept, as that applies if there are no redirections.)
src/cmd/ksh93/sh/io.c: sh_redirect():
- Edit comments to better explain what the flag values do.
src/cmd/ksh93/bltins/misc.c:
- Add a dummy b_redirect() function declaration "for the dictionary
generator" as has historically been done for other builtins that
share one C function. I'm not sure what that dictionary generator
is supposed to be, but this also improves greppability.
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/sh.1:
- Fix misleading "I/O redirection arguments" term. I/O redirections
are not arguments at all; no argument parser ever sees them.
src/cmd/ksh93/tests/io.sh:
- Test both conditions that should make 'redirect' produce an
"incorrect syntax" error.
- Test that any redirections are not executed if erroneous
non-redirection arguments exist.
src/cmd/ksh93/tests/builtins.sh:
- "... should show usage info on unrecognized options" test:
Because 'redirect' now refuses to process redirections on error,
the error message was not captured. The fix is to run the builtin
in a braces block and add the redirection to the block.
2020-08-08 22:57:57 +00:00
|
|
|
2020-08-08:
|
|
|
|
|
|
|
|
|
|
- Argument checking in the 'redirect' builtin command (see 2020-06-11) has
|
|
|
|
|
been improved to error out before executing redirections. For example, an
|
|
|
|
|
error like 'redirect ls >foo.txt' now will not create 'foo.txt' and will
|
|
|
|
|
not leave your standard output permanently redirected to it.
|
|
|
|
|
|
2020-08-07 20:09:01 +00:00
|
|
|
2020-08-06:
|
|
|
|
|
|
2020-08-07 01:53:25 +00:00
|
|
|
- Added the '${.sh.pid}' variable as an alternative to Bash's '$BASHPID'.
|
|
|
|
|
This variable is set to the current shell's PID, unlike '$$' (which is
|
2021-12-12 21:40:38 +00:00
|
|
|
set to the main shell's PID). In virtual subshells '${.sh.pid}' is not
|
2020-08-07 01:53:25 +00:00
|
|
|
changed from its previous value, while in forked subshells '${.sh.pid}'
|
|
|
|
|
is set to the subshell's process ID.
|
|
|
|
|
|
2020-08-05 17:06:16 +00:00
|
|
|
2020-08-05:
|
|
|
|
|
|
2020-08-05 17:14:30 +00:00
|
|
|
- Fixed a bug in functions that caused ksh to crash when an array with an
|
|
|
|
|
unset method was turned into a multidimensional array.
|
|
|
|
|
|
2020-08-05 17:06:16 +00:00
|
|
|
- Fixed a bug that caused scripts to continue running after over-shifting
|
|
|
|
|
in a function when the function call had a redirection.
|
|
|
|
|
|
Fix shellquoting of invalid multibyte char (re: f9d28935, 8c7c60ec)
This commit fixes two bugs in the generation of $'...' shellquoted
strings:
1. A bug introduced in f9d28935. In UTF-8 locales, a byte that is
invalid in UTF-8, e.g. hex byte 86, would be shellquoted as
\u[86], which is not the same as the correct quoting, \x86.
2. A bug inherited from 93u+. Single bytes (e.g. hex 11) were
always quoted as \x11 and not \x[11], even if a subsequent
character was a hexadecimal digit. However, the parser reads
past two hexadecimal digits, so we got:
$ printf '%q\n' $'\x[11]1'
$'\x111'
$ printf $'\x111' | od -t x1
0000000 c4 91
0000002
After the bug fix, this works correctly:
$ printf '%q\n' $'\x[11]1'
$'\x[11]1'
$ printf $'\x[11]1' | od -t x1
0000000 11 31
0000002
src/cmd/ksh93/sh/string.c: sh_fmtq():
- Make the multibyte code for $'...' more readable, eliminating the
'isbyte' flag.
- When in a multibyte locale, make sure to shellquote both invalid
multibyte characters and unprintable ASCII characters as
hexadecimal bytes (\xNN). This reinstates 93u+ behaviour.
- When quoting bytes, use isxdigit(3) to determine if the next
character is a hex digit, and if so, protect the quoted byte with
square brackets.
src/cmd/ksh93/tests/quoting2.sh:
- Move the 'printf %q' shellquoting regression tests here from
builtins.sh; they test the shellquoting algorithm, not so much
the printf builtin itself.
- Add regression tests for these bugs.
2020-08-05 17:22:22 +00:00
|
|
|
- When generating shellquoted strings (such as with 'printf %q'), the
|
|
|
|
|
hexadecimal value of a quoted unprintable character was not protected with
|
|
|
|
|
square braces, e.g. 0x12 followed by '3' would be quoted as '\x123', which
|
|
|
|
|
is a different value. Such strings are now quoted like '\x[12]3' if the
|
|
|
|
|
next character is a hexadecimal digit.
|
|
|
|
|
|
2020-07-31 16:32:09 +00:00
|
|
|
2020-07-31:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused multidimensional associative arrays to be created
|
|
|
|
|
with an extra array member.
|
|
|
|
|
|
2020-08-01 00:12:45 +00:00
|
|
|
- Fixed a bug that caused the expansions of positional parameters $1 - $9,
|
|
|
|
|
as well as special parameters such as $? and $-, to corrupt any multibyte
|
|
|
|
|
characters immediately following the expansion if a UTF-8 locale is active.
|
|
|
|
|
|
2020-07-30 00:22:11 +00:00
|
|
|
2020-07-29:
|
|
|
|
|
|
|
|
|
|
- On a ksh compiled to use fork(2) to run external commands, a bug has been
|
|
|
|
|
fixed that caused signals (such as SIGINT, Ctrl+C) to be ignored within a
|
|
|
|
|
non-forked subshell after running an external command within that subshell.
|
|
|
|
|
|
2020-07-25 18:46:11 +00:00
|
|
|
2020-07-25:
|
|
|
|
|
|
|
|
|
|
- Fixed BUG_MULTIBIFS: Multibyte characters can now be used as IFS
|
|
|
|
|
delimiters. "$*" was incorrectly joining positional parameters on
|
|
|
|
|
the first byte of a multibyte character. This was due to truncation
|
|
|
|
|
based on the incorrect assumption the IFS would never be larger
|
|
|
|
|
than a single byte.
|
|
|
|
|
|
2020-07-26 01:18:49 +00:00
|
|
|
- Fixed a bug that caused the sleep builtin to continue after being given
|
|
|
|
|
an unrecognized option. 'sleep -: 1' will now show a usage message and
|
|
|
|
|
exit instead of sleep for one second.
|
|
|
|
|
|
2020-07-25 15:39:12 +00:00
|
|
|
- Fixed a bug that caused the 'typeset' variable attributes -a, -A, -l, and
|
|
|
|
|
-u to leak out of a subshell if they were set without assigning a value.
|
|
|
|
|
|
2020-07-23 23:03:57 +00:00
|
|
|
2020-07-23:
|
|
|
|
|
|
2020-07-24 00:20:26 +00:00
|
|
|
- Fixed an infinite loop that could occur when ksh is the system's /bin/sh.
|
|
|
|
|
|
2020-07-23 23:03:57 +00:00
|
|
|
- A command substitution that is run on the same line as a here-document
|
|
|
|
|
will no longer cause a syntax error.
|
|
|
|
|
|
2020-07-22 11:47:04 +00:00
|
|
|
2020-07-22:
|
|
|
|
|
|
|
|
|
|
- Fixed two race conditions when running external commands on
|
|
|
|
|
interactive shells with job control active.
|
|
|
|
|
|
2020-07-20 18:49:32 +00:00
|
|
|
2020-07-20:
|
|
|
|
|
|
|
|
|
|
- If a shell function and a built-in command by the same name exist,
|
|
|
|
|
'whence -a' and 'type -a' now report both.
|
|
|
|
|
|
2020-07-21 03:02:47 +00:00
|
|
|
- Fixed a bug that caused file descriptors opened with 'redirect' or 'exec'
|
|
|
|
|
to survive a subshell environment after exiting it.
|
|
|
|
|
|
2020-07-19 22:42:12 +00:00
|
|
|
2020-07-19:
|
|
|
|
|
|
2020-08-06 23:50:11 +00:00
|
|
|
- Fixed a crash that occurred in the '.' command when using kshdb.
|
2020-07-19 22:42:12 +00:00
|
|
|
|
2020-08-06 23:50:11 +00:00
|
|
|
- Fixed a crash that occurred when attempting to use redirection with an
|
2020-07-19 22:42:12 +00:00
|
|
|
invalid file descriptor.
|
|
|
|
|
|
2020-07-16 17:56:49 +00:00
|
|
|
2020-07-16:
|
|
|
|
|
|
|
|
|
|
- The 'history' and 'r' default aliases have been made regular built-ins,
|
|
|
|
|
leaving zero default aliases.
|
|
|
|
|
|
2020-07-17 04:00:28 +00:00
|
|
|
- Fixed a bug that caused 'sleep -s' to have no effect with intervals longer
|
|
|
|
|
than 30 seconds.
|
|
|
|
|
|
|
|
|
|
- The accuracy of the sleep builtin has been improved. It no longer ignores
|
|
|
|
|
microseconds and doesn't add extra milliseconds when the interval is less
|
|
|
|
|
than 31 seconds.
|
|
|
|
|
|
Convert default typeset aliases to regular builtins
This converts the 'autoload', 'compound', 'float', 'functions',
'integer' and 'nameref' default aliases into regular built-in
commands, so that 'unalias -a' does not remove them. Shell
functions can now use these names, which improves compatibility
with POSIX shell scripts.
src/cmd/ksh93/data/aliases.c:
- Remove default typeset aliases.
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/include/builtins.h:
- Add corresponding built-in command declarations. Typeset-style
commands are now defined by a pointer range, SYSTYPESET ..
SYSTYPESET_END. A couple need their own IDs (SYSCOMPOUND,
SYSNAMEREF) for special-casing in sh/xec.c.
- Update 'typeset --man'.
src/cmd/ksh93/bltins/typeset.c: b_typeset():
- Recognise the new builtin commands by argv[0]. Implement them by
inserting the corresponding 'typeset' options into the argument
list before parsing options. This may seem like a bit of a hack,
but it is simpler, shorter, more future-proof and less
error-prone than manually copying and adapting all the complex
flaggery from the option parsing loop.
src/cmd/ksh93/sh/parse.c,
src/cmd/ksh93/sh/xec.c:
- Recognise typeset-style commands by SYSTYPESET .. SYSTYPESET_END
pointer range.
- Special-case 'compound' (SYSCOMPOUND) and 'nameref' (SYSNAMEREF)
along with recognising the corresponding 'typeset' options.
src/cmd/ksh93/sh.1:
- Update to document the new built-ins.
- Since not all declaration commands are special built-ins now,
identify declaration commands using a double-dagger "\(dd"
character (which renders as '=' in ASCII) and disassociate their
definition from that of special built-ins.
src/cmd/ksh93/tests/variables.sh:
- Adapt a regression test as there is no more 'integer' alias.
2020-07-15 18:52:01 +00:00
|
|
|
2020-07-15:
|
|
|
|
|
|
|
|
|
|
- The 'autoload', 'compound', 'float', 'functions', 'integer' and 'nameref'
|
|
|
|
|
default aliases have been converted into regular built-in commands, so
|
|
|
|
|
that 'unalias -a' does not remove them. Shell functions can now use
|
|
|
|
|
these names, which improves compatibility with POSIX shell scripts.
|
|
|
|
|
|
2020-07-15 22:38:44 +00:00
|
|
|
- The End key escape sequence '^[[F' is now handled in the emacs and vi editing
|
|
|
|
|
modes. The End key moves the cursor to the end of the line (in contrast to
|
|
|
|
|
the Home key doing the opposite).
|
|
|
|
|
|
2020-07-14 21:00:28 +00:00
|
|
|
2020-07-14:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused 'set -b' to have no effect.
|
|
|
|
|
|
2020-07-15 03:11:33 +00:00
|
|
|
- Following the 'time' keyword, the 'times' builtin command now also
|
|
|
|
|
supports millisecond precision.
|
|
|
|
|
|
2020-07-13 18:10:23 +00:00
|
|
|
2020-07-13:
|
|
|
|
|
|
|
|
|
|
- Fixed a fork bomb that could occur when the vi editor was sent SIGTSTP
|
|
|
|
|
while running in a ksh script.
|
|
|
|
|
|
2020-07-14 21:48:04 +00:00
|
|
|
- Appending a lone percent to the end of a format specifier no longer
|
|
|
|
|
causes a syntax error. The extra percent will be treated as a literal
|
|
|
|
|
'%', like in Bash and zsh.
|
|
|
|
|
|
|
|
|
|
- The 'time' keyword now has proper support for millisecond precision.
|
|
|
|
|
Although this feature was previously documented, the 'time' keyword
|
|
|
|
|
only supported up to centisecond precision, which caused a command
|
|
|
|
|
like the one below to return '0.000' on certain operating systems:
|
|
|
|
|
$ TIMEFORMAT='%3R'; time sleep .003
|
|
|
|
|
|
|
|
|
|
- The 'time' keyword now zero-pads seconds less than ten (like mksh).
|
|
|
|
|
|
2020-07-10 16:52:47 +00:00
|
|
|
2020-07-10:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused types created with 'typeset -T' to throw an error
|
|
|
|
|
when used if the type name started with a lowercase 'a'.
|
|
|
|
|
|
2020-07-10 17:26:28 +00:00
|
|
|
- A potential crash due to memory corruption when using many file
|
|
|
|
|
descriptors has been fixed.
|
|
|
|
|
|
2020-07-09 14:09:52 +00:00
|
|
|
2020-07-09:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash on syntax error when sourcing/dotting multiple files.
|
|
|
|
|
|
2020-07-09 15:02:15 +00:00
|
|
|
- Fixed a crash when listing indexed arrays.
|
|
|
|
|
|
Fix hash table memory leak when restoring PATH
There is a bug in path_alias() that may cause a memory leak when
clearing the hash table while setting/restoring PATH.
This applies a fix from Siteshwar Vashist:
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01945.html
Note that, contrary to Siteshwar's analysis linked above, this bug
has nothing directly to do with subshells, forked or otherwise; it
can also be reproduced by temporarily setting PATH for a command,
for example, 'PATH=/dev/null true', and then doing a PATH search.
Modified analysis:
ksh maintains the value of PATH as a linked list. When a local
scope for PATH is created (e.g. in a virtual subshell or when doing
something like PATH=/foo/bar command ...), ksh duplicates PATH by
increasing the refcount for every element in the linked list by
calling the path_dup() and path_alias() functions. However, when
the state of PATH is restored, this refcount is not decreased. Next
time when PATH is reset to a new value, ksh calls the path_delete()
function to delete the linked list that stored the older path. But
the path_delete() function does not free elements whose refcount is
greater than 1, causing a memory leak.
src/cmd/ksh93/sh/path.c: path_alias():
- Decrease refcount and free old item if needed.
(The 'old' variable was already introduced in 99065353, but
its value was never used there; this fixes that as well.)
src/cmd/ksh93/tests/leaks.sh:
- Add regression test. With the bug, setting/restoring PATH
(which clears the hash table) and doing a PATH search 16 times
causes about 1.5 KiB of memory to be leaked.
2020-07-09 17:34:15 +00:00
|
|
|
- Fixed a memory leak when restoring PATH when temporarily setting PATH
|
|
|
|
|
for a command (e.g. PATH=/foo/bar command ...) or in a virtual subshell.
|
|
|
|
|
|
2020-07-09 21:12:04 +00:00
|
|
|
- Combining ((...)) with redirections no longer causes a syntax error
|
|
|
|
|
due to the parser handling '>' incorrectly.
|
|
|
|
|
|
2020-07-09 21:59:45 +00:00
|
|
|
- Fixed a bug that corrupted KIA/CQL cross-reference databases created using
|
|
|
|
|
ksh's -R option; shell warnings were wrongly included in the database file.
|
|
|
|
|
|
Fix UTF-8 shellquoting for xtrace, printf %q, etc.
This fixes an annoying issue in the shell's quoting algorithm
(used for xtrace (set -x), printf %q, and other things) for UTF-8
locales, that caused it to encode perfectly printable UTF-8
characters unnecessarily and inconsistently. For example:
$ (set -x; : 'aeu aéu')
+ : $'aeu a\u[e9]u'
$ (set -x; : 'aéu aeu')
+ : 'aéu aeu'
$ (set -x; : '正常終了 aeu')
+ : '正常終了 aeu'
$ (set -x; : 'aeu 正常終了')
+ : $'aeu \u[6b63]\u[5e38]\u[7d42]\u[4e86]'
This issue was originally reported by lijo george in May 2017:
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01958.html
src/cmd/ksh93/sh/string.c:
- Add is_invisible() function that returns true if a character is a
Unicode invisible (non-graph) character, excluding ASCII space.
Ref.: https://unicode.org/charts/PDF/U2000.pdf
- Use a fallback in is_invisible() if we cannot use the system's
iswprint(3); this is the case for the ksh C.UTF-8 locale if the
OS doesn't support that. Fall back to a hardcoded blacklist of
invisible and control characters and put up with not encoding
nonexistent characters into \u[xxxx] escapes.
Ref.: https://unicode.org/charts/PDF/U2000.pdf
- When deciding whether to switch to $'...' quoting mode (state=2),
use is_invisible() instead of testing for ASCII 0-127 range.
- In $'...' quoting mode, use is_invisible() to decide whether to
encode wide characters into \u[xxxx] escapes.
src/cmd/ksh93/tests/builtins.sh:
- Add regression tests for shellquoting Arabic, Japanese and Latin
UTF-8 characters, to be run only in a UTF-8 locale. The Arabic
sample text[*] contains a couple of direction markers that are
expected to be encoded into \u[xxxx] escapes.
[*] source: https://r12a.github.io/scripts/tutorial/summaries/arabic
2020-07-10 00:38:13 +00:00
|
|
|
- The shell's quoting algorithm (used in xtrace, printf %q, and more) has been
|
|
|
|
|
fixed for UTF-8 (Unicode) locales; it no longer needlessly and inconsistently
|
|
|
|
|
encodes normal printable UTF-8 characters into hexadecimal \u[xxxx] codes.
|
|
|
|
|
|
2020-07-09 04:08:28 +00:00
|
|
|
2020-07-07:
|
|
|
|
|
|
|
|
|
|
- Four of the date formats accepted by 'printf %()T' have had their
|
|
|
|
|
functionality altered to the common behavior of date(1):
|
|
|
|
|
- '%k' and '%l' print the current hour with blank padding, the former
|
2022-07-30 22:11:20 +00:00
|
|
|
based on a 24-hour clock and the latter a 12-hour clock. These are
|
|
|
|
|
common extensions present on Linux and *BSD.
|
2020-07-09 04:08:28 +00:00
|
|
|
- '%f' prints a date with the format string '%Y.%m.%d-%H:%M:%S' (BusyBox).
|
|
|
|
|
- '%q' prints the quarter of the year (GNU).
|
|
|
|
|
|
2020-07-06 20:51:44 +00:00
|
|
|
2020-07-06:
|
|
|
|
|
|
|
|
|
|
- 'notty' is now written to the ksh auditing file instead of '(null)' if
|
|
|
|
|
the user's tty could not be determined.
|
|
|
|
|
|
2020-07-09 00:09:40 +00:00
|
|
|
- Unsetting an associative array no longer causes a memory leak to occur.
|
|
|
|
|
|
Fix corrupt UTF-8 char processing & shellquoting after aborted read
If the processing of a multibyte character was interrupted in UTF-8
locales, e.g. by reading just one byte of a two-byte character 'ü'
(\303\274) with a command like:
print -nr $'\303\274' | read -n1 g
then the shellquoting algorithm was corrupted in such a way that
the final quote in simple single-quoted string was missing. This
bug may have had other, as yet undiscovered, effects as well. The
problem was with corrupted multibyte character processing and not
with the shell-quoting routine sh_fmtq() itself.
Full trace and discussion at: https://github.com/ksh93/ksh/issues/5
(which is also an attempt to begin to understand the esoteric
workings of the libast mb* macros that process UTF-8 characters).
src/lib/libast/comp/setlocale.c: utf8_mbtowc():
- If called from the mbinit() macro (i.e. if both pointer
parameters are null), reset the global multibyte character
synchronisation state variable. This fixes the problem with
interrupted processing leaving an inconsistent state, provided
that mbinit() is called before processing multibyte characters
(which it is, in most (?) places that do this). Before this fix,
calling mbinit() in UTF-8 locales was a no-op.
src/cmd/ksh93/sh/string.c: sh_fmtq():
- Call mbinit() before potentially processing multibyte characters.
Testing suggests that this could be superfluous, but at worst,
it's harmless; better be sure.
src/cmd/ksh93/tests/builtins.sh:
- Add regression test for shellquoting with 'printf %q' after
interrupting the processing of a multibyte characeter with
'read -n1'. This test only fails in a UTF-8 locale, e.g. when
running: bin/shtests -u builtins SHELL=/buggy/ksh-2012-08-01
Fixes #5.
2020-07-05 17:24:41 +00:00
|
|
|
2020-07-05:
|
|
|
|
|
|
|
|
|
|
- In UTF-8 locales, fix corruption of the shell's internal string quoting
|
|
|
|
|
algorithm (as used by xtrace, 'printf %q', and more) that occurred when
|
|
|
|
|
the processing of a multibyte character was interrupted.
|
|
|
|
|
|
2020-07-03 19:08:00 +00:00
|
|
|
2020-07-03:
|
|
|
|
|
|
|
|
|
|
- Backslashes are no longer escaped in the raw Bourne Shell-like editing
|
|
|
|
|
mode in multibyte locales, i.e. backslashes are no longer treated like
|
|
|
|
|
Control-V if the emacs and vi modes are disabled.
|
|
|
|
|
|
|
|
|
|
- Deleting a backslash in vi mode with Control-H or Backspace now only
|
|
|
|
|
escapes a backslash if it was the previous input. This means erasing a
|
|
|
|
|
string such as 'ab\\\' will only cause the first backslash to escape a
|
|
|
|
|
Backspace as '^?', like in emacs mode.
|
|
|
|
|
|
|
|
|
|
- An odd interaction with Backspace when the last character of a separate
|
|
|
|
|
buffer created with Shift-C was '\' has been fixed. '^?' will no longer
|
|
|
|
|
be output repeatedly when attempting to erase a separate buffer with
|
|
|
|
|
a Backspace. Note that buffers created with Shift-C are not meant to be
|
|
|
|
|
erasable:
|
|
|
|
|
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/vi.html#tag_20_152_13_49
|
|
|
|
|
|
2020-07-04 14:57:47 +00:00
|
|
|
- The 'kill' builtin now supports the SIGINFO signal (on operating systems
|
|
|
|
|
with support for SIGINFO).
|
|
|
|
|
|
2020-07-02 22:29:07 +00:00
|
|
|
2020-07-02:
|
|
|
|
|
|
|
|
|
|
- Fixed a crash that occurred if a directory named '.paths' existed in any
|
|
|
|
|
directory listed in $PATH. The fix was to only read '.paths' if it is a
|
|
|
|
|
regular file or a symlink to a regular file.
|
|
|
|
|
|
2020-07-01 17:14:10 +00:00
|
|
|
2020-06-30:
|
|
|
|
|
|
|
|
|
|
- 'read -u' will no longer crash with a memory fault when given an out of
|
|
|
|
|
range or negative file descriptor.
|
2020-07-02 17:40:15 +00:00
|
|
|
|
|
|
|
|
- The '=~' operator no longer raises an error if a regular expression
|
|
|
|
|
combines the '{x}' quantifier with a sub-expression.
|
Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
|
2020-06-28 22:30:27 +00:00
|
|
|
2020-06-28:
|
|
|
|
|
|
|
|
|
|
- Variables created with 'typeset -RF' no longer cause a memory fault
|
|
|
|
|
when accessed.
|
|
|
|
|
|
2020-06-29 17:08:28 +00:00
|
|
|
- Unsetting an array that was turned into a compound variable will no
|
|
|
|
|
longer cause silent memory corruption.
|
|
|
|
|
|
2020-06-29 18:09:20 +00:00
|
|
|
- Variables created with 'readonly' in functions are now set to the
|
|
|
|
|
specified value instead of nothing. Note that 'readonly' does not
|
|
|
|
|
create a function-local scope, unlike 'typeset -r' which does.
|
|
|
|
|
|
2020-06-26 22:36:29 +00:00
|
|
|
2020-06-26:
|
|
|
|
|
|
|
|
|
|
- Changing to a directory that has a name starting with a '.' will no
|
|
|
|
|
longer fail if preceded by '../' (i.e. 'cd ../.local' will now work).
|
|
|
|
|
|
2020-06-25 21:08:43 +00:00
|
|
|
2020-06-24:
|
|
|
|
|
|
|
|
|
|
- Fixed buggy tab completion of tilde-expanded paths such as
|
|
|
|
|
~/some in 'vi' mode.
|
|
|
|
|
|
2020-06-26 05:19:58 +00:00
|
|
|
- In the raw/default Bourne Shell-like editing mode that occurs when neither
|
|
|
|
|
the 'emacs' nor the 'vi' shell option is active:
|
|
|
|
|
* tab completion is now correctly disabled, instead of enabled and broken;
|
|
|
|
|
* entering tab characters now moves the cursor the correct amount.
|
|
|
|
|
|
2020-06-23 22:02:16 +00:00
|
|
|
2020-06-23:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused combining process substitution with redirection
|
|
|
|
|
to create a bizarre file in the user's current working directory.
|
|
|
|
|
|
|
|
|
|
- Using process substitution while the shell is interactive no longer
|
|
|
|
|
causes the process ID of the asynchronous process to be printed.
|
|
|
|
|
|
2020-06-22 12:59:24 +00:00
|
|
|
2020-06-22:
|
|
|
|
|
|
|
|
|
|
- The 'stop' and 'suspend' default aliases have been converted into regular
|
|
|
|
|
built-in commands, so that 'unalias -a' does not remove them, 'suspend'
|
|
|
|
|
can do a couple of sanity checks, and something like
|
|
|
|
|
cmd=stop; $cmd $!
|
|
|
|
|
will now work. See 'stop --man' and 'suspend --man' for more information.
|
|
|
|
|
|
2020-06-22 17:11:19 +00:00
|
|
|
- Fixed a bug that caused the kill and stop commands to segfault when given
|
|
|
|
|
a non-existent job.
|
|
|
|
|
|
2020-06-22 23:27:05 +00:00
|
|
|
- Nested functions no longer ignore variable assignments that were prefixed
|
|
|
|
|
to their parent function, i.e. 'VAR=foo func' will now set $VAR to 'foo'
|
|
|
|
|
in the scope of any nested function 'func' runs.
|
|
|
|
|
|
2020-06-20 17:08:41 +00:00
|
|
|
2020-06-20:
|
|
|
|
|
|
|
|
|
|
- Fixed a bug that caused setting the following variables as readonly in
|
|
|
|
|
a virtual subshell to affect the environment outside of the subshell:
|
|
|
|
|
$_
|
|
|
|
|
${.sh.name}
|
|
|
|
|
${.sh.subscript}
|
|
|
|
|
${.sh.level}
|
|
|
|
|
$RANDOM
|
|
|
|
|
$LINENO
|
|
|
|
|
|
2021-03-21 14:39:03 +00:00
|
|
|
- Fixed two bugs that caused 'unset .sh.lineno' to always produce a memory
|
|
|
|
|
fault and '(unset .sh.level)' to memory fault when run in nested
|
2020-06-20 17:08:41 +00:00
|
|
|
functions.
|
|
|
|
|
|
2020-06-19 12:01:55 +00:00
|
|
|
2020-06-18:
|
|
|
|
|
|
|
|
|
|
- A two decade old bug that caused 'whence -a' to base the path of
|
|
|
|
|
tracked aliases on the user's current working directory has been
|
2020-06-25 15:21:40 +00:00
|
|
|
fixed. Now the real path to tracked aliases is shown when '-a' is
|
|
|
|
|
passed to the whence command.
|
2020-06-19 12:01:55 +00:00
|
|
|
|
2020-06-18 00:48:51 +00:00
|
|
|
2020-06-17:
|
|
|
|
|
|
|
|
|
|
- A bug in 'unset -f' was fixed that prevented shell functions from
|
|
|
|
|
unsetting themselves while they were running. A POSIX function no longer
|
|
|
|
|
crashes when doing so, and a KornShell-style function no longer silently
|
|
|
|
|
ignores an 'unset -f' on itself. A function of either form now continues
|
|
|
|
|
running after unsetting itself, and is removed at the end of the run.
|
|
|
|
|
|
2020-06-16 11:21:42 +00:00
|
|
|
2020-06-16:
|
|
|
|
|
|
|
|
|
|
- Passing the '-d' flag to the read builtin will no longer cause the '-r'
|
|
|
|
|
flag to be discarded when 'read -r -d' is run.
|
|
|
|
|
|
2020-06-16 21:58:05 +00:00
|
|
|
- Fix BUG_CMDSPASGN: preceding a "special builtin"[*] with 'command' now
|
|
|
|
|
prevents preceding invocation-local variable assignments from becoming global.
|
|
|
|
|
[*] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_14
|
|
|
|
|
|
2020-06-15 07:03:44 +00:00
|
|
|
2020-06-15:
|
|
|
|
|
|
|
|
|
|
- The 'source' alias has been converted into a regular built-in command.
|
|
|
|
|
|
2020-06-15 13:43:37 +00:00
|
|
|
- Functions that set variables in a virtual subshell will no longer affect
|
|
|
|
|
variables of the same name outside of the virtual subshell's environment.
|
|
|
|
|
|
2020-06-15 14:56:11 +00:00
|
|
|
- Terse usage messages written by builtin commands now point the user to
|
|
|
|
|
the --help and --man options for more information.
|
|
|
|
|
|
2020-06-14 16:28:22 +00:00
|
|
|
2020-06-14:
|
2020-06-15 07:03:44 +00:00
|
|
|
|
2020-06-14 16:28:22 +00:00
|
|
|
- 'read -S' is now able to correctly handle strings with double quotes
|
|
|
|
|
nested inside of double quotes.
|
|
|
|
|
|
2020-06-13 21:16:08 +00:00
|
|
|
2020-06-13:
|
2020-06-14 04:28:38 +00:00
|
|
|
|
2020-06-13 21:16:08 +00:00
|
|
|
- Fixed a timezone name determination bug on FreeBSD that caused the
|
2021-03-21 14:39:03 +00:00
|
|
|
output from "LC_ALL=C printf '%T\n' now" to print the wrong time zone name.
|
2020-06-13 21:16:08 +00:00
|
|
|
|
2020-06-11 10:43:17 +00:00
|
|
|
2020-06-11:
|
|
|
|
|
|
2020-06-12 03:18:01 +00:00
|
|
|
- Fixed a bug that caused running 'builtin -d' on a special builtin to
|
|
|
|
|
delete it. The man page for the 'builtin' command documents that special
|
|
|
|
|
builtins cannot be deleted.
|
|
|
|
|
|
2020-06-11 10:43:17 +00:00
|
|
|
- POSIX compliance fix: It is now possible to set shell functions named
|
|
|
|
|
'alias' or 'unalias', overriding the commands by the same names. In
|
|
|
|
|
technical terms, they are now regular builtins, not special builtins.
|
|
|
|
|
|
2020-06-12 02:17:14 +00:00
|
|
|
- The redirect='command exec' alias has been converted to a regular
|
|
|
|
|
'redirect' builtin command that only accepts I/O redirections, which
|
|
|
|
|
persist as in 'exec'. This means that:
|
|
|
|
|
* 'unlias -a' no longer removes the 'redirect' command;
|
|
|
|
|
* users no longer accidentally get logged out of their shells if
|
|
|
|
|
they type something intuitive but wrong, like 'redirect ls >file'.
|
|
|
|
|
|
2020-06-12 04:57:57 +00:00
|
|
|
- The undocumented 'login' and 'newgrp' builtin commands have been removed.
|
|
|
|
|
These replaced your shell session with the external commands by the same
|
|
|
|
|
name, as in 'exec'. If an error occurred (e.g. due to a typo), you would
|
|
|
|
|
end up immediately logged out.
|
|
|
|
|
If you do want this behaviour, you can restore it by setting:
|
|
|
|
|
alias login='exec login'
|
|
|
|
|
alias newgrp='exec newgrp'
|
|
|
|
|
|
2020-06-10 11:00:35 +00:00
|
|
|
2020-06-10:
|
|
|
|
|
|
|
|
|
|
- The 'hash' utility is now a regular builtin instead of an alias to
|
|
|
|
|
'alias -t --'. The functionality of the old command has been removed
|
|
|
|
|
from the alias builtin.
|
|
|
|
|
|
2020-06-25 15:21:40 +00:00
|
|
|
- 'set +r' is no longer able to unset the restricted option. This change
|
|
|
|
|
makes the behavior of 'set +r' identical to 'set +o restricted'.
|
|
|
|
|
|
2020-06-09 15:31:00 +00:00
|
|
|
2020-06-09:
|
|
|
|
|
|
|
|
|
|
- The 'unalias' builtin will now return a non-zero status if it tries
|
|
|
|
|
to remove a previously set alias that is not currently set.
|
|
|
|
|
|
2020-06-07 22:01:24 +00:00
|
|
|
2020-06-08:
|
|
|
|
|
|
2020-06-10 12:16:53 +00:00
|
|
|
- Fix an issue with the up arrow key in Emacs editing mode.
|
|
|
|
|
Emacs editing mode is bugged in ksh93u+ and ksh2020. Let's
|
|
|
|
|
say you were to run the following commands after starting
|
|
|
|
|
a fresh instance of ksh:
|
2021-03-21 14:39:03 +00:00
|
|
|
$ alias foo=true
|
2020-06-10 12:16:53 +00:00
|
|
|
$ unalias foo
|
|
|
|
|
If you type 'a' and then press the up arrow on your keyboard,
|
2021-03-21 14:39:03 +00:00
|
|
|
ksh will complete 'a' to 'alias foo=true' by doing a reverse
|
2020-06-10 12:16:53 +00:00
|
|
|
search for the last command that starts with 'a'.
|
|
|
|
|
Run the alias command again, then type 'u' and press the up
|
2021-03-21 14:39:03 +00:00
|
|
|
arrow key again. If ksh is in Vi mode, you will get 'unalias foo',
|
|
|
|
|
but in Emacs mode you will get 'alias foo=true' again.
|
2020-06-10 12:16:53 +00:00
|
|
|
All subsequent commands were ignored as ksh was saving the first
|
|
|
|
|
command and only based later searches off of it.
|
2020-06-08 10:22:51 +00:00
|
|
|
|
2020-06-07 22:01:24 +00:00
|
|
|
- If 'set -u'/'set -o nounset' is active, then the shell now errors out if a
|
|
|
|
|
nonexistent positional parameter such as $1, $2, ... is accessed, as other
|
|
|
|
|
shells do and POSIX requires. (This does *not* apply to "$@" and "$*".)
|
|
|
|
|
|
2020-06-07 23:05:27 +00:00
|
|
|
- If 'set -u'/'set -o nounset' is active, then the shell now errors out if $!
|
|
|
|
|
is accessed before the shell has launched any background process.
|
|
|
|
|
|
2020-06-08 01:28:36 +00:00
|
|
|
- Removed support for an obscure early 1990s Bell Labs file system research
|
|
|
|
|
project called 3DFS, which has not existed for decades. This removes:
|
|
|
|
|
- an obnoxious default alias 2d='set -f;_2d' that turned off your file name
|
|
|
|
|
wildcard expansion and then tried to run a nonexistent '_2d' command
|
|
|
|
|
- undocumented builtins 'vmap' and 'vpath' that only printed error messages
|
|
|
|
|
- a non-functional -V unary operator for the test and [[ commands
|
|
|
|
|
|
Fix signal handling due to exit status > 256
This fixes two bugs: issuing the 'exit' command with a value > 256
would cause ksh 93u+ to kill itself with the corresponding signal
(try 'exit 265' to SIGKILL your interactive shell), and, if the
last command of a script exits due to a signal, the shell would
repeat that signal to itself, causing any parent ksh to also be
killed.
Discussion:
https://bugzilla.redhat.com/show_bug.cgi?id=1469624
https://rainbow.chard.org/2017/03/21/ksh-deliberately-segfaults-if-the-last-command-in-a-script-crashes/
This commit is loosely based on a patch applied to the 93v- beta
and the abandoned ksh2020, but that patch was incomplete & broken:
$ ksh-2020.0.0 -c 'exit 265'; echo $?
137
Expected: 9. Since the exit was *not* due to a signal, the value
should simply be cropped to the 8 bits supported by the OS.
src/cmd/ksh93/bltins/cflow.c: b_exit():
- For the 'exit' builtin command, bitwise-AND the argument to
'exit' with SH_EXITMASK (8 bits, crop to 0-255) before passing it
on to sh_exit(). This restores the behaviour of <=2011 ksh93
versions and is in line with all other POSIX shells.
It also fixes this bogosity:
$ (exit 265); echo $? # non-forked subshell
265
$ (ulimit -t unlimited; exit 265); echo $? # forked subshell
9
Forked or non-forked should make no difference at all
(see commit message a0e0e29e for why).
src/cmd/ksh93/sh/fault.c: sh_done():
- If the current exit status is equal to the status for the last
signal that was received from a child process, remove the
SH_EXITSIG (9th) bit, so that the shell doesn't kill itself.
- If the shell's last child process exits due to a signal, exit
with a portable 8-bit exit status (128 + signal number). This
avoids the exit status being < 128 by being cropped to 8 bits.
src/cmd/ksh93/tests/signal.sh:
- Add regression test for exit with status > 256.
- Add regression test verifying the shell no longer kills itself.
(cherry picked from commit 98e0fc94393e175ce6adfee390327c320795bf12)
2020-06-08 10:23:37 +00:00
|
|
|
- If the last program run by a ksh script exits with a signal (e.g. crashed),
|
|
|
|
|
ksh itself now exits normally instead of repeating that same signal.
|
|
|
|
|
In addition, using 'exit x' for x > 256 no longer makes ksh issue a signal.
|
|
|
|
|
|
2020-06-06 19:25:59 +00:00
|
|
|
2020-06-06:
|
|
|
|
|
|
|
|
|
|
- The 'times' command is now a builtin command that conforms to POSIX
|
|
|
|
|
instead of an alias for the 'time' command. It displays the accumulated
|
|
|
|
|
user and system CPU times, one line with the times used by the shell and
|
|
|
|
|
another with those used by all of the shell's child processes.
|
|
|
|
|
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_27
|
|
|
|
|
|
POSIX compliance: rm harmful default aliases 'command '/'nohup '
Continuing alias substitution after 'command' (due to the final
space in the alias) is inherently broken and doing so by default is
incompatible with the POSIX standard, as aliases may contain
arbitrary shell grammar.
For instance, until the previous commit, the POSIX standard 'times'
command was an alias: times='{ { time;} 2>&1;}' -- and so, of
course, 'command times' gave a syntax error, although this is
a perfectly valid POSIX idiom that must be supported.
'command' is specified by POSIX as a regular builtin, not an alias.
Therefore it should always bypass aliases just as it bypasses
functions to expose standard builtin and external commands.
I can only imagine that the reason for this command='command '
alias was that some standard commands themselves were implemented
as aliases, and POSIX requires that standard commands are
accessible with the 'command' prefix. But implementing standard
commands as aliases is itself inherently broken. It never worked
for 'command times', as shown; and in any case, removing all
aliases with 'unalias -a' should not get rid of standard commands.
Similarly, the default alias nohup='nohup ' is also harmful.
Anyone who really wants to keep this behaviour can just define
these aliases themselves in their script or ~/.kshrc file.
src/cmd/ksh93/data/aliases.c:
- Remove default alias command='command '.
- Remove default alias nohup='nohup '.
src/cmd/ksh93/sh.1
- Remove the above default aliases from the list.
- Mention that the 'command' builtin does not search for aliases.
(cherry picked from commit 5cfe7c4e2015b7445da24983af5008035c4b6e1e)
2020-06-07 00:36:54 +00:00
|
|
|
- The default aliases command='command ' and nohup='nohup ' have been
|
|
|
|
|
removed because they caused breakage in an attempt to circumvent other
|
|
|
|
|
breakage which is being fixed. In the unlikely even that anyone still
|
|
|
|
|
needs alias substitution to continue on the command argument following
|
|
|
|
|
'command' or 'nohup', it's easy to set these aliases yourself.
|
|
|
|
|
|
Fix unsetting special vars in subshells (re: efa31503, 8b7f8f9b)
This fixes (or at least works around) a bug that caused special
variables such as PATH, LANG, LC_ALL, LINENO, etc. to lose their
effect after being unset in a subshell.
For example:
(unset PATH; PATH=/dev/null; ls); : wrongly ran 'ls'
(unset LC_ALL; LC_ALL=badlocale); : failed to print a diagnostic
This is yet another problem with non-forking/virtual subshells. If
you forced the subshell to fork (one way of doing this is using the
'ulimit' builtin, e.g. ulimit -t unlimited) before unsetting the
special variable, the problem vanished.
I've tried to localise the problem. I suspect the sh_assignok()
function, which is called from unall(), is to blame. This function
is supposed to make a copy of a variable node in the virtual
subshell's variable tree. Apparently, it fails to copy the
associated permanent discipline function settings (stored in the
np->nvfun->disc pointer) that gave these variables their special
effect, and which survive unset. However, my attempts to fix that
have been unsuccessful. If anyone can figure out a fix, please send
a patch/pull request!
Data point: This bug existed in 93u 2011-02-08, but did not yet
exist in M-1993-12-28-s+. So it is a regression.
Meanwhile, pending a proper fix, this commit adds a safe
workaround: it forces a non-forked subshell to fork before
unsetting such a special variable.
src/cmd/ksh93/bltins/typeset.c: unall():
- If we're in a non-forked, non-${ ...; } subshell, then before
unsetting any variable, check for variables with internal
trap/discipline functions, and call sh_subfork() if any are
found. To avoid crashing, this must be done before calling
sh_pushcontext(), so we need to loop through the args separately.
src/cmd/ksh93/tests/variables.sh:
- Remove the 'ulimit' that forced the fork; we do this in C now.
(cherry picked from commit 21b1a67156582e3cbd36936f4af908bb45211a4b)
2020-06-05 13:31:26 +00:00
|
|
|
2020-06-05:
|
|
|
|
|
|
|
|
|
|
- Fix a bug that caused special variables such as PATH, LANG, LC_ALL,
|
|
|
|
|
etc. to lose their effect after being unset in a subshell. For example:
|
|
|
|
|
(unset PATH; PATH=/dev/null; ls); : wrongly ran 'ls'
|
|
|
|
|
(unset LC_ALL; LC_ALL=badlocale); : failed to print a diagnostic
|
2020-06-24 17:38:09 +00:00
|
|
|
This also fixes BUG_KUNSETIFS: unsetting IFS in a subshell failed if IFS
|
|
|
|
|
was set to the empty value in the parent shell.
|
Fix unsetting special vars in subshells (re: efa31503, 8b7f8f9b)
This fixes (or at least works around) a bug that caused special
variables such as PATH, LANG, LC_ALL, LINENO, etc. to lose their
effect after being unset in a subshell.
For example:
(unset PATH; PATH=/dev/null; ls); : wrongly ran 'ls'
(unset LC_ALL; LC_ALL=badlocale); : failed to print a diagnostic
This is yet another problem with non-forking/virtual subshells. If
you forced the subshell to fork (one way of doing this is using the
'ulimit' builtin, e.g. ulimit -t unlimited) before unsetting the
special variable, the problem vanished.
I've tried to localise the problem. I suspect the sh_assignok()
function, which is called from unall(), is to blame. This function
is supposed to make a copy of a variable node in the virtual
subshell's variable tree. Apparently, it fails to copy the
associated permanent discipline function settings (stored in the
np->nvfun->disc pointer) that gave these variables their special
effect, and which survive unset. However, my attempts to fix that
have been unsuccessful. If anyone can figure out a fix, please send
a patch/pull request!
Data point: This bug existed in 93u 2011-02-08, but did not yet
exist in M-1993-12-28-s+. So it is a regression.
Meanwhile, pending a proper fix, this commit adds a safe
workaround: it forces a non-forked subshell to fork before
unsetting such a special variable.
src/cmd/ksh93/bltins/typeset.c: unall():
- If we're in a non-forked, non-${ ...; } subshell, then before
unsetting any variable, check for variables with internal
trap/discipline functions, and call sh_subfork() if any are
found. To avoid crashing, this must be done before calling
sh_pushcontext(), so we need to loop through the args separately.
src/cmd/ksh93/tests/variables.sh:
- Remove the 'ulimit' that forced the fork; we do this in C now.
(cherry picked from commit 21b1a67156582e3cbd36936f4af908bb45211a4b)
2020-06-05 13:31:26 +00:00
|
|
|
|
2020-06-05 14:03:07 +00:00
|
|
|
- Fix crashes on some systems, including at least a crash in 'print -v' on
|
|
|
|
|
macOS, by eliminating an invalid/undefined use of memccpy() on overlapping
|
|
|
|
|
buffers in the commonly used sfputr() function.
|
|
|
|
|
|
Fix ${.sh.subshell} counter to actually count level of subshells
This counter is documented as follows:
"The current depth for subshells and command substitution."
But before this commit, the actual behaviour was that the counter
was reset to zero whenever a subshell forked for any reason: a
pipe, background job, running 'ulimit', redirecting stdout in a
command substitution, and more. This behaviour was:
1. Not consistent with the documentation. Non-forked (a.k.a.
virtual) subshells are an internal implementation detail which
scripts should not have to be concerned with. The manual page
doesn't mention them at all.
2. Inherently broken. Since a subshell may fork for any number of
reasons, even mid-run, and those reasons may change with
bugfixes and further development, scripts have never actually
been able to rely on the value of ${.sh.subshell}.
So, this commit fixes the counter to count the levels of all
subshells, both virtual and forked.
src/cmd/ksh93/sh/xec.c: _sh_fork():
- Increase ${.sh.subshell} whenever we fork.
src/cmd/ksh93/sh/subshell.c:
- sh_subfork():
* Fix comment to properly explain what it does. It doesn't
"create" a subshell, it forks off an existing virtual subshell.
* Don't zero ${.sh.subshell}. Instead, since sh_fork() increases
it but we're forking an existing subshell, undo the increase.
- sh_subshell():
* Remove 'int16_t subshell' variable. It was unnecessary and
mostly unused. It was also the wrong type: it was assigned the
value from shp->subshell which is of type short.
* Increase and decrease the level of virtual subshells and
${.sh.subshell} independently.
src/cmd/ksh93/tests/variables.sh:
- Add regression tests for ${.sh.subshell} in virtual and forked
subshells of several kinds: comsub, parentheses, pipe, bg job.
- Undo wrong error test count fix from 04b4aef0.
(cherry picked from commit a0e0e29e7e0dbf21e4b3958ae02bde6665fb2696)
2020-06-05 19:39:47 +00:00
|
|
|
- Fix the ${.sh.subshell} level counter; it is no longer reset to zero when a
|
|
|
|
|
non-forked subshell happens to fork into a separate process for some reason
|
|
|
|
|
(an internal implementation detail that should be unnoticeable to scripts).
|
|
|
|
|
|
Fix BUG_KBGPID: $! was not updated under certain conditions
The $! special parameter was not set if a background job
(somecommand &) or co-process (somecommand |&) was launched as the
only command within a braces block with an attached redirection,
for example:
{
somecommand &
} >&2
With the bug, $! was unchanged; now it contains the PID of
somecommand.
Ref.: https://github.com/att/ast/issues/1357
src/cmd/ksh93/sh/parse.c: item():
- When processing redirections following a compound command, always
create a parent node with the TSETIO (I/O redirection) token.
Before this commit, if the last command was of type TFORK (and
the last command only tested as TFORK if the bg job or coprocess
was the only command in a braces block, because the ksh parser
optimises away the braces in that case), then the parent node was
created with the TFORK token instead.
I have no idea what David Korn's intention was with that, but
this is clearly very wrong. Creating another TFORK node when
parsing the redirection caused sh_exec() in sh/xec.c to execute
the redirection in an extra forked, non-background subshell.
Since redirections are executed before anything else, this
subshell is what then launched the background job between the
braces, so $! (a.k.a. shp->bckpid) was updated in that subshell
only, and never in the main shell. The extra subshell also
prevented the background job from being noticed by job control
on interactive shells.
So, the fix is simply to remove the broken test for TFORK.
src/cmd/ksh93/tests/variables.sh:
- Add regression tests for a bg job and a co-process as the only
command within a braces block with attached redirection.
(cherry picked from commit ffe5df30e69f7b596941a98498014d8e838861f2)
2020-06-04 03:19:59 +00:00
|
|
|
2020-06-04:
|
|
|
|
|
|
|
|
|
|
- Fix BUG_KBGPID: the $! special parameter was not set if a background job
|
|
|
|
|
(somecommand &) or co-process (somecommand |&) was launched as the only
|
|
|
|
|
command within a braces block with an attached redirection, for example:
|
|
|
|
|
{
|
|
|
|
|
somecommand &
|
|
|
|
|
} >&2
|
|
|
|
|
With the bug, $! was unchanged; now it contains the PID of somecommand.
|
|
|
|
|
|
2020-05-31 13:29:15 +00:00
|
|
|
2020-05-31:
|
|
|
|
|
|
|
|
|
|
- Fix a bug in autoloading functions. Directories in the path search list
|
|
|
|
|
which should be skipped (e.g. because they don't exist) did not interact
|
|
|
|
|
correctly with autoloaded functions, so that a function to autoload was
|
|
|
|
|
not always found correctly.
|
|
|
|
|
Details: https://github.com/att/ast/issues/1454
|
|
|
|
|
|
2020-05-30 14:21:30 +00:00
|
|
|
2020-05-30:
|
|
|
|
|
|
|
|
|
|
- Fix POSIX compliance of 'test'/'[' exit status on error. The command now
|
|
|
|
|
returns status 2 instead of 1 when given an invalid number or arithmetic
|
|
|
|
|
expression, e.g.:
|
|
|
|
|
[ 123 -eq 123x ]; echo $?
|
|
|
|
|
now outputs 2 instead of 1.
|
|
|
|
|
|
Fix redefining & unsetting functions in subshells (BUG_FNSUBSH)
Functions can now be correctly redefined and unset in subshell
environments (such as ( ... ), $(command substitutions), etc).
Before this fix, attempts to do this were silently ignored (!!!),
causing the wrong code (i.e.: the function by the same name from
the parent shell environment) to be executed.
Redefining and unsetting functions within "shared" command
substitutions of the form '${ ...; }' is also fixed.
Prior discussion: https://github.com/att/ast/issues/73
src/cmd/ksh93/sh/parse.c:
- A fix from George Koelher (URL above). He writes:
| The parser can set t->comnamp to the wrong function.
| Suppose that the shell has executed
| foo() { echo WRONG; }
| and is now parsing
| (foo() { echo ok; } && foo)
| The parser was setting t->comnamp to the wrong foo. [This
| fix] doesn't set t->comnamp unless it was a builtin. Now the
| subshell can't call t->comnamp, so it looks for foo and finds
| the ok foo in the subshell's function tree.
src/cmd/ksh93/bltins/typeset.c:
- Unsetting functions in a virtual/non-forked subshell still
doesn't work: nv_open() fails to find the function. To work
around this problem, make 'unset -f' fork the subshell into its
own process with sh_subfork().
- The workaround exposed another bug: if we unset a function in a
subshell tree that overrode a function by the same name in the
main shell, then nv_delete() exposes the function from the main
shell scope. Since 'unset -f' now always forks a subshell, the
fix is to simply walk though troot's parent views and delete any
such zombie functions as well. (Without this, the 4 'more fun'
tests in tests/subshell.sh fail.)
src/cmd/ksh93/sh/subshell.c: sh_subfuntree():
- Fix function (re)definitions and unsetting in "shared" command
substitutions of the form '${ commandlist; }' (i.e.: if
sp->shp->subshare is true). Though internally this is a weird
form of virtual subshell, the manual page says it does not
execute in a subshell (meaning, all changes must survive it), so
a subshell function tree must not be created for these.
src/cmd/ksh93/tests/subshell.sh:
- Add regression tests related to these bugfixes. Test unsetting
and redefining a function in all three forms of virtual subshell.
(cherry picked from commit dde387825ab1bbd9f2eafc5dc38d5fd0bf9c3652)
2020-05-29 07:27:20 +00:00
|
|
|
2020-05-29:
|
|
|
|
|
|
|
|
|
|
- Fix BUG_FNSUBSH: functions can now be correctly redefined and unset in
|
|
|
|
|
subshell environments (such as ( ... ), $(command substitutions), etc).
|
|
|
|
|
Before this fix, this was silently ignored, causing the function by the
|
|
|
|
|
same name from the parent shell environment to be executed instead.
|
|
|
|
|
fn() { echo mainsh; }
|
|
|
|
|
(fn() { echo subsh; }; fn); fn
|
|
|
|
|
This now correctly outputs "subsh mainsh" instead of "mainsh mainsh".
|
|
|
|
|
ls() { echo "ls executed"; }
|
|
|
|
|
(unset -f ls; ls); ls
|
|
|
|
|
This now correctly lists your directory and then prints "ls executed",
|
|
|
|
|
instead of printing "ls executed" twice.
|
|
|
|
|
|
Fix unsetting aliases in subshells
Aliases can now be correctly unset within subshell environments
(such as ( ... ), $(command substitutions), etc), as well as
non-subshell "shared" command substitutions (${ ...; }). Before,
attempts to unset aliases within these were silently ignored.
Prior discussion: https://github.com/att/ast/issues/108
Subshell alias trees are only referenced in a few places in the
code, *and* have always been broken, so this commit gets rid of the
whole notion of a subshell alias tree. Instead, there is now just
one flat alias tree, and subshells fork into a separate process
when aliases are set or unset within them. It is not really
conceivable that this could be a performance-sensitive operation,
or even a common one, so this is a clean fix with no downside.
src/cmd/ksh93/include/defs.h:
- Remove sh_subaliastree() definition.
src/cmd/ksh93/sh/subshell.c:
- Remove salias element (pointer to subshell alias tree) from
subshell struct.
- Remove sh_subaliastree() function.
- sh_subshell(): Remove alias subshell tree cleanup.
src/cmd/ksh93/bltins/typeset.c:
- b_alias(): If in subshell, fork before setting alias.
- b_unalias(): If in subshell, fork before unsetting alias.
- unall(): Remove sh_subaliastree() call.
src/cmd/ksh93/sh/name.c:
- nv_open(): Remove sh_subaliastree() call.
src/cmd/ksh93/tests/subshell.sh:
- Add regression tests for unsetting or redefining aliases within
subshells.
(cherry picked from commit 12a15605b9521a2564a6e657905705a060e89095)
2020-05-29 07:27:53 +00:00
|
|
|
- Fix a similar bug with aliases. These can now be correctly unset
|
|
|
|
|
in subshell environments.
|
|
|
|
|
|
2020-05-20 21:42:00 +00:00
|
|
|
2020-05-21:
|
|
|
|
|
|
|
|
|
|
- Fix truncating of files with the combined redirections '<>;file' and
|
|
|
|
|
'<#pattern'. The bug was caused by out-of-sync streams.
|
|
|
|
|
Details and discussion: https://github.com/att/ast/issues/61
|
|
|
|
|
|
2020-05-30 14:55:30 +00:00
|
|
|
- Patched code injection vulnerability CVE-2019-14868. As a result, you can
|
2020-05-21 12:27:51 +00:00
|
|
|
no longer use expressions in imported numeric environment variables; only
|
|
|
|
|
integer literals are allowed.
|
|
|
|
|
|
Fix bugs in testing if a parameter is set
This fixes three related bugs:
1. Expansions like ${var+set} remained static when used within a
'for', 'while' or 'until' loop; the expansions din't change
along with the state of the variable, so they could not be used
to check whether a variable is set within a loop if the state of
that variable changed in the course of the loop. (BUG_ISSETLOOP)
2. ${IFS+s} always yielded 's', and [[ -v IFS ]] always yielded
true, even if IFS is unset. (BUG_IFSISSET)
3. IFS was incorrectly exempt from '-u' ('-o nounset') checks.
src/cmd/ksh93/sh/macro.c: varsub():
- When getting a node pointer (np) to the parameter to test,
special-case IFS by checking if it has a value and not setting
the pointer if not. The node to IFS always exists, even after
'unset -v IFS', so before this fix it always followed the code
path for a parameter that is set. This fixes BUG_IFSISSET for
${IFS+s} and also fixes set -u (-o nounset) with IFS.
- Before using the 'nv_isnull' macro to check if a regular variable
is set, call nv_optimize() if needed. This fixes BUG_ISSETLOOP.
Idea from Kurtis Rader: https://github.com/att/ast/issues/1090
Of course this only works if SHOPT_OPTIMIZE==1 (the default),
but if not, then this bug is not triggered in the first place.
- Add some comments for future reference.
src/cmd/ksh93/bltins/test.c: test_unop():
- Fix BUG_IFSISSET for [[ -v IFS ]]. The nv_optimize() method
doesn't seem to have any effect here, so the only way that I can
figure out is to special-case IFS, nv_getval()'ing it to check if
IFS has a value in the current scope.
src/cmd/ksh93/tests/variables.sh:
- Add regression tests for checking if a varariable is set within a
loop, within and outside a function with that variable made local
(to check if the scope is honoured). Repeat these tests for a
regular variable and for IFS, for ${foo+set} and [[ -v foo ]].
(cherry picked from commit a2cf79cb98fa3e47eca85d9049d1d831636c9b16)
2020-05-20 14:50:43 +00:00
|
|
|
2020-05-20:
|
|
|
|
|
|
|
|
|
|
- Fix BUG_ISSETLOOP. Expansions like ${var+set} remained static when used
|
|
|
|
|
within a 'for', 'while' or 'until' loop; the expansions din't change along
|
|
|
|
|
with the state of the variable, so they could not be used to check whether a
|
|
|
|
|
variable is set within a loop if the state of that variable changed in the
|
|
|
|
|
course of the loop.
|
|
|
|
|
|
|
|
|
|
- Fix BUG_IFSISSET. ${IFS+s} always yielded 's', and [[ -v IFS ]] always
|
|
|
|
|
yielded true, even if IFS is unset. This applied to IFS only.
|
|
|
|
|
|
2020-05-19 14:49:56 +00:00
|
|
|
2020-05-19:
|
|
|
|
|
|
|
|
|
|
- Fix 'command -p'. The -p option causes the operating system's standard
|
|
|
|
|
utilities path (as output by 'getconf PATH') to be searched instead of $PATH.
|
|
|
|
|
Before this fix, this was broken on non-interactive shells as the internal
|
|
|
|
|
variable holding the default PATH value was not correctly initialised.
|
|
|
|
|
|
Fix 'test -t 1' in $(command substitutions)
Standard output (FD 1) tested as being on a terminal within a
command substitution, which makes no sense as the command
substitution is supposed to be catching standard output.
ksh -c 'v=$(echo begincomsub
[ -t 1 ] && echo oops
echo endcomsub)
echo "$v"'
This should not output "oops".
This is one of the many bugs with ksh93 virtual (non-forked)
subshells. On the abandoned Vashist/Rader ksh2020 branch, this bug
was fixed by changing quite a lot of code, which introduced and/or
exposed another bug:
https://github.com/att/ast/issues/1079
https://github.com/att/ast/commit/8e1e405e
https://github.com/att/ast/issues/1088
That issue was unresolved when the ksh2020 branch was abandoned.
The safer and more conservative fix is simply forcing the subshell
to fork if we're in a non-forked command substitution and testing
'-t 1'. It is hard to imagine a situation where this would cause a
noticable performance hit.
Note that this fix does not affect ksh93-specific "shared"
non-subshell ${ command substitutions; } which are executed in the
main shell environment, so that variables survive, etcetera.
'test -t 1' continues to wrongly return true there, but command
substitutions of that form cannot be forked because that would
defeat their purpose.
src/cmd/ksh93/bltins/test.c:
- Fix 'test -t 1', '[ -t 1 ]' and '[[ -t 1 ]]' by forking the
current subshell if it is a virtual/non-forked subshell
(shp->subshell), and a command substitution (shp->comsub), but
NOT a "shared" ${ command substitution; } (!shp->subshare).
src/cmd/ksh93/tests/bracket.sh:
- Add two regression tests for this issue, which were adapted from
the Vashist/Rader ksh2020 branch.
NEWS, src/cmd/ksh93/include/version.h:
- Update.
(cherry picked from commit b8ef05e457ead65b83417699b8dd8632f855e2fa)
2020-05-16 19:06:49 +00:00
|
|
|
2020-05-16:
|
|
|
|
|
|
|
|
|
|
- Fix 'test -t 1', '[ -t 1 ]', '[[ -t 1 ]]' in command substitutions.
|
|
|
|
|
Standard output (file descriptor 1) tested as being on a terminal within a
|
|
|
|
|
command substitution, which makes no sense as the command substitution is
|
|
|
|
|
supposed to be catching standard output.
|
|
|
|
|
v=$(echo begincomsub
|
|
|
|
|
[ -t 1 ] && echo oops
|
|
|
|
|
echo endcomsub)
|
|
|
|
|
echo "$v"
|
|
|
|
|
This now does not output "oops".
|
|
|
|
|
|
2020-05-14 10:36:16 +00:00
|
|
|
2020-05-14:
|
|
|
|
|
|
|
|
|
|
- Fix syncing history when print -s -f is used. For example, the
|
|
|
|
|
following now correctly adds a 'cd' command to the history:
|
|
|
|
|
print -s -f 'cd -- %q\n' "$PWD"
|
|
|
|
|
Ref.: https://github.com/att/ast/issues/425
|
|
|
|
|
https://github.com/att/ast/pull/442
|
|
|
|
|
|
2020-06-09 19:57:05 +00:00
|
|
|
- Fix BUG_PUTIOERR: Output builtins now correctly detect
|
2020-05-16 14:04:35 +00:00
|
|
|
input/output errors. This allows scripts to check for a nonzero exit
|
|
|
|
|
status on the 'print', 'printf' and 'echo' builtins and prevent possible
|
|
|
|
|
infinite loops if SIGPIPE is ignored.
|
|
|
|
|
|
2020-05-14 16:41:26 +00:00
|
|
|
- Add a convenient bin/run_ksh_tests script to the source tree that
|
|
|
|
|
sets up the necessary environment and runs the ksh regression tests.
|
|
|
|
|
|
Fix BUG_CASELIT: pattern matching as literal string in 'case'
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
2020-06-11 15:14:31 +00:00
|
|
|
2020-05-13:
|
|
|
|
|
|
|
|
|
|
- Fix BUG_CASELIT: an undocumented 'case' pattern matching misbehaviour that
|
|
|
|
|
goes back to the original Bourne shell, but wasn't discovered until 2018.
|
|
|
|
|
If a pattern doesn't match as a pattern, it was tried again as a literal
|
|
|
|
|
string. This broke common validation use cases, e.g.:
|
|
|
|
|
n='[0-9]'
|
|
|
|
|
case $n in
|
|
|
|
|
( [0-9] ) echo "$n is a number" ;;
|
|
|
|
|
esac
|
|
|
|
|
would output "[0-9] is a number" as the literal string fallback matches the
|
|
|
|
|
pattern. As this misbehaviour was never documented anywhere (not for Bourne,
|
|
|
|
|
ksh88, or ksh93), and it was never replicated in other shells (not even in
|
|
|
|
|
ksh88 clones pdksh and mksh), it is unlikely any scripts rely on it.
|
|
|
|
|
Of course, a literal string fallback, should it be needed, is trivial to
|
|
|
|
|
implement correctly without this breakage:
|
|
|
|
|
case $n in
|
|
|
|
|
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
|
|
|
|
|
esac
|
2020-05-13 14:00:33 +00:00
|
|
|
Ref.: https://github.com/att/ast/issues/476
|
|
|
|
|
|
|
|
|
|
- Fix BUG_REDIRIO: ksh used to redirect standard output by default when no
|
|
|
|
|
file descriptor was specified with the rarely used '<>' reading/writing
|
|
|
|
|
redirection operator. It now redirects standard input by default, as POSIX
|
|
|
|
|
specifies and as all other POSIX shells do. To redirect standard output
|
|
|
|
|
for reading and writing, you now need '1<>'.
|
|
|
|
|
Ref.: https://github.com/att/ast/issues/75
|
|
|
|
|
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_07
|
2022-08-01 19:26:14 +00:00
|
|
|
|
|
|
|
|
2020-05-12:
|
|
|
|
|
|
|
|
|
|
- ksh development rebooted based on 93u+ 2012-08-01.
|
