This documents significant changes in the 93u+m branch of AT&T ksh93. For full details, see the git log at: https://github.com/ksh93/ksh Any uppercase BUG_* names are modernish shell bug IDs. 2020-07-22: - Fixed two race conditions when running external commands on interactive shells with job control active. 2020-07-20: - If a shell function and a built-in command by the same name exist, 'whence -a' and 'type -a' now report both. - Fixed a bug that caused file descriptors opened with 'redirect' or 'exec' to survive a subshell environment after exiting it. 2020-07-19: - Fixed a crash that occured in the '.' command when using kshdb. - Fixed a crash that occured when attempting to use redirection with an invalid file descriptor. 2020-07-16: - The 'history' and 'r' default aliases have been made regular built-ins, leaving zero default aliases. - Fixed a bug that caused 'sleep -s' to have no effect with intervals longer than 30 seconds. - The accuracy of the sleep builtin has been improved. It no longer ignores microseconds and doesn't add extra milliseconds when the interval is less than 31 seconds. 2020-07-15: - The 'autoload', 'compound', 'float', 'functions', 'integer' and 'nameref' default aliases have been converted into regular built-in commands, so that 'unalias -a' does not remove them. Shell functions can now use these names, which improves compatibility with POSIX shell scripts. - The End key escape sequence '^[[F' is now handled in the emacs and vi editing modes. The End key moves the cursor to the end of the line (in contrast to the Home key doing the opposite). 2020-07-14: - Fixed a bug that caused 'set -b' to have no effect. - Following the 'time' keyword, the 'times' builtin command now also supports millisecond precision. 2020-07-13: - Fixed a fork bomb that could occur when the vi editor was sent SIGTSTP while running in a ksh script. - Appending a lone percent to the end of a format specifier no longer causes a syntax error. The extra percent will be treated as a literal '%', like in Bash and zsh. - The 'time' keyword now has proper support for millisecond precision. Although this feature was previously documented, the 'time' keyword only supported up to centisecond precision, which caused a command like the one below to return '0.000' on certain operating systems: $ TIMEFORMAT='%3R'; time sleep .003 - The 'time' keyword now zero-pads seconds less than ten (like mksh). 2020-07-10: - Fixed a bug that caused types created with 'typeset -T' to throw an error when used if the type name started with a lowercase 'a'. - A potential crash due to memory corruption when using many file descriptors has been fixed. 2020-07-09: - Fixed a crash on syntax error when sourcing/dotting multiple files. - Fixed a crash when listing indexed arrays. - Fixed a memory leak when restoring PATH when temporarily setting PATH for a command (e.g. PATH=/foo/bar command ...) or in a virtual subshell. - Combining ((...)) with redirections no longer causes a syntax error due to the parser handling '>' incorrectly. - Fixed a bug that corrupted KIA/CQL cross-reference databases created using ksh's -R option; shell warnings were wrongly included in the database file. - The shell's quoting algorithm (used in xtrace, printf %q, and more) has been fixed for UTF-8 (Unicode) locales; it no longer needlessly and inconsistently encodes normal printable UTF-8 characters into hexadecimal \u[xxxx] codes. 2020-07-07: - Four of the date formats accepted by 'printf %()T' have had their functionality altered to the common behavior of date(1): - '%k' and '%l' print the current hour with blank padding, the former based on a 24-hour clock and the latter a twelve hour clock. These are common extensions present on Linux and *BSD. - '%f' prints a date with the format string '%Y.%m.%d-%H:%M:%S' (BusyBox). - '%q' prints the quarter of the year (GNU). 2020-07-06: - 'notty' is now written to the ksh auditing file instead of '(null)' if the user's tty could not be determined. - Unsetting an associative array no longer causes a memory leak to occur. 2020-07-05: - In UTF-8 locales, fix corruption of the shell's internal string quoting algorithm (as used by xtrace, 'printf %q', and more) that occurred when the processing of a multibyte character was interrupted. 2020-07-03: - Backslashes are no longer escaped in the raw Bourne Shell-like editing mode in multibyte locales, i.e. backslashes are no longer treated like Control-V if the emacs and vi modes are disabled. - Deleting a backslash in vi mode with Control-H or Backspace now only escapes a backslash if it was the previous input. This means erasing a string such as 'ab\\\' will only cause the first backslash to escape a Backspace as '^?', like in emacs mode. - An odd interaction with Backspace when the last character of a separate buffer created with Shift-C was '\' has been fixed. '^?' will no longer be output repeatedly when attempting to erase a separate buffer with a Backspace. Note that buffers created with Shift-C are not meant to be erasable: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/vi.html#tag_20_152_13_49 - The 'kill' builtin now supports the SIGINFO signal (on operating systems with support for SIGINFO). 2020-07-02: - Fixed a crash that occurred if a directory named '.paths' existed in any directory listed in $PATH. The fix was to only read '.paths' if it is a regular file or a symlink to a regular file. 2020-06-30: - 'read -u' will no longer crash with a memory fault when given an out of range or negative file descriptor. - The '=~' operator no longer raises an error if a regular expression combines the '{x}' quantifier with a sub-expression. 2020-06-28: - Variables created with 'typeset -RF' no longer cause a memory fault when accessed. - Unsetting an array that was turned into a compound variable will no longer cause silent memory corruption. - Variables created with 'readonly' in functions are now set to the specified value instead of nothing. Note that 'readonly' does not create a function-local scope, unlike 'typeset -r' which does. 2020-06-26: - Changing to a directory that has a name starting with a '.' will no longer fail if preceded by '../' (i.e. 'cd ../.local' will now work). 2020-06-24: - Fixed buggy tab completion of tilde-expanded paths such as ~/some in 'vi' mode. - In the raw/default Bourne Shell-like editing mode that occurs when neither the 'emacs' nor the 'vi' shell option is active: * tab completion is now correctly disabled, instead of enabled and broken; * entering tab characters now moves the cursor the correct amount. 2020-06-23: - Fixed a bug that caused combining process substitution with redirection to create a bizarre file in the user's current working directory. - Using process substitution while the shell is interactive no longer causes the process ID of the asynchronous process to be printed. 2020-06-22: - The 'stop' and 'suspend' default aliases have been converted into regular built-in commands, so that 'unalias -a' does not remove them, 'suspend' can do a couple of sanity checks, and something like cmd=stop; $cmd $! will now work. See 'stop --man' and 'suspend --man' for more information. - Fixed a bug that caused the kill and stop commands to segfault when given a non-existent job. - Nested functions no longer ignore variable assignments that were prefixed to their parent function, i.e. 'VAR=foo func' will now set $VAR to 'foo' in the scope of any nested function 'func' runs. 2020-06-20: - Fixed a bug that caused setting the following variables as readonly in a virtual subshell to affect the environment outside of the subshell: $_ ${.sh.name} ${.sh.subscript} ${.sh.level} $RANDOM $LINENO - Fixed two bugs that caused `unset .sh.lineno` to always produce a memory fault and `(unset .sh.level)` to memory fault when run in nested functions. 2020-06-18: - A two decade old bug that caused 'whence -a' to base the path of tracked aliases on the user's current working directory has been fixed. Now the real path to tracked aliases is shown when '-a' is passed to the whence command. 2020-06-17: - A bug in 'unset -f' was fixed that prevented shell functions from unsetting themselves while they were running. A POSIX function no longer crashes when doing so, and a KornShell-style function no longer silently ignores an 'unset -f' on itself. A function of either form now continues running after unsetting itself, and is removed at the end of the run. 2020-06-16: - Passing the '-d' flag to the read builtin will no longer cause the '-r' flag to be discarded when 'read -r -d' is run. - Fix BUG_CMDSPASGN: preceding a "special builtin"[*] with 'command' now prevents preceding invocation-local variable assignments from becoming global. [*] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_14 2020-06-15: - The 'source' alias has been converted into a regular built-in command. - Functions that set variables in a virtual subshell will no longer affect variables of the same name outside of the virtual subshell's environment. - Terse usage messages written by builtin commands now point the user to the --help and --man options for more information. 2020-06-14: - 'read -S' is now able to correctly handle strings with double quotes nested inside of double quotes. 2020-06-13: - Fixed a timezone name determination bug on FreeBSD that caused the output from `LC_ALL=C printf '%T' now` to print the wrong time zone name. 2020-06-11: - Fixed a bug that caused running 'builtin -d' on a special builtin to delete it. The man page for the 'builtin' command documents that special builtins cannot be deleted. - POSIX compliance fix: It is now possible to set shell functions named 'alias' or 'unalias', overriding the commands by the same names. In technical terms, they are now regular builtins, not special builtins. - The redirect='command exec' alias has been converted to a regular 'redirect' builtin command that only accepts I/O redirections, which persist as in 'exec'. This means that: * 'unlias -a' no longer removes the 'redirect' command; * users no longer accidentally get logged out of their shells if they type something intuitive but wrong, like 'redirect ls >file'. - The undocumented 'login' and 'newgrp' builtin commands have been removed. These replaced your shell session with the external commands by the same name, as in 'exec'. If an error occurred (e.g. due to a typo), you would end up immediately logged out. If you do want this behaviour, you can restore it by setting: alias login='exec login' alias newgrp='exec newgrp' 2020-06-10: - The 'hash' utility is now a regular builtin instead of an alias to 'alias -t --'. The functionality of the old command has been removed from the alias builtin. - Changing the hash table in a subshell will no longer affect the parent shell's hash table. This fix applies to the hash utility and when the PATH is reset manually. - 'set +r' is no longer able to unset the restricted option. This change makes the behavior of 'set +r' identical to 'set +o restricted'. 2020-06-09: - The 'unalias' builtin will now return a non-zero status if it tries to remove a previously set alias that is not currently set. 2020-06-08: - Fix an issue with the up arrow key in Emacs editing mode. Emacs editing mode is bugged in ksh93u+ and ksh2020. Let's say you were to run the following commands after starting a fresh instance of ksh: $ alias foo='true' $ unalias foo If you type 'a' and then press the up arrow on your keyboard, ksh will complete 'a' to `alias foo='true'` by doing a reverse search for the last command that starts with 'a'. Run the alias command again, then type 'u' and press the up arrow key again. If ksh is in Vi mode, you will get `unalias foo`, but in Emacs mode you will get `alias foo='true'` again. All subsequent commands were ignored as ksh was saving the first command and only based later searches off of it. - If 'set -u'/'set -o nounset' is active, then the shell now errors out if a nonexistent positional parameter such as $1, $2, ... is accessed, as other shells do and POSIX requires. (This does *not* apply to "$@" and "$*".) - If 'set -u'/'set -o nounset' is active, then the shell now errors out if $! is accessed before the shell has launched any background process. - Removed support for an obscure early 1990s Bell Labs file system research project called 3DFS, which has not existed for decades. This removes: - an obnoxious default alias 2d='set -f;_2d' that turned off your file name wildcard expansion and then tried to run a nonexistent '_2d' command - undocumented builtins 'vmap' and 'vpath' that only printed error messages - a non-functional -V unary operator for the test and [[ commands - If the last program run by a ksh script exits with a signal (e.g. crashed), ksh itself now exits normally instead of repeating that same signal. In addition, using 'exit x' for x > 256 no longer makes ksh issue a signal. 2020-06-06: - The 'times' command is now a builtin command that conforms to POSIX instead of an alias for the 'time' command. It displays the accumulated user and system CPU times, one line with the times used by the shell and another with those used by all of the shell's child processes. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_27 - The default aliases command='command ' and nohup='nohup ' have been removed because they caused breakage in an attempt to circumvent other breakage which is being fixed. In the unlikely even that anyone still needs alias substitution to continue on the command argument following 'command' or 'nohup', it's easy to set these aliases yourself. 2020-06-05: - Fix a bug that caused special variables such as PATH, LANG, LC_ALL, etc. to lose their effect after being unset in a subshell. For example: (unset PATH; PATH=/dev/null; ls); : wrongly ran 'ls' (unset LC_ALL; LC_ALL=badlocale); : failed to print a diagnostic This also fixes BUG_KUNSETIFS: unsetting IFS in a subshell failed if IFS was set to the empty value in the parent shell. - Fix crashes on some systems, including at least a crash in 'print -v' on macOS, by eliminating an invalid/undefined use of memccpy() on overlapping buffers in the commonly used sfputr() function. - Fix the ${.sh.subshell} level counter; it is no longer reset to zero when a non-forked subshell happens to fork into a separate process for some reason (an internal implementation detail that should be unnoticeable to scripts). 2020-06-04: - Fix BUG_KBGPID: the $! special parameter was not set if a background job (somecommand &) or co-process (somecommand |&) was launched as the only command within a braces block with an attached redirection, for example: { somecommand & } >&2 With the bug, $! was unchanged; now it contains the PID of somecommand. 2020-05-31: - Fix a bug in autoloading functions. Directories in the path search list which should be skipped (e.g. because they don't exist) did not interact correctly with autoloaded functions, so that a function to autoload was not always found correctly. Details: https://github.com/att/ast/issues/1454 2020-05-30: - Fix POSIX compliance of 'test'/'[' exit status on error. The command now returns status 2 instead of 1 when given an invalid number or arithmetic expression, e.g.: [ 123 -eq 123x ]; echo $? now outputs 2 instead of 1. 2020-05-29: - Fix BUG_FNSUBSH: functions can now be correctly redefined and unset in subshell environments (such as ( ... ), $(command substitutions), etc). Before this fix, this was silently ignored, causing the function by the same name from the parent shell environment to be executed instead. fn() { echo mainsh; } (fn() { echo subsh; }; fn); fn This now correctly outputs "subsh mainsh" instead of "mainsh mainsh". ls() { echo "ls executed"; } (unset -f ls; ls); ls This now correctly lists your directory and then prints "ls executed", instead of printing "ls executed" twice. - Fix a similar bug with aliases. These can now be correctly unset in subshell environments. 2020-05-21: - Fix truncating of files with the combined redirections '<>;file' and '<#pattern'. The bug was caused by out-of-sync streams. Details and discussion: https://github.com/att/ast/issues/61 - Patched code injection vulnerability CVE-2019-14868. As a result, you can no longer use expressions in imported numeric environment variables; only integer literals are allowed. 2020-05-20: - Fix BUG_ISSETLOOP. Expansions like ${var+set} remained static when used within a 'for', 'while' or 'until' loop; the expansions din't change along with the state of the variable, so they could not be used to check whether a variable is set within a loop if the state of that variable changed in the course of the loop. - Fix BUG_IFSISSET. ${IFS+s} always yielded 's', and [[ -v IFS ]] always yielded true, even if IFS is unset. This applied to IFS only. 2020-05-19: - Fix 'command -p'. The -p option causes the operating system's standard utilities path (as output by 'getconf PATH') to be searched instead of $PATH. Before this fix, this was broken on non-interactive shells as the internal variable holding the default PATH value was not correctly initialised. 2020-05-16: - Fix 'test -t 1', '[ -t 1 ]', '[[ -t 1 ]]' in command substitutions. Standard output (file descriptor 1) tested as being on a terminal within a command substitution, which makes no sense as the command substitution is supposed to be catching standard output. v=$(echo begincomsub [ -t 1 ] && echo oops echo endcomsub) echo "$v" This now does not output "oops". 2020-05-14: - Fix syncing history when print -s -f is used. For example, the following now correctly adds a 'cd' command to the history: print -s -f 'cd -- %q\n' "$PWD" Ref.: https://github.com/att/ast/issues/425 https://github.com/att/ast/pull/442 - Fix BUG_PUTIOERR: Output builtins now correctly detect input/output errors. This allows scripts to check for a nonzero exit status on the 'print', 'printf' and 'echo' builtins and prevent possible infinite loops if SIGPIPE is ignored. - Add a convenient bin/run_ksh_tests script to the source tree that sets up the necessary environment and runs the ksh regression tests. 2020-05-13: - Fix BUG_CASELIT: an undocumented 'case' pattern matching misbehaviour that goes back to the original Bourne shell, but wasn't discovered until 2018. If a pattern doesn't match as a pattern, it was tried again as a literal string. This broke common validation use cases, e.g.: n='[0-9]' case $n in ( [0-9] ) echo "$n is a number" ;; esac would output "[0-9] is a number" as the literal string fallback matches the pattern. As this misbehaviour was never documented anywhere (not for Bourne, ksh88, or ksh93), and it was never replicated in other shells (not even in ksh88 clones pdksh and mksh), it is unlikely any scripts rely on it. Of course, a literal string fallback, should it be needed, is trivial to implement correctly without this breakage: case $n in ( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;; esac Ref.: https://github.com/att/ast/issues/476 - Fix BUG_REDIRIO: ksh used to redirect standard output by default when no file descriptor was specified with the rarely used '<>' reading/writing redirection operator. It now redirects standard input by default, as POSIX specifies and as all other POSIX shells do. To redirect standard output for reading and writing, you now need '1<>'. Ref.: https://github.com/att/ast/issues/75 http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_07