cde/NEWS

This documents significant changes in the 93u+m branch of AT&T ksh93.
For full details, see the git log at: https://github.com/ksh93/ksh

Any uppercase BUG_* names are modernish shell bug IDs.

2020-06-16:

- Passing the '-d' flag to the read builtin will no longer cause the '-r'
  flag to be discarded when 'read -r -d' is run.

- Fix BUG_CMDSPASGN: preceding a "special builtin"[*] with 'command' now
  prevents preceding invocation-local variable assignments from becoming global.
  [*] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_14

2020-06-15:

- The 'source' alias has been converted into a regular built-in command.

- Functions that set variables in a virtual subshell will no longer affect
  variables of the same name outside of the virtual subshell's environment.

- Terse usage messages written by builtin commands now point the user to
  the --help and --man options for more information.

2020-06-14:

- 'read -S' is now able to correctly handle strings with double quotes
  nested inside of double quotes.

2020-06-13:

- Fixed a timezone name determination bug on FreeBSD that caused the
  output from `LC_ALL=C printf '%T' now` to print the wrong time zone name.

2020-06-11:

- Fixed a bug that caused running 'builtin -d' on a special builtin to
  delete it. The man page for the 'builtin' command documents that special
  builtins cannot be deleted.

- POSIX compliance fix: It is now possible to set shell functions named
  'alias' or 'unalias', overriding the commands by the same names. In
  technical terms, they are now regular builtins, not special builtins.

- The redirect='command exec' alias has been converted to a regular
  'redirect' builtin command that only accepts I/O redirections, which
  persist as in 'exec'. This means that:
  * 'unlias -a' no longer removes the 'redirect' command;
  * users no longer accidentally get logged out of their shells if
    they type something intuitive but wrong, like 'redirect ls >file'.

- The undocumented 'login' and 'newgrp' builtin commands have been removed.
  These replaced your shell session with the external commands by the same
  name, as in 'exec'. If an error occurred (e.g. due to a typo), you would
  end up immediately logged out.
  If you do want this behaviour, you can restore it by setting:
	alias login='exec login'
	alias newgrp='exec newgrp'

2020-06-10:

- The 'hash' utility is now a regular builtin instead of an alias to
  'alias -t --'. The functionality of the old command has been removed
  from the alias builtin.

- Changing the hash table in a subshell will no longer affect the parent
  shell's hash table. This fix applies to the hash utility and when the
  PATH is reset manually.

2020-06-09:

- The 'unalias' builtin will now return a non-zero status if it tries
  to remove a previously set alias that is not currently set.

2020-06-08:

- Fix an issue with the up arrow key in Emacs editing mode.
  Emacs editing mode is bugged in ksh93u+ and ksh2020. Let's
  say you were to run the following commands after starting
  a fresh instance of ksh:
  $ alias foo='true'
  $ unalias foo
  If you type 'a' and then press the up arrow on your keyboard,
  ksh will complete 'a' to `alias foo='true'` by doing a reverse
  search for the last command that starts with 'a'.
  Run the alias command again, then type 'u' and press the up
  arrow key again. If ksh is in Vi mode, you will get `unalias foo`,
  but in Emacs mode you will get `alias foo='true'` again.
  All subsequent commands were ignored as ksh was saving the first
  command and only based later searches off of it.

- If 'set -u'/'set -o nounset' is active, then the shell now errors out if a
  nonexistent positional parameter such as $1, $2, ... is accessed, as other
  shells do and POSIX requires. (This does *not* apply to "$@" and "$*".)

- If 'set -u'/'set -o nounset' is active, then the shell now errors out if $!
  is accessed before the shell has launched any background process.

- Removed support for an obscure early 1990s Bell Labs file system research
  project called 3DFS, which has not existed for decades. This removes:
  - an obnoxious default alias 2d='set -f;_2d' that turned off your file name
    wildcard expansion and then tried to run a nonexistent '_2d' command
  - undocumented builtins 'vmap' and 'vpath' that only printed error messages
  - a non-functional -V unary operator for the test and [[ commands

- If the last program run by a ksh script exits with a signal (e.g. crashed),
  ksh itself now exits normally instead of repeating that same signal.
  In addition, using 'exit x' for x > 256 no longer makes ksh issue a signal.

2020-06-06:

- The 'times' command is now a builtin command that conforms to POSIX
  instead of an alias for the 'time' command. It displays the accumulated
  user and system CPU times, one line with the times used by the shell and
  another with those used by all of the shell's child processes.
  https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_27

- The default aliases command='command ' and nohup='nohup ' have been
  removed because they caused breakage in an attempt to circumvent other
  breakage which is being fixed. In the unlikely even that anyone still
  needs alias substitution to continue on the command argument following
  'command' or 'nohup', it's easy to set these aliases yourself.

2020-06-05:

- Fix a bug that caused special variables such as PATH, LANG, LC_ALL,
  etc. to lose their effect after being unset in a subshell. For example:
    (unset PATH; PATH=/dev/null; ls); : wrongly ran 'ls'
    (unset LC_ALL; LC_ALL=badlocale); : failed to print a diagnostic

- Fix crashes on some systems, including at least a crash in 'print -v' on
  macOS, by eliminating an invalid/undefined use of memccpy() on overlapping
  buffers in the commonly used sfputr() function.

- Fix the ${.sh.subshell} level counter; it is no longer reset to zero when a
  non-forked subshell happens to fork into a separate process for some reason
  (an internal implementation detail that should be unnoticeable to scripts).

2020-06-04:

- Fix BUG_KBGPID: the $! special parameter was not set if a background job
  (somecommand &) or co-process (somecommand |&) was launched as the only
  command within a braces block with an attached redirection, for example:
	{
		somecommand &
	} >&2
  With the bug, $! was unchanged; now it contains the PID of somecommand.

2020-05-31:

- Fix a bug in autoloading functions. Directories in the path search list
  which should be skipped (e.g. because they don't exist) did not interact
  correctly with autoloaded functions, so that a function to autoload was
  not always found correctly.
  Details: https://github.com/att/ast/issues/1454

2020-05-30:

- Fix POSIX compliance of 'test'/'[' exit status on error. The command now
  returns status 2 instead of 1 when given an invalid number or arithmetic
  expression, e.g.:
	[ 123 -eq 123x ]; echo $?
  now outputs 2 instead of 1.

2020-05-29:

- Fix BUG_FNSUBSH: functions can now be correctly redefined and unset in
  subshell environments (such as ( ... ), $(command substitutions), etc).
  Before this fix, this was silently ignored, causing the function by the
  same name from the parent shell environment to be executed instead.
	fn() { echo mainsh; }
	(fn() { echo subsh; }; fn); fn
  This now correctly outputs "subsh mainsh" instead of "mainsh mainsh".
	ls() { echo "ls executed"; }
	(unset -f ls; ls); ls
  This now correctly lists your directory and then prints "ls executed",
  instead of printing "ls executed" twice.

- Fix a similar bug with aliases. These can now be correctly unset
  in subshell environments.

2020-05-21:

- Fix truncating of files with the combined redirections '<>;file' and
  '<#pattern'. The bug was caused by out-of-sync streams.
  Details and discussion: https://github.com/att/ast/issues/61

- Patched code injection vulnerability CVE-2019-14868. As a result, you can
  no longer use expressions in imported numeric environment variables; only
  integer literals are allowed.

2020-05-20:

- Fix BUG_ISSETLOOP. Expansions like ${var+set} remained static when used
  within a 'for', 'while' or 'until' loop; the expansions din't change along
  with the state of the variable, so they could not be used to check whether a
  variable is set within a loop if the state of that variable changed in the
  course of the loop.

- Fix BUG_IFSISSET. ${IFS+s} always yielded 's', and [[ -v IFS ]] always
  yielded true, even if IFS is unset. This applied to IFS only.

2020-05-19:

- Fix 'command -p'. The -p option causes the operating system's standard
  utilities path (as output by 'getconf PATH') to be searched instead of $PATH.
  Before this fix, this was broken on non-interactive shells as the internal
  variable holding the default PATH value was not correctly initialised.

2020-05-16:

- Fix 'test -t 1', '[ -t 1 ]', '[[ -t 1 ]]' in command substitutions.
  Standard output (file descriptor 1) tested as being on a terminal within a
  command substitution, which makes no sense as the command substitution is
  supposed to be catching standard output.
	v=$(echo begincomsub
	    [ -t 1 ] && echo oops
	    echo endcomsub)
	echo "$v"
  This now does not output "oops".

2020-05-14:

- Fix syncing history when print -s -f is used. For example, the
  following now correctly adds a 'cd' command to the history:
	print -s -f 'cd -- %q\n' "$PWD"
  Ref.:	https://github.com/att/ast/issues/425
	https://github.com/att/ast/pull/442

- Fix BUG_PUTIOERR: Output builtins now correctly detect
  input/output errors. This allows scripts to check for a nonzero exit
  status on the 'print', 'printf' and 'echo' builtins and prevent possible
  infinite loops if SIGPIPE is ignored.

- Add a convenient bin/run_ksh_tests script to the source tree that
  sets up the necessary environment and runs the ksh regression tests.

2020-05-13:

- Fix BUG_CASELIT: an undocumented 'case' pattern matching misbehaviour that
  goes back to the original Bourne shell, but wasn't discovered until 2018.
  If a pattern doesn't match as a pattern, it was tried again as a literal
  string. This broke common validation use cases, e.g.:
      n='[0-9]'
      case $n in
      ( [0-9] )  echo "$n is a number" ;;
      esac
  would output "[0-9] is a number" as the literal string fallback matches the
  pattern. As this misbehaviour was never documented anywhere (not for Bourne,
  ksh88, or ksh93), and it was never replicated in other shells (not even in
  ksh88 clones pdksh and mksh), it is unlikely any scripts rely on it.
     Of course, a literal string fallback, should it be needed, is trivial to
  implement correctly without this breakage:
      case $n in
      ( [0-9] | "[0-9]")  echo "$n is a number or the number pattern" ;;
      esac
  Ref.: https://github.com/att/ast/issues/476

- Fix BUG_REDIRIO: ksh used to redirect standard output by default when no
  file descriptor was specified with the rarely used '<>' reading/writing
  redirection operator. It now redirects standard input by default, as POSIX
  specifies and as all other POSIX shells do. To redirect standard output
  for reading and writing, you now need '1<>'.
  Ref.:	https://github.com/att/ast/issues/75
	http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_07