1
0
Fork 0
mirror of git://git.code.sf.net/p/cdesktopenv/code synced 2025-02-24 15:04:13 +00:00
Commit graph

51 commits

Author SHA1 Message Date
Johnothan King
49ae483574
Make liblist an extern to fix dtksh compile (#108)
The liblist variable needs to be an extern for dtksh to build.
Quote from CDE developer Chase:
we use an old function that no longer appears in kornshell,
sh_getliblist, it seems to be replaced by the function sh_getlib,
which is fine, but it seems to return a "Shbltin_f" type, which I
can't seem to find any information on what it is. We need the void
pointer dlsym provides for some widget init stuff, I tried making
liblist an extern, but it kept giving me an error about libcomp_t
being undefined.

src/cmd/ksh93/bltins/typeset.c,
src/cmd/ksh93/include/shell.h:
- Fix the compiler error reported above by moving the type definition
  for Libcomp_t to shell.h.
- Make liblist an extern since findsym.c in dtksh needs it to build.
  The old sh_getliblist function doesn't need to be reintroduced
  since the only purpose it served was to workaround the problem
  of liblist being a static variable. Now that liblist is an extern,
  dtksh fsym can use liblist directly to avoid sh_getliblist.

dtksh findsym.c:
https://sourceforge.net/p/cdesktopenv/code/ci/2.3.2/tree/cde/programs/dtksh/findsym.c
2020-08-05 22:18:22 +01:00
Martijn Dekker
a2f13c19f2 Fix typeset attributes -a, -A, -l, -u leaking out of subshells
If an array or upper/lowercase variable was declared with a null
initial value within a virtual/non-forked subshell, like:
	( typeset -a foo; ... )
	( typeset -A foo; ... )
	( typeset -l foo; ... )
	( typeset -u foo; ... )
then the type declaration leaked out of the subshell into the
parent shell environment, though without any values that may
subsequently have been assigned.

src/cmd/ksh93/bltins/typeset.c: setall():
- When deciding whether to create a virtual subshell scope for a
  variable, use sh_assignok(), which was actually designed for the
  purpose, instead of _nv_unset(). This allows getting rid of a
  tangled mess of special-casing that never worked quite right.

src/cmd/ksh93/tests/arrays.sh:
- Add regression tests checking that array declarations don't leak
  out of virtual subshells.

src/cmd/ksh93/tests/attributes.sh:
- Add regression tests for combining the 'export' and 'readonly'
  attributes with every other possible typeset attribute on unset
  variables. This also includes a subshell leak test for each one.

Fixes: https://github.com/ksh93/ksh/issues/88
2020-07-26 02:41:12 +01:00
Johnothan King
1bc2c74c74
Fix how unrecognized options are handled in 'sleep' and 'suspend' (#93)
When a builtin is given an unrecognized option, the usage information
for that builtin should be shown as 'Usage: builtin-name options'. The
sleep and suspend builtins were an exception to this. 'suspend' would
not show usage information and sleep wouldn't exit on error:

$ suspend -e
/usr/bin/ksh: suspend: -e: unknown option
$ time sleep -e 1
sleep: -e: unknown option

real	0m1.00s
user	0m0.00s
sys	0m0.00s

src/cmd/ksh93/bltins/sleep.c:
- Show usage information and exit when sleep is given an unknown
  option. This bugfix was backported from ksh2020: https://github.com/att/ast/pull/1024

src/cmd/ksh93/bltins/trap.c:
- Use the normal method of parsing options with optget to fix the
  suspend builtin's test failure.

src/cmd/ksh93/tests/builtins.sh:
- Add the ksh2020 regression test for getting the usage information
  of each builtin. Enable all /opt/ast/bin builtins in a subshell
  since those should be tested as well (aside from getconf and uname
  because those builtins fallback to the real commands on error).
2020-07-26 02:18:49 +01:00
Martijn Dekker
bc8b36faba whence -a/type -a: report both function and built-in by same name
'whence -a' is documented to list all possible interpretations of a
command, but failed to list a built-in command if a shell function
by the same name exists or is marked undefined using 'autoload'.

src/cmd/ksh93/bltins/whence.c: whence():
- Refactor and separate the code for reporting functions and
  built-in commands so that both can be reported for one name.

src/cmd/ksh93/data/builtins.c: sh_optwhence[]:
- Correct 'whence --man' to document that:
  * 'type' is equivalent to 'whence -v'
  * '-a' output is like '-v'

src/cmd/ksh93/tests/builtins.sh:
- Test 'whence -a' with these combinations:
  * a function, built-in and external command
  * an undefined/autoload function, built-in and external command

Fixes https://github.com/ksh93/ksh/issues/83
2020-07-20 21:16:24 +01:00
Martijn Dekker
01c25cb14b whence -a: fix spurious 'undefined function' message
$ ksh -c 'whence -a printf'
	printf is a shell builtin
	printf is /usr/bin/printf
	printf is an undefined function

The third line should not appear.

src/cmd/ksh93/bltins/whence.c:
- Remove faulty extra check for undefined (= autoload) functions.
  This was already handled earlier, on lines 192-193.

src/cmd/ksh93/tests/builtins.sh:
- Add regression test.
- For previous 'whence -a' test, don't bother with shell function.

Fixes https://github.com/ksh93/ksh/issues/26
2020-07-20 17:03:04 +01:00
Johnothan King
bd88cc7f4f
Fix two crashes related to kshdb (#82)
This commit fixes two different crashes related to kshdb:
- When redirect is given an invalid file descriptor, a segfault
  no longer occurs. Reproducer:
  $ ksh -c 'redirect 9>&200000000000'

- Fix a crash due to free(3) being used on an invalid pointer.
  This can be reproduced with kshdb (commands from att/ast#582):
  $ git clone https://github.com/rocky/kshdb.git
  $ cd kshdb
  $ ksh autogen.sh
  $ echo "print hi there" > $HOME/.kshdbrc
  $ ./kshdb -L . test/example/dbg-test1.sh

src/cmd/ksh93/bltins/misc.c: b_dot_cmd():
- The string pointed to by shp->st.filename must be able to be
  freed from memory with free(3), so duplicate the string with
  strdup(3).

src/cmd/ksh93/sh/io.c: sh_redirect():
- Show an error message when a file descriptor is invalid to
  fix a memory fault.
2020-07-19 23:42:12 +01:00
Martijn Dekker
36f55f1f85 bltins/whence.c: Revert accidentally included test (re: 3613da42)
Some temp debug code that tests a possible fix for #26 accidentally
snuck in to a completely unrelated commit. Sorry about that.
2020-07-19 06:42:53 +01:00
Martijn Dekker
e9c7ac70a7 remove unused 'is an exported alias' message (re: 80d9ae2b)
Commit 80d9ae2b removed the line that set the NV_EXPORT flag on an
alias when the obsolete ksh88 'alias -x' option was used. But it
turns out that flag actually did something: it caused 'whence -v'
to report the alias as an exported alias -- misleadingly, because
exported aliases have never actually exised in ksh93. Since '-x' no
longer sets that flag, that message is never printed.

src/cmd/ksh93/data/msg.c,
src/cmd/ksh93/include/path.h:
- Remove is_xalias[] = "%s is an exported alias for " message.

src/cmd/ksh93/bltins/whence.c:
- Remove dead code to check for (formerly) exported alias.
2020-07-19 06:21:14 +01:00
Martijn Dekker
3613da4240 Remove unused libcoshell
The coshell(1) command, which is required for libcoshell to be
useful, is not known to be shipped by any distribution. It was
removed by the ksh-community fork and hence also by 93u+m (in
2940b3f5). The coshell facility as a whole is obsolete and
insecure. For a long time now, the statically linked libcoshell
library has been 40+ kilobytes of dead weight in the ksh binary.

Prior discussion (ksh2020): https://github.com/att/ast/issues/619

src/lib/libcoshell/*:
- Removed.

src/cmd/ksh93/*:
- Remove the SHOPT_COSHELL compiler option (which was enabled) and
  a lot of code that was conditional upon #ifdef SHOPT_COSHELL.

- init.c: e_version[]: Removing SHOPT_COSHELL changed the "J"
  feature identifier in ${.sh.version} to a lowercase "j", which
  was conditional upon SHOPT_BGX (background job extensions).
  But src/cmd/ksh93/RELEASE documents (at 08-12-04, on line 1188):
    | +SHOPT_BGX enables background job extensions. Noted by "J" in
    |  the version string when enabled. [...]
  That is the only available documentation. So change that "j" back
  to a "J", leaving the version string unchanged after this commit.

- jobs.c: job_walk(): We need to keep one 'job_waitsafe(SIGCHLD);'
  call that was conditional upon SHOPT_COSHELL; removing it caused
  a regression test failure in tests/sigchld.sh, 'SIGCHLD blocked
  for script at end of pipeline' (which means that until now, a ksh
  compiled without libcoshell had broken SIGCHLD handling.)

bin/package, src/cmd/INIT/package.sh:
- Don't export COSHELL variable.
2020-07-17 19:28:52 +01:00
Martijn Dekker
fbc6cd4286 Remove vestigial 3DFS support code (re: f88f302c)
Support for the long-dead 3DFS userland versioning file system was
already removed from ksh93 (although I'd overlooked some minor
things), but libast still supported it. This removes that too.

src/lib/libast/include/fs3d.h,
src/lib/libast/man/fs3d.3,
src/lib/libast/misc/fs3d.c:
- Removed.

bin/package,
src/cmd/INIT/package.sh:
- Remove attempted use of removed vpath builtin.

src/cmd/ksh93/*:
- Remove minor 3dfs vestiges.

src/lib/lib{ast,cmd,coshell}/*:
- Remove code supporting 3dfs.
2020-07-17 05:04:03 +01:00
Johnothan King
2db9953ae0
Fix three bugs in the sleep builtin (#77)
This commit backports the main changes to sh_delay from ksh93v-
and ksh2020, which fixes the following bugs:

- Microsecond amounts of less than one millisecond are no longer
  ignored. The following loop will now take a minimum of one
  second to complete:
  for ((i = 0; i != 10000; i++)) do
    sleep PT100U
  done

- 'sleep 30' no longer adds an extra 30 milliseconds to the total
  amount of time to sleep. This bug is hard to notice since 30
  milliseconds can be considered within the margin of error. The
  only reason why longer delays weren't affected is because the old
  code masked the bug when the interval is greater than 30 seconds:
  else if(n > 30)
  {
      sleep(n);
      t -= n;
  }
  This caused 'sleep -s' to break with intervals greater than 30
  seconds, so an actual fix is used instead of a workaround.

- 'sleep -s' now functions correctly with intervals of more than
  30 seconds as the new code doesn't need the old workaround. This
  is done by handling '-s' in sh_delay.

src/cmd/ksh93/bltins/sleep.c:
- Remove the replacement for sleep(3) from the sleep builtin.
- Replace the old sh_delay function with the newer one from ksh2020.
  The new function uses tvsleep, which uses nanosleep(3) internally.

src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/edit/edit.c,
src/cmd/ksh93/sh/jobs.c,
src/cmd/ksh93/sh/xec.c,
src/cmd/ksh93/shell.3:
- Update sh_delay documentation and usage since the function now
  requires two arguments.

src/cmd/ksh93/tests/builtins.sh:
- Add a regression test for 'sleep -s' when the interval is greater
  than 30 seconds. The other bugs can't be tested for in a feasible
  manner across all systems:
  https://github.com/ksh93/ksh/pull/72#issuecomment-657215616
2020-07-17 05:00:28 +01:00
Johnothan King
03224ae3af
Make the 'history' and 'r' commands builtins (#76)
With this change no more preset aliases exist, so the preset alias
tables can be safely removed. All ksh commands can now be used
without 'unalias -a' removing them, even in interactive shells.
Additionally, the history and r commands are no longer limited to
being used in interactive shells.

src/cmd/ksh93/bltins/hist.c:
- Implement the history and r commands as builtins. Also guarantee
  lflag is set to one by avoiding 'lflag++'.

src/cmd/ksh93/Makefile,
src/cmd/ksh93/Mamfile,
src/cmd/ksh93/sh/main.c,
src/cmd/ksh93/sh/init.c,
src/cmd/ksh93/data/aliases.c:
- Remove the table of predefined aliases because the last few have
  been removed. During init the alias tree is now initialized the
  same way as the function tree.

src/cmd/ksh93/bltins/typeset.c:
- Remove the bugfix for unsetting predefined aliases because it is
  now a no-op. Aliases are no longer able to have the NV_NOFREE
  attribute.

src/cmd/ksh93/tests/alias.sh:
- Remove the regression test for unsetting predefined aliases since
  those no longer exist.

src/cmd/ksh93/data/builtins.c:
- Update sh_opthist[] for 'hist --man', etc.

src/cmd/ksh93/sh.1:
- Remove the list of preset aliases since those no longer exist.
- Document history and r as builtins instead of preset aliases.

Co-authored-by: Martijn Dekker <martijn@inlv.org>
2020-07-16 18:56:49 +01:00
Martijn Dekker
1fbbeaa19d Convert default typeset aliases to regular builtins
This converts the 'autoload', 'compound', 'float', 'functions',
'integer' and 'nameref' default aliases into regular built-in
commands, so that 'unalias -a' does not remove them. Shell
functions can now use these names, which improves compatibility
with POSIX shell scripts.

src/cmd/ksh93/data/aliases.c:
- Remove default typeset aliases.

src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/include/builtins.h:
- Add corresponding built-in command declarations. Typeset-style
  commands are now defined by a pointer range, SYSTYPESET ..
  SYSTYPESET_END. A couple need their own IDs (SYSCOMPOUND,
  SYSNAMEREF) for special-casing in sh/xec.c.
- Update 'typeset --man'.

src/cmd/ksh93/bltins/typeset.c: b_typeset():
- Recognise the new builtin commands by argv[0]. Implement them by
  inserting the corresponding 'typeset' options into the argument
  list before parsing options. This may seem like a bit of a hack,
  but it is simpler, shorter, more future-proof and less
  error-prone than manually copying and adapting all the complex
  flaggery from the option parsing loop.

src/cmd/ksh93/sh/parse.c,
src/cmd/ksh93/sh/xec.c:
- Recognise typeset-style commands by SYSTYPESET .. SYSTYPESET_END
  pointer range.
- Special-case 'compound' (SYSCOMPOUND) and 'nameref' (SYSNAMEREF)
  along with recognising the corresponding 'typeset' options.

src/cmd/ksh93/sh.1:
- Update to document the new built-ins.
- Since not all declaration commands are special built-ins now,
  identify declaration commands using a double-dagger "\(dd"
  character (which renders as '=' in ASCII) and disassociate their
  definition from that of special built-ins.

src/cmd/ksh93/tests/variables.sh:
- Adapt a regression test as there is no more 'integer' alias.
2020-07-15 20:54:06 +01:00
Martijn Dekker
b1a4131123 Millisecond precision for 'times' builtin (re: 65d363fd, 5c677a4c)
Now that we have an iffe feature test for getrusage(3), introduced
in 70fc1da7, the millisecond-precision 'times' command from the
last version of ksh2020 can easily be backported.

src/cmd/ksh93/bltins/misc.c:
- Incorporate ksh2020 'times' command, with a couple of tweaks:
  * Use locale's radix point instead of '.'.
  * Pad seconds with initial zero if < 10.

src/cmd/ksh93/data/builtins.c:
- Update version date for 'times --man'.

src/cmd/ksh93/tests/builtins.sh:
- Update 'times' test for 3 digits after radix point.
2020-07-15 04:22:45 +01:00
Martijn Dekker
5e7d335f2f Fix crash when listing indexed arrays with 'typeset -a'
There is a bug in print_scan() function that may cause ksh to crash
while listing indexed arrays. The crash happens in nv_search() when
called from print_scan().

This applies a fix from Siteshwar Vashist:
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01944.html

src/cmd/ksh93/bltins/typeset.c:
- Call nv_scan() without the NV_IARRAY flag, even for a null scan.

src/cmd/ksh93/tests/arrays.sh:
- Add regression test for 'typeset -a' crash and check output.
2020-07-09 16:42:16 +01:00
Johnothan King
120aec25ba
Fix a crash when 'read -u' is given an invalid fd (#53)
File descriptors that are too far out of range will cause the
read builtin to crash. The following example will generate
two crashes:

$ ksh -c 'read -u 2000000' || ksh -c 'read -u-2000000'

The fix is to error out when the given file descriptor is out
of range. This bugfix is from Tomas Klacko, although it was
modified to use 'sh_iovalidfd' and reject numbers greater
than 'INT_MAX':
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01912.html
The question about 'shp->fdstatus[-1]' only applies to ksh93v-
(ksh93u+ doesn't have any references to 'shp->fdstatus[-1]').

src/cmd/ksh93/bltins/read.c:
- File descriptors that are out of range should be rejected
  with an error message (like invalid file descriptors that
  are in range). The seemingly redundant check for negative
  numbers is there because out of range negative numbers also
  cause memory faults despite the later 'fd<0' check.

src/cmd/ksh93/tests/io.sh:
- Add three tests for attempting 'read -u' on various invalid
  file descriptor numbers.
2020-07-01 18:14:10 +01:00
Johnothan King
1b5bc1802a
Fix the readonly builtin's scope in functions (#51)
* Fix the readonly builtin's scope in functions

This bug was first reported at https://github.com/att/ast/issues/881

'tdata.sh->prefix' is only set to the correct value when
'b_readonly' is called as 'export', which breaks 'readonly' in
functions because the correct scope isn't set. As a result, the
following example will only print a newline:

$ function show_bar { readonly foo=bar; echo $foo; }; show_bar

The fix is to move the required code out of the if statement for
'export', as it needs to be run for 'readonly' as well. This bugfix
is from https://github.com/att/ast/pull/906

src/cmd/ksh93/bltins/typeset.c:
- Set 'tdata.sh->prefix' to the correct value, otherwise 'readonly'
  uses the wrong scope.

src/cmd/ksh93/tests/builtins.sh:
- Add the regression test from ksh2020, modified to run in a
  subshell.

src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/sh.1:
- Add documentation of 'readonly' vs. 'typeset -r' difference:
  'readonly' does not create a function-local scope.

Co-authored-by: Martijn Dekker <martijn@inlv.org>
2020-06-29 19:09:20 +01:00
Johnothan King
bb4745e897
Fix incorrect behavior of 'cd ../.foo' (#46)
The cd builtin was removing '.' from directory names when combined
with a preceding '../', which caused commands like 'cd ../.local'
to become 'cd ../local'. This patch fixes the problem by limiting
the extra handling to leading '..'. The bugfix comes from ksh93v-
2013-10-10-alpha, although this version is a shortened patch from
Solaris (as ksh93v- refactored a decent amount of the code for the
cd builtin).

src/cmd/ksh93/bltins/cd_pwd.c:
- cd should only check for leading '..', as trying to handle a lone
  '.' only causes problems.

src/cmd/ksh93/tests/builtins.sh:
- Add a regression test for this problem based on the test present in
  ksh93v- 2013-10-10-alpha.

Patch from Solaris:
https://github.com/oracle/solaris-userland/blob/860d27f/components/ksh93/patches/270-23319761.patch
2020-06-26 23:36:29 +01:00
Johnothan King
d41ec674c7 Fix some errors in the documentation and other minor issues (#42)
Somewhat notable changes in this commit:
- The 'set +r' bugfix (re: 74b41621) is now documented in the
  changelog.
- Missing options have been added to the synopsis section of the
  ksh man page.
- The minor formatting fix from https://github.com/ksh-community/ksh/pull/5
  has been applied to the ksh man page.
- A few fixes from https://github.com/att/ast/commit/5e747cfb
  have been applied to the ksh man page.
- The man page fixes from https://github.com/att/ast/pull/353
  have been applied, being:
  - An addition to document the behavior of 'set -H'.
  - A fix for the cd section appending rksh93.
  - A fix for some options being indented too far.
  - Removal of a duplicate section documenting '-D'.
  - Reordering the options for 'set' in alphabetical order.
  - A minor fix for the documentation of 'ksh -i'.
2020-06-25 19:31:51 +02:00
Martijn Dekker
5c677a4c6c Refactor the new 'times' builtin; zero-pad seconds (re: 65d363fd)
The output format is now identical to mksh's except for
the locale-dependent radix point ('.' or ',').

src/cmd/ksh93/bltins/misc.c:
- Output format tweak: pad seconds with initial zero if < 10.
- Use "too many operands" (e_toomanyops) error msg from 3ba4900e
  if there are operands, instead of "bad syntax" (e_badsyntax).
- Consolidate repetitive calculating and printing code
  into print_times().
- Get rid of some excessive variables.

src/cmd/ksh93/tests/builtins.sh:
- Update regression tests to match the above.

src/cmd/ksh93/data/builtins.c:
- Update sh_opttimes[] version string.
2020-06-24 14:32:20 +02:00
Martijn Dekker
57ff4676eb Fix very silly bug in times builtin (re: 65d363fd)
Well, that's what I get for backporting code without properly
checking it over. There was an elementary math error in how the
times builtin calculated seconds:
	utime_sec = utime - utime_min;
which could cause output such as "1m98.38s" or "3m234.77s".

src/cmd/ksh93/bltins/misc.c: b_times():
- Use fmod(), i.e. floating point modulus, to calculate seconds.
2020-06-24 07:47:57 +02:00
Martijn Dekker
3ba4900e9c Make 'stop' and 'suspend' regular built-ins
The 'stop' and 'suspend' default aliases are now converted into
regular built-in commands so that 'unalias -a' does not remove
them, 'suspend' can do some sanity checks, and something like
	cmd=stop; $cmd $!
will now work.

src/cmd/ksh93/bltins/trap.c:
- b_kill(): Incorporate 'stop' functionality, which is simply
  setting the same flag and variable as '-s STOP' would have done.
- b_suspend(): Add simple builtin function that sends SIGSTOP to
  the main shell. Check for no operands, and refuse to suspend a
  login shell (which would leave the user stuck with no way out).
  Also check that 'kill' succeeds; if we're in an asynchronous
  subshell, it is possible the main shell no longer exists.

src/cmd/ksh93/data/aliases.c:
- Remove "stop" and "suspend" default aliases. (Why were these
  conditional upon SIGTSTP when they actually issued SIGSTOP?)

src/cmd/ksh93/include/builtins.h,
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/data/msg.c:
- Add declarations of "stop" and "suspend" regular built-ins.
- Add option strings (AST manual/--man pages) for them.
- Add e_toomanyops ("too many operands") reusable error message for
  b_suspend(). Other new commands may want this at some point.

src/cmd/ksh93/sh.1:
- Remove "stop" and "suspend" default aliases.
- Document "stop" and "suspend" regular built-in commands.
2020-06-22 15:36:29 +02:00
Johnothan King
bd3e2a8001
Fix unreliable behavior when special vars are readonly or unset (#27)
src/cmd/ksh93/data/variables.c:
 - Running 'unset .sh.lineno' creates a memory fault, so fix that
   by giving it the NV_NOFREE attribute. This crash was happening
   because ${.sh.lineno} is an integer that cannot be freed from
   memory with free(3).

src/cmd/ksh93/sh/init.c:
 - Tell _nv_unset to ignore NV_RDONLY when $RANDOM and $LINENO are
   restored from the subshell scope. This is required to fully
   restore the original state of these variables after a virtual
   subshell finishes.

src/cmd/ksh93/bltins/typeset.c,
src/cmd/ksh93/sh/subshell.c:
 - Disabled some optimizations for two instances of 'sh_assignok' to
   fix 'readonly' in virtual subshells and '(unset .sh.level)' in
   nested functions. This fixes the following variables when
   '(readonly $varname); enum varname=' is run:

   $_
   ${.sh.name}
   ${.sh.subscript}
   ${.sh.level}

   The optimization in question prevents sh_assignok from saving the
   original state of these variables by making the sh_assignok call
   a no-op. Ksh needs the original state of a variable for it to be
   properly restored after a virtual subshell has run, otherwise ksh
   will simply carry over any new flags (being NV_RDONLY in this case)
   from the subshell into the main shell.

src/cmd/ksh93/tests/variables.sh:
 - Add regression tests from Martijn Dekker for setting special
   variables as readonly in virtual subshells and for unsetting
   special variables in general.

Fixes #4
2020-06-20 18:08:41 +01:00
Johnothan King
99065353b3 Fix 'whence -a' to print correct path for tracked alias (#25)
'whence -a' bases the path for tracked aliases on the user's
current working directory if an enabled ksh builtin of the same
name is also available. The following example will claim 'cat'
is in the user's current working directory:

$ whence -a cat
cat is a tracked alias for /usr/bin/cat
$ builtin cat
$ whence -a cat
cat is a shell builtin
cat is /usr/bin/cat
cat is a tracked alias for /current/working/directory/cat

This patch from ksh2020 fixes this problem by properly saving the
path of the tracked alias for use with 'whence -a', since
'path_pwd' (as implied by the function's name) only gets the users
current working directory, not the location of tracked aliases.
Ref.: https://github.com/att/ast/issues/1049

This bug was originally reported by David Morano about two decades
ago to the AST team: https://github.com/att/ast/issues/954

src/cmd/ksh93/bltins/whence.c:
 - Print the actual path of a tracked alias, path_pwd doesn't
   have this functionality.

src/cmd/ksh93/include/name.h:
 - Add 'pathcomp' for saving the value of tracked aliases.

src/cmd/ksh93/sh/path.c:
 - Save the value of tracked aliases for use by whence.

src/cmd/ksh93/tests/builtins.sh:
 - Add a regression test for using 'whence -a' on tracked
   aliases with a builtin equivalent.
2020-06-19 14:03:58 +02:00
Johnothan King
764acefaf1 read -r -d should not ignore -r
This bug was previously reported in att/ast#37.
Ksh ignores `-r` when `read -r -d` is run because when
the bit for `D_FLAG` is set, the bit for `R_FLAG` is unset
as a side effect of setting `D_FLAG`. The following set
of commands fails to print a backslash:

$ printf '\\\000' | read -r -d ''
$ echo $REPLY

The fix for this bug is to set `D_FLAG` with `D_FLAG + 1`,
which prevents `R_FLAG` from being unset. This bugfix
has been backported from ksh93v- 2013-10-10-alpha.

src/cmd/ksh93/bltins/read.c:
 - Set `D_FLAG` with `D_FLAG + 1` to prevent the bit for
   `R_FLAG` from being unset.

src/cmd/ksh93/tests/builtins.sh:
 - Add the regression test for `read -r -d` from ksh93v-.
2020-06-16 13:49:23 +01:00
Martijn Dekker
ad349c7668 silence macro redefinition warnings (re: 7003aba4)
src/cmd/ksh93/bltins/test.c,
src/cmd/ksh93/sh/arith.c,
src/cmd/ksh93/sh/streval.c:
- #undef ERROR_exit before redefining it, so clang stops nagging.
2020-06-16 04:51:21 +02:00
Johnothan King
af0bd6ad70 read -S now correctly handles nested double quotes
Prior to this bugfix, the following set of commands would
fail to print two double quotes:

IFS=',' read -S a b c <<<'foo,"""title"" data",bar'
echo $b

This fix is from ksh93v- 2013-10-10-alpha, although it has
been revised to use stakputc to put the required double quote
into the buffer for consistency with the ksh93u+ codebase.

src/cmd/ksh93/bltins/read.c:
 - When handling nested double quotes, put the required double
   quote in read's buffer with stakputc.

src/cmd/ksh93/tests/builtins.sh:
 - Add the regression test for `read -S` from ksh93v-.

src/cmd/ksh93/sh.1:
 - Fix a minor formatting error to highlight '-S' in the ksh(1)
   man page.
2020-06-14 10:40:30 -07:00
Johnothan King
7b994b6a7e Implement a better fix for unsetting special env vars
The regression this commit fixes was first introduced in ksh93t
2008-07-25. It was previously worked around in 6f0e008c by forking
subshells if any special environment variable is unset.

The reason why this problem doesn't occur in ksh93s+ is because in
that version of ksh sh_assignok never moves nodes, it only clones
them. The second argument doesn't set NV_MOVE, which makes
`sh_assignok(np,0)` is similar to `sh_assignok(np,1)`. In ksh93t and
higher, setting the second argument to zero causes the node to be moved
with NV_MOVE, which causes the discipline function associated with
the variable node to be removed when `np->nvfun` is set to zero (i.e.
NULL). This is why a command like `(unset LC_NUMERIC; LC_NUMERIC=invalid)`
doesn't print a diagnostic, as it looses its discipline function.

This patch fixes the problem by cloning the node with sh_assignok
if it is a special variable with a discipline function. This allows
special variables to work as expected in virtual subshells. The
original workaround has been kept for the $PATH variable only, as
hash tables are still broken in virtual subshells. It has been updated
accordingly to only fork subshells if it detects the variable node
for PATH. I have added two more regression tests for changing the
PATH in subshells to make sure hash tables continue working as
expected with this fix.

src/cmd/ksh93/bltins/typeset.c:
 - Only fork virtual subshells if the PATH will be changed. If a
   variable is a special variable with a discipline function, it
   should be just be cloned, not moved.

src/cmd/ksh93/sh/nvdisc.c:
 - Add a comment to clarify that NV_MOVE will delete the discipline
   function associated with the node.

src/cmd/ksh93/tests/subshells.sh:
 - Add two more regression tests for unsetting the PATH in subshells,
   one for if PATH is being pointed to by a nameref. Condense the
   hash table tests by moving the main test into a single function.
2020-06-13 12:55:48 -07:00
Martijn Dekker
d8eba9d112 Remove 'login' and 'newgrp' builtins: not sane default behaviour
This commit removes the undocumented 'login' and 'newgrp' builtin
commands. They already stopped blocking shell functions by that
name by changing from special to regular builtins in 04b91718 (a
change I forgot to mention in that commit message), but there is
another obnoxious aspect to these: being glorified hooks into
'exec', they replaced your shell session with the external commands
by the same name. This makes argument and error checking
impossible, so if you made so much as a typo, you would be
immediately logged out.

Even if that behaviour is wanted by a few, having it as the default
is user-hostile enough to be called a bug. It also violates the
POSIX definition of the 'newgrp' utility which explicitly says that
it "shall create a new shell execution environment", not replace
the existing one.
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/newgrp.html

Users who do want this behaviour can easily restore it by setting:
	alias login='exec login'
	alias newgrp='exec newgrp'

src/cmd/ksh93/bltins/misc.c:
- As there is no more 'login' builtin, combine b_exec() and
  B_login() functions, which allows eliminating a few variables.
  Note that most of 'exec' was actually implemented in B_login()!

src/cmd/ksh93/data/builtins.c:
- Remove "login" and "newgrp" table entries.

src/cmd/ksh93/include/builtins.h:
- Remove SYSLOGIN parser ID. As this was the first, all the others
  needed renumbering.

src/cmd/ksh93/sh/xec.c:
- Remove SYSLOGIN parser check that made 'login' and 'newgrp' act
  like 'exec' and replace the shell.
2020-06-12 06:57:57 +02:00
Martijn Dekker
7b82c338da Make 'redirect' a regular builtin instead of an alias of 'exec'
This commit converts the redirect='command exec' alias to a regular
'redirect' builtin command that only accepts I/O redirections, which
persist as in 'exec'. This means that:
* 'unlias -a' no longer removes the 'redirect' command;
* users no longer accidentally get logged out of their shells if
  they type something intuitive but wrong, like 'redirect ls >file'.

This should not introduce any legitimate change in behaviour. If
someone did accidentally pass non-redirection arguments to
'redirect', unexpected behaviour would occur; this now produces
an 'incorrect syntax' error.

src/cmd/ksh93/bltins/misc.c: b_exec():
- Recognise 'redirect' when parsing options.
- If invoked as 'redirect', produce error if there are arguments.

src/cmd/ksh93/data/aliases.c:
- Remove redirect='command exec' alias.

src/cmd/ksh93/data/builtins.c:
- Update/improve comments re ordering.
- Add 'redirect' builtin entry.
- sh_optexec[]: Abbreviate redirection-related documentation;
  refer to redirect(1) instead.
- sh_optredirect[]: Add documentation.

src/cmd/ksh93/include/builtins.h:
- Add SYSREDIR parser ID, renumbering those following it.
- Improve comments.
- Add extern sh_optredirect[].

src/cmd/ksh93/sh.1:
- exec: Abbreviate redirection-related documentation; refer to
  'redirect' instead.
- redirect: Add documentation.

src/cmd/ksh93/sh/xec.c:
- Recognise SYSREDIR parser ID in addition to SYSEXEC when
  determining whether to make redirections persistent.

src/cmd/ksh93/tests/io.sh:
- To regress-test the new builtin, change most 'command exec' uses
  to 'redirect'.
- Add tests verifying the exit behaviour of 'exec', 'command exec',
  'redirect' on redirections.
2020-06-12 04:54:33 +02:00
Martijn Dekker
80d9ae2b1c Refactor new b_hash(); better hash table clear (re: d8428a83)
The b_hash() function duplicated much of its code from b_alias(),
while b_alias() retained some code to support being called as
'hash'. There is no reason why 'hash' and 'alias' can't be handled
with a single function, as is the case several other builtins. Note
that option parsing can easily be made dependent on the name the
command was invoked with (in this case, argv[0]=='h').

The new hash builtin's -r option cleared the hash table by
assigning to PATH its existing value, triggering its associated
discipline function (put_restricted() in init.c) which then
actually cleared the hash table. That's a bit of a hack. It's nicer
if we can just do that directly. This requires taking a static
handler function rehash() from init.c, which invalidates one hash
table entry, and making it available to the builtin.

src/cmd/ksh93/bltins/typeset.c,
src/cmd/ksh93/include/builtins.h,
src/cmd/ksh93/include/nval.h,
src/cmd/ksh93/sh/init.c,
src/cmd/ksh93/sh/name.c:
- Merge b_hash() into b_alias().
- The -x option was still uselessly setting the NV_EXPORT flag.
  Exported aliases were in ksh88 but were removed in ksh93.
- Rename rehash() handler function from init.c to nv_rehash
  (avoiding a possible conflict with another rehash() in cd_pwd.c)
  and move it to name.c just above nv_scan(), which it's meant to
  be used with. Make it an extern so typeset.c can use it.
- b_alias(): Replace the PATH assignment by an nv_scan() call to
  clear the hash table directly using the nv_rehash() handler.

src/cmd/ksh93/data/builtins.c:
- POSIX compliance fix: Remove BLT_SPC (special builtin) flag from
  "alias" definition. 'alias' is specified as a regular builtin.
- sh_optalias[]: Fix uninformative -t option documentation.
- sh_opthash[]: Edit for conciseness and clarity.

src/cmd/ksh93/sh.1:
- Edit the 'alias -t' and 'hash' documentation.
- Remove the -- prefix from the 'alias' entry, which indicated that
  it was supposed to be a declaration builtin like 'typeset', with
  assignment-arguments expanding tildes and not being subject to
  field splitting. However, my testing shows that 'alias' has never
  actually behaved that way on ksh93. Even adding the BLT_DCL flag
  in data/builtins.c doesn't seem to change that.

(cherry picked from commit afa68dca5c786daa13213973e8b0f9bf3a1dadf6)
2020-06-12 01:45:18 +02:00
Johnothan King
102868f850 Replace the hash alias with a proper builtin
This commit replaces the old hash alias with a proper builtin.
I based this builtin off of the code alias uses for handling
`alias -t --`, but with the hack for `--` removed as it has
no use in the new builtin. `alias -t --` will no longer work,
that hack is now gone.

While I was testing this builtin, I found a bug with hash tables
in non-forking subshells. If the hash table of a non-forking
subshell is changed, the parent shell's hash table is also changed.
As an example, running `(hash -r)` was resetting the parent shell's
hash table. The workaround is to force the subshell to fork if the
hash table will be changed.

src/cmd/ksh93/bltins/typeset.c:
 - Move the code for hash out of the alias builtin into a dedicated
   hash builtin. `alias -t --` is no longer supported.

src/cmd/ksh93/data/aliases.c:
 - Remove the old alias for hash from the table of predefined aliases.

src/cmd/ksh93/data/builtins.c:
 - Fix the broken entry for the hash builtin and add a man page for
   the new builtin.

src/cmd/ksh93/sh.1:
 - Replace the entry for the hash alias with a more detailed entry
   for the hash builtin.

src/cmd/ksh93/sh/name.c:
 - Force non-forking subshells to fork if the PATH is being reset
   to workaround a bug with the hash tree.

src/cmd/ksh93/tests/alias.sh:
 - Add a regression test for resetting a hash table, then adding
   a utility to the refreshed hash table.

src/cmd/ksh93/tests/subshell.sh:
 - Add regression tests for changing the hash table in subshells.

(cherry picked from commit d8428a833afe9270b61745ba3d6df355fe1d5499)
2020-06-12 01:45:18 +02:00
Martijn Dekker
a1f46d785f rm "I/O error" error msg; just keep >0 exit status (re: 9011fa93)
The bug was really that I/O errors in output builtins were
undetectable by any means. Having a >0 exit status is sufficient.
Adding an error message risks making existing ksh scripts noisier,
or even breaking them if they redirect stderr to stdout.

Note to self: in future, implement the minimum change necessary to
fix bugs, nothing more. The fact that I needed to add four extra
2>/dev/null to the regression tests should have been a hint.

src/cmd/ksh93/bltins/print.c,
src/cmd/ksh93/data/msg.c,
src/cmd/ksh93/include/io.h:
- Remove "I/O error" message.

src/cmd/ksh93/tests/builtins.sh:
- Update to check for exit status only.

src/cmd/ksh93/tests/basic.sh,
src/cmd/ksh93/tests/coprocess.sh:
- Revert four new '2>/dev/null' to suppress the error message.

(cherry picked from commit 5e17be24d18455b575b6e98bc631c6935ffc795a)
2020-06-12 01:45:18 +02:00
Martijn Dekker
b883c7a7f8 remove extra 'isalias' variable (re: 16d5ea9b)
src/cmd/ksh93/bltins/typeset.c: unall():
- There's no need to keep track of whether we're unsetting an alias
  using a new 'isalias' vaariable, as we can just repeat the check
  for 'troot==shp->alias_tree'.

(cherry picked from commit 68f3ae3acd0d19dd80805049a1c9afc1ae1e75c3)
2020-06-12 01:45:18 +02:00
Johnothan King
5d50f825e4 The unalias builtin should return an error for non-existent aliases
This commit fixes a bug that caused unalias to return a zero status
when it tries to remove an alias twice. The following set of commands
will no longer end with an error:

$ alias foo=bar
$ unalias foo
$ unalias foo && echo 'Error'

This commit is based on the fix present in ksh2020, but it has been
extended with another bugfix. The initial fix for this problem tried to
remove aliases from the alias tree without accounting for NV_NOFREE. This
caused any attempt to remove a predefined aliases (e.g. `unalias float`)
to trigger an error with free, as all predefined aliases are in read-only
memory. The fix for this problem is to set NV_NOFREE when removing aliases
from the alias tree, but only if the alias is in read-only memory. All
other aliases must be freed from memory to prevent memory leaks.

I'll also note that I am using an `isalias` variable rather than the `type`
enum from ksh2020, as the `VARIABLE` value is never used and was replaced
with a bool called `aliases` in the ksh2020 release. The `isalias` variable
is an int as the ksh93u+ codebase does not use C99 bools.

Previous discussion: https://github.com/att/ast/issues/909

- src/cmd/ksh93/bltins/typeset.c:
  Remove aliases from the alias tree by using nv_delete. NV_NOFREE
  is only used when it is necessary.

- src/cmd/ksh93/tests/alias.sh:
  Add two regression tests for the bugs fixed by this commit.

(cherry picked from commit 16d5ea9b52ba51f9d1bca115ce8f4f18e97abbc4)
2020-06-12 01:45:18 +02:00
Johnothan King
ba1f2ba9c3 Fix a build error on Linux due to wrong include ordering
This regression was caused by the commit that introduced
the times builtin. The ordering of the #include directives
was causing the build to fail on Ubuntu 20.04 and Arch Linux.

src/cmd/ksh93/bltins/misc.c:
 - Move the includes for standard headers to the bottom
   of the includes to fix build failures on Linux.

(cherry picked from commit 3a42b206a0cc71bedc4773ce545435570e862e61)
2020-06-12 01:45:17 +02:00
Martijn Dekker
40d0bdc287 Fix regression in 'return' introduced by 98e0fc94
This fixes two regression test failures in tests/functions.sh.

src/cmd/ksh93/bltins/cflow.c: b_exit():
- The exit status should of course only be cropped to 8 bits if
  b_exit() will actually exit, not if it returns from a function.

(cherry picked from commit 2d1e7f87551159c942b16de2a98dc72697988d26)
2020-06-12 01:45:17 +02:00
Martijn Dekker
d024d4c895 Fix signal handling due to exit status > 256
This fixes two bugs: issuing the 'exit' command with a value > 256
would cause ksh 93u+ to kill itself with the corresponding signal
(try 'exit 265' to SIGKILL your interactive shell), and, if the
last command of a script exits due to a signal, the shell would
repeat that signal to itself, causing any parent ksh to also be
killed.

Discussion:
https://bugzilla.redhat.com/show_bug.cgi?id=1469624
https://rainbow.chard.org/2017/03/21/ksh-deliberately-segfaults-if-the-last-command-in-a-script-crashes/

This commit is loosely based on a patch applied to the 93v- beta
and the abandoned ksh2020, but that patch was incomplete & broken:
  $ ksh-2020.0.0 -c 'exit 265'; echo $?
  137
Expected: 9. Since the exit was *not* due to a signal, the value
should simply be cropped to the 8 bits supported by the OS.

src/cmd/ksh93/bltins/cflow.c: b_exit():
- For the 'exit' builtin command, bitwise-AND the argument to
  'exit' with SH_EXITMASK (8 bits, crop to 0-255) before passing it
  on to sh_exit(). This restores the behaviour of <=2011 ksh93
  versions and is in line with all other POSIX shells.
  It also fixes this bogosity:
    $ (exit 265); echo $?                   # non-forked subshell
    265
    $ (ulimit -t unlimited; exit 265); echo $?  # forked subshell
    9
  Forked or non-forked should make no difference at all
  (see commit message a0e0e29e for why).

src/cmd/ksh93/sh/fault.c: sh_done():
- If the current exit status is equal to the status for the last
  signal that was received from a child process, remove the
  SH_EXITSIG (9th) bit, so that the shell doesn't kill itself.
- If the shell's last child process exits due to a signal, exit
  with a portable 8-bit exit status (128 + signal number). This
  avoids the exit status being < 128 by being cropped to 8 bits.

src/cmd/ksh93/tests/signal.sh:
- Add regression test for exit with status > 256.
- Add regression test verifying the shell no longer kills itself.

(cherry picked from commit 98e0fc94393e175ce6adfee390327c320795bf12)
2020-06-12 01:45:17 +02:00
Martijn Dekker
f88f302c38 Remove code related to long-dead 3DFS research project
This commit gets rid of dead weight related to an obscure early
1990s Bell Labs versioning file system research project called
3DFS, which has not existed for decades and for which I have not
managed to find any evidence that it was ever used outside the lab.

This removes:
- the SHOPT_FS_3D compile option (which was forced on even when 0)
- the obnoxious default alias 2d='set -f;_2d' that turned off your
  globbing and then tried to run a nonexistent _2d command
- undocumented builtins 'vmap' and 'vpath' that only errored out
- a non-functional -V unary operator for the test and [[ commands
- some specific code for Apollo workstations (last made in 1997),
  which was inseparably intertwined with the 3DFS code

(cherry picked from commit 20cdf3709f4fb4e468057b534dcee819b1961fb6)
2020-06-12 01:45:17 +02:00
Martijn Dekker
65d363fd34 POSIX compliance fix: make 'times' a proper builtin
As of this commit, the 'times' command is a POSIX-compliant special
builtin command instead of an alias that doesn't produce the
required output. It displays the accumulated user and system CPU
times, one line with the times used by the shell and another with
those used by all of the shell's child processes.
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_27

This was originally written by Kurtis Rader and is now backported
and tweaked from the abandoned ksh2020 branch. I chose an earlier
and simpler version[*1] that uses times(3), with a precision of
hundredths of seconds, so it outputs the same precision as mksh and
zsh. Rader later wrote another version[*2] that uses getrusage(2),
giving it the same millisecond precision as bash. But that required
adding a feature test and a fallback to the old version, which is
non-trivial in the old INIT/iffe system. This simpler version is
enough to gain POSIX compliance and I think it will do very nicely
in this stable bugfix branch.

[*1] https://github.com/att/ast/pull/1332
[*2] https://github.com/att/ast/commit/038045f6

src/cmd/ksh93/bltins/misc.c
- Add b_times() function for 'times' builtin.
- Note we include <times.h>, not <sys/times.h>, so that we use the
  AST feature-tested version with fallback on systems that need it.

src/cmd/ksh93/data/aliases.c:
- Remove times='{ { time;} 2>&1;}' builtin alias.

src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/include/builtins.h:
- Add entry for 'times' special builtin.
- Add --help/--man info for same.

src/cmd/ksh93/sh.1:
- Update manual page.

src/cmd/ksh93/tests/builtins.sh:
- Add a couple of simple regression tests.

(cherry picked from commit ebf71e619eb298ec1cf6b81d1828fa7cdf6e9203)
2020-06-12 01:45:16 +02:00
Martijn Dekker
6f0e008cf7 Fix unsetting special vars in subshells (re: efa31503, 8b7f8f9b)
This fixes (or at least works around) a bug that caused special
variables such as PATH, LANG, LC_ALL, LINENO, etc. to lose their
effect after being unset in a subshell.

For example:
(unset PATH; PATH=/dev/null; ls); : wrongly ran 'ls'
(unset LC_ALL; LC_ALL=badlocale); : failed to print a diagnostic

This is yet another problem with non-forking/virtual subshells. If
you forced the subshell to fork (one way of doing this is using the
'ulimit' builtin, e.g. ulimit -t unlimited) before unsetting the
special variable, the problem vanished.

I've tried to localise the problem. I suspect the sh_assignok()
function, which is called from unall(), is to blame. This function
is supposed to make a copy of a variable node in the virtual
subshell's variable tree. Apparently, it fails to copy the
associated permanent discipline function settings (stored in the
np->nvfun->disc pointer) that gave these variables their special
effect, and which survive unset. However, my attempts to fix that
have been unsuccessful. If anyone can figure out a fix, please send
a patch/pull request!
Data point: This bug existed in 93u 2011-02-08, but did not yet
exist in M-1993-12-28-s+. So it is a regression.

Meanwhile, pending a proper fix, this commit adds a safe
workaround: it forces a non-forked subshell to fork before
unsetting such a special variable.

src/cmd/ksh93/bltins/typeset.c: unall():
- If we're in a non-forked, non-${ ...; } subshell, then before
  unsetting any variable, check for variables with internal
  trap/discipline functions, and call sh_subfork() if any are
  found. To avoid crashing, this must be done before calling
  sh_pushcontext(), so we need to loop through the args separately.

src/cmd/ksh93/tests/variables.sh:
- Remove the 'ulimit' that forced the fork; we do this in C now.

(cherry picked from commit 21b1a67156582e3cbd36936f4af908bb45211a4b)
2020-06-12 01:45:16 +02:00
Martijn Dekker
759157bdb2 Fix hang in unsetting functions in subshells (re: dde38782)
This fixes a really stupid bug in my own code for unsetting a
function in a subshell. The algorithm for walking through the
subshell tree was broken, resulting in an infinite loop if there
were multiple levels of subshell.

src/cmd/ksh93/bltins/typeset.c:
- Correct the subshell function tree walk that deletes functions
  from zombie parent scopes.

src/cmd/ksh93/tests/subshell.sh:
- Add a regression test for setting and unsetting identically named
  functions in multiple levels of subshell.

(cherry picked from commit 972a7999c7f16469138daf3d86dfd6c0db3f4879)
2020-06-12 01:45:15 +02:00
Martijn Dekker
7003aba487 Fix 'test'/'[' exit status >1 on error in arithmetic expression
Fix BUG_TESTERR1A: POSIX non-compliance of 'test'/'[' exit status
on error. The command now returns status 2 instead of 1 when given
an invalid number or arithmetic expression, e.g.: [ 123 -eq 123x ]

The problem was that the test builtin (b_test()) calls the generic
arithmetic evaluation subsystem (sh/arith.c, sh/streval.c) which
has no awareness of the test builtin. A simple solution would be to
always make the arithmetic subsystem use an exit status > 1 for
arithmetic errors, but globally changing this may cause backwards
compatibility issues. So it's best to change the behaviour of the
'test' builtin only. This requires the arithmetic subsystem to be
aware of whether it was called from the 'test' builtin or not. To
that end, this commit adds a global flag and overrides the
ERROR_exit macro where needed.

src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/defs.c:
- Declare and initialise a global sh_in_test_builtin flag.
- Declare internal function for ERROR_exit override in test.c.

src/cmd/ksh93/bltins/test.c:
- Add override for ERROR_exit macro using a function that checks if
  the exit status is at least 2 if the error occurred while running
  the test builtin.
- b_test(): Set sh_in_test_builtin flag while running test builtin.

src/cmd/ksh93/sh/arith.c,
src/cmd/ksh93/sh/streval.c:
- Override ERROR_exit macro using function from test.c.

src/cmd/ksh93/tests/bracket.sh:
- Add regression test verifying status > 1 on arith error in test.

(cherry picked from commit 5eeae5eb9fd5ed961a5096764ad11ab870a223a9)
2020-06-12 01:45:15 +02:00
Martijn Dekker
ec888867fd Fix unsetting aliases in subshells
Aliases can now be correctly unset within subshell environments
(such as ( ... ), $(command substitutions), etc), as well as
non-subshell "shared" command substitutions (${ ...; }). Before,
attempts to unset aliases within these were silently ignored.

Prior discussion: https://github.com/att/ast/issues/108

Subshell alias trees are only referenced in a few places in the
code, *and* have always been broken, so this commit gets rid of the
whole notion of a subshell alias tree. Instead, there is now just
one flat alias tree, and subshells fork into a separate process
when aliases are set or unset within them. It is not really
conceivable that this could be a performance-sensitive operation,
or even a common one, so this is a clean fix with no downside.

src/cmd/ksh93/include/defs.h:
- Remove sh_subaliastree() definition.

src/cmd/ksh93/sh/subshell.c:
- Remove salias element (pointer to subshell alias tree) from
  subshell struct.
- Remove sh_subaliastree() function.
- sh_subshell(): Remove alias subshell tree cleanup.

src/cmd/ksh93/bltins/typeset.c:
- b_alias(): If in subshell, fork before setting alias.
- b_unalias(): If in subshell, fork before unsetting alias.
- unall(): Remove sh_subaliastree() call.

src/cmd/ksh93/sh/name.c:
- nv_open(): Remove sh_subaliastree() call.

src/cmd/ksh93/tests/subshell.sh:
- Add regression tests for unsetting or redefining aliases within
  subshells.

(cherry picked from commit 12a15605b9521a2564a6e657905705a060e89095)
2020-06-12 01:45:14 +02:00
Martijn Dekker
047cb3303c Fix redefining & unsetting functions in subshells (BUG_FNSUBSH)
Functions can now be correctly redefined and unset in subshell
environments (such as ( ... ), $(command substitutions), etc).
Before this fix, attempts to do this were silently ignored (!!!),
causing the wrong code (i.e.: the function by the same name from
the parent shell environment) to be executed.

Redefining and unsetting functions within "shared" command
substitutions of the form '${ ...; }' is also fixed.

Prior discussion: https://github.com/att/ast/issues/73

src/cmd/ksh93/sh/parse.c:
- A fix from George Koelher (URL above). He writes:
  | The parser can set t->comnamp to the wrong function.
  | Suppose that the shell has executed
  |     foo() { echo WRONG; }
  | and is now parsing
  |     (foo() { echo ok; } && foo)
  | The parser was setting t->comnamp to the wrong foo. [This
  | fix] doesn't set t->comnamp unless it was a builtin. Now the
  | subshell can't call t->comnamp, so it looks for foo and finds
  | the ok foo in the subshell's function tree.

src/cmd/ksh93/bltins/typeset.c:
- Unsetting functions in a virtual/non-forked subshell still
  doesn't work: nv_open() fails to find the function. To work
  around this problem, make 'unset -f' fork the subshell into its
  own process with sh_subfork().
- The workaround exposed another bug: if we unset a function in a
  subshell tree that overrode a function by the same name in the
  main shell, then nv_delete() exposes the function from the main
  shell scope. Since 'unset -f' now always forks a subshell, the
  fix is to simply walk though troot's parent views and delete any
  such zombie functions as well. (Without this, the 4 'more fun'
  tests in tests/subshell.sh fail.)

src/cmd/ksh93/sh/subshell.c: sh_subfuntree():
- Fix function (re)definitions and unsetting in "shared" command
  substitutions of the form '${ commandlist; }' (i.e.: if
  sp->shp->subshare is true). Though internally this is a weird
  form of virtual subshell, the manual page says it does not
  execute in a subshell (meaning, all changes must survive it), so
  a subshell function tree must not be created for these.

src/cmd/ksh93/tests/subshell.sh:
- Add regression tests related to these bugfixes. Test unsetting
  and redefining a function in all three forms of virtual subshell.

(cherry picked from commit dde387825ab1bbd9f2eafc5dc38d5fd0bf9c3652)
2020-06-12 01:45:14 +02:00
Martijn Dekker
952944197f Fix bugs in testing if a parameter is set
This fixes three related bugs:
1. Expansions like ${var+set} remained static when used within a
   'for', 'while' or 'until' loop; the expansions din't change
   along with the state of the variable, so they could not be used
   to check whether a variable is set within a loop if the state of
   that variable changed in the course of the loop. (BUG_ISSETLOOP)
2. ${IFS+s} always yielded 's', and [[ -v IFS ]] always yielded
   true, even if IFS is unset. (BUG_IFSISSET)
3. IFS was incorrectly exempt from '-u' ('-o nounset') checks.

src/cmd/ksh93/sh/macro.c: varsub():
- When getting a node pointer (np) to the parameter to test,
  special-case IFS by checking if it has a value and not setting
  the pointer if not. The node to IFS always exists, even after
  'unset -v IFS', so before this fix it always followed the code
  path for a parameter that is set. This fixes BUG_IFSISSET for
  ${IFS+s} and also fixes set -u (-o nounset) with IFS.
- Before using the 'nv_isnull' macro to check if a regular variable
  is set, call nv_optimize() if needed. This fixes BUG_ISSETLOOP.
  Idea from Kurtis Rader: https://github.com/att/ast/issues/1090
  Of course this only works if SHOPT_OPTIMIZE==1 (the default),
  but if not, then this bug is not triggered in the first place.
- Add some comments for future reference.

src/cmd/ksh93/bltins/test.c: test_unop():
- Fix BUG_IFSISSET for [[ -v IFS ]]. The nv_optimize() method
  doesn't seem to have any effect here, so the only way that I can
  figure out is to special-case IFS, nv_getval()'ing it to check if
  IFS has a value in the current scope.

src/cmd/ksh93/tests/variables.sh:
- Add regression tests for checking if a varariable is set within a
  loop, within and outside a function with that variable made local
  (to check if the scope is honoured). Repeat these tests for a
  regular variable and for IFS, for ${foo+set} and [[ -v foo ]].

(cherry picked from commit a2cf79cb98fa3e47eca85d9049d1d831636c9b16)
2020-06-12 01:45:14 +02:00
Martijn Dekker
cafe33f048 Fix 'test -t 1' in $(command substitutions)
Standard output (FD 1) tested as being on a terminal within a
command substitution, which makes no sense as the command
substitution is supposed to be catching standard output.
    ksh -c 'v=$(echo begincomsub
                [ -t 1 ] && echo oops
                echo endcomsub)
            echo "$v"'
This should not output "oops".

This is one of the many bugs with ksh93 virtual (non-forked)
subshells. On the abandoned Vashist/Rader ksh2020 branch, this bug
was fixed by changing quite a lot of code, which introduced and/or
exposed another bug:
	https://github.com/att/ast/issues/1079
	https://github.com/att/ast/commit/8e1e405e
	https://github.com/att/ast/issues/1088
That issue was unresolved when the ksh2020 branch was abandoned.

The safer and more conservative fix is simply forcing the subshell
to fork if we're in a non-forked command substitution and testing
'-t 1'. It is hard to imagine a situation where this would cause a
noticable performance hit.

Note that this fix does not affect ksh93-specific "shared"
non-subshell ${ command substitutions; } which are executed in the
main shell environment, so that variables survive, etcetera.
'test -t 1' continues to wrongly return true there, but command
substitutions of that form cannot be forked because that would
defeat their purpose.

src/cmd/ksh93/bltins/test.c:
- Fix 'test -t 1', '[ -t 1 ]' and '[[ -t 1 ]]' by forking the
  current subshell if it is a virtual/non-forked subshell
  (shp->subshell), and a command substitution (shp->comsub), but
  NOT a "shared" ${ command substitution; } (!shp->subshare).

src/cmd/ksh93/tests/bracket.sh:
- Add two regression tests for this issue, which were adapted from
  the Vashist/Rader ksh2020 branch.

NEWS, src/cmd/ksh93/include/version.h:
- Update.

(cherry picked from commit b8ef05e457ead65b83417699b8dd8632f855e2fa)
2020-06-12 01:45:13 +02:00
Martijn Dekker
93e15a3035 Fix BUG_PUTIOERR: Check for and report I/O error in output builtins
This allows scripts to check for a nonzero exit status on the
'print', 'printf' and 'echo' builtins and prevent possible infinite
loops if SIGPIPE is ignored.

sfsync() was already returning a negative value on I/O error, so
all we need to do is add a check. The stream buffer will need to be
filled before an I/O error can be detected, but this is the same on
other shells. See manual page: src/lib/libast/man/sfio.3

Ref.: https://github.com/att/ast/issues/1093
      https://github.com/att/ast/pull/1363

src/cmd/ksh93/bltins/print.c: b_print():
- Make sure an error result from sfsync() is reflected in the
  output builtin's exit status (exitval).
- Write an I/O error message (e_io) if the exit status is nonzero.

src/cmd/ksh93/data/msg.c, src/cmd/ksh93/include/io.h:
- Add the e_io[] error message.

src/cmd/ksh93/tests/builtins.sh:
- Add I/O error regression test, checking for the correct error
  message and exit status. All three output builtins use the same
  b_print() function so we only need to test one.

src/cmd/ksh93/tests/basic.sh,
src/cmd/ksh93/tests/coprocess.sh:
- Redirect stderr on a few 'print' commands to /dev/null; these
  now issue an expected I/O error. This does not cause failures.

NEWS, TODO:
- Update.

(cherry picked from commit 9011fa933552e483dab460f7dd1593d64e059d94)
2020-06-12 01:45:13 +02:00
Martijn Dekker
846ad93272 Fix 'print -s -f'
This fix is backported from the Vashisht/Rader ksh2020 branch.

src/cmd/ksh93/bltins/print.c:
- Fix syncing history when print -s -f is used. For example, the
  following now correctly adds a 'cd' command to the history:
	print -s -f 'cd -- %q\n' "$PWD"
  Ref.:	https://github.com/att/ast/issues/425
	https://github.com/att/ast/pull/442

src/cmd/ksh93/include/version.h:
- Version date bump.

(cherry picked from commit 46ba7ecdc5c14cd73f6cb24b16c50bdc331a000e)
2020-06-12 01:45:13 +02:00
Martijn Dekker
53443bb981 Fix 47 typos in user-facing help and error messages
(cherry picked from commit df9b2ea03efdf82c0eb6bb53ed9b128c1513a299)
2020-06-12 01:45:12 +02:00