1
0
Fork 0
mirror of git://git.code.sf.net/p/cdesktopenv/code synced 2025-02-13 19:52:20 +00:00
Commit graph

127 commits

Author SHA1 Message Date
Martijn Dekker
6109f54a5e Fix for no-argument use of 'cd' in .../fun/dirs
'cd' with no argument is supposed to go to your home directory, but
in this override function it produced an error due to an incorrect
use of $@ within another parameter substitution, preventing $@ from
expanding to zero words.

Ref.: https://groups.google.com/d/msgid/korn-shell/781ed34b-6860-5d47-dfe8-be21280a6b31%40inlv.org

src/cmd/ksh93/fun/dirs: _cd():
- Fix the bug by setting a positional parameter if needed,
  then using a normal and correct "$@" expansion for \cd.
- While we're here, restore a historic comment about using
  'command' to bypass aliases that exists in the listing of this
  function in the official 1995 KornShell book (pp. 274-276).
  Presumably, that comment was deleted when they decided to add the
  obnoxious and broken default alias command='command ', whenever
  that was. This branch got rid of that default alias in 61d9bca5.
2020-06-14 10:39:48 +02:00
Martijn Dekker
d66f2a80d8 sh.1: default PS1 is "$ ", not "$" 2020-06-14 08:42:07 +02:00
Martijn Dekker
1a6f37dc86 NEWS: update repo URI 2020-06-14 06:28:38 +02:00
Martijn Dekker
76f7d15a93
Merge pull request #8 from JohnoKing/fix-freebsd-date
Backport the ksh2020 fix for timezone name determination

Partial fix for #6.
2020-06-14 00:07:51 +01:00
Martijn Dekker
a739115ffe
Merge pull request #7 from JohnoKing/fix-special-vars
Implement a better fix for unsetting special env vars.

See #7 for more explanations and ksh source code history :)
2020-06-14 00:05:40 +01:00
Johnothan King
d7c9470704 Backport the ksh2020 fix for timezone name determination
This fix for `printf '%T' now` on FreeBSD was written by
@krader1961. This is from https://github.com/att/ast/pull/591:

On FreeBSD calling tzset() does not guarantee the tzname array will
be correctly populated. On most systems that works but on FreeBSD you
have to call localtime() or a related function (e.g., ctime()).

This change also eliminates a potential, very small, memory leak due to
the strdup()'ed tznames not being freed.

src/lib/libast/tm/tminit.c:
 - Fix timezone name determination on FreeBSD and a memory leak.
2020-06-13 14:28:10 -07:00
Johnothan King
7b994b6a7e Implement a better fix for unsetting special env vars
The regression this commit fixes was first introduced in ksh93t
2008-07-25. It was previously worked around in 6f0e008c by forking
subshells if any special environment variable is unset.

The reason why this problem doesn't occur in ksh93s+ is because in
that version of ksh sh_assignok never moves nodes, it only clones
them. The second argument doesn't set NV_MOVE, which makes
`sh_assignok(np,0)` is similar to `sh_assignok(np,1)`. In ksh93t and
higher, setting the second argument to zero causes the node to be moved
with NV_MOVE, which causes the discipline function associated with
the variable node to be removed when `np->nvfun` is set to zero (i.e.
NULL). This is why a command like `(unset LC_NUMERIC; LC_NUMERIC=invalid)`
doesn't print a diagnostic, as it looses its discipline function.

This patch fixes the problem by cloning the node with sh_assignok
if it is a special variable with a discipline function. This allows
special variables to work as expected in virtual subshells. The
original workaround has been kept for the $PATH variable only, as
hash tables are still broken in virtual subshells. It has been updated
accordingly to only fork subshells if it detects the variable node
for PATH. I have added two more regression tests for changing the
PATH in subshells to make sure hash tables continue working as
expected with this fix.

src/cmd/ksh93/bltins/typeset.c:
 - Only fork virtual subshells if the PATH will be changed. If a
   variable is a special variable with a discipline function, it
   should be just be cloned, not moved.

src/cmd/ksh93/sh/nvdisc.c:
 - Add a comment to clarify that NV_MOVE will delete the discipline
   function associated with the node.

src/cmd/ksh93/tests/subshells.sh:
 - Add two more regression tests for unsetting the PATH in subshells,
   one for if PATH is being pointed to by a nameref. Condense the
   hash table tests by moving the main test into a single function.
2020-06-13 12:55:48 -07:00
Martijn Dekker
289f56cd4c tests/pty.sh: fix regress fail due to $TMPDIR
Test 137(C) was failing on some systems because $TMPDIR was set and
the local vi(1) honours it, so that the expected '/tmp/' string was
never output by vi. For compatibility with vi programs that honour
$TMPDIR and those that always use /tmp, we must export TMPDIR=/tmp.

src/cmd/ksh93/tests/pty.sh:

- Export TMPDIR=/tmp for test 137(C).
     Note that this exports TMPDIR to the environment for the
  duration of the 'tst' function run because the function was
  defined using the ksh 'function tst { ...; }' syntax.
2020-06-13 01:48:13 +02:00
Martijn Dekker
881a5be07c Remove obsolete docs, keeping official description
The lib/package/ast-open.* are almost entirely obsolete after
most of the AST utilities were removed to concentrate on ksh93
development. The only part that is both relevant and not repeated
elsewhere is AT&T's introductory blurb on ksh93.

README.md:
- Add the official AT&T ksh93 description from ast-open.html,
  reformatted to Markdown with a couple of added hyperlinks.

lib/package/ast-open.README,
lib/package/ast-open.html:
- Removed. (Note that the historic ksh93 changelog in these files
  is also included in src/cmd/ksh93/RELEASE).
2020-06-12 20:15:20 +02:00
Martijn Dekker
d600cf61fa use 'ksh93' org identifier for new/changed builtins
We cannot identify the new builtins as coming from "AT&T Research"
as they don't. 'ksh community' is accurate but a bit long. Since
we're now at a GitHub org called ksh93, let's just use that.
2020-06-12 18:54:08 +02:00
Martijn Dekker
a5e2fa8db7
README.md: tweak policy 2020-06-12 14:42:23 +01:00
Martijn Dekker
5aea9bbe08
README.md: update NEWS link after transfer to ksh93 org 2020-06-12 13:52:51 +01:00
Martijn Dekker
e500479ede
Merge pull request #1 from JohnoKing/fix-builtin-delete
`builtin -d` should not delete special builtins
2020-06-12 12:36:42 +01:00
Johnothan King
017d088c39 builtin -d should not delete special builtins
The man page for the builtin command says special builtins cannot
be deleted. This wasn't the case though, running `builtin -d` on
a special builtin was deleting it. As an example, the following
set of commands was ending with 'export: not found':

$ builtin -d export
$ export foo=bar

This commit backports the bugfix from ksh93v- (2014-12-24-beta),
which added an error check to prevent special builtins from being
deleted.

src/cmd/ksh93/sh/nvdisc.c:
 - Add an error check to prevent special builtins from being deleted.

src/cmd/ksh93/tests/builtins.sh
 - Add a regression test for using `builtin -d` on special builtins.
2020-06-12 04:26:40 -07:00
Martijn Dekker
cd5c181a39 Restore AST 'pty' pseudoterminal command (re: reboot)
This enables the tests/pty.sh regression tests for the interactive
shell to work. I've managed to restore these without restoring
nmake (which would cause unresolved build failures). It involved
hand-editing the Mamfile originally generated by nmake.

src/cmd/builtin/Makefile:
- Restored. Even though nmake is not installed, apparently mamake
  still needs the Makefile for the 'install' target to work. This
  will probably need editing if nmake is ever restored.

src/cmd/builtin/Mamfile:
- Restored and edited by hand to remove the commands to build all
  the utilities except pty, as well as all test targets (pty has no
  test). Most of this was systematically deleting blocks delimited
  by 'make FOO' and 'done FOO generated' until the build stopped
  failing -- a task made so much easier by the build now failing
  gracefully on error (see d0dfb37c). To see the diff, do:
	git diff cc1f2bf8 src/cmd/builtin/Mamfile

src/cmd/builtin/RELEASE:
- Restored for completeness' sake with all non-pty entries removed.

src/cmd/builtin/features/pty,
src/cmd/builtin/pty.c:
- Restored.
2020-06-12 12:35:58 +02:00
Martijn Dekker
802ea67afb tests/io.sh: don't abort entire suite on failure
src/cmd/ksh93/tests/io.sh:
- 16 tests for the 'redirect' builtin used the ((arithmetic
  command)) to check the result. This does not tolerate the result
  being a non-number, such as the empty string, which may occur on
  test failure. So use [[ ... -eq ArithExpression ]] to check these
  results instead; it simply returns false, failing gracefully.
2020-06-12 11:34:10 +02:00
Martijn Dekker
d8eba9d112 Remove 'login' and 'newgrp' builtins: not sane default behaviour
This commit removes the undocumented 'login' and 'newgrp' builtin
commands. They already stopped blocking shell functions by that
name by changing from special to regular builtins in 04b91718 (a
change I forgot to mention in that commit message), but there is
another obnoxious aspect to these: being glorified hooks into
'exec', they replaced your shell session with the external commands
by the same name. This makes argument and error checking
impossible, so if you made so much as a typo, you would be
immediately logged out.

Even if that behaviour is wanted by a few, having it as the default
is user-hostile enough to be called a bug. It also violates the
POSIX definition of the 'newgrp' utility which explicitly says that
it "shall create a new shell execution environment", not replace
the existing one.
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/newgrp.html

Users who do want this behaviour can easily restore it by setting:
	alias login='exec login'
	alias newgrp='exec newgrp'

src/cmd/ksh93/bltins/misc.c:
- As there is no more 'login' builtin, combine b_exec() and
  B_login() functions, which allows eliminating a few variables.
  Note that most of 'exec' was actually implemented in B_login()!

src/cmd/ksh93/data/builtins.c:
- Remove "login" and "newgrp" table entries.

src/cmd/ksh93/include/builtins.h:
- Remove SYSLOGIN parser ID. As this was the first, all the others
  needed renumbering.

src/cmd/ksh93/sh/xec.c:
- Remove SYSLOGIN parser check that made 'login' and 'newgrp' act
  like 'exec' and replace the shell.
2020-06-12 06:57:57 +02:00
Martijn Dekker
7b82c338da Make 'redirect' a regular builtin instead of an alias of 'exec'
This commit converts the redirect='command exec' alias to a regular
'redirect' builtin command that only accepts I/O redirections, which
persist as in 'exec'. This means that:
* 'unlias -a' no longer removes the 'redirect' command;
* users no longer accidentally get logged out of their shells if
  they type something intuitive but wrong, like 'redirect ls >file'.

This should not introduce any legitimate change in behaviour. If
someone did accidentally pass non-redirection arguments to
'redirect', unexpected behaviour would occur; this now produces
an 'incorrect syntax' error.

src/cmd/ksh93/bltins/misc.c: b_exec():
- Recognise 'redirect' when parsing options.
- If invoked as 'redirect', produce error if there are arguments.

src/cmd/ksh93/data/aliases.c:
- Remove redirect='command exec' alias.

src/cmd/ksh93/data/builtins.c:
- Update/improve comments re ordering.
- Add 'redirect' builtin entry.
- sh_optexec[]: Abbreviate redirection-related documentation;
  refer to redirect(1) instead.
- sh_optredirect[]: Add documentation.

src/cmd/ksh93/include/builtins.h:
- Add SYSREDIR parser ID, renumbering those following it.
- Improve comments.
- Add extern sh_optredirect[].

src/cmd/ksh93/sh.1:
- exec: Abbreviate redirection-related documentation; refer to
  'redirect' instead.
- redirect: Add documentation.

src/cmd/ksh93/sh/xec.c:
- Recognise SYSREDIR parser ID in addition to SYSEXEC when
  determining whether to make redirections persistent.

src/cmd/ksh93/tests/io.sh:
- To regress-test the new builtin, change most 'command exec' uses
  to 'redirect'.
- Add tests verifying the exit behaviour of 'exec', 'command exec',
  'redirect' on redirections.
2020-06-12 04:54:33 +02:00
Martijn Dekker
936802f92a Simplify exit status fix, restore interruptability
The exit status fix introduced in cherry-picked commit 22c3a6e1
introduced a problem: interrupting the build only interrupted the
main process, and the background job would happily continue.

bin/package, src/cmd/INIT/package.sh:
- Give up on the idea of a background job and put up with not
  cleaning up the FIFO. It's in a build directory that is going to
  get cleaned up anyway.
2020-06-12 02:11:46 +02:00
Martijn Dekker
7a8a46700c Build on more darwin/macOS versions
src/cmd/INIT/cc.darwin*:
- Build on ancient Mac OS X gcc versions (darwin 7+, Mac OS X
  10.3+), more recent gcc versions (darwin 11+, Mac OS X 10.7+), as
  well as current clang versions. So there are now three cc.darwin*
  wrapper scripts for these different darwin versions.

bin/package, src/cmd/INIT/package.sh:
- Differentiate the host ID and pick the correct Darwin script based on
  the detected OS release version.
2020-06-12 01:45:19 +02:00
Martijn Dekker
04b9171858 POSIX compliance fix: make 'unalias' a regular builtin
Both 'alias' and 'unalias' are specified as regular builtins. Among
a few other things, that means it ought to be portable to use these
names for shell functions. But ksh93 disallowed that until now.

src/cmd/ksh93/data/builtins.c:
- Make 'unalias' a regular builtin by removing the BLT_SPC flag.
- (same fix for 'alias' was already done in afa68dca)
- Add the BLT_ENV flag to the 'alias' and 'hash' commands. In
  include/name.h, this flag is commented: "non-stoppable, can
  modify environment". The "non-stoppable" bit seems like a good
  idea: these operations should not be interruptable as that would
  cause an inconsistent state.

src/cmd/ksh93/sh.1:
- Remove the '-', indicating special builtin, from 'alias' entry.
- Minor cosmetic fix: space after the '-' for 'unset'.

(cherry picked from commit a4315d7672204acb543010b4d4916b22dcb9cb08)
2020-06-12 01:45:18 +02:00
Martijn Dekker
2da22f4023 data/builtins.c: minor cosmetic 'alias --man' fix (re: d8428a833)
(cherry picked from commit 98be5130d3a46a565d0ad601aac838531765fc43)
2020-06-12 01:45:18 +02:00
Martijn Dekker
80d9ae2b1c Refactor new b_hash(); better hash table clear (re: d8428a83)
The b_hash() function duplicated much of its code from b_alias(),
while b_alias() retained some code to support being called as
'hash'. There is no reason why 'hash' and 'alias' can't be handled
with a single function, as is the case several other builtins. Note
that option parsing can easily be made dependent on the name the
command was invoked with (in this case, argv[0]=='h').

The new hash builtin's -r option cleared the hash table by
assigning to PATH its existing value, triggering its associated
discipline function (put_restricted() in init.c) which then
actually cleared the hash table. That's a bit of a hack. It's nicer
if we can just do that directly. This requires taking a static
handler function rehash() from init.c, which invalidates one hash
table entry, and making it available to the builtin.

src/cmd/ksh93/bltins/typeset.c,
src/cmd/ksh93/include/builtins.h,
src/cmd/ksh93/include/nval.h,
src/cmd/ksh93/sh/init.c,
src/cmd/ksh93/sh/name.c:
- Merge b_hash() into b_alias().
- The -x option was still uselessly setting the NV_EXPORT flag.
  Exported aliases were in ksh88 but were removed in ksh93.
- Rename rehash() handler function from init.c to nv_rehash
  (avoiding a possible conflict with another rehash() in cd_pwd.c)
  and move it to name.c just above nv_scan(), which it's meant to
  be used with. Make it an extern so typeset.c can use it.
- b_alias(): Replace the PATH assignment by an nv_scan() call to
  clear the hash table directly using the nv_rehash() handler.

src/cmd/ksh93/data/builtins.c:
- POSIX compliance fix: Remove BLT_SPC (special builtin) flag from
  "alias" definition. 'alias' is specified as a regular builtin.
- sh_optalias[]: Fix uninformative -t option documentation.
- sh_opthash[]: Edit for conciseness and clarity.

src/cmd/ksh93/sh.1:
- Edit the 'alias -t' and 'hash' documentation.
- Remove the -- prefix from the 'alias' entry, which indicated that
  it was supposed to be a declaration builtin like 'typeset', with
  assignment-arguments expanding tildes and not being subject to
  field splitting. However, my testing shows that 'alias' has never
  actually behaved that way on ksh93. Even adding the BLT_DCL flag
  in data/builtins.c doesn't seem to change that.

(cherry picked from commit afa68dca5c786daa13213973e8b0f9bf3a1dadf6)
2020-06-12 01:45:18 +02:00
Johnothan King
74b4162178 Fix set +r so that it cannot unset the restricted option
The ksh man page documents that the restricted option cannot be
unset once it is set, which means `set +r` should be invalid.
While this was true for `set +o restricted`, `set +r` was causing
the restricted option to be unset. The fix for this problem comes
from one of Solaris' patches, which adds an error check to prevent
this behavior.

Solaris' patch:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/020-CR6919590.patch

src/cmd/ksh93/sh/args.c:
 - Add an error check to stop `set +r` from unsetting the
   restricted option.

src/cmd/ksh93/tests/restricted.sh:
 - Add two regression tests to make sure the restricted option
   cannot be unset.

(cherry picked from commit bef4fee404d8e24b38fce66420c14a39ac4a123e)
2020-06-12 01:45:18 +02:00
Martijn Dekker
9cef2d534a shtests: make tests more interruptable with Ctrl+C
Sometimes you have to put ^C on rapid repeat to get out of running
the tests. This is because each test scripts is launch as a child
shell (not a subshell) in the foreground, which thus handle the ^C.
It then exits with a status indicating SIGINT, but the shtest script
wasn't handling this and just kept going.

src/cmd/ksh93/tests/shtests:
- For reasons I don't understand yet, interrupting tests with ^C
  tends to make them exit with status 130 (128+2) instead of what
  I would expect for ksh93, 258 (256+2). Not a big deal: POSIX
  specifies that any exit > 128 signifies a signal in any case,
  and 'kill -l' works even on those values. But the checks need
  changing to '> 128'.
- Check each result for SIGINT, and issue SIGINT to self if found.

(cherry picked from commit d0dfb37c6c71ac7b157060249125e0959130927d)
2020-06-12 01:45:18 +02:00
Johnothan King
102868f850 Replace the hash alias with a proper builtin
This commit replaces the old hash alias with a proper builtin.
I based this builtin off of the code alias uses for handling
`alias -t --`, but with the hack for `--` removed as it has
no use in the new builtin. `alias -t --` will no longer work,
that hack is now gone.

While I was testing this builtin, I found a bug with hash tables
in non-forking subshells. If the hash table of a non-forking
subshell is changed, the parent shell's hash table is also changed.
As an example, running `(hash -r)` was resetting the parent shell's
hash table. The workaround is to force the subshell to fork if the
hash table will be changed.

src/cmd/ksh93/bltins/typeset.c:
 - Move the code for hash out of the alias builtin into a dedicated
   hash builtin. `alias -t --` is no longer supported.

src/cmd/ksh93/data/aliases.c:
 - Remove the old alias for hash from the table of predefined aliases.

src/cmd/ksh93/data/builtins.c:
 - Fix the broken entry for the hash builtin and add a man page for
   the new builtin.

src/cmd/ksh93/sh.1:
 - Replace the entry for the hash alias with a more detailed entry
   for the hash builtin.

src/cmd/ksh93/sh/name.c:
 - Force non-forking subshells to fork if the PATH is being reset
   to workaround a bug with the hash tree.

src/cmd/ksh93/tests/alias.sh:
 - Add a regression test for resetting a hash table, then adding
   a utility to the refreshed hash table.

src/cmd/ksh93/tests/subshell.sh:
 - Add regression tests for changing the hash table in subshells.

(cherry picked from commit d8428a833afe9270b61745ba3d6df355fe1d5499)
2020-06-12 01:45:18 +02:00
Martijn Dekker
d1bd8f89b7 shtests: print CPU times used at end (re: ebf71e61)
Since we now have a shiny new POSIX compliant 'times' builtin,
we might as well use it.

src/cmd/ksh93/tests/shtests:
- Run 'times' at end of test run.
- Skip the pretty-printing until #7 is fixed.

(cherry picked from commit 2c27d9fbc239583004ec70377db98627eea5e294)
2020-06-12 01:45:18 +02:00
Johnothan King
e92faddbf9 Fix 39 spelling errors and a formatting issue
A column of whitespace in the NEWS file was removed for consistent
formatting. Most of the spelling errors were found with this
codespell dictionary:
https://github.com/orbitcowboy/codespell_dictionary

(cherry picked from commit 0e36b17abe5609c461a3e4da7041eb0fdf9991b7)
2020-06-12 01:45:18 +02:00
Martijn Dekker
90fd867171 Update manual after a0e0e29e
An oversight...

src/cmd/ksh93/sh.1:
- Undocument the fixed ${.sh.subshell} breakage.

(cherry picked from commit 42be8a3c6dd5a76d03e0456d7f53e38536757aa5)
2020-06-12 01:45:18 +02:00
Martijn Dekker
30b111d51a TODO: moar update
(cherry picked from commit 3bce00165113ed22e22c99d6347cc8211bea4db1)
2020-06-12 01:45:18 +02:00
Martijn Dekker
12a079ea2b TODO: update
(cherry picked from commit 1e6bf87c2c435799034c9d87236939ab8bd61fb9)
2020-06-12 01:45:18 +02:00
Martijn Dekker
a1f46d785f rm "I/O error" error msg; just keep >0 exit status (re: 9011fa93)
The bug was really that I/O errors in output builtins were
undetectable by any means. Having a >0 exit status is sufficient.
Adding an error message risks making existing ksh scripts noisier,
or even breaking them if they redirect stderr to stdout.

Note to self: in future, implement the minimum change necessary to
fix bugs, nothing more. The fact that I needed to add four extra
2>/dev/null to the regression tests should have been a hint.

src/cmd/ksh93/bltins/print.c,
src/cmd/ksh93/data/msg.c,
src/cmd/ksh93/include/io.h:
- Remove "I/O error" message.

src/cmd/ksh93/tests/builtins.sh:
- Update to check for exit status only.

src/cmd/ksh93/tests/basic.sh,
src/cmd/ksh93/tests/coprocess.sh:
- Revert four new '2>/dev/null' to suppress the error message.

(cherry picked from commit 5e17be24d18455b575b6e98bc631c6935ffc795a)
2020-06-12 01:45:18 +02:00
Martijn Dekker
b883c7a7f8 remove extra 'isalias' variable (re: 16d5ea9b)
src/cmd/ksh93/bltins/typeset.c: unall():
- There's no need to keep track of whether we're unsetting an alias
  using a new 'isalias' vaariable, as we can just repeat the check
  for 'troot==shp->alias_tree'.

(cherry picked from commit 68f3ae3acd0d19dd80805049a1c9afc1ae1e75c3)
2020-06-12 01:45:18 +02:00
Johnothan King
5d50f825e4 The unalias builtin should return an error for non-existent aliases
This commit fixes a bug that caused unalias to return a zero status
when it tries to remove an alias twice. The following set of commands
will no longer end with an error:

$ alias foo=bar
$ unalias foo
$ unalias foo && echo 'Error'

This commit is based on the fix present in ksh2020, but it has been
extended with another bugfix. The initial fix for this problem tried to
remove aliases from the alias tree without accounting for NV_NOFREE. This
caused any attempt to remove a predefined aliases (e.g. `unalias float`)
to trigger an error with free, as all predefined aliases are in read-only
memory. The fix for this problem is to set NV_NOFREE when removing aliases
from the alias tree, but only if the alias is in read-only memory. All
other aliases must be freed from memory to prevent memory leaks.

I'll also note that I am using an `isalias` variable rather than the `type`
enum from ksh2020, as the `VARIABLE` value is never used and was replaced
with a bool called `aliases` in the ksh2020 release. The `isalias` variable
is an int as the ksh93u+ codebase does not use C99 bools.

Previous discussion: https://github.com/att/ast/issues/909

- src/cmd/ksh93/bltins/typeset.c:
  Remove aliases from the alias tree by using nv_delete. NV_NOFREE
  is only used when it is necessary.

- src/cmd/ksh93/tests/alias.sh:
  Add two regression tests for the bugs fixed by this commit.

(cherry picked from commit 16d5ea9b52ba51f9d1bca115ce8f4f18e97abbc4)
2020-06-12 01:45:18 +02:00
Martijn Dekker
d18469d608 bin/package: exit with nonzero status on build failure
Build scripts and the like ought to be able to check the results of
the build using the exit status of the build command.

bin/package,
src/cmd/INIT/package.sh:
- Add global error_status variable to keep track of the highest
  exit status of all commands using capture(). Exit with that.
- capture():
  * When capturing build output using 'tee', change the regular
    pipe construct (which doesn't preserve the exit status of pipe
    elements except the last one) to a couple of background jobs
    connected with a FIFO (named pipe). This allows getting the
    build's exit status using the "wait" command.
  * Update error_status on every invocation of capture().

(cherry picked from commit 22c3a6e1b32f3ab7eeca45adb76863982dc3e334)
2020-06-12 01:45:17 +02:00
Johnothan King
22d6b4527d Avoid brute-forcing during the build process to prevent failed builds
The default behavior of the package script is to ignore
errors. This has disastrous effects if any part of ksh
or libast fails to build due to something like a syntax
error, as the build will stop long after the first error
occurred.

This commit removes the -k flag from the $makeflags variable
in both copies of the package script. This forces the build
to fail if a compiler error occurs.

bin/package and src/cmd/INIT/package.sh:
 - Remove the -k assignment to $makeflags so that compiler
   errors cause the build to fail.

(cherry picked from commit 87bfaa19f3d336feb7b7bd4f5d4caf8d553f2547)
2020-06-12 01:45:17 +02:00
Martijn Dekker
c2eabc57e0 shtests: report total errors on terminal and in exit status
Setting the exit status allows build scripts and the like
to actually detect regression test failures.

src/cmd/ksh93/tests/shtests:
- Add counter for total number of errors.
- Report total to stdout at the end.
- Set the exit status to the total number of errors, up to 125
  (higher statuses have special meanings).

(cherry picked from commit c0dd80b14d4d316b4e9eff4b1b67d4e47f23a6ba)
2020-06-12 01:45:17 +02:00
Martijn Dekker
fb652a7e50 shtests: fix loose ends
src/cmd/ksh93/tests/variables.sh:
- Tolerate a bit more time for the SECONDS verification test.

src/cmd/ksh93/tests/subshell.sh:
- Replace unportable 'head -c 1' by 'dd bs=1 count=1'
- Remove unnecessary uses of 'whence'.

src/cmd/ksh93/tests/builtins.sh:
- Add a regression test for a weirdly specific 'whence' bug exposed
  by the aforementioned unneccessary uses of 'whence', which only
  shows up on my old Power Mac G5 running Mac OS X 10.3. For all I
  know it's a compiler bug, but let's add a more clear failure for
  it here, in case that happens anywhere else.

(cherry picked from commit c3898bd1e6e40874845771d33a5b37220ef0b06e)
2020-06-12 01:45:17 +02:00
Martijn Dekker
712261c89b shtests: More speedups; also fix xtrace (re: 734e5953)
This reduces a bunch more unnecessarily long sleeps to give
asynchronous processes time to run, etc. (No, we don't need to be
compatible anymore with your cool 1985 Intel 80386DX 16 MHz
battlestation...) Running the test suite is almost tolerable now,
making me much more likely to actually run the regression test
suite and catch my own regressions.

In addition, there are various fixes to make the test suite
compatible with 'set -x' ('set -o xtrace') so that you can now
actually *use* the documented 'bin/shtests -x' option. Recommend
combining with '-p' to avoid tracing everything three times.

I've also added a really useful $PS4 trace prompt to shtests that
traces pretty much everything there is to trace. (It does use
expansions that modify ${.sh.match}, which affected several tests,
particularly in tests/substring.sh; for those we need to set a
temporary simpler $PS4.)

(cherry picked from commit c3a5d47cfe880b526cabb5370ddaced0e8626acd)
2020-06-12 01:45:17 +02:00
Johnothan King
ba1f2ba9c3 Fix a build error on Linux due to wrong include ordering
This regression was caused by the commit that introduced
the times builtin. The ordering of the #include directives
was causing the build to fail on Ubuntu 20.04 and Arch Linux.

src/cmd/ksh93/bltins/misc.c:
 - Move the includes for standard headers to the bottom
   of the includes to fix build failures on Linux.

(cherry picked from commit 3a42b206a0cc71bedc4773ce545435570e862e61)
2020-06-12 01:45:17 +02:00
Martijn Dekker
40d0bdc287 Fix regression in 'return' introduced by 98e0fc94
This fixes two regression test failures in tests/functions.sh.

src/cmd/ksh93/bltins/cflow.c: b_exit():
- The exit status should of course only be cropped to 8 bits if
  b_exit() will actually exit, not if it returns from a function.

(cherry picked from commit 2d1e7f87551159c942b16de2a98dc72697988d26)
2020-06-12 01:45:17 +02:00
Johnothan King
b5e52703e9 Fix an issue with the up arrow key in Emacs editing mode.
Emacs editing mode is bugged in ksh93u+ and ksh2020. Let's
say you were to run the following commands after starting
a fresh instance of ksh:

$ alias foo='true'
$ unalias foo

If you type 'a' and then press the up arrow on your keyboard,
ksh will complete 'a' to `alias foo='true'` by doing a reverse
search for the last command that starts with 'a'.
Run the alias command again, then type 'u' and press the up
arrow key again. If ksh is in Vi mode, you will get `unalias foo`,
but in Emacs mode you will get `alias foo='true'` again.

This bug was occurring because ksh was only doing a reverse search
based on the first command that was completed using the up arrow.
All subsequent commands were ignored as ksh was saving the first
command and only based later searches off of it.

NEWS:
 - Add instructions for reproducing this bug with the up arrow
   key and information about why this bug was happening in the
   first place.

src/cmd/ksh93/edit/emacs.c:
 - Remove a bad check that was preventing ksh from using the
   latest input for reverse search.

(cherry picked from commit 745b065487ad6bac693ec6f44752f96e87f9a63b)
2020-06-12 01:45:17 +02:00
Martijn Dekker
8f90d98280 tests/signal.sh: slash sleeps by 10 to reduce a long wait
The signal.sh regression tests took over a minute and a half to
complete, so with the default three runs (for different locales)
they took about five minutes. Annoying and unnecessary. As ksh
supports floating point shell arithmetic plus a 'sleep' builtin
that supports fractional seconds, this is easy to fix.

src/cmd/ksh93/tests/signal.sh:
- Slash all the 'sleep' arguments and associated factors by a
  factor of 10. Since computers are now hundreds or thousands of
  times faster than they were in the early 1990s, this should be
  plenty safe enough and all the tests should still be valid.
- Change one unportable direct invocation of /bin/sleep to use the
  builtin.

(cherry picked from commit 734e5953650e9a62048c45eb9a3b827bb9efcfb5)
2020-06-12 01:45:17 +02:00
Martijn Dekker
d024d4c895 Fix signal handling due to exit status > 256
This fixes two bugs: issuing the 'exit' command with a value > 256
would cause ksh 93u+ to kill itself with the corresponding signal
(try 'exit 265' to SIGKILL your interactive shell), and, if the
last command of a script exits due to a signal, the shell would
repeat that signal to itself, causing any parent ksh to also be
killed.

Discussion:
https://bugzilla.redhat.com/show_bug.cgi?id=1469624
https://rainbow.chard.org/2017/03/21/ksh-deliberately-segfaults-if-the-last-command-in-a-script-crashes/

This commit is loosely based on a patch applied to the 93v- beta
and the abandoned ksh2020, but that patch was incomplete & broken:
  $ ksh-2020.0.0 -c 'exit 265'; echo $?
  137
Expected: 9. Since the exit was *not* due to a signal, the value
should simply be cropped to the 8 bits supported by the OS.

src/cmd/ksh93/bltins/cflow.c: b_exit():
- For the 'exit' builtin command, bitwise-AND the argument to
  'exit' with SH_EXITMASK (8 bits, crop to 0-255) before passing it
  on to sh_exit(). This restores the behaviour of <=2011 ksh93
  versions and is in line with all other POSIX shells.
  It also fixes this bogosity:
    $ (exit 265); echo $?                   # non-forked subshell
    265
    $ (ulimit -t unlimited; exit 265); echo $?  # forked subshell
    9
  Forked or non-forked should make no difference at all
  (see commit message a0e0e29e for why).

src/cmd/ksh93/sh/fault.c: sh_done():
- If the current exit status is equal to the status for the last
  signal that was received from a child process, remove the
  SH_EXITSIG (9th) bit, so that the shell doesn't kill itself.
- If the shell's last child process exits due to a signal, exit
  with a portable 8-bit exit status (128 + signal number). This
  avoids the exit status being < 128 by being cropped to 8 bits.

src/cmd/ksh93/tests/signal.sh:
- Add regression test for exit with status > 256.
- Add regression test verifying the shell no longer kills itself.

(cherry picked from commit 98e0fc94393e175ce6adfee390327c320795bf12)
2020-06-12 01:45:17 +02:00
Martijn Dekker
f88f302c38 Remove code related to long-dead 3DFS research project
This commit gets rid of dead weight related to an obscure early
1990s Bell Labs versioning file system research project called
3DFS, which has not existed for decades and for which I have not
managed to find any evidence that it was ever used outside the lab.

This removes:
- the SHOPT_FS_3D compile option (which was forced on even when 0)
- the obnoxious default alias 2d='set -f;_2d' that turned off your
  globbing and then tried to run a nonexistent _2d command
- undocumented builtins 'vmap' and 'vpath' that only errored out
- a non-functional -V unary operator for the test and [[ commands
- some specific code for Apollo workstations (last made in 1997),
  which was inseparably intertwined with the 3DFS code

(cherry picked from commit 20cdf3709f4fb4e468057b534dcee819b1961fb6)
2020-06-12 01:45:17 +02:00
Martijn Dekker
5f8b0512f0 POSIX compliance fix: apply 'set -u' to $!
POSIX requires[*] that expanding any unset parameter other than $@
and $* is an error when 'set -u'/'set -o nounset' is active.
However, on ksh93, $! was exempt as well. That is a bug.
[*] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_25

src/cmd/ksh93/sh/macro.c:
- special(): Handle 'set -u' for special parameters if/when it is
  about to return NIL. That code path is currently only possible to
  reach for "$!", but this is future-proof and will do the right
  thing if any other special parameter can ever have no value.

src/cmd/ksh93/tests/options.sh:
- Add and tweak 'set -u' regression tests.

(cherry picked from commit 75cc7a38cafe3a9929e1ed17d8b952babda22a09)
2020-06-12 01:45:17 +02:00
Martijn Dekker
36da314c9e POSIX compliance fix: apply 'set -u' to $1, $2, ...
POSIX requires[*] that expanding any unset parameter other than $@
and $* is an error when 'set -u'/'set -o nounset' is active.
However, on ksh93, $1, $2, ... were exempt as well. That is a bug.
[*] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_25

src/cmd/ksh93/sh/macro.c:
- varsub(): Backport code for handling 'set -u' for positional
  parameters from the ast 2016-10-01-beta branch.

src/cmd/ksh93/tests/options.sh:
- Add relevant regression tests.

src/cmd/ksh93/sh.1:
- Document that $@ and $* are exempt from 'set -u'.

(cherry picked from commit f954c6be0748c4c38a680a75f27564965fbd328e)
2020-06-12 01:45:17 +02:00
Martijn Dekker
e2a648b41f tests/options.sh: fix child shell test by exporting variable
src/cmd/ksh93/tests/options.sh:
- Pipe hang bugfix test: The child shell has no idea how long to
  sleep for if we don't export the variable telling it so.

(cherry picked from commit ede0960c4ec84f0f934b17072a892a4e40798e97)
2020-06-12 01:45:17 +02:00
Martijn Dekker
781f0a39aa Fix parsing of multibyte characters
According to Red Hat, this fixes "a bug on input buffer boundary
and/or temporary composing buffer of multibyte characters".
The patch was credited to Paulo Andrade <pandrade@redhat.com>.

To be honest, I don't know how to trigger this bug or what the code
removed by this fix really does, but this patch is in production
use at Red Hat, removes some smelly stuff, and is not triggering
any regression test failures, so I'll just take this one on faith.

https://bugzilla.redhat.com/show_bug.cgi?id=1417886
https://github.com/att/ast/commit/4fa2020b

src/cmd/ksh93/sh/fcin.c:
- _fcmbget(): Remove some dodgy-looking buffer-fiddling code that
  is marked as "for testing purposes with small buffers".

(cherry picked from commit 407760fdbddcb7f8ac92b5d1da29d3e09dac0369)
2020-06-12 01:45:17 +02:00
Martijn Dekker
7559f83eff Fix ksh hanging in pipes
Under certain conditions, ksh scripts may hang when piped through
a program that terminates before EOF.

Discussion and reproducer:
https://bugzilla.redhat.com/show_bug.cgi?id=1457990
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01961.html

src/cmd/ksh93/sh/jobs.c:
- In job_wait(), correctly set the terminal foreground process
  group when resuming a process.

(cherry picked from commit 99ef2ca5309da91fdead74bea00743960d470c6a)
2020-06-12 01:45:16 +02:00