src/cmd/ksh93/tests/bracket.sh:
- -r and -w are always true when running as root, so skip these.
- Fix several typos in the [[ tests (apparent copy/paste errors).
(cherry picked from commit b9325e9753b38f087d0c5501d474f7d1f15c4d36)
Fix a bug in autoloading functions. Directories in the path search
list which should be skipped (e.g. because they don't exist) did
not interact correctly with autoloaded functions, so that a
function to autoload was not always found.
Details:
https://github.com/att/ast/issues/1454
Fix backported (and cleaned up) from:
https://github.com/att/ast/commit/3bc58164
src/cmd/ksh93/sh/path.c:
- path_opentype(): Fix the path search loop so that entries marked
with PATH_SKIP are handled correctly.
src/cmd/ksh93/tests/functions.sh:
- Add regression test verifying an autoloaded function with a PATH
that triggered the bug.
The bug in path_opentype() fixed by this commit may affect other
scenarios but we know it affects autoloaded functions. Hence the
test for that scenario.
(cherry picked from commit a27903165775309f4f032de5d42ec1785f14cfbc)
Fix BUG_TESTERR1A: POSIX non-compliance of 'test'/'[' exit status
on error. The command now returns status 2 instead of 1 when given
an invalid number or arithmetic expression, e.g.: [ 123 -eq 123x ]
The problem was that the test builtin (b_test()) calls the generic
arithmetic evaluation subsystem (sh/arith.c, sh/streval.c) which
has no awareness of the test builtin. A simple solution would be to
always make the arithmetic subsystem use an exit status > 1 for
arithmetic errors, but globally changing this may cause backwards
compatibility issues. So it's best to change the behaviour of the
'test' builtin only. This requires the arithmetic subsystem to be
aware of whether it was called from the 'test' builtin or not. To
that end, this commit adds a global flag and overrides the
ERROR_exit macro where needed.
src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/defs.c:
- Declare and initialise a global sh_in_test_builtin flag.
- Declare internal function for ERROR_exit override in test.c.
src/cmd/ksh93/bltins/test.c:
- Add override for ERROR_exit macro using a function that checks if
the exit status is at least 2 if the error occurred while running
the test builtin.
- b_test(): Set sh_in_test_builtin flag while running test builtin.
src/cmd/ksh93/sh/arith.c,
src/cmd/ksh93/sh/streval.c:
- Override ERROR_exit macro using function from test.c.
src/cmd/ksh93/tests/bracket.sh:
- Add regression test verifying status > 1 on arith error in test.
(cherry picked from commit 5eeae5eb9fd5ed961a5096764ad11ab870a223a9)
Passing a SHELL=/path/to/ksh assignment-argument after the
bin/shtest command (as documented in 'shtest --man') made the
wrapper script produce inconsistent results: it would launch and
claim to test the default shell, but src/cmd/ksh93/tests/shtests
would actually test the shell indicated in the argument.
bin/shtests:
- Scan arguments for a SHELL= (or KSH=) assignment-argument,
setting $KSH (and then exporting $SHELL) based on that if found.
(cherry picked from commit 1e78fbbd094dfc84989ecadf06231c6515fb7412)
Aliases can now be correctly unset within subshell environments
(such as ( ... ), $(command substitutions), etc), as well as
non-subshell "shared" command substitutions (${ ...; }). Before,
attempts to unset aliases within these were silently ignored.
Prior discussion: https://github.com/att/ast/issues/108
Subshell alias trees are only referenced in a few places in the
code, *and* have always been broken, so this commit gets rid of the
whole notion of a subshell alias tree. Instead, there is now just
one flat alias tree, and subshells fork into a separate process
when aliases are set or unset within them. It is not really
conceivable that this could be a performance-sensitive operation,
or even a common one, so this is a clean fix with no downside.
src/cmd/ksh93/include/defs.h:
- Remove sh_subaliastree() definition.
src/cmd/ksh93/sh/subshell.c:
- Remove salias element (pointer to subshell alias tree) from
subshell struct.
- Remove sh_subaliastree() function.
- sh_subshell(): Remove alias subshell tree cleanup.
src/cmd/ksh93/bltins/typeset.c:
- b_alias(): If in subshell, fork before setting alias.
- b_unalias(): If in subshell, fork before unsetting alias.
- unall(): Remove sh_subaliastree() call.
src/cmd/ksh93/sh/name.c:
- nv_open(): Remove sh_subaliastree() call.
src/cmd/ksh93/tests/subshell.sh:
- Add regression tests for unsetting or redefining aliases within
subshells.
(cherry picked from commit 12a15605b9521a2564a6e657905705a060e89095)
Functions can now be correctly redefined and unset in subshell
environments (such as ( ... ), $(command substitutions), etc).
Before this fix, attempts to do this were silently ignored (!!!),
causing the wrong code (i.e.: the function by the same name from
the parent shell environment) to be executed.
Redefining and unsetting functions within "shared" command
substitutions of the form '${ ...; }' is also fixed.
Prior discussion: https://github.com/att/ast/issues/73
src/cmd/ksh93/sh/parse.c:
- A fix from George Koelher (URL above). He writes:
| The parser can set t->comnamp to the wrong function.
| Suppose that the shell has executed
| foo() { echo WRONG; }
| and is now parsing
| (foo() { echo ok; } && foo)
| The parser was setting t->comnamp to the wrong foo. [This
| fix] doesn't set t->comnamp unless it was a builtin. Now the
| subshell can't call t->comnamp, so it looks for foo and finds
| the ok foo in the subshell's function tree.
src/cmd/ksh93/bltins/typeset.c:
- Unsetting functions in a virtual/non-forked subshell still
doesn't work: nv_open() fails to find the function. To work
around this problem, make 'unset -f' fork the subshell into its
own process with sh_subfork().
- The workaround exposed another bug: if we unset a function in a
subshell tree that overrode a function by the same name in the
main shell, then nv_delete() exposes the function from the main
shell scope. Since 'unset -f' now always forks a subshell, the
fix is to simply walk though troot's parent views and delete any
such zombie functions as well. (Without this, the 4 'more fun'
tests in tests/subshell.sh fail.)
src/cmd/ksh93/sh/subshell.c: sh_subfuntree():
- Fix function (re)definitions and unsetting in "shared" command
substitutions of the form '${ commandlist; }' (i.e.: if
sp->shp->subshare is true). Though internally this is a weird
form of virtual subshell, the manual page says it does not
execute in a subshell (meaning, all changes must survive it), so
a subshell function tree must not be created for these.
src/cmd/ksh93/tests/subshell.sh:
- Add regression tests related to these bugfixes. Test unsetting
and redefining a function in all three forms of virtual subshell.
(cherry picked from commit dde387825ab1bbd9f2eafc5dc38d5fd0bf9c3652)
These always fail when running as root, as root can access files
regardless of permission bits.
(cherry picked from commit a821fe13906ac8ef56162bebd7c3e976b973f91c)
Certain environment variables were interpreted as arithmetic
expressions on startup, leading to code injection.
Ref.:
https://bugzilla.redhat.com/show_bug.cgi?id=1757324c7de8b6412
(cherry picked from commit ee6b001d0611ad2e00b6da2c2b42051995c0a678)
It was annoying me that I always had to type out the .sh in
'shtests builtins.sh', etc. to execute a specific regression test
script. All the test scripts end in .sh, so this is superflous.
src/cmd/ksh93/tests/shtests:
- Add .sh to test script arguments if not already present.
(cherry picked from commit d780e701fbf6f27b6e79f88542a3175bf5dd85e9)
The issue with truncating files was caused by out-of-sync streams.
Details and discussion: https://github.com/att/ast/issues/61
src/cmd/ksh93/sh/io.c: sh_iorestore():
- To be safe, sync all streams before restoring file descriptors.
src/cmd/ksh93/tests/io.sh:
- Add two regression tests for truncating files with this
combination of redirections.
- The second test, which invokes a -c script, is disabled for now
as this triggers another corner case bug involving the SH_NOFORK
optimisaton for -c scripts. That fix is for another commit.
(cherry picked from commit 18fb64840365c2ff4608188e5487bd79d08f67d1)
src/cmd/ksh93/tests/io.sh:
- After BUG_REDIRIO was fixed in 29afc16c, '<>' now defaults to
'0<>' instead of '1<>', as POSIX specifies. Update '<>' to '1<>'.
- After BUG_PUTIOERR was fixed in 9011fa93, one 'print' command
with pipe buffering disabled (piped into 'read -n3') printed an
expected I/O error to standard error. Redirect this to /dev/null.
(cherry picked from commit d4b6510fdf155a3d75a98860a09c0ebedb6c93d2)
This fixes three related bugs:
1. Expansions like ${var+set} remained static when used within a
'for', 'while' or 'until' loop; the expansions din't change
along with the state of the variable, so they could not be used
to check whether a variable is set within a loop if the state of
that variable changed in the course of the loop. (BUG_ISSETLOOP)
2. ${IFS+s} always yielded 's', and [[ -v IFS ]] always yielded
true, even if IFS is unset. (BUG_IFSISSET)
3. IFS was incorrectly exempt from '-u' ('-o nounset') checks.
src/cmd/ksh93/sh/macro.c: varsub():
- When getting a node pointer (np) to the parameter to test,
special-case IFS by checking if it has a value and not setting
the pointer if not. The node to IFS always exists, even after
'unset -v IFS', so before this fix it always followed the code
path for a parameter that is set. This fixes BUG_IFSISSET for
${IFS+s} and also fixes set -u (-o nounset) with IFS.
- Before using the 'nv_isnull' macro to check if a regular variable
is set, call nv_optimize() if needed. This fixes BUG_ISSETLOOP.
Idea from Kurtis Rader: https://github.com/att/ast/issues/1090
Of course this only works if SHOPT_OPTIMIZE==1 (the default),
but if not, then this bug is not triggered in the first place.
- Add some comments for future reference.
src/cmd/ksh93/bltins/test.c: test_unop():
- Fix BUG_IFSISSET for [[ -v IFS ]]. The nv_optimize() method
doesn't seem to have any effect here, so the only way that I can
figure out is to special-case IFS, nv_getval()'ing it to check if
IFS has a value in the current scope.
src/cmd/ksh93/tests/variables.sh:
- Add regression tests for checking if a varariable is set within a
loop, within and outside a function with that variable made local
(to check if the scope is honoured). Repeat these tests for a
regular variable and for IFS, for ${foo+set} and [[ -v foo ]].
(cherry picked from commit a2cf79cb98fa3e47eca85d9049d1d831636c9b16)
src/cmd/INIT/cc.darwin:
- Since the flags added from Apple's ksh-27.tar.gz Makefile
explicitly redefine a macro, let's disable the warning on
redefined macros.
(cherry picked from commit 4b21a0abce8406e82d3afa4a68ef6bb56d98c797)
'command -p' was broken for non-interactive shells as the variable
used to store the default system PATH, std_path, was not
initialised correctly. For instance:
$ ksh -c 'command -p ls'
ksh: ls: not found
This fix by Siteshwar Vashisht is backported from ksh2020.
Ref.:
https://github.com/att/ast/issues/426https://github.com/att/ast/pull/448
src/cmd/ksh93/sh/path.c:
- Correctly initialise std_path (the default PATH) when ksh is
started as a non-interactive shell.
src/cmd/ksh93/sh.1:
- Fix vague explanation of 'command -p'.
src/cmd/ksh93/tests/path.sh:
- Add regression test.
(cherry picked from commit a76439d60b70c18cf44d84c1962fcd8df84c947c)
${.sh.subshell} only counts the nesting level of virtual (non-forked)
subshell environments relative to the current process, whether that
is the main shell environment or a real (forked) subshell.
This means ${.sh.subshell} is not sufficient to test if your code
is running in a subshell environment, so it's important to mention
this in the manual.
E.g.:
(echo ${.sh.subshell}; ulimit -t 1; echo ${.sh.subshell}); :
outputs 1 followed by 0.
(cherry picked from commit 63dad8863fd26bf3aa4d7a1cf60b743f4169f7c4)
The number of errors in each tests/*.sh test set is reported using
its exit code (up to 125). The main test script saves this code in
$e to report the number of errors. But if a set had errors, $e was
then erroneously overwritten before reporting the number of errors,
so it was always 1.
src/cmd/ksh93/tests/shtests:
- Don't overwrite $e with the exit code of ((e == 0)).
(cherry picked from commit b9ab95488b23167316f5b21f338368c9490ebc50)
Standard output (FD 1) tested as being on a terminal within a
command substitution, which makes no sense as the command
substitution is supposed to be catching standard output.
ksh -c 'v=$(echo begincomsub
[ -t 1 ] && echo oops
echo endcomsub)
echo "$v"'
This should not output "oops".
This is one of the many bugs with ksh93 virtual (non-forked)
subshells. On the abandoned Vashist/Rader ksh2020 branch, this bug
was fixed by changing quite a lot of code, which introduced and/or
exposed another bug:
https://github.com/att/ast/issues/1079https://github.com/att/ast/commit/8e1e405ehttps://github.com/att/ast/issues/1088
That issue was unresolved when the ksh2020 branch was abandoned.
The safer and more conservative fix is simply forcing the subshell
to fork if we're in a non-forked command substitution and testing
'-t 1'. It is hard to imagine a situation where this would cause a
noticable performance hit.
Note that this fix does not affect ksh93-specific "shared"
non-subshell ${ command substitutions; } which are executed in the
main shell environment, so that variables survive, etcetera.
'test -t 1' continues to wrongly return true there, but command
substitutions of that form cannot be forked because that would
defeat their purpose.
src/cmd/ksh93/bltins/test.c:
- Fix 'test -t 1', '[ -t 1 ]' and '[[ -t 1 ]]' by forking the
current subshell if it is a virtual/non-forked subshell
(shp->subshell), and a command substitution (shp->comsub), but
NOT a "shared" ${ command substitution; } (!shp->subshare).
src/cmd/ksh93/tests/bracket.sh:
- Add two regression tests for this issue, which were adapted from
the Vashist/Rader ksh2020 branch.
NEWS, src/cmd/ksh93/include/version.h:
- Update.
(cherry picked from commit b8ef05e457ead65b83417699b8dd8632f855e2fa)
Different implementations of the 'od' command may legitimately vary
the whitespace they output between fields. One of the regression
tests checked 'od' output without tolerating whitespace varieties,
which is not portable. For example, the BSD 'od' implementation
outputs more whitespace than the variant used to write the test.
src/cmd/ksh93/tests/locale.sh:
- In test "incorrect string from printf %q", use 'awk' to eliminate
whitespace from 'od' output before checking results.
(cherry picked from commit 7aa830bd3a2284f7228ee7189f5671749c3f2682)
This test used an unportable invocation of an external 'join'
command. The options used were Solaris-specific. Not all OSs
support UTF-8 in their external utilities so using an external
utility is not appropriate.
src/cmd/ksh93/tests/locale.sh:
- Rewrite the 'join test script' to use ksh builtin 'cut' instead.
This should suffice to check whether the locale is correctly set,
which is the point of this test.
(cherry picked from commit fb16f33ff1e9d33d5fccc5cf9633d508eb6eecc7)
The -C flag to 'wc' is not portable; '-m' is the portable variant.
However, this whole test is not appropriate here, as it tests the
external 'wc' utility and not ksh's builtin version -- so any test
failure represents a bug in the OS and not in ksh.
src/cmd/ksh93/tests/locale.sh:
- Remove test that counts two € (euro) character using the OS's
external 'wc' utility. Keep the test that uses builtin 'wc'.
(cherry picked from commit e1b25e0ace85ee6e62734fb236900fe069dccb6e)
The regression test suite used 'mktemp -dt' to create temporary
directories, but that is not portable, e.g. it does not work on
macOS, FreebSD or NetBSD. Installing a compatible 'mktemp' in
another location in $PATH did not work either, as the regression
test suite resets $PATH.
src/cmd/ksh93/tests/*:
- Replace many uses of 'mktemp -dt' by a portable and reasonably
safe way to create a temporary directory.
(cherry picked from commit 71f4c43703e3eff034009b4c8f4110bd32f4e209)
This new wrapper script sets up the correct environment for running
the ksh93 regression test suite. It allows running the tests
without AST nmake, which is not maintained in this repository.
An alternative ksh to test may be passed in the $KSH env var.
bin/shtests:
- Added. Sets up environment before passing control to
'src/cmd/ksh93/tests/shtest'. Passes on any options given.
NEWS:
- Updated.
(cherry picked from commit 14ced94ed83991687c645a09bd2e45a5c2ffe8dc)
This allows scripts to check for a nonzero exit status on the
'print', 'printf' and 'echo' builtins and prevent possible infinite
loops if SIGPIPE is ignored.
sfsync() was already returning a negative value on I/O error, so
all we need to do is add a check. The stream buffer will need to be
filled before an I/O error can be detected, but this is the same on
other shells. See manual page: src/lib/libast/man/sfio.3
Ref.: https://github.com/att/ast/issues/1093https://github.com/att/ast/pull/1363
src/cmd/ksh93/bltins/print.c: b_print():
- Make sure an error result from sfsync() is reflected in the
output builtin's exit status (exitval).
- Write an I/O error message (e_io) if the exit status is nonzero.
src/cmd/ksh93/data/msg.c, src/cmd/ksh93/include/io.h:
- Add the e_io[] error message.
src/cmd/ksh93/tests/builtins.sh:
- Add I/O error regression test, checking for the correct error
message and exit status. All three output builtins use the same
b_print() function so we only need to test one.
src/cmd/ksh93/tests/basic.sh,
src/cmd/ksh93/tests/coprocess.sh:
- Redirect stderr on a few 'print' commands to /dev/null; these
now issue an expected I/O error. This does not cause failures.
NEWS, TODO:
- Update.
(cherry picked from commit 9011fa933552e483dab460f7dd1593d64e059d94)
This fix is backported from the Vashisht/Rader ksh2020 branch.
src/cmd/ksh93/bltins/print.c:
- Fix syncing history when print -s -f is used. For example, the
following now correctly adds a 'cd' command to the history:
print -s -f 'cd -- %q\n' "$PWD"
Ref.: https://github.com/att/ast/issues/425https://github.com/att/ast/pull/442
src/cmd/ksh93/include/version.h:
- Version date bump.
(cherry picked from commit 46ba7ecdc5c14cd73f6cb24b16c50bdc331a000e)
This updates the cc.darwin* compiler compatibility wrapper scripts
and patches bin/package to identify a 64-bit system, allowing ksh
to build using a simple 'bin/package make' command on at least on
macOS Mojave (10.14.6). Testing on other macOS versions is invited.
src/cmd/INIT/cc.darwin:
- Remove obsolete compatibility cruft
- Update compiler flags:
* remove obsolete/unneeded -DCLK_TCK=100 (this is already defined
in the system header i386/limits.h included by time.h)
* add necessary flags from Apple's Makefile in ksh-27.tar.gz:
-DSHOPT_SPAWN=0 -D_ast_int8_t=int64_t -D_lib_memccpy
source: https://opensource.apple.com/tarballs/ksh/
* add -Wno-unused-value and -Wno-parentheses to suppress excessive
clang warnings that object to David Korn's coding style.
- Update linker flags:
* remove obsolete -Wl,-m
* add flag from Apple's Makefile: -Wl,-search_paths_first
src/cmd/INIT/cc.darwin.i386,
src/cmd/INIT/cc.darwin.i386-64:
- Removed as superfluous (was identical to generic cc.darwin).
bin/package,
src/cmd/INIT/package.sh:
- Fix the recognition of a 64-bit system from the output of a 'file
/bin/sh' command (what an ugly hack... never mind), so that the
architecture is correctly identified as darwin.i386-64.
(cherry picked from commit e74e98b886a2bc5760166c124412474293f85fd8)
ksh used to redirect standard output by default when no file
descriptor was specified with the rarely used '<>' reading/writing
redirection operator. It now redirects standard input by default,
as POSIX specifies and as all other POSIX shells do. To redirect
standard output for reading and writing, you now need '1<>'.
Ref.: https://github.com/att/ast/issues/75http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_07
(cherry picked from commit 29afc16c47824fc79ed092ae7704c525b1db6a0a)
Let's call this series of ksh93 the '93u+m' branch, for 'minor'
conservative bug fixes from 'modernish'. :)
src/cmd/ksh93/include/version.h:
- Rename release ID: 93u+ => 93u+m
- Update release date (2020-05-13)
(cherry picked from commit ea1c7ff8da597ff9057235a53e84c3c4cc944ee4)
This fixes an undocumented 'case' pattern matching misbehaviour
(labelled BUG_CASELIT in modernish) that goes back to the original
Bourne shell, but wasn't discovered until 2018.
If a pattern doesn't match as a pattern, it's tried again as a
literal string. This breaks common validation use cases, such as:
n='[0-9]'
case $n in
( [0-9] ) echo "$n is a number" ;;
esac
would output "[0-9] is a number" as the literal string fallback
matches the pattern. As this misbehaviour was never documented
anywhere (not for Bourne, ksh88, or ksh93), and it was never
replicated in other shells (not even in ksh88 clones pdksh and
mksh), it is unlikely any scripts rely on it.
Of course, a literal string fallback, should it be needed, is
trivial to implement correctly without this breakage:
case $n in
( [0-9] | "[0-9]") echo "$n is a number or the number pattern" ;;
esac
src/cmd/ksh93/sh/xec.c:
- Remove trim_eq() function responsible for implementing the
misbehaviour described above.
NEWS:
- Added. Document this bugfix.
Ref.:
- The problem: thread starting at
https://www.mail-archive.com/austin-group-l@opengroup.org/msg02127.html
- The solution, thanks to George Koehler: comments/commits in
https://github.com/att/ast/issues/476
- Modernish BUG_CASELIT bug test & documentation:
https://github.com/modernish/modernish/commit/b2024ae3
(cherry picked from commit 8d6c8ce69884767a160c1e20049e77bdd849c248
with some extra edits to NEWS to upate the info for this reboot)
Description: Correcting typo in man page
Bug-Debian: https://bugs.debian.org/755486
Forwarded: no
Author: Nicholas Bamber <nicholas@peiapt.co.uk>
Last-Update: 2015-11-01
(cherry picked from commit 1f4e043f395760a19dc3690b9027551a8c2c66a1)
This commit reboots ksh 93u+m development with a new fork based off
the archived official AT&T ast repo, master branch.
README:
- Removed. Having two READMEs is a botch.
README.md:
- Imported and edited from the archived fork,
https://github.com/modernish/ksh -- the newer instructions will
become applicable as I cherry-pick all the changes from there :)
- Copy-edits.
- Added policy and rationale for this 93u+m fork.
- Incorporated info from old README that is still relevant.
.gitignore:
- Added.
bin/execrate:
- Removed. This one keeps changing its license header, causing git
to show an uncommitted file. It's always re-copied anyway, and
not for direct invocation, so, exclude and add to .gitignore.
The source file is: src/cmd/INIT/execrate.sh
src/cmd/*,
src/lib/*:
- Remove historical baggage: everything except ksh93 and
dependencies. This is the same stuff the ksh-community fork
removed. I wouldn't mind keeping it, but some of this stuff
causes build failures on macOS (and probably other systems), and
I need to get on with fixing ksh93.
Hopefully, we can figure out how to re-add stuff we can use
later -- particularly the pty command (pseudo-terminal utility
for scripting interactive sessions) which is needed for
regression-testing the interactive shell.
- added cc and ar scripts for freebsd12.amd64
- fix for sfio macro SF_FLAGS also being defined for sendfile(2) in socket.h
-- fix is to hide the sendfile(2) macro
- defined _STDFILE_DECLARED to handle FILE macro incompatibility
- added test in std/features/procfs to deal with freebsd kvm interface
-- being different from what AST expects (for ps.c)
- fix for std/banner.c to clean up main() definition
- changed comment in ar.* scripts to match cc.* scripts
