1
0
Fork 0
mirror of git://git.code.sf.net/p/cdesktopenv/code synced 2025-03-09 15:50:02 +00:00
Commit graph

281 commits

Author SHA1 Message Date
Martijn Dekker
c709868572 process substitution: improve fifo error handling (re: ab5dedde)
src/cmd/ksh93/sh/args.c: sh_argprocsub():
- Fix compiler warnings with SHOPT_DEVFD on by including "io.h".
- Without SHOPT_DEVFD, the FIFO code didn't consider that libast's
  pathtemp(3) may also fail and return null. Add a check for this.
2021-02-04 20:47:45 +00:00
Martijn Dekker
a410bc480f autoload: Add loop detection
It was trivial to crash ksh by making an autoloaded function
definition file autoload itself, causing a stack overflow due to
infinite recursion. This commit adds loop detection that stops a
function that is being autoloaded from autoloading itself either
directly or indirectly, without removing the ability of autoloaded
function definition files to autoload other functions.

src/cmd/ksh93/sh/path.c: funload():
- Detect loops by checking if the path of a function to be
  autoloaded was already added to a new internal static tree,
  and if not, adding it while the function is being loaded.

src/cmd/ksh93/tests/path.sh:
- Add regression test.
- Tweak a couple of others to be freeze- and crash-proof.

NEWS:
- Add this fix + a forgotten entry for the previous fix (6f3b23e6).

Fixes: https://github.com/ksh93/ksh/issues/136
2021-02-04 16:28:19 +00:00
Martijn Dekker
6f3b23e6f4 Fix crash on trying a very long nonexistent command
Reproducer from @Saikiran-m:
| ~# sh -c `perl -e 'print "a"x100000'`
|  genunix: NOTICE: core_log: sh[1221] core dumped: /var/cores/core.sh.0.1602153496
| Memory fault(coredump)

The crash was in trying to decide whether the name was suitable for
autoloading as a function on $FPATH. This calls strmatch() to check
the name against a regex for valid function name. But the libast
regex code is not designed optimally and uses too much recursion,
limiting the length of the strings it's able to cope with.

src/cmd/ksh93/sh/path.c: path_search():
- Before calling strmatch(), check that the name is shorter than
  256 bytes. The maximum length of file names on Linux and macOS is
  255 bytes, so an autoload function can't have a name longer than
  that anyway.

src/cmd/ksh93/tests/path.sh:
- Add test for this bug.
- Tweak 'command -x' test to not leave a hanging process on Ctrl+C.

Fixes: https://github.com/ksh93/ksh/issues/144
2021-02-04 05:03:40 +00:00
Martijn Dekker
9ddb45b12d command -x: fix 'args list too long' on Linux, again (re: 8f5235a5)
This time it was failing on a 64-bit Debian Linux system with very
few and short environment variables. Sigh.

src/cmd/ksh93/sh/path.c:
- Combine the strategy from 63979488 with that of 8f5235a5.
2021-02-03 20:07:47 +00:00
Martijn Dekker
8f5235a53f fix 'args list too long' on Linux, again (re: 63979488)
That fix turned out to be insufficient as NixOS has huge
environment variable lists because (due to each software package
being installed in its own directory tree) it has to keep dozens
of directories in variables like XDG_CONFIG_DIRS and others.
The 'command -x' regression test was failing on NixOS.

src/cmd/ksh93/sh/path.c:
- Different strategy. Leave twice the size of the existing
  environment free.
2021-02-03 19:22:35 +00:00
Martijn Dekker
52067c3d37 Backport atomic job locking from ksh 93v- beta
Something similar was previously done in 07cc71b8 from a Debian
patch, and eventually reverted; it redefined the ast atomic
functions asoincint() and asodecint() to be gcc-specific. This
imports the upstream version from the ksh 93v- beta instead.

This commit is based on an OpenSUSE patch:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-joblock.dif

src/cmd/ksh93/include/jobs.h:
- Replace job locking mechanism with the 93v- version which uses
  the atomic libast functions asoincint(), asogetint() and
  asodecint(). See: src/lib/libast/man/aso.3

src/cmd/ksh93/sh/jobs.c: job_subsave():
- Revert gcc optimiser bug workaround from c258a04f.
  It should now be unnecessary.
2021-02-02 17:38:19 +00:00
Martijn Dekker
63979488e6 command -x: fix 'arg list too long' on Linux (re: 66e1d446)
I got one intermittent regression test failure due to 'argument
list too long' on a Debian x86_64 system.

src/cmd/ksh93/sh/path.c: path_xargs():
- Leave extra argument space for systems that need extra bytes:
  1KiB per extra byte, with a minimum of 2KiB (the old value).
2021-02-02 15:23:21 +00:00
Martijn Dekker
4645e6ad33 job_reset(): fix for resetting foreground process group
Patch from OpenSUSE:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-foreground-prgrp.dif

src/cmd/ksh93/sh/jobs.c: job_reset():
- Only reset the foreground process group associated with the
  current job if the current job's process ID is different from the
  terminal's foreground process group ID.
2021-02-02 12:49:00 +00:00
Martijn Dekker
9f980ba65a sh_setmatch(): fix node size calculation
This fixes the function that sets ${.sh.match}. Patch from OpenSUSE:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-limit-name-len.dif

src/cmd/ksh93/sh/init.c: sh_setmatch():
- Fix node size calculation, possibly preventing data corruption.

src/cmd/ksh93/include/ulimit.h: Limit_t:
- Defining the 'name' struct member as 'char name[16]' makes
  no sense as the name is being initialised statically in
  data/limits.c; just make it a 'char *name' pointer.
2021-02-02 11:52:54 +00:00
Martijn Dekker
fa3a3d7955 sh_ntfork(): handle SIGTSTP along with SIGTTIN and SIGTTOU
This applies a patch from OpenSUSE that makes SIGTSTP handling
consistent with that of SIGTTIN and SIGTTOU.
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-signals.dif
2021-02-02 04:25:46 +00:00
Martijn Dekker
0e4c4d619d Fix minor typeset attribute regressions (re: 95fe07d8, fdb9781e)
This fixes the following regressions marked TODO in attributes.sh:

$ typeset -L 13 bar; readonly bar; typeset -p bar
typeset -r -L 0 foo		# exp.: typeset -r -L 13 foo
$ typeset -R 13 bar; readonly bar; typeset -p bar
typeset -r -R 0 bar		# exp.: typeset -r -R 13 bar
$ typeset -Z 13 baz; readonly baz; typeset -p baz
typeset -r -Z 0 -R 0 baz	# exp.: typeset -r Z 13 -R 13 baz

I've discovered that these were briefly fixed between fdb9781e (Red
Hat patch for typeset -xu/-xl) and 95fe07d8 (reversal of patch,
different -xu/-xl fix, but reintroduced these regressions).

src/cmd/ksh93/sh/name.c: nv_newattr():
- Replace check from 95fe07d8 with a new one that combines its
  approach with that of fdb9781e: do not change size (and hence
  return early) if NV_RDONLY and/or NV_EXPORT are the only
  attributes that are changing.

src/cmd/ksh93/tests/attributes.sh:
- Enable the TODO regression tests.
2021-02-02 02:20:37 +00:00
Martijn Dekker
5491fe9724 Correctly block invalid values for arrays of an enum type
This fixes part of https://github.com/ksh93/ksh/issues/87:

Scalar arrays (-a) and associative arrays (-A) of a type created by
'enum' did not consistently block values not specified by the enum
type, yielding corrupted results.

An expansion of type "${array[@]}" yielded random numbers instead
of values for associative arrays of a type created by 'enum'.

This does not yet fix another problem: ${array[@]} does not yield
all values for associative enum arrays.

src/cmd/ksh93/bltins/enum.c: put_enum():
- Always throw an error if the value is not in the list of possible
  values for an enum type. Remove incorrect check for the NV_NOFREE
  flag. Whatever that was meant to accomplish, I've no idea.

src/cmd/ksh93/sh/array.c: nv_arraysettype():
- Instead of sh_eval()ing a shell assignment, use nv_putval()
  directly. Also use the stack (see src/lib/libast/man/stk.3)
  instead of malloc to save the value; it's faster and will be
  auto-freed at some point. This shortens the function and makes it
  faster by not entering into a whole new shell context -- which
  also fixes another problem: the error message from put_enum()
  didn't cause the shell to exit for indexed enum arrays.

src/cmd/ksh93/sh/name.c: nv_setlist():
- Apply a patch from David Korn that correctly sets the data type
  for associative arrays, fixing the ${array[@]} expansion yielding
  random numbers. Thanks to @JohnoKing for the pointer.
  https://github.com/ksh93/ksh/issues/87#issuecomment-662613887
  https://www.mail-archive.com/ast-developers@lists.research.att.com/msg00697.html

src/cmd/ksh93/tests/enum.sh:
- Add tests checking that invalid values are correctly blocked for
  indexed and associative arrays of an enum type.

Makes progress on: https://github.com/ksh93/ksh/issues/87
2021-02-01 16:57:43 +00:00
Martijn Dekker
6a0e9a1a75 Tweak and regress-test 'command -x' (re: 66e1d446)
Turns out the assumption I was operating on, that Linux and macOS
align arguments on 32 or 64 bit boundaries, is incorrect -- they
just need some extra bytes per argument. So we can use a bit more
of the arguments buffer on these systems than I thought.

src/cmd/ksh93/features/externs:
- Change the feature test to simply detect the # of extra bytes per
  argument needed. On *BSD and commercial Unices, ARG_EXTRA_BYTES
  shows as zero; on Linux and macOS (64-bit), this yields 8. On
  Linux (32-bit), this yields 4.

src/cmd/ksh93/sh/path.c: path_xargs():
- Do not try to calculate alignment, just add ARG_EXTRA_BYTES to
  each argument.
- Also add this when substracting the length of environment
  variables and leading and trailing static command arguments.

src/cmd/ksh93/tests/path.sh:
- Test command -v/-V with -x.
- Add a robust regression test for command -x.

src/cmd/ksh93/data/builtins.c, src/cmd/ksh93/sh.1:
- Tweak docs. Glob patterns also expand to multiple words.
2021-02-01 02:19:02 +00:00
Martijn Dekker
ede479967f resolve/remove USAGE_LICENSE macros; remove repetitive (c) strings
This takes another small step towards disentangling the build
system from the old AT&T environment. The USAGE_LICENSE macros with
author and copyright information, which was formerly generated
dynamically for each file from a database, are eliminated and the
copyright/author information is instead inserted into the AST
getopt usage strings directly.

Repetitive license/copyright information is also removed from the
getopt strings in the builtin commands (src/lib/libcmd/*.c and
src/cmd/ksh93/data/builtins.c). There's no need to include 55
identical license/copyright strings in the ksh binary; one (in the
main ksh getopt string, shown by ksh --man) ought to be enough!
This makes the ksh binary about 10k smaller.

It does mean that something like 'enum --author', 'typeset
--license' or 'shift --copyright' will now not show those notices
for those builtins, but I doubt anyone will care.
2021-01-31 11:00:49 +00:00
Martijn Dekker
66e1d44642 command -x: fix efficiency; always run external cmd (re: acf84e96)
This commit fixes 'command -x' to adapt to OS limitations with
regards to data alignment in the arguments list. A feature test is
added that detects if the OS aligns the argument on 32-bit or
64-bit boundaries or not at all, allowing 'command -x' to avoid
E2BIG errors while maximising efficiency.

Also, as of now, 'command -x' is a way to bypass built-ins and
run/query an external command. Built-ins do not limit the length of
their argument list, so '-x' never made sense to use for them. And
because '-x' hangs on Linux and macOS on every ksh93 release
version to date (see acf84e96), few use it, so there is little
reason not to make this change.

Finally, this fixes a longstanding bug that caused the minimum exit
status of 'command -x' to be 1 if a command with many arguments was
divided into several command invocations. This is done by replacing
broken flaggery with a new SH_XARG state flag bit.

src/cmd/ksh93/features/externs:
- Add new C feature test detecting byte alignment in args list.
  The test writes a #define ARG_ALIGN_BYTES with the amount of
  bytes the OS aligns arguments to, or zero for no alignment.

src/cmd/ksh93/include/defs.h:
- Add new SH_XARG state bit indicating 'command -x' is active.

src/cmd/ksh93/sh/path.c: path_xargs():
- Leave extra 2k in the args buffer instead of 1k, just to be sure;
  some commands add large environment variables these days.
- Fix a bug in subtracting the length of existing arguments and
  environment variables. 'size -= strlen(cp)-1;' subtracts one less
  than the size of cp, which makes no sense; what is necessary is
  to substract the length plus one to account for the terminating
  zero byte, i.e.: 'size -= strlen(cp)+1'.
- Use the ARG_ALIGN_BYTES feature test result to match the OS's
  data alignment requirements.
- path_spawn(): E2BIG: Change to checking SH_XARG state bit.

src/cmd/ksh93/bltins/whence.c: b_command():
- Allow combining -x with -p, -v and -V with the expected results
  by setting P_FLAG to act like 'whence -p'. E.g., as of now,
	command -xv printf
  is equivalent to
	whence -p printf
  but note that 'whence' has no equivalent of 'command -pvx printf'
  which searches $(getconf PATH) for a command.
- When -x will run a command, now set the new SH_XARG state flag.

src/cmd/ksh93/sh/xec.c: sh_exec():
- Change to using the new SH_XARG state bit.
- Skip the check for built-ins if SH_XARG is active, so that
  'command -x' now always runs an external command.

src/lib/libcmd/date.c, src/lib/libcmd/uname.c:
- These path-bound builtins sometimes need to run the external
  system command by the same name, but they did that by hardcoding
  an unportable direct path. Now that 'command -x' runs an external
  command, change this to using 'command -px' to guarantee using
  the known-good external system utility in the default PATH.
- In date.c, fix the format string passed to 'command -px date'
  when setting the date; it was only compatible with BSD systems.
  Use the POSIX variant on non-BSD systems.
2021-01-30 06:53:19 +00:00
Martijn Dekker
674a0c3559 Another lexical fix for here-documents (re: 6e515f1d)
OpenSUSE patch from:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-heredoc.dif

src/cmd/ksh93/sh/lex.c: here_copy():
- Do not potentially seek back a zero or negative length.
2021-01-28 06:32:42 +00:00
Martijn Dekker
4604df9ada Stack robustness fixes from OpenSUSE
Three OpenSUSE patches from:
https://build.opensuse.org/package/show/shells/ksh

As usual, the relevant bug is not currently public:
https://bugzilla.opensuse.org/show_bug.cgi?id=844071

src/cmd/ksh93/sh/xec.c: sh_debug()/sh_exec():
- Fix stk restoration. [bnc#844071]

src/lib/libast/misc/stk.c:
- Fix stk aliasing code. [bnc#844071]
  (ksh93-stkalias.dif)
- Make a unknown location fatal in stkset() so that we get a core
  dump right away instead of later in an unrelated part of code.
  (ksh93-stkset-abort.dif)

src/lib/libast/man/stk.3,
src/lib/libast/man/stak.3:
- Update manual with new stkset() behaviour. (93u+m addition)
  (Note that stak is implemented as macros that translate to stk)
2021-01-28 06:18:44 +00:00
Martijn Dekker
288b6c6517 Fix various possible uses of uninitialised variables
Patch from OpenSUSE, slightly adapted for 93u+m. Source:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-uninitialized.dif
2021-01-28 04:54:41 +00:00
Martijn Dekker
c52cb93999 sh_funscope(): Fix possible dereference of null pointer
Patch from OpenSUSE:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-unset-f.dif
2021-01-28 04:38:48 +00:00
Martijn Dekker
3ad4307054 name.c: fix possible crash in attstore()
From OpenSUSE:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-env.dif
(the init.c fix there is already done differently in 3654ee73)

src/cmd/ksh93/sh/name.c: attstore():
- Check nv_mapchar() returns a non-null pointer before using it.
2021-01-28 04:01:49 +00:00
Martijn Dekker
cc4927529b libast: Update cdt(3): Allow empty strings in (dt)trees
This backports most of the Cdt (container data types) mechanism
from the ksh 93v- beta, based on ground work done by OpenSUSE:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-dttree-crash.dif
plus adaptations to match ksh 93u+m and an updated manual page
(src/lib/libast/man/cdt.3) added directly from the 93v- sources.

| Thu Dec 20 12:48:02 UTC 2012 - werner@suse.de
|
| - Add ksh93-dttree-crash.dif - Allow empty strings in (dt)trees
|   (bnc#795324)
|
| Fri Oct 25 14:07:57 UTC 2013 - werner@suse.de
|
| - Rework patch ksh93-dttree-crash.dif

As usual, precious little information is available because the
OpenSUSE bug report is currently closed to the public:
https://bugzilla.opensuse.org/show_bug.cgi?id=795324

However, a cursory inspection suggests that this code contains
improvements to do with concurrent processing and related
robustness. The new cdt.3 manual page adds a lot about that.

This has been in production use on OpenSUSE for a long time,
so hopefully this will make ksh a little more stable again.
Only one way to find out: let's commit and test this...

BTW, to get a nice manual, use groff and ghostscript's ps2pdf:
$ groff -tman src/lib/libast/man/cdt.3 | ps2pdf - cdt.3.pdf
2021-01-28 02:44:52 +00:00
Martijn Dekker
9b5ff0f833 Fix leak in optimize processing
This applies a patch from OpenSUSE. Source:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-optimizeleak.dif

| Tue Jul  5 14:49:03 CEST 2016 - mls@suse.de
|
| - fix leak in optimize processing [bnc#982423]
|   new patch: ksh93-optimizeleak.dif

Unfortunately the bug report is not currently public:
https://bugzilla.opensuse.org/show_bug.cgi?id=982423
but this one seems sensible enough and is in production use,
so I'll take it on faith.
2021-01-27 05:51:47 +00:00
Martijn Dekker
6d1352699e Fix locking error in spawn implementation
This applies a patch from OpenSUSE. Source:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-spawnlock.dif

| Wed Oct 12 13:23:14 CEST 2016 - mls@suse.de
|
| - fix locking error in spawn implementation [bnc#988213]
|   new patch: ksh93-spawnlock.dif

Unfortunately the bug report is not currently public:
https://bugzilla.opensuse.org/show_bug.cgi?id=988213
but this one seems sensible enough and is in production use,
so I'll take it on faith.
2021-01-27 05:32:24 +00:00
Martijn Dekker
70368c57d6 Fix field splitting bug triggered by DEBUG trap
An unquoted variable expansion evaluated in a DEBUG trap action
caused IFS field splitting to be deactivated in code executed after
the trap action. Thanks to Koichi Nakashima for the reproducer:

| v=''
| trap ': $v' DEBUG
| A="a b c"
| set -- $A
| printf '%s\n' "$@"
|
| Expected
|
| a
| b
| c
|
| Actual
|
| a b c

src/cmd/ksh93/sh/fault.c: sh_trap():
- Remove incorrect save/restore of sh.ifstable, the internal state
  table for field splitting. This reverts three lines added in ksh
  93t+ 2009-11-30. Analysis: As an expansion is split into fields
  (macro.c, lines 2367-2471), sh.ifstable is modified. If that
  happens within a DEBUG trap, any modifications in ifstable are
  undone by the restoring memccpy, leaving an inconsistent state.

src/cmd/ksh93/COMPATIBILITY:
- Document the DEBUG trap fixes, particularly the incorrect
  inheritance by subshells and functions that some scripts may now
  rely on because this bug is so longstanding. (re: 2a835a2d)

src/cmd/ksh93/tests/basic.sh:
- Add relevant tests.

Resolves: https://github.com/ksh93/ksh/issues/155

TODO: add a -T (-o functrace) option as in bash, which should allow
subshells and ksh-style functions to inherit DEBUG traps.

P.S.: The very handy multishell repo allows us to use 'git blame'
to trace the origin of the recently fixed DEBUG trap bugs.

The off-by-one error causing various bugs, reverted in 2a835a2d,
was introduced in ksh 93t 2008-07-25:
8e947ccf
(fault.c, line 321)

The incorrect check causing the exit status bug, reverted in
d00b4b39, was introduced in ksh 93t 2008-11-04:
b1ade268
(fault.c, line 459)

The ifstable save/restore causing the field splitting bug, reverted
in this commit, was introduced in ksh 93t+ 2009-11-30:
53d9f009
(fault.c, lines 440, 444, 482)

So all the bugs reported in #155 were fixed by simply reverting
these specific changes. I think that they are some experiments that
the developers simply forgot to remove. I've suspected such a thing
multiple times before. ksh93 was developed by researchers who were
genius innovators, but incredibly sloppy maintainers.
2021-01-24 16:09:02 +00:00
Martijn Dekker
2a835a2d8a Fix restoring DEBUG trap upon exiting virtual subshell
This trap failed to be restored correctly when being trapped in
a subshell, causing corruption or a crash when restoring the
parent shell environment's trap upon leaving the subshell.

Thanks to Koichi Nakashima for the report and reproducer.

src/cmd/ksh93/sh/fault.c: sh_sigreset():
- Fix an off-by-one error in the loop that restores the
  pseudosignal traps.

src/cmd/ksh93/tests/basic.sh:
- Test overwriting the main shell trap in a subshell for all
  pseudosignals.

Makes progress on: https://github.com/ksh93/ksh/issues/155
2021-01-24 01:06:11 +00:00
Lev Kujawski
e73696fd87
Reenable the clobber method on UnixWare. (#160)
UnixWare's ps prefers to read psinfo (from the proc structure in
kernel memory) within /proc as an anti-Trojan horse measure.
Updates to argv[0] are still reflected within /proc/$pid/cmdline,
which is useful for diagnostic purposes.

src/cmd/ksh93/sh/main.c:
- Remove __USLC__ from the list of platforms excluded from the
  fixargs method.

src/cmd/ksh93/tests/basic.sh:
- Read /proc/$pid/cmdline instead of ps on UnixWare.
2021-01-23 21:31:47 +00:00
Martijn Dekker
d2cc520883 Disable SHOPT_KIA (ksh -R) by default
SHOPT_KIA enables the -R option that generates a cross-reference
database from a script. However, no tool to analyse this database
is shipped or seems to be available anywhere (in spite of multiple
people looking for one), and the format is very opaque. No usage
examples are known or findable on the internet. This seems like it
should not be compiled in by default, although we'll keep the code
in case some way to use it is found.

src/cmd/ksh93/SHOPT.sh:
- Disable SHOPT_KIA by default by removing the default 1 value.

src/cmd/ksh93/sh/args.c, src/cmd/ksh93/sh/parse.c:
- Fix a couple of preprocessor logic bugs that made it impossible
  to compile ksh without SHOPT_KIA.

src/cmd/ksh93/data/builtins.c:
- Fix typo in -R doc in ksh --man (in case SHOPT_KIA is enabled).

src/cmd/ksh93/sh.1:
- Since sh.1 is not generated dynamically, remove the -R doc.
2021-01-23 18:26:38 +00:00
Martijn Dekker
47468f56c2 main.c: fixargs(): add support for *BSD using setproctitle(3)
What is this for? See cefe087d

src/cmd/ksh93/Mamfile:
- Make iffe generate a test for the presence of setproctitle(3).

src/cmd/ksh93/sh/main.c:
- Include setproctitle test result.
- Re-enable fixargs() for FreeBSD and DragonFly BSD.
  Disable it for UnixWare.
- fixargs(): Add _lib_setproctitle version. Keep it simple with a
  128-character buffer array -- should be plenty for 'ps' output.
- fixargs(): Fix an off-by-one in zeroing the rest of the buffer.

src/cmd/ksh93/tests/basic.sh:
- Update the relevant regression test to run on FreeBSD/DragonFly
  and tolerate the "ksh: " prefix added by setproctitle(3).
2021-01-21 22:59:31 +01:00
Martijn Dekker
7fdeadd4f1 Increase shcomp bytecode header version; doc updates
src/cmd/ksh93/include/version.h:
- Centrally define the 93u+m copyright (SH_RELEASE_CPYR) for adding
  to the original AT&T copyright in 'ksh --man' and 'shcomp --man'.
- Centrally define the binary header version number for bytecode
  generated by shcomp: SHCOMP_HDR_VERSION.
- Bump SHCOMP_HDR_VERSION from 3 to 4. Converting all the preset
  aliases to builtin commands has caused new bytecode to be
  incompatible with old ksh. (However, old bytecode runs fine on
  93u+m, because shcomp pre-expands the preset aliases.)

src/cmd/ksh93/sh/shcomp.c:
- Instead of keeping its own version date (not changed since 2003),
  use the same version string as ksh itself (SH_RELEASE).
- Use SH_RELEASE_CPYR for the extra 93u+m copyright string.
- Use SHCOMP_HDR_VERSION for the bytecode header.

src/cmd/ksh93/sh/parse.c: sh_parse():
- Use SHCOMP_HDR_VERSION for the bytecode version check.

src/cmd/ksh93/data/builtins.c: opt_ksh[]:
- Use SH_RELEASE_CPYR for the extra 93u+m copyright string.

src/cmd/ksh93/COMPATIBILITY:
- Mention that 93u+m shcomp bytecode won't run on older ksh.
- Document changes in printf %T (re: 9526b3fa).

src/cmd/ksh93/README:
- Mention that we run on UnixWare (with major regressions).
  https://github.com/ksh93/ksh/pull/159#issuecomment-764667929
2021-01-21 17:04:14 +00:00
Martijn Dekker
d00b4b39f6 Fix side effect to exit status of DEBUG trap in comsub
This fixes the following:

trap ':' DEBUG
r=$(exit 123)
echo $? # Expected 123, but actually 0.

Thanks to Koichi Nakashima for the report and reproducer.

src/cmd/ksh93/sh/fault.c: sh_trap():
- Restore the saved current exit status (exitval) for all traps.
  Do not except the DEBUG trap from doing that. I've no idea why
  this exception was made, but it's not correct.

src/cmd/ksh93/tests/basic.sh:
- Add tests.

Makes progress on: https://github.com/ksh93/ksh/issues/155
2021-01-20 17:48:09 +00:00
Martijn Dekker
7bab9508aa Fix crash on subshell exit if PWD is inaccessible (re: dd9bc229)
This commit also further mitigates the problems with restoring an
inaccessible or nonexistent PWD on exiting a virtual subshell.

Harald van Dijk writes:
> On a build of ksh with -fsanitize=undefined to help diagnose
> problems:
>
> $ mkdir deleted
> $ cd deleted
> $ rmdir ../deleted
> $ ksh -c '(cd /; (cd /)); :'
> /home/harald/ksh/src/cmd/ksh93/sh/subshell.c:561:22: runtime
> error: null pointer passed as argument 1, which is declared to
> never be null
> Segmentation fault (core dumped)
>
> Note that it segfaults the same with default compilation flags,
> but it does not print out the useful extra message. The code
> assumes that pwd is non-null and passes it to strcmp without
> checking, but it will be null if the current directory cannot be
> determined, for instance because it has been deleted.

src/cmd/ksh93/sh/subshell.c: sh_subshell():
- Avoid the null pointer dereference reported above.

src/cmd/ksh93/bltins/cd_pwd.c: b_cd():
- Fork a virtual subshell even on systems with fchdir(2) if the
  present working directory tests as inaccessible on invoking 'cd';
  it may no longer exist and fchdir would fail to get a handle.
  (For the test we have to opendir(3) the full path to the PWD and
  not ".", as the latter may succeed even if the PWD is gone.)

src/cmd/ksh93/data/builtins.c:
- Update 'cd' version string.

Fixes:   https://github.com/ksh93/ksh/issues/153
Related: https://github.com/ksh93/ksh/issues/141
2021-01-19 18:47:41 +00:00
Martijn Dekker
82847bba3f sh_main(): do not decrease null pointer
src/cmd/ksh93/sh/main.c: sh_main():
- Reading the code makes it obvious that the shp->comdiv-- decrease
  in the 'else' block is never reached unless that pointer is still
  null, in which case it makes no sense to decrease it. Must be
  some kind of missed leftover from old code. Remove the decrease.
2021-01-19 18:39:43 +00:00
Martijn Dekker
de7eb85255 sh_subsavefd(): avoid a negative shift operand
This change is backported from the abandoned ksh 93v- beta.

src/cmd/ksh93/sh/subshell.c: sh_subsavefd():
- Do not subtract 1 from fd, as this would cause a negative shift
  operand for stdin (fd==0).
2021-01-19 18:30:10 +00:00
Martijn Dekker
5f6371f078 main.c: tweak fixargs() macros: single point of truth 2021-01-18 09:33:26 +00:00
Martijn Dekker
4dcf5c5066 Apply patches to build on DragonFly BSD and (older) FreeBSD
This now makes ksh build on DragonFly BSD.

bin/package, src/cmd/INIT/package.sh:
- DragonFly also needs the -lm hack for LDFLAGS.

src/cmd/ksh93/sh/main.c, src/cmd/ksh93/tests/basic.sh:
- fixargs() doesn't work on DragonFly either
  (re: 9b7c392a, 159fb9ee, cefe087d).

The following are backported from:
https://github.com/att/ast/issues/26#issuecomment-313927854
https://github.com/att/ast/pull/19

src/lib/libast/comp/setlocale.c:
- Add missing #include <errno.h> since errno is used.

src/lib/libast/features/standards:
- Do not set any standards macros (_POSIX_SOURCE etc) on FreeBSD or
  DragonflyBSD; they disable too much functionality on those.

src/lib/libast/features/wchar:
- Set _STDFILE_DECLARED on DragonFly, too.

src/lib/libast/include/sfio.h, src/lib/libast/include/sfio_t.h,
src/lib/libast/sfio/_sfopen.c, src/lib/libast/sfio/sfclrlock.c,
src/lib/libast/sfio/sfhdr.h, src/lib/libast/sfio/sfnew.c,
src/lib/libast/sfio/sfset.c:
- Rename SF_* macros to SFIO_* to avoid a conflict with system
  headers.

src/lib/libast/string/strexpr.c:
- Rename error() to err() to avoid a conflict.
2021-01-18 09:08:48 +00:00
Martijn Dekker
8361e065e6 job_unpost(): fix segfault
This function could segfault under certain conditions (in macOS
Terminal, when ksh received SIGWINCH and a complex PS1 prompt is
defined; see 61437b27)

0   ksh            job_unpost + 49 (jobs.c:1703)
1   ksh            job_reap + 1632 (jobs.c:468)
2   ksh            job_wait + 942 (jobs.c:1525)
3   ksh            sh_exec + 19579 (xec.c:1627)
4   ksh            sh_eval + 545 (xec.c:763)
5   ksh            sh_trap + 427
6   ksh            ed_emacsread + 3598 (emacs.c:1072)
7   ksh            slowread + 489 (io.c:1962)
8   ksh            sfrd + 1026 (sfrd.c:253)
9   ksh            _sffilbuf + 587 (sffilbuf.c:105)
10  ksh            sfreserve + 662
11  ksh            exfile + 1922 (main.c:527)
12  ksh            sh_main + 1071 (main.c:351)
13  libdyld.dylib  start + 1

src/cmd/ksh93/sh/jobs.c: job_unpost():
- Fix a dereference of a possible null pointer returned by
  job_byjid(). Add a check and return if that pointer is null.
2021-01-18 07:14:06 +00:00
Martijn Dekker
e25d9f4190 nv_newattr(): fix potential invalid free
src/cmd/ksh93/sh/name.c:
- Zero the 'cp' pointer after freeing it, as the next loop
  iteration may otherwise re-use the old address.
2021-01-17 03:55:46 +00:00
Martijn Dekker
9b7c392a7c Disable fixargs() on Solaris (re: 159fb9ee, cefe087d)
It doesn't work on Solaris either.
2021-01-10 18:47:12 +00:00
Martijn Dekker
159fb9ee27 main.c: Tweak fixargs() (re: cefe087d)
src/cmd/ksh93/sh/main.c: fixargs():
- Erase the entire length of the command arguments buffer (the
  space from argv[0] until environ[0]) so that remnants of longer
  command arguments aren't left in 'ps' output when executing a
  hashbang-ess script with a shorter command line.
- Disable fixargs() on FreeBSD. It has never had any effect on that
  system; apparently it either requires another method to rewrite
  arguments for 'ps' output purposes (which?) or it's not possible.

src/cmd/ksh93/tests/basic.sh:
- Skip the test if running on FreeBSD.
2021-01-10 06:34:49 +00:00
Martijn Dekker
4d0b77d398 Revert "Fix SIGALRM core dump (Solaris patch 230-18229654)"
This reverts commit 13e7b262. It caused the regression test for the
'alarm' builtin, introduced in 18b3f4aa, to hang on FreeBSD.
2021-01-09 13:18:00 +00:00
Martijn Dekker
096f46eee5 Fix for memory mgmt in variable expansion (Solaris 105-CR7032068)
This upstreams a Solaris patch:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/105-CR7032068.patch

No other information is publicly available but this has been in
production use on Solaris for a long time. It looks like this is
intended to avoid an invalid free().
2021-01-09 00:28:11 +00:00
Martijn Dekker
aa7713c2a9 sh_init(): rm directoryless '.profile' login file path
This applies a patch from Solaris:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/160-CR7175995.patch

There is no public information on why it's needed, but it seems
sensible on the face of it. Using a file called '.profile' in the
PWD on login, without a directory path, is redundant at best, since
"$HOME/.profile" (e_profile, see data/msg.c) is already used. And
if the PWD is not $HOME at login time, it seems to me there are
serious problems and the last thing you want is to read some
random and probably dodgy '.profile' from the PWD.

src/cmd/ksh93/sh/init.c: sh_init(): login_files[]:
- Remove redundant/problematic ".profile" entry.
2021-01-08 23:53:04 +00:00
Martijn Dekker
86fc4c6d0a init: Refuse to import $KSH_VERSION from environment
$KSH_VERSION is initialised as a nameref to ${.sh.version}, but it
was not realiable as it could be overridden from the environment.
Some scripts do version checking so this would allow influencing
their execution.

This fix is inspired by the following Solaris patch:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/200-17435456.patch
but a different approach was needed, because the code has changed
(see 960a1a99).

src/cmd/ksh93/sh/init.c: env_init():
- Refuse to import $KSH_VERSION. Using strncmp(3) might be crude,
  but it's effective and I can't figure out another way.
2021-01-08 21:59:30 +00:00
Martijn Dekker
13e7b26202 Fix SIGALRM core dump (Solaris patch 230-18229654)
This should fix the following Solaris bug:
18229654 ksh93 read not reentrant in alarm context dumps core
with the patch taken from:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/230-18229654.patch

Unfortunately the link to the details is inaccessible
as lists.research.att.com is dead.
2021-01-08 18:50:34 +00:00
Martijn Dekker
62cf88d0df Fix SIGHUP on termination (Solaris patch 260-22964338)
This fixes the following bug filed with Solaris: "22964338 ksh93
appears to send SIGHUP to unrelated processes on occasion". It is
fixed by applying this patch by Lijo George from the Solaris repo:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/260-22964338.patch

The ksh2020 upstream rejected this, but if it's in production use
in Solaris, Solaris, it's probably good enough for 93u+m. If any
breakage is left, it can be fixed later.
https://github.com/att/ast/pull/1

src/cmd/ksh93/include/jobs.h,
src/cmd/ksh93/sh/fault.c,
src/cmd/ksh93/sh/jobs.c:
- Use a new job_hup() function instead of job_kill() to send SIGHUP
  to job processes on termination. The new function checks if a job
  is in fact still live before issuing SIGHUP to it.
2021-01-08 17:33:04 +00:00
Martijn Dekker
ab98ec65e4 Replace safe FD fix with Solaris/ksh2020 version (re: 045fe6a1)
This pulls a new version of sh_iosafefd() from:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/285-30771135.patch

It was written by Kurtis Rader for ksh2020:
https://github.com/att/ast/issues/198
https://github.com/att/ast/pull/199
It is presumably better than the Red Hat version and also comes
with more regression test cases (although it still doesn't fix
modernish BUG_CSUBSTDO, which remains in the TODO file).

This commit does not go along with other peripheral changes from
that patch, i.e. a different name and location of this function.

src/cmd/ksh93/sh/io.c:
- Replace sh_iosafefd() as above.

src/cmd/ksh93/tests/subshell.sh:
- Add and tweak tests from the patch.
2021-01-08 16:35:26 +00:00
Martijn Dekker
17ebfbf6a3 Fix I/O redirection in -c script (Solaris patch 280-23332860)
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/280-23332860.patch

Info and reproducers:
https://github.com/att/ast/issues/36

In a -c script (like ksh -c 'commands'), the last command
misredirects standard output if an EXIT or ERR trap is set.
This appears to be a side effect of the optimisation that
runs the last command without forking.

This applies a patch by George Lijo that flags these specific
cases and disables the optimisation.

src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/bltins/trap.c,
src/cmd/ksh93/sh/init.c,
src/cmd/ksh93/sh/main.c,
src/cmd/ksh93/sh/xec.c:
- Apply patch as above.

src/cmd/ksh93/tests/io.sh:
- Add the reproducers from the bug report as regression tests.
2021-01-08 15:15:53 +00:00
Martijn Dekker
7c47ab56fe I/O: Properly handle EIO error (Solaris patch 275-20855453)
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/275-20855453.patch
https://github.com/att/ast/issues/30

George Lijo wrote on 17 Feb 2017:
> Here's a reproducible testcase on a Solaris11 host running
> ksh93u+(2012-08-01).
> $ cat a.sh
> #!/bin/sh
>
> AAA="aaa"
> echo 'insert character'
> BBB=`echo ${AAA} | sed "s/aaa/bbb/g"`
> logger "variable BBB = ${BBB}"
>
> $ cat t.sh
> #!/bin/ksh
>
> sleep 10
> /bin/ksh ./a.sh
> exit 0
>
> $
>
> $ ./t.sh
>
> The expected result is:
>
> Apr 9 12:43:34 lab user: [ID 702911 user.notice] variable BBB = bbb
>
> because variable "BBB" is supposed to be set to 'bbb' in a.sh.
>
> But if the parent shell is terminated, the variable is wrongly set.
>
> user@xxxxx$ telnet lab
> ...
> $ ./t.sh & <--- Run t.sh in background.
> [1] 2067
> $ logout <--- CTRL + D to exit while t.sh is running.
> Connection to lab closed by foreign host.
>
> Again, access the system and check the output:
>
> user@xxxxx$ telnet lab
> ...
> $ tail -f /var/adm/messages
> :
> Apr 9 12:47:47 lab user: [ID 702911 user.notice] variable BBB =
> insert character <--- !!!
> Apr 9 12:47:47 lab bbb
> <--- !!!
>
> Thus the variable is wrongly set. (The previous echo string was
> not cleared.)
>
> The issue happens because the EIO error during the logout is not
> handled properly.

src/cmd/ksh93/sh/io.c,
src/lib/libast/include/error.h:
- Amend the ERROR_PIPE() macro to check for EIO as well as EPIPE
  and ECONNRESET.
2021-01-08 13:28:45 +00:00
Martijn Dekker
ad9ea0ba7d Fix off-by-one in nv_mktype() (Solaris patch 210-Bug15993811)
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/210-Bug15993811.patch

Unfortunately there is no publicly available documentation on why
this change was needed. We just have to assume the Solaris people
knew what they were doing. ksh2020 upstreamed this too (as well as
all the other Solaris patches applied here).
2021-01-08 11:56:04 +00:00
Martijn Dekker
bae02c39b6 Fix for argv for setuid scripts (Solaris patch 115-CR6934836)
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/115-CR6934836.patch

Unfortunately there is no publicly available documentation on what
this does or why it was needed. We just have to assume the Solaris
people knew what they were doing. ksh2020 upstreamed this too (as
well as all the other Solaris patches applied here).
2021-01-08 11:28:33 +00:00