1
0
Fork 0
mirror of git://git.code.sf.net/p/cdesktopenv/code synced 2025-02-15 04:32:24 +00:00
Commit graph

448 commits

Author SHA1 Message Date
Martijn Dekker
ccd98fe754 src/cmd/ksh93/{COMPATIBILITY,README}: updates in prep for release 2021-01-19 01:12:04 +00:00
Martijn Dekker
33b6718bf2 iffe: avoid pointless reruns: update result timestamp if unchanged
If iffe re-ran a test because the source test script changed, but
the result file is unchanged, it didn't update the timestamp of the
result, so the source script remained newer. The build system then
kept pointlessly re-running the test on each rebuild. If a central
test script such as src/lib/libast/features/standards was changed,
this had cascading effects, e.g., causing libast to be rebuilt over
and over as I recompiled small changes elsewhere. Until now, my
workaround was to delete the entire 'arch' directory and start
over. Hopefully that will now no longer be needed.

src/cmd/INIT/iffe.sh:
- If a test output file has not changed after rerunning a test that
  has changed, still update the output file's timestamp using
  touch(1) to signal that the test has already been run.
2021-01-18 21:55:57 +00:00
Martijn Dekker
dd0d03b973 Eliminate LDFLAGS hack to compile on certain OSs
Instead, we now link to the libm system math library where needed
by adding -lm to the relevant compile commands in the Mamfiles.
This is not needed on every system but never does any harm.

(This adds more custom edits to the Mamfiles, which were originally
generated from the nmake Makefiles. This takes us further from
restoring nmake, but that already wasn't going to happen anyway,
due to its many problems... the path forward will be to translate
the Mamfiles to some other, current make system such as GNU make.)

bin/package, src/cmd/INIT/package.sh:
- Remove LDFLAGS=-lm hack for DragonFly BSD, NetBSD and Solaris.

src/cmd/builtin/Mamfile,
src/cmd/ksh93/Mamfile,
src/lib/libdll/Mamfile:
- Add -lm where linking failed on any of the three mentioned OSs.

src/lib/libdll/features/dll:
- In the output test program, add missing #include <math.h>, fixing
  unknown identifier errors on NetBSD (ldexp, ldexpl).

src/cmd/builtin/features/pty:
- Add missing #include <stdio.h> to make printf work on all systems
  (this is just a feature test, no need to bother with sfio here).

src/lib/libast/features/stdio:
- Undef __FILE_T to avoid interference from system headers on QNX.
  (There are still other problems preventing a build on QNX 6.5.0.
  The shipped version of gcc seems to be broken.)
2021-01-18 19:16:17 +00:00
Martijn Dekker
5f6371f078 main.c: tweak fixargs() macros: single point of truth 2021-01-18 09:33:26 +00:00
Martijn Dekker
4dcf5c5066 Apply patches to build on DragonFly BSD and (older) FreeBSD
This now makes ksh build on DragonFly BSD.

bin/package, src/cmd/INIT/package.sh:
- DragonFly also needs the -lm hack for LDFLAGS.

src/cmd/ksh93/sh/main.c, src/cmd/ksh93/tests/basic.sh:
- fixargs() doesn't work on DragonFly either
  (re: 9b7c392a, 159fb9ee, cefe087d).

The following are backported from:
https://github.com/att/ast/issues/26#issuecomment-313927854
https://github.com/att/ast/pull/19

src/lib/libast/comp/setlocale.c:
- Add missing #include <errno.h> since errno is used.

src/lib/libast/features/standards:
- Do not set any standards macros (_POSIX_SOURCE etc) on FreeBSD or
  DragonflyBSD; they disable too much functionality on those.

src/lib/libast/features/wchar:
- Set _STDFILE_DECLARED on DragonFly, too.

src/lib/libast/include/sfio.h, src/lib/libast/include/sfio_t.h,
src/lib/libast/sfio/_sfopen.c, src/lib/libast/sfio/sfclrlock.c,
src/lib/libast/sfio/sfhdr.h, src/lib/libast/sfio/sfnew.c,
src/lib/libast/sfio/sfset.c:
- Rename SF_* macros to SFIO_* to avoid a conflict with system
  headers.

src/lib/libast/string/strexpr.c:
- Rename error() to err() to avoid a conflict.
2021-01-18 09:08:48 +00:00
Martijn Dekker
8361e065e6 job_unpost(): fix segfault
This function could segfault under certain conditions (in macOS
Terminal, when ksh received SIGWINCH and a complex PS1 prompt is
defined; see 61437b27)

0   ksh            job_unpost + 49 (jobs.c:1703)
1   ksh            job_reap + 1632 (jobs.c:468)
2   ksh            job_wait + 942 (jobs.c:1525)
3   ksh            sh_exec + 19579 (xec.c:1627)
4   ksh            sh_eval + 545 (xec.c:763)
5   ksh            sh_trap + 427
6   ksh            ed_emacsread + 3598 (emacs.c:1072)
7   ksh            slowread + 489 (io.c:1962)
8   ksh            sfrd + 1026 (sfrd.c:253)
9   ksh            _sffilbuf + 587 (sffilbuf.c:105)
10  ksh            sfreserve + 662
11  ksh            exfile + 1922 (main.c:527)
12  ksh            sh_main + 1071 (main.c:351)
13  libdyld.dylib  start + 1

src/cmd/ksh93/sh/jobs.c: job_unpost():
- Fix a dereference of a possible null pointer returned by
  job_byjid(). Add a check and return if that pointer is null.
2021-01-18 07:14:06 +00:00
Martijn Dekker
7222ba3af7 tests/basic.sh: fix intermittent spurious regress fail
~- and ~+ are ksh93-specific tilde expansions that expand to
$OLDPWD and $PWD, respectively. On some systems, $OLDPWD is not set
on entry to the test script, because it is not exported to the
environment. This made it unset before any 'cd' was executed,
which (correctly) disabled ~- expansion.

src/cmd/ksh93/tests/basic.sh:
- Before testing 'cd ~-', make sure $OLDPWD is set by cd'ing to
  /dev first (a directory guaranteed by POSIX).
2021-01-18 06:27:08 +00:00
Martijn Dekker
8e8ff5f6f6 Disable SHOPT_PFSH in feature test (re: f089d799)
src/cmd/ksh93/features/options:
- To make sure SHOPT_PFSH stays disabled on Solaris, we
  also need to stop this feature test from re-enabling it.
2021-01-18 06:11:45 +00:00
Martijn Dekker
f089d7990a Solaris: disable deprecated SHOPT_PFSH by default
Solaris /bin/ksh disables the SHOPT_PFSH compile option ("solaris
exec_attr(4) profile execution") with a patch. Since this option
applies to Solaris and variants only, let's upstream that change.

(Solaris now provides pfksh93 as a wrapper around ksh93, and does
the same for other shells, so profiling functionality is no longer
ksh-specific.)

If you want to re-enable it, add -DSHOPT_PFSH to your $CCFLAGS.

Original patch:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/150-CR7168611.patch

src/cmd/ksh93/Makefile:
- Add note that edits in Makefile are ineffective as we do not ship
  nmake.
- Disable SHOPT_PFSH, cosmetically.

src/cmd/ksh93/Mamfile:
- Remove -DSHOPT_PFSH from all compiler commands.
2021-01-18 05:29:09 +00:00
Martijn Dekker
4cfe49aebb package: involve $CCFLAGS when determing 64-bit arch (re: 9a48ba15)
bin/package, src/cmd/INIT/package.sh:
- It can depend on the compiler flags passed whether the compiler
  produces code for a 64-bit architecture, so pass $CCFLAGS to
  the compiler when testing whether it creates 64-bit object code.

README.md:
- Copy-edit of build instructions.
2021-01-18 03:56:03 +00:00
Martijn Dekker
580ff61617 Fix release and standards build flags (re: 35672208, aa4669ad)
bin/package, src/cmd/INIT/package.sh:
- CCFLAGS overwrites the autodetected optimisation flags (e.g. -Os)
  if set. Unfortunately, that also happened when we added something
  to CCFLAGS for a release build or to add an extra flag needed by
  Solaris. The fix is to use a new flags variable (KSH_RELFLAGS)
  instead. This needs to be done in a different place as it needs
  to be added to the mamake command as an assignment argument.
- Remove the Solaris CCFLAGS hack; see features/common below.

src/*/*/Mamfile:
- Add ${KSH_RELFLAGS} to all the compiler commands.

src/lib/libast/features/common:
- Enable POSIX standard on Solaris (i.e.: if __sun is defined) by
  defining _XPG6 directly in the feature test that generates
  ast_std.h, which is indirectly included by everything. This
  removes the need to pass -D_XPG6 via CCFLAGS. (Doing so
  automatically with gcc was not otherwise possible.)

src/cmd/INIT/cc.sol11.*:
- No longer pass -D_XPG6, as per above.
2021-01-18 01:07:45 +00:00
Martijn Dekker
e25d9f4190 nv_newattr(): fix potential invalid free
src/cmd/ksh93/sh/name.c:
- Zero the 'cp' pointer after freeing it, as the next loop
  iteration may otherwise re-use the old address.
2021-01-17 03:55:46 +00:00
Martijn Dekker
9a48ba1557 package: fix code for detecting 64-bit compiler
bin/package, src/cmd/INIT/package.sh:
- The code for detecting a 64-bit object file was seriously broken:
  the temporary file name could contain '64' because it included $$,
  the current PID, and 64-bit was detected if the output of 'file'
  (which includes the complete file name) contained '64'. Fix by
  removing the file name from 'file' output before testing.
- Also refactor that code a bit and remove the nonsensical test if
  /bin/sh is a 64-bit binary, which is neither here nor there. It's
  what the compiler produces that we need to care about.
2021-01-16 22:53:35 +00:00
Martijn Dekker
6025c8125e make.probe: add fallback optimisation flags
src/cmd/INIT/make.probe:
- probe_optimize: Also try -O2 and -O, for compilers (such as
  Solaris Studio cc) that do not support -Os.
- Use more robust code to loop through possible optimiser flags.
2021-01-16 22:01:07 +01:00
Martijn Dekker
68a6f9a6e2 Fix Solaris cc wrappers (re: 4e67234a)
The versions from the Solaris patch require $CC_EXPLICIT to be set,
which is specific to the internal Solaris build environment.

src/cmd/INIT/cc.sol11.*:
- Cope without $CC_EXPLICIT set in environment; fall back to $CC
  and if that is not set either, detect whether to use cc or gcc.
- Set appropriate flags for cc (Solaris Studio) or gcc, including
  the necessary -D_XPG6 flag, without which ksh crashes on Solaris.

bin/package, src/cmd/INIT/package.sh:
- Update hack to add the -D_XPG6 flag so it applies to gcc only
  (note: the src/cmd/INIT/cc.* scripts are never used for gcc).
2021-01-16 19:25:39 +01:00
Martijn Dekker
649f4b047b tests/builtins.sh: tweak for HP-UX 2021-01-12 18:18:39 +00:00
Martijn Dekker
4d5e21de80 Fix for compiling with SHOPT_DYNAMIC disabled
src/cmd/ksh93/bltins/typeset.c:
- Correct faulty preprocessor directive logic causing a build
  failure if SHOPT_DYNAMIC is disabled.
2021-01-10 23:02:15 +00:00
Martijn Dekker
e981f7c8b8 Regression test tweaks to avoid false fails on Solaris
src/cmd/ksh93/tests/path.sh:
- Re-export PATH after unsetting it.

src/cmd/ksh93/tests/pty.sh:
- Increase some delays.
2021-01-10 20:53:41 +01:00
Martijn Dekker
9b7c392a7c Disable fixargs() on Solaris (re: 159fb9ee, cefe087d)
It doesn't work on Solaris either.
2021-01-10 18:47:12 +00:00
Martijn Dekker
159fb9ee27 main.c: Tweak fixargs() (re: cefe087d)
src/cmd/ksh93/sh/main.c: fixargs():
- Erase the entire length of the command arguments buffer (the
  space from argv[0] until environ[0]) so that remnants of longer
  command arguments aren't left in 'ps' output when executing a
  hashbang-ess script with a shorter command line.
- Disable fixargs() on FreeBSD. It has never had any effect on that
  system; apparently it either requires another method to rewrite
  arguments for 'ps' output purposes (which?) or it's not possible.

src/cmd/ksh93/tests/basic.sh:
- Skip the test if running on FreeBSD.
2021-01-10 06:34:49 +00:00
Martijn Dekker
e7202832fd Revert "bin/package: don't test-compile using possibly broken dev shell"
This reverts commit 600cb182.
$cc may be a system compiler binary, it is not necessarily a
src/cmd/INIT/cc.* wrapper script; so prefixing 'sh' is wrong.
2021-01-09 16:16:28 +00:00
Martijn Dekker
4d0b77d398 Revert "Fix SIGALRM core dump (Solaris patch 230-18229654)"
This reverts commit 13e7b262. It caused the regression test for the
'alarm' builtin, introduced in 18b3f4aa, to hang on FreeBSD.
2021-01-09 13:18:00 +00:00
Martijn Dekker
4e67234ae8 INIT: Add Solaris 11 compiler wrappers (Solaris patch 005-compiler)
This upstreams a Solaris patch:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/005-compiler.patch
2021-01-09 01:03:08 +00:00
Martijn Dekker
096f46eee5 Fix for memory mgmt in variable expansion (Solaris 105-CR7032068)
This upstreams a Solaris patch:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/105-CR7032068.patch

No other information is publicly available but this has been in
production use on Solaris for a long time. It looks like this is
intended to avoid an invalid free().
2021-01-09 00:28:11 +00:00
Martijn Dekker
aa7713c2a9 sh_init(): rm directoryless '.profile' login file path
This applies a patch from Solaris:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/160-CR7175995.patch

There is no public information on why it's needed, but it seems
sensible on the face of it. Using a file called '.profile' in the
PWD on login, without a directory path, is redundant at best, since
"$HOME/.profile" (e_profile, see data/msg.c) is already used. And
if the PWD is not $HOME at login time, it seems to me there are
serious problems and the last thing you want is to read some
random and probably dodgy '.profile' from the PWD.

src/cmd/ksh93/sh/init.c: sh_init(): login_files[]:
- Remove redundant/problematic ".profile" entry.
2021-01-08 23:53:04 +00:00
Martijn Dekker
5d7e00a109 cd: validate $OLDPWD (Solaris patch 185-Bug17714341)
This change was pulled in from:
https://raw.githubusercontent.com/oracle/solaris-userland/master/components/ksh93/patches/185-Bug17714341.patch

No public information about the reasons for this change is
available, but it seems reasonable to trust that the Solaris people
found a legitimate need for it.

src/cmd/ksh93/bltins/cd_pwd.c: b_cd():
- When determining the old PWD before 'cd', do not trust shp->pwd
  but get and validate the current PWD using path_pwd().
2021-01-08 22:31:16 +00:00
Martijn Dekker
1de20d65a8 Fix crash on long PS1 prompt (Solaris patch 195-17824699)
Original report and info:
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01677.html
https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01679.html

Patch pulled in from:
https://raw.githubusercontent.com/oracle/solaris-userland/master/components/ksh93/patches/195-17824699.patch

src/cmd/ksh93/edit/edit.c: ed_setup():
- Prevent the ed_setup() function from writing past ep->e_prompt,
  which is set to the local char prompt[PRSIZE] variable in
  ed_emacsread().

src/cmd/ksh93/include/edit.h:
- Increase maximum prompt size, PRSIZE, to 256.
2021-01-08 22:22:47 +00:00
Martijn Dekker
86fc4c6d0a init: Refuse to import $KSH_VERSION from environment
$KSH_VERSION is initialised as a nameref to ${.sh.version}, but it
was not realiable as it could be overridden from the environment.
Some scripts do version checking so this would allow influencing
their execution.

This fix is inspired by the following Solaris patch:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/200-17435456.patch
but a different approach was needed, because the code has changed
(see 960a1a99).

src/cmd/ksh93/sh/init.c: env_init():
- Refuse to import $KSH_VERSION. Using strncmp(3) might be crude,
  but it's effective and I can't figure out another way.
2021-01-08 21:59:30 +00:00
Martijn Dekker
13e7b26202 Fix SIGALRM core dump (Solaris patch 230-18229654)
This should fix the following Solaris bug:
18229654 ksh93 read not reentrant in alarm context dumps core
with the patch taken from:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/230-18229654.patch

Unfortunately the link to the details is inaccessible
as lists.research.att.com is dead.
2021-01-08 18:50:34 +00:00
Martijn Dekker
99cbb7f794 Add reproducer from https://github.com/att/ast/issues/7 as regress
ksh 93u+ has a subshell leak bug where a variable exported in
function in a subshell is also visible in a different subshell.
This is long fixed in 93u+m, but there wasn't a regression test for
this particular bug yet, so this commit adds one.
2021-01-08 18:15:11 +00:00
Martijn Dekker
62cf88d0df Fix SIGHUP on termination (Solaris patch 260-22964338)
This fixes the following bug filed with Solaris: "22964338 ksh93
appears to send SIGHUP to unrelated processes on occasion". It is
fixed by applying this patch by Lijo George from the Solaris repo:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/260-22964338.patch

The ksh2020 upstream rejected this, but if it's in production use
in Solaris, Solaris, it's probably good enough for 93u+m. If any
breakage is left, it can be fixed later.
https://github.com/att/ast/pull/1

src/cmd/ksh93/include/jobs.h,
src/cmd/ksh93/sh/fault.c,
src/cmd/ksh93/sh/jobs.c:
- Use a new job_hup() function instead of job_kill() to send SIGHUP
  to job processes on termination. The new function checks if a job
  is in fact still live before issuing SIGHUP to it.
2021-01-08 17:33:04 +00:00
Martijn Dekker
ab98ec65e4 Replace safe FD fix with Solaris/ksh2020 version (re: 045fe6a1)
This pulls a new version of sh_iosafefd() from:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/285-30771135.patch

It was written by Kurtis Rader for ksh2020:
https://github.com/att/ast/issues/198
https://github.com/att/ast/pull/199
It is presumably better than the Red Hat version and also comes
with more regression test cases (although it still doesn't fix
modernish BUG_CSUBSTDO, which remains in the TODO file).

This commit does not go along with other peripheral changes from
that patch, i.e. a different name and location of this function.

src/cmd/ksh93/sh/io.c:
- Replace sh_iosafefd() as above.

src/cmd/ksh93/tests/subshell.sh:
- Add and tweak tests from the patch.
2021-01-08 16:35:26 +00:00
Martijn Dekker
17ebfbf6a3 Fix I/O redirection in -c script (Solaris patch 280-23332860)
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/280-23332860.patch

Info and reproducers:
https://github.com/att/ast/issues/36

In a -c script (like ksh -c 'commands'), the last command
misredirects standard output if an EXIT or ERR trap is set.
This appears to be a side effect of the optimisation that
runs the last command without forking.

This applies a patch by George Lijo that flags these specific
cases and disables the optimisation.

src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/bltins/trap.c,
src/cmd/ksh93/sh/init.c,
src/cmd/ksh93/sh/main.c,
src/cmd/ksh93/sh/xec.c:
- Apply patch as above.

src/cmd/ksh93/tests/io.sh:
- Add the reproducers from the bug report as regression tests.
2021-01-08 15:15:53 +00:00
Martijn Dekker
7c47ab56fe I/O: Properly handle EIO error (Solaris patch 275-20855453)
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/275-20855453.patch
https://github.com/att/ast/issues/30

George Lijo wrote on 17 Feb 2017:
> Here's a reproducible testcase on a Solaris11 host running
> ksh93u+(2012-08-01).
> $ cat a.sh
> #!/bin/sh
>
> AAA="aaa"
> echo 'insert character'
> BBB=`echo ${AAA} | sed "s/aaa/bbb/g"`
> logger "variable BBB = ${BBB}"
>
> $ cat t.sh
> #!/bin/ksh
>
> sleep 10
> /bin/ksh ./a.sh
> exit 0
>
> $
>
> $ ./t.sh
>
> The expected result is:
>
> Apr 9 12:43:34 lab user: [ID 702911 user.notice] variable BBB = bbb
>
> because variable "BBB" is supposed to be set to 'bbb' in a.sh.
>
> But if the parent shell is terminated, the variable is wrongly set.
>
> user@xxxxx$ telnet lab
> ...
> $ ./t.sh & <--- Run t.sh in background.
> [1] 2067
> $ logout <--- CTRL + D to exit while t.sh is running.
> Connection to lab closed by foreign host.
>
> Again, access the system and check the output:
>
> user@xxxxx$ telnet lab
> ...
> $ tail -f /var/adm/messages
> :
> Apr 9 12:47:47 lab user: [ID 702911 user.notice] variable BBB =
> insert character <--- !!!
> Apr 9 12:47:47 lab bbb
> <--- !!!
>
> Thus the variable is wrongly set. (The previous echo string was
> not cleared.)
>
> The issue happens because the EIO error during the logout is not
> handled properly.

src/cmd/ksh93/sh/io.c,
src/lib/libast/include/error.h:
- Amend the ERROR_PIPE() macro to check for EIO as well as EPIPE
  and ECONNRESET.
2021-01-08 13:28:45 +00:00
Martijn Dekker
3f38f8a285 emacs: Fix crash on inputting Asian chars (Solaris 240-22461939)
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/240-22461939.patch

Information:
https://github.com/att/ast/issues/6

George Lijo wrote on 14 Mar 2016:
> I observed this issue in a Solaris 11 system on ksh2012-08-01
> [...]. The issue can be reproduced if we add Asian locales to
> ibus (such as Korean). In the ksh93 shell prompt, input some
> Asian character. ksh promptly dumps core [...].
>
> The coredump happens at the following line no 320 in
> src/cmd/ksh93/edit/emacs.c
>	if(c!='\t' && c!=ESC && !isdigit(c)).
>
> I referred the vi.c code and added the digit(c) macro, i.e.
> ((c&~STRIP)==0 && isdigit(c)) and replaced the isdigit(c) usage
> with the "digit(c)" macro.
2021-01-08 12:55:05 +00:00
Martijn Dekker
a3ccff6c75 cd: fix an invalid free() call (Solaris patch 211-21547336)
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/211-21547336.patch

src/cmd/ksh93/bltins/cd_pwd.c: b_cd():
- The functions path_pwd() and path_relative() in sh/path.c may
  return a pointer to e_dot[] (".") as a fallback if they fail to
  determine a path. This is a string in read-only memory
  (data/msg.c), so must not be freed. A pointer to that string may
  end up in sh.pwd (== shp->pwd), so b_cd() needs a check for that.
2021-01-08 12:43:19 +00:00
Martijn Dekker
ad9ea0ba7d Fix off-by-one in nv_mktype() (Solaris patch 210-Bug15993811)
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/210-Bug15993811.patch

Unfortunately there is no publicly available documentation on why
this change was needed. We just have to assume the Solaris people
knew what they were doing. ksh2020 upstreamed this too (as well as
all the other Solaris patches applied here).
2021-01-08 11:56:04 +00:00
Martijn Dekker
744e68e7be rm obsolete /usr/5bin paths (Solaris patch 130-CR7019368)
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/130-CR7019368.patch
2021-01-08 11:47:05 +00:00
Martijn Dekker
bae02c39b6 Fix for argv for setuid scripts (Solaris patch 115-CR6934836)
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/115-CR6934836.patch

Unfortunately there is no publicly available documentation on what
this does or why it was needed. We just have to assume the Solaris
people knew what they were doing. ksh2020 upstreamed this too (as
well as all the other Solaris patches applied here).
2021-01-08 11:28:33 +00:00
Martijn Dekker
3f15067272 setdisc(): Return null pointer if no event (Solaris 110-CR7061011)
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/110-CR7061011.patch

Unfortunately there is no publicly available documentation on why
this change was needed. We just have to assume the Solaris people
knew what they were doing. ksh2020 upstreamed this too (as well as
all the other Solaris patches applied here).

src/cmd/ksh93/sh/nvdisc.c: setdisc():
- If no <event> is known for <np>, return a null pointer instead
  of a pointer to the empty string.
2021-01-08 11:27:30 +00:00
Martijn Dekker
e20db01247 Apply Solaris patch 065-CR7110983
This change is pulled from here:
https://github.com/oracle/solaris-userland/blob/master/components/ksh93/patches/065-CR7110983.patch

Unfortunately there is no publicly available documentation on what
it does. We just have to assume the Solaris people knew what they
were doing. It looks like this fixes a memory leak in nv_putval().

This patch was also applied by ksh2020:
056386400a
2021-01-08 04:59:54 +00:00
Martijn Dekker
222515bf08 Implement hash tables for virtual subshells (re: 102868f8, 9d428f8f)
The forking fix implemented in 102868f8 and 9d428f8f, which stops
the main shell's hash table from being cleared if PATH is changed
in a subshell, can cause a significant performance penalty for
certain scripts that do something like

    ( PATH=... command foo )

in a subshell, especially if done repeatedly. This is because the
hash table is cleared (and hence a subshell forks) even for
temporary PATH assignments preceding commands.

It also just plain doesn't work. For instance:

    $ hash -r; (ls) >/dev/null; hash
    ls=/bin/ls

Simply running an external command in a subshell caches the path in
the hash table that is shared with a main shell. To remedy this, we
would have to fork the subshell before forking any external
command. And that would be an unacceptable performance regression.

Virtual subshells do not need to fork when changing PATH if they
get their own hash tables. This commit adds these. The code for
alias subshell trees (which was removed in ec888867 because they
were broken and unneeded) provided the beginning of a template for
their implementation.

src/cmd/ksh93/sh/subshell.c:
- struct subshell: Add strack pointer to subshell hash table.
- Add sh_subtracktree(): return pointer to subshell hash table.
- sh_subfuntree(): Refactor a bit for legibility.
- sh_subshell(): Add code for cleaning up subshell hash table.

src/cmd/ksh93/sh/name.c:
- nv_putval(): Remove code to fork a subshell upon resetting PATH.
- nv_rehash(): When in a subshell, invalidate a hash table entry
  for a subshell by creating the subshell scope if needed, then
  giving that entry the NV_NOALIAS attribute to invalidate it.

src/cmd/ksh93/sh/path.c: path_search():
- To set a tracked alias/hash table entry, use sh_subtracktree()
  and pass the HASH_NOSCOPE flag to nv_search() so that any new
  entries are added to the current subshell table (if any) and do
  not influence any parent scopes.

src/cmd/ksh93/bltins/typeset.c: b_alias():
- b_alias(): For hash table entries, use sh_subtracktree() instead
  of forking a subshell. Keep forking for normal aliases.
- setall(): To set a tracked alias/hash table entry, pass the
  HASH_NOSCOPE flag to nv_search() so that any new entries are
  added to the current subshell table (if any) and do not influence
  any parent scopes.

src/cmd/ksh93/sh/init.c: put_restricted():
- Update code for clearing the hash table (when changing $PATH) to
  use sh_subtracktree().

src/cmd/ksh93/bltins/cd_pwd.c:
- When invalidating path name bindings to relative paths, use the
  subshell hash tree if applicable by calling sh_subtracktree().
- rehash(): Call nv_rehash() instead of _nv_unset()ting the hash
  table entry; this is needed to work correctly in subshells.

src/cmd/ksh93/tests/leaks.sh:
- Add leak tests for various PATH-related operations in the main
  shell and in a virtual subshell.
- Several pre-existing memory leaks are exposed by the new tests
  (I've confirmed these in 93u+). The tests are disabled and marked
  TODO for now, as these bugs have not yet been fixed.

src/cmd/ksh93/tests/subshell.sh:
- Update.

Resolves: https://github.com/ksh93/ksh/issues/66
2021-01-07 22:18:25 +00:00
Martijn Dekker
a95d107ee5 Fix segfault while updating ${.sh.match}
The SHOPT_2DMATCH code block in sh_setmatch() modifies the 'ap'
pointer, which is initialised as nv_arrayptr(SH_MATCHNOD). This
caused a (rarely occurring) segfault in the following line near the
end of the function:
	ap->nelem -= x;
as this line assumed that 'ap' still had the initial value.

src/cmd/ksh93/sh/init.c: sh_setmatch():
- On init, save ap in ap_save and use ap_save instead of ap where
  it should be pointing to SH_MATCHNOD. This also allows removing
  two redundant nv_arrayptr(SH_MATCHNOD) calls, slightly increasing
  the efficiency of this function.
2021-01-07 17:34:47 +00:00
Martijn Dekker
c8513fcb7a Arith: informative err msg on '.' radix point in non-'.' locales
Scripts that use floating point shell arithmetic confusingly fail
with 'arithmetic syntax error' when running in a locale that uses
',' as the radix point, because '.' is generally assumed by
scripts. The error message is confounding as the problem is not a
syntax error but a locale that is incompatible with the script.

src/cmd/ksh93/sh/arith.c:
- If the locale's radix point is not '.' but a '.' is found in its
  place, issue an informative error message that instructs setting
  LC_NUMERIC=C.

Resolves: https://github.com/ksh93/ksh/issues/145
2021-01-05 23:16:53 +00:00
Martijn Dekker
d1483150ab 'cd': properly ignore $CDPATH if initial component is '.' or '..'
@stephane-chazelas writes:
> Per POSIX[*], cd should skip the $CDPATH processing if the first
> component of the directory given to cd is . or ...
>
> Yet, with ksh93u+m 2021-01-03 at least, while that's OK with ..,
> it's not with . with or without the posix option:
>
> $ CDPATH=/ ./ksh -o posix -c 'cd -P ./etc && pwd'
> /etc
> /etc
>
> It seems to be a regression introduced with ksh93u+ as I can't
> reproduce it with ksh93u or any version prior to that. I can also
> reproduce in u+, v- and the ksh2020 from the Ubuntu 20.04
> package.

src/cmd/ksh93/bltins/cd_pwd.c: b_cd():
- Skip $CDPATH processing not only if the path is absolute, but
  also if the initial path component is '.' or '..' (in the latter
  case the $CDPATH processing was done but appeared to be a no-op).

src/cmd/ksh93/tests/builtins.sh:
- Add regression test.

[*] https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/utilities/cd.html

Fixes: https://github.com/ksh93/ksh/issues/151
2021-01-05 05:04:24 +00:00
Martijn Dekker
3567220898 New semantic versioning scheme; disable vmalloc in release builds
As of this commit, ksh 93u+m has a standard semantic version number
<https://semver.org/>, beginning with 1.0.0-alpha. This is added to
the version string in a way that should be compatible with scripts
parsing ${.sh.version} or $(ksh --version). This addition does not
replace the release date and does not affect $((.sh.version)).

For non-release builds, the version string will be:
	FORK/VERSION+HASH YYYY-MM-DD
e.g.:	93u+m/1.0.0-alpha+41ef7f76 2021-01-03

For release builds, it will be:
	FORK/VERSION YYYY-MM-DD
e.g.:	93u+m/1.0.0 2021-01-03

It is now automatically decided by bin/package whether to build a
release or development build. When building from a directory that
is not a git repository, or if the current git branch name starts
with a number (e.g. '1.0'), the release build is enabled; otherwise
a development build is the default. This is arranged by adding -D
flags to $CCFLAGS as described below. These flags are prepended to
$CCFLAGS, so they can be overridden by adding your own -D or -U
flags via the environment.

In addition, AST vmalloc is disabled for release builds as of this
commit, forcing the use of the OS's standard malloc(3). In 2021,
this is generally more reliable, faster, and more economical with
memory than AST vmalloc. Several memory leaks and crashing bugs are
avoided, e.g.: <https://github.com/ksh93/ksh/issues/95>.

For development builds, vmalloc stays enabled (along with its known
bugs) because this allows the use of the vmstat builtin, making it
much more efficient to test for memory leaks. For more info, see
the regression test script: src/cmd/ksh93/tests/leaks.sh

bin/package, src/cmd/INIT/package.sh:
- Add flags for build type. In $CCFLAGS, define _AST_ksh_release if
  we're not on any git branch or on a git branch whose name starts
  with a number. Otherwise, define _AST_git_commit as the first 8
  characters of the current git commit hash.

src/lib/libast/features/vmalloc:
- If _AST_ksh_release is defined, disable vmalloc and force use of
  the operating system's malloc. Discussion:
  https://github.com/ksh93/ksh/issues/95

src/cmd/ksh93/include/version.h:
- Define new format for version string, adding a semantic version
  number as well as (for non-release builds) the git commit hash.

src/cmd/ksh93/sh/init.c: e_version[]:
- Add a 'v' to the ${.sh.version} feature string if ksh was
  compiled with vmalloc enabled. This allows scripts, such as
  regression tests, to detect this.

src/cmd/ksh93/data/builtins.c: sh_optksh[]:
- Add a copyright line crediting the contributors to ksh 93u+m.

Resolves: https://github.com/ksh93/ksh/issues/95
2021-01-05 04:52:42 +00:00
Harald van Dijk
41ef7f76cf Invocation: fix infinite loop on 'ksh +s'
When starting ksh +s, it gets stuck in an infinite loop continually
trying to parse its own binary as a shell script and rejecting it:

$ arch/linux.i386-64/bin/ksh +s
arch/linux.i386-64/bin/ksh: arch/linux.i386-64/bin/ksh: cannot execute [Exec format error]
arch/linux.i386-64/bin/ksh: arch/linux.i386-64/bin/ksh: cannot execute [Exec format error]
arch/linux.i386-64/bin/ksh: arch/linux.i386-64/bin/ksh: cannot execute [Exec format error]
arch/linux.i386-64/bin/ksh: arch/linux.i386-64/bin/ksh: cannot execute [Exec format error]
arch/linux.i386-64/bin/ksh: arch/linux.i386-64/bin/ksh: cannot execute [Exec format error]
[...]
$ echo 'echo "this is stdin"' | arch/linux.i386-64/bin/ksh +s
arch/linux.i386-64/bin/ksh: arch/linux.i386-64/bin/ksh: cannot execute [Exec format error]
(no loop, but still ksh trying to parse itself)

src/cmd/ksh93/sh/init.c: sh_init():
- When forcing on the '-s' option upon finding no command
  arguments, also update sh.offoptions, a.k.a. shp->offoptions.
  This avoids the inconsistent state causing this problem.

  In main.c, there is:

  if(sh_isoption(SH_SFLAG))
      fdin = 0;
  else
      (code to open $0 as a file)

  This was entering the else block because sh_isoption(SH_SFLAG)
  was returning 0, and $0 is set to the ksh binary as it is
  supposed to when no other script is provided. When I looked for
  why sh_isoption was returning 0, I found main.c's

  for(i=0; i<elementsof(shp->offoptions.v); i++)
      shp->options.v[i] &= ~shp->offoptions.v[i];

  Before this loop, shp->offoptions tracks which options were
  explicitly disabled by the user on the command line. The effect
  of this loop is to make "explicitly disabled" take precedence
  over "implicitly enabled". My patch removes the registration of
  the +s option.

Fixes: https://github.com/ksh93/ksh/issues/150
Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-01-03 23:54:36 +00:00
Martijn Dekker
737438a30f tests/path.sh: fix spurious 'whence -a' test failures
A recent change in Github's Ubuntu test runners exposed a problem
in the way the all_paths test function replicates 'whence -a'
functionality, causing spurious regression test failures.
The problem occurs when e.g. /bin is a symlink to /usr/bin, but
both are in $PATH. The all_paths function treats them as separate
but 'whence -a' detects a duplicate and will not output /usr/bin/*.
I have not succeeded in making all_paths match 'whence -a' exactly.

src/cmd/ksh93/tests/path.sh: function all_paths:
- Using the -ef test expression operator, remove entries from $PATH
  that point to the same directory as another entry in $PATH (e.g.
  when /bin is a symlink to /usr/bin but both are in $PATH).
  Avoiding such dupes works around the problem.
2020-12-20 20:10:02 +00:00
Chase
cda1976e4c Properly clean and ignore flat make binaries and libs
bin/package, src/cmd/INIT/package.sh:
- When running bin/package flat make clean, also clean the flat
  hierarchy binaries.

.gitignore:
- Ignore flat hierarchy binaries.
2020-12-20 01:31:26 +00:00
Martijn Dekker
77111310aa name.c: rm duplicate #define Empty
src/cmd/ksh93/sh/name.c:
- The Empty macro (a constant pointer to the empty string) is already
  defined in include/defs.h, so does not need to be repeated here.
2020-12-04 03:49:23 +00:00