Now that the Make Abstract Machine files are maintained manually
and not generated automatically, unused variables are an annoying
distraction -- and there are many.
But the language/format is very simple and very parseable using
shell, awk, etc. -- so this was easy to automate. All variables are
declared with 'setv' and they are used if an expansion of the form
${varname} exists (the braces are mandatory in Mamfiles).
bin/Mamfile_rm_unused_vars:
- Added for reference and future use.
src/*/*/Mamfile:
- Remove all unused 'setv' variable declarations.
src/cmd/INIT/iffe.sh:
- Fix "standard system directories" for the cmd test, which were
hardcoded as bin, /etc, /usr/bin, /usr/etc, /usr/ucb. That's both
unportable and antiquated. Replace this with the path output by
'getconf PATH'.
- Add fixes from modernish for 'getconf PATH' output to compensate
for bugs/shortcomigns in NixOS and AIX. Source:
https://github.com/modernish/modernish/blob/9e4bf5eb/lib/modernish/aux/defpath.sh
Ref.: https://github.com/NixOS/nixpkgs/issues/65512
src/lib/libast/comp/conf.tab: PATH:
- Add the NixOS and AIX default path fixes here too; this fixes
'command -p' and the builtin 'getconf PATH' on these systems.
bin/package, src/cmd/INIT/package.sh:
- Re-support being launched with just the command name 'package' in
the command line (if the 'package' command is in $PATH). At least
one other script in the build system does this. (re: 6cc2f6a0)
- Go back three levels (../../..) if we were invoked from
arch/*/bin/package, otherwise we won't find src/cmd/ksh93/SHOPT.sh.
iffe feature test that add a -D_LARGEFILE64_SOURCE compiler flag to
detect the presence of 64-bit types like off64_t are very
incorrect; they always find the type even if the rest of the source
is not compiled with that flag, causing an inconsistent compilation
environment. This was the cause of mysterious failures to compile
some feature tests on Linux i386 -- it tried to use an off64_t type
that was wrongly detected.
A flag like -D_LARGEFILE64_SOURCE needs to be added to the compiler
flags consistently so it is used for compiling all files and tests.
src/lib/libast/features/dirent,
src/lib/libast/features/fs,
src/lib/libast/features/lib,
src/lib/libast/features/mmap,
src/cmd/ksh93/features/rlimits:
- Remove the -D_LARGEFILE64_SOURCE flag from all the tests that
used it.
- Fix some preprocessor directives for compiling without
_LARGEFILE64_SOURCE. We cannot rely on the result of the _lib_*64
tests because those functions are still found in glibc even if
_LARGEFILE64_SOURCE is not defined; we have to check for the
existence of the type definitions before using them.
src/cmd/INIT/cc.linux.i386,
src/cmd/INIT/cc.linux.i386-icc:
- Add/update compiler wrappers to hardcode -D_LARGEFILE64_SOURCE
in the flags for the default compiler. If it is overriden with
$CC, then it needs to be added manually if desired.
This takes another small step towards disentangling the build
system from the old AT&T environment. The USAGE_LICENSE macros with
author and copyright information, which was formerly generated
dynamically for each file from a database, are eliminated and the
copyright/author information is instead inserted into the AST
getopt usage strings directly.
Repetitive license/copyright information is also removed from the
getopt strings in the builtin commands (src/lib/libcmd/*.c and
src/cmd/ksh93/data/builtins.c). There's no need to include 55
identical license/copyright strings in the ksh binary; one (in the
main ksh getopt string, shown by ksh --man) ought to be enough!
This makes the ksh binary about 10k smaller.
It does mean that something like 'enum --author', 'typeset
--license' or 'shift --copyright' will now not show those notices
for those builtins, but I doubt anyone will care.
Commit 308696ec caused the build to fail on macOS Catalina.
src/cmd/INIT/iffe.sh:
- Fix a blatantly unportable practice of passing multiple
"|"-separated 'case' patterns through a variable. This was a way
of grepping for some headers including stdio.h, but it only works
this way on ksh93 and possibly the original Bourne shell, and not
on *any* other shell (not even pdksh or mksh) -- and the fact
that it works on ksh93 is arguably a bug. Fix by eliminating the
"noext" variable (which is init'ed once and never changes) and
using the pattern in the relevant 'case' statement directly.
src/cmd/builtin/features/pty:
- No matter what I try, including <stdio.h> causes the build to
fail on Gentoo Linux (i386) with mysterious "invalid identifier:
off64_t" errors -- this is probably some AST preprocessor hackery
gone awry, but I've no idea where to even begin with that. This
works around the problem by using AST sfio instead, which is
built and functional by the time this feature test is run.
- Remove explicit extern declaration for ptsname(2) that was never
used because it depended on an npt_ptsname feature test that
doesn't exist (or no longer exists).
- Add missing <fcntl.h>, <stdlib.h>, and <unistd.h> for open(2),
ptsname(2) and close(2), respectively.
src/lib/libast/features/float,
src/lib/libast/features/sfio,
src/lib/libast/features/stdio:
- Re-include <stdio.h>.
Fixes: https://github.com/ksh93/ksh/issues/164 (I hope)
This makes ksh build at least on AIX 7.1 on RISC (PowerPC).
There are 4 regression test failures:
leaks.sh[159]: memory leak on PATH reset before PATH search
(leaked approx 220 KiB after 16384 iterations)
pty.sh[351]: POSIX sh 104(C): line 364: expected
"^done\r?\n$", got EOF
signal.sh[280]: subshell ignoring signal does not send
signal to parent (expected 'SIGUSR1', got 'done')
signal.sh[282]: parent does not wait for child to complete
before handling signal
src/cmd/INIT/iffe.sh:
- Unset LIBPATH on AIX. The features/pty output{ ... }end will fail
to link to libiconv otherwise, causing a build failure. See:
https://www.ibm.com/support/pages/member-libiconvso2-not-found-archive
src/cmd/builtin/pty.c:
- CMIN is not defined on AIX, so set it to 1 if it's not defined.
src/cmd/ksh93/README:
- Update list of tested OSs.
The only proper documentation of the MAM language is in Glenn
Fowler's paper, which is unfortunately copyrighted so we can't
include it. But we can at least provide a link to it.
src/**/Mamfile:
- Add header comment.
src/cmd/INIT/mamake.c:
- Re-enable clang warnings on unused values (there aren't any).
When building old code for debugging purposes (e.g. when doing 'git
bisect' runs), it's best to use the current build system even with
the old code, because the old build system was very broken. E.g.:
git checkout (some old commit)
git checkout master bin src/cmd/INIT # use new build system
bin/package make
However, that became impossible in 6cc2f6a0 because the new
SHOPT.sh script was unconditionally sourced. The error caused the
script to exit because '.' is a special builtin.
bin/package, src/cmd/INIT/package.sh:
- If src/cmd/ksh93/SHOPT.sh doesn't exist, issue a warning instasd
of trying to source it.
The build system is adapted to make SHOPT_* compile-time options
editable without nmake. We can now easily change ksh's compile-time
options by editing src/cmd/ksh93/SHOPT.sh. The bin/package script
is adapted to turn these into compile flags. This resolves the most
important drawback of not using nmake.
Also, mamake now has support for indented Mam (Make Abstract
Machine) code. Only one type of block (make...done) is supported in
Mamfiles, so they are easy to indent automatically. A script to
(re)do this is included.
Since nmake is not going to be restored (it has too many problems
that no one is interested in fixing), this at least makes mamake
significantly easier to work with.
The Makefiles are deleted. They may still be handy for reference to
understand the Mamfiles, but they haven't actually matched the
Mamfiles for a while -- and you can still look in the git history.
Deleting them requires some adaptations to bin/package and mamake.c
because, even though they do not use those files, they still looked
for them to decide whether to build code in a directory.
Finally, this commit incorporates some #pragmas for clang to
suppress annoying warnings about the coding style used in this
historic code base. (gcc does not complain so much.)
src/cmd/ksh93/SHOPT.sh:
- Added.
bin/package, src/cmd/INIT/package.sh:
- cd into our own directory in case we were run from another dir.
- $makefiles: only look for Mamfiles.
- Add ksh compile-options via KSH_SHOPTFLAGS. Include SHOPT.sh.
- make_recurse(): Do not write a missing Makefile.
- finalize environment: Look for Mamfiles instead of Makefiles.
src/cmd/INIT/mamake.c:
- Tell clang to suppress annoying warnings about coding style.
- Update version string and self-documentation.
- input(): Add support for indented Mam code by skipping initial
whitespace on each input line.
- files[]: Instead of looking for various of Makefiles to decide
where to build, only look for Mamfiles.
src/Makefile, src/cmd/INIT/Makefile, src/cmd/Makefile,
src/cmd/builtin/Makefile, src/cmd/ksh93/Makefile, src/lib/Makefile,
src/lib/libast/Makefile, src/lib/libcmd/Makefile,
src/lib/libdll/Makefile, src/lib/libsum/Makefile:
- Removed.
src/Mamfile, src/cmd/INIT/Mamfile, src/cmd/Mamfile,
src/cmd/builtin/Mamfile, src/cmd/ksh93/Mamfile, src/lib/Mamfile,
src/lib/libast/Mamfile, src/lib/libcmd/Mamfile,
src/lib/libdll/Mamfile, src/lib/libsum/Mamfile:
- Indent the code with tabs.
- In ksh93/Mamfile, add ${KSH_SHOPT_FLAGS} to every $CC command.
- In ksh93/Mamfile, add "prev SHOPT.sh" for every *.o file
so they are rebuilt whenever SHOPT.sh changes.
bin/Mamfile_indent:
- Added, in case someone wants to re-indent a Mamfile.
src/cmd/INIT/proto.c, src/cmd/INIT/ratz.c, src/cmd/INIT/release.c,
src/lib/libast/features/common, src/lib/libast/include/ast.h:
- Tell clang to suppress annoying warnings about coding style that
it disapproves of (mainly concerning the use of parentheses).
src/cmd/INIT/cc.darwin, src/cmd/INIT/cc.freebsd,
src/cmd/INIT/cc.openbsd:
- Remove now-redundant clang warning suppression flags.
Resolves: https://github.com/ksh93/ksh/issues/60
src/cmd/ksh93/bltins/typeset.c:
- The new_argv[] array was one item too short (should be argc+2).
- Use AST stakalloc(3) to allocate it instead of a dynamic array;
this restores compatibility with ISO C90.
src/lib/libast/features/standards, src/cmd/INIT/cc.unixware.i386:
- Add support for UnixWare.
- Do not define any standards macros on this system, as on FreeBSD
and DragonFly BSD.
AIX on ibm.risc comes with a broken version of ksh88 as /bin/sh
where the following causes breakage in the parser (spurious syntax
errors):
(set -o posix) 2>/dev/null && set -o posix
However, prefixing it with 'command' (while keeping the subshell)
circumvents the problem. So, why not.
(command set -o posix) 2>/dev/null && set -o posix
Solaris Studio 12.5 cc seems to produce incorrect code at -O2
(a.k.a. -xO2) optimisation level; integer variables initialise at
random values, and the behaviour of the shell is so incorrect it
can't even run the test scripts. It does not support -Os so that
is skipped for that compiler. At -O it works fine.
src/cmd/INIT/make.probe:
- By default, only try -Os and -O optimisation flags.
If iffe re-ran a test because the source test script changed, but
the result file is unchanged, it didn't update the timestamp of the
result, so the source script remained newer. The build system then
kept pointlessly re-running the test on each rebuild. If a central
test script such as src/lib/libast/features/standards was changed,
this had cascading effects, e.g., causing libast to be rebuilt over
and over as I recompiled small changes elsewhere. Until now, my
workaround was to delete the entire 'arch' directory and start
over. Hopefully that will now no longer be needed.
src/cmd/INIT/iffe.sh:
- If a test output file has not changed after rerunning a test that
has changed, still update the output file's timestamp using
touch(1) to signal that the test has already been run.
Instead, we now link to the libm system math library where needed
by adding -lm to the relevant compile commands in the Mamfiles.
This is not needed on every system but never does any harm.
(This adds more custom edits to the Mamfiles, which were originally
generated from the nmake Makefiles. This takes us further from
restoring nmake, but that already wasn't going to happen anyway,
due to its many problems... the path forward will be to translate
the Mamfiles to some other, current make system such as GNU make.)
bin/package, src/cmd/INIT/package.sh:
- Remove LDFLAGS=-lm hack for DragonFly BSD, NetBSD and Solaris.
src/cmd/builtin/Mamfile,
src/cmd/ksh93/Mamfile,
src/lib/libdll/Mamfile:
- Add -lm where linking failed on any of the three mentioned OSs.
src/lib/libdll/features/dll:
- In the output test program, add missing #include <math.h>, fixing
unknown identifier errors on NetBSD (ldexp, ldexpl).
src/cmd/builtin/features/pty:
- Add missing #include <stdio.h> to make printf work on all systems
(this is just a feature test, no need to bother with sfio here).
src/lib/libast/features/stdio:
- Undef __FILE_T to avoid interference from system headers on QNX.
(There are still other problems preventing a build on QNX 6.5.0.
The shipped version of gcc seems to be broken.)
This now makes ksh build on DragonFly BSD.
bin/package, src/cmd/INIT/package.sh:
- DragonFly also needs the -lm hack for LDFLAGS.
src/cmd/ksh93/sh/main.c, src/cmd/ksh93/tests/basic.sh:
- fixargs() doesn't work on DragonFly either
(re: 9b7c392a, 159fb9ee, cefe087d).
The following are backported from:
https://github.com/att/ast/issues/26#issuecomment-313927854https://github.com/att/ast/pull/19
src/lib/libast/comp/setlocale.c:
- Add missing #include <errno.h> since errno is used.
src/lib/libast/features/standards:
- Do not set any standards macros (_POSIX_SOURCE etc) on FreeBSD or
DragonflyBSD; they disable too much functionality on those.
src/lib/libast/features/wchar:
- Set _STDFILE_DECLARED on DragonFly, too.
src/lib/libast/include/sfio.h, src/lib/libast/include/sfio_t.h,
src/lib/libast/sfio/_sfopen.c, src/lib/libast/sfio/sfclrlock.c,
src/lib/libast/sfio/sfhdr.h, src/lib/libast/sfio/sfnew.c,
src/lib/libast/sfio/sfset.c:
- Rename SF_* macros to SFIO_* to avoid a conflict with system
headers.
src/lib/libast/string/strexpr.c:
- Rename error() to err() to avoid a conflict.
bin/package, src/cmd/INIT/package.sh:
- It can depend on the compiler flags passed whether the compiler
produces code for a 64-bit architecture, so pass $CCFLAGS to
the compiler when testing whether it creates 64-bit object code.
README.md:
- Copy-edit of build instructions.
bin/package, src/cmd/INIT/package.sh:
- CCFLAGS overwrites the autodetected optimisation flags (e.g. -Os)
if set. Unfortunately, that also happened when we added something
to CCFLAGS for a release build or to add an extra flag needed by
Solaris. The fix is to use a new flags variable (KSH_RELFLAGS)
instead. This needs to be done in a different place as it needs
to be added to the mamake command as an assignment argument.
- Remove the Solaris CCFLAGS hack; see features/common below.
src/*/*/Mamfile:
- Add ${KSH_RELFLAGS} to all the compiler commands.
src/lib/libast/features/common:
- Enable POSIX standard on Solaris (i.e.: if __sun is defined) by
defining _XPG6 directly in the feature test that generates
ast_std.h, which is indirectly included by everything. This
removes the need to pass -D_XPG6 via CCFLAGS. (Doing so
automatically with gcc was not otherwise possible.)
src/cmd/INIT/cc.sol11.*:
- No longer pass -D_XPG6, as per above.
bin/package, src/cmd/INIT/package.sh:
- The code for detecting a 64-bit object file was seriously broken:
the temporary file name could contain '64' because it included $$,
the current PID, and 64-bit was detected if the output of 'file'
(which includes the complete file name) contained '64'. Fix by
removing the file name from 'file' output before testing.
- Also refactor that code a bit and remove the nonsensical test if
/bin/sh is a 64-bit binary, which is neither here nor there. It's
what the compiler produces that we need to care about.
src/cmd/INIT/make.probe:
- probe_optimize: Also try -O2 and -O, for compilers (such as
Solaris Studio cc) that do not support -Os.
- Use more robust code to loop through possible optimiser flags.
The versions from the Solaris patch require $CC_EXPLICIT to be set,
which is specific to the internal Solaris build environment.
src/cmd/INIT/cc.sol11.*:
- Cope without $CC_EXPLICIT set in environment; fall back to $CC
and if that is not set either, detect whether to use cc or gcc.
- Set appropriate flags for cc (Solaris Studio) or gcc, including
the necessary -D_XPG6 flag, without which ksh crashes on Solaris.
bin/package, src/cmd/INIT/package.sh:
- Update hack to add the -D_XPG6 flag so it applies to gcc only
(note: the src/cmd/INIT/cc.* scripts are never used for gcc).
This reverts commit 600cb182.
$cc may be a system compiler binary, it is not necessarily a
src/cmd/INIT/cc.* wrapper script; so prefixing 'sh' is wrong.
As of this commit, ksh 93u+m has a standard semantic version number
<https://semver.org/>, beginning with 1.0.0-alpha. This is added to
the version string in a way that should be compatible with scripts
parsing ${.sh.version} or $(ksh --version). This addition does not
replace the release date and does not affect $((.sh.version)).
For non-release builds, the version string will be:
FORK/VERSION+HASH YYYY-MM-DD
e.g.: 93u+m/1.0.0-alpha+41ef7f76 2021-01-03
For release builds, it will be:
FORK/VERSION YYYY-MM-DD
e.g.: 93u+m/1.0.0 2021-01-03
It is now automatically decided by bin/package whether to build a
release or development build. When building from a directory that
is not a git repository, or if the current git branch name starts
with a number (e.g. '1.0'), the release build is enabled; otherwise
a development build is the default. This is arranged by adding -D
flags to $CCFLAGS as described below. These flags are prepended to
$CCFLAGS, so they can be overridden by adding your own -D or -U
flags via the environment.
In addition, AST vmalloc is disabled for release builds as of this
commit, forcing the use of the OS's standard malloc(3). In 2021,
this is generally more reliable, faster, and more economical with
memory than AST vmalloc. Several memory leaks and crashing bugs are
avoided, e.g.: <https://github.com/ksh93/ksh/issues/95>.
For development builds, vmalloc stays enabled (along with its known
bugs) because this allows the use of the vmstat builtin, making it
much more efficient to test for memory leaks. For more info, see
the regression test script: src/cmd/ksh93/tests/leaks.sh
bin/package, src/cmd/INIT/package.sh:
- Add flags for build type. In $CCFLAGS, define _AST_ksh_release if
we're not on any git branch or on a git branch whose name starts
with a number. Otherwise, define _AST_git_commit as the first 8
characters of the current git commit hash.
src/lib/libast/features/vmalloc:
- If _AST_ksh_release is defined, disable vmalloc and force use of
the operating system's malloc. Discussion:
https://github.com/ksh93/ksh/issues/95
src/cmd/ksh93/include/version.h:
- Define new format for version string, adding a semantic version
number as well as (for non-release builds) the git commit hash.
src/cmd/ksh93/sh/init.c: e_version[]:
- Add a 'v' to the ${.sh.version} feature string if ksh was
compiled with vmalloc enabled. This allows scripts, such as
regression tests, to detect this.
src/cmd/ksh93/data/builtins.c: sh_optksh[]:
- Add a copyright line crediting the contributors to ksh 93u+m.
Resolves: https://github.com/ksh93/ksh/issues/95
The package script searches for a good shell to run the build
scripts, preferring a ksh. But it also finds any recently compiled
development version of ksh in arch/*/bin that may be broken, or
have debug code, etc. -- and uses that in preference to anything
else. This is quite capable of breaking the build process.
The way to get around it is to do something like
bin/package make SHELL=/bin/ksh
which is annoying to have to keep doing.
bin/package,
src/cmd/INIT/package.sh:
- When finding a good shell, use the saved user path ($path), not
the current $PATH which includes arch/$HOSTTYPE/bin. Prefix this
temporary path with `getconf PATH`, the system's default path,
so that known-good system shells are found first.
All changed files:
- Put the shell in POSIX mode if it has an '-o posix' option.
- Remove nonsense disabling 'set -x' on bash. It's not broken.
bin/package, src/cmd/INIT/package.sh:
- Add check blocking native zsh mode (e.g., "$path" conflicts).
Using a 'sh -> zsh' symlink works, so recommend that.
- Remove old ksh93 version check for a supposed conflict with
libcmd. It was broken; it would revert to /bin/sh, but on illumos
distributions, /bin/sh is a ksh93 of a version that is supposedly
affected. It builds fine anyway.
- Rewrite checksh() to incorporate the shell compatibility checks
that were previously in two different places in 'package'.
bin/ignore, src/cmd/INIT/ignore.sh,
bin/silent, src/cmd/INIT/silent.sh:
- Change bad check for a full POSIX 'export' command (no, $RANDOM
has nothing to do with that) with a proper feature test.
The shell detection test assumed that any shell that has $RANDOM
and isn't bash must be ksh and have the 'print' built-in command.
This broke iffe on NetBSD 9.0 sh, which has $RANDOM but not 'print'.
src/cmd/INIT/iffe.sh:
- Rewrite shell detection test to test for features actually used
by iffe. "$shell" can now have the values 'bsh' for an ancient
pre-POSIX Bourne shell, 'bash' for bash, 'ksh' for a shell with
'let', 'typeset -u' and 'print' (ksh88, ksh93, mksh, pdksh, zsh),
and 'posix' for another POSIX or POSIX-ish shell with modern
command substitutions and parameter substitutions. (The 'posix'
value currently is not actively checked for anywhere, but avoids
unnecessary use of bsh fallbacks.)
The INIT scripts are supposed to be Bourne shell scripts, not
POSIX, so we can't use $( ... ) command substitutions.
bin/package,
src/cmd/INIT/package.sh:
- Change a POSIX command substitution to old-style backticks.
Some systems disallow executing files in /tmp and there is nothing
regular users can do about it. The build would fail with a
misleading error message about cc being a cross-compiler.
This commit makes the build system consistently use $TMPDIR with
/tmp as a fallback if that variable is not defined. This allows the
user to use another temporary directory with execute permission.
The error message in bin/package is also extended to signal the
possibility of a noexec temp dir.
It was working on Solaris 11.3, but there were still problems
building on Solaris 11.4 with GCC (as on the evaluation VM
downloaded directly from Oracle):
1. ksh immediately segfaulted. Experimenting with the compiler
flags Oracle uses revealed that we need to define _XPG6 for ksh
not to segfault. Why is a mystery.
2. The default path logic used by 'command -p' and the 'getconf
PATH' builtin command was still broken: the result did not
include any of the /usr/xpg?/bin directories where the standard
POSIX utilities actually live. Testing shows that the result of
the C language probe 'confstr(_CS_PATH,name,length)' is broken
on Solaris (it only yields the paths to the historic
non-standard utilities, defeating the purpose) unless _XPG7 is
defined; but the latter makes ksh segfault again. So another
solution is needed.
src/cmd/INIT/package.sh, bin/package:
- Add another hack to add the -D_XPG6 flag to CCFLAGS if we're
running SunOS aka Solaris. (I've tried to add a 'cc.sol11' script
to src/cmd/INIT/ instead, but for some reason that I just don't
have time to figure out, the INIT system ignores that on Solaris
with gcc, so this is the only way I could come up with. Any
patches for less hacky alternatives would be welcome.)
src/lib/libast/comp/conf.sh:
- Sanitise the code for finding the best 'getconf' utility.
src/lib/libast/comp/conf.tab: PATH:
- Since the C-languge getconf(_CS_PATH,...) is broken on Solaris
11.4, replace the C language probe with a shell script probe that
uses the external 'getconf' utility.
- To avoid ksh overriding the result of this probe with the result
of its own getconf(_CS_PATH,...) call, which would make Solaris
use the wrong value again, specify this as an AST configuration
entry instead of a POSIX entry. This should be good enough for
all systems; the OS 'getconf' utility should be reliable and the
default path value is constant for each OS, so can be hardcoded.
src/cmd/ksh93/tests/builtins.sh:
- Add another 'sleep .1' to the 'sleep -s 31' test as it was still
intermittently failing on Solaris and possibly other systems.
Solaris, Illumos distributions, and NetBSD need LDFLAGS set to link
explicitly to libm, otherwise, due to as-yet unknown reasons, the
src/lib/libdll/features/dll fails to write a valid header file and
compilation fails due to unknown identifiers such as Dllscan_t.
This commit adds the flag on those systems.
NixOS is a Linux distro that uses very different paths from the
usual Unix conventions (though it's POSIX compliant), and the
regression tests still needed a lot of tweaks to be compatible.
src/cmd/INIT/package.sh, bin/package:
- On SunOS (Solaris and illumos distros) and NetBSD, add '-lm' to
LDFLAGS before compiling.
src/cmd/INIT/mamprobe.sh, bin/mamprobe,
src/cmd/INIT/execrate.sh, bin/execrate:
- Instead of only in /bin, /usr/bin, /sbin and /usr/sbin, search
utilities in the path given by the OS 'getconf PATH', and use the
user's original $PATH as a fallback.
src/cmd/ksh93/tests/*.sh:
- Miscellaneous portability fixes, mainly elimination of unportable
hardcoded paths to commands.
- basic.sh: Remove test for 'time' keyword millisecond precision.
It was racy and could fail depending on system and system load.
The following is quoted from Marcin Cieślak [*]:
When running under FreeBSD /bin/sh (and not ksh) we get spurious
file named '=' created in the root. This is because the "checksh"
function runs /bin/sh -c '(( .sh.version >= 20111111 ))' which
produces a "=" file with /bin/sh as a side effect.
Fixes https://github.com/ksh93/ksh/issues/13
bin/package,
src/cmd/INIT/package.sh:
- Fix the creation of a spurious '=' file by making sure the shell
has support for (( ... )) expressions.
.gitignore:
- Remove the '=' file entry since it no longer has a purpose.
[*]: https://bsd.network/@saper/103196289917156347
This bugfix is from Marcin Cieślak's fork of the INIT build
system. Before this bugfix, running 'bin/package host cpu'
on FreeBSD would always report one CPU core, even if the CPU
is multi-core:
$ ./bin/package host cpu
1
bin/package,
src/cmd/INIT/package.sh:
- Correctly report the number of CPUs on FreeBSD by using
'sysctl -n hw.ncpu'.
src/cmd/INIT/mamake.c:
- Fix a rare build error by applying Oracle's patch to increase
mamake's buffer size[*]. Description from the original patch:
The build of KornShell might spuriously fail
with the following error.
...
/usr/bin/ksh: line 40: syntax error at line 44: `else unmatched
mamake [lib/libast]: *** exit code 3 making ast.req
mamake: *** exit code 139 making lib/libast
The patch increases the buffer size of mamake to avoid
spurious build failures.
I can't reproduce build error, but this patch should be merged
anyway because OpenSUSE also increases mamake's buffer size
in a patch titled 'workaround-stupid-build-system.diff'[**].
This indicates that the build failure is a heisenbug that can
occur on at least Linux and Solaris.
[*]: 7cad9dae78
[**]: https://build.opensuse.org/package/view_file/shells/ksh/workaround-stupid-build-system.diff?expand=1
When ksh is compiled with SHOPT_SPAWN (the default), which uses
posix_spawn(3) or vfork(2) (via sh_ntfork()) to launch external
commands, at least two race conditions occur when launching
external commands while job control is active. See:
https://bugs.launchpad.net/ubuntu/+source/ksh/+bug/1887863/comments/3https://www.mail-archive.com/ast-developers@research.att.com/msg00717.html
The basic issue is that this performance optimisation is
incompatible with job control, because it uses a spawning mechanism
that doesn't copy the parent process' memory pages into the child
process, therefore no state that involves memory can be set before
exec-ing the external program. This makes it impossible to
correctly set the terminal's process group ID in the child process,
something that is essential for job control to work.
src/cmd/ksh93/sh/xec.c:
- Use sh_fork() instead of sh_ntfork() if job control is active.
This uses fork(2), which is 30%-ish slower on most sytems, but
allows for correctly setting the terminal process group.
src/cmd/ksh93/tests/basic.sh:
- Add regression test for the race condition reported in #79.
src/cmd/INIT/cc.darwin:
- Remove hardcoded flag to disable SHOPT_SPAWN on the Mac.
It should be safe to use now.
Fixes https://github.com/ksh93/ksh/issues/79
My tests with running shbench[*] on ksh binaries compiled by clang
and gcc yielded no performance difference between compiling with
'-O2' and '-Os'. So we might as well reduce ksh's size and memory
footprint by default.
[*] http://fossil.0branch.com/csb/https://github.com/ksh-community/shbench
src/cmd/INIT/make.probe:
- Change default gcc optimisation level from -O2 to -Os.
- Change default non-gcc optimisation level from -O to -Os.
The coshell(1) command, which is required for libcoshell to be
useful, is not known to be shipped by any distribution. It was
removed by the ksh-community fork and hence also by 93u+m (in
2940b3f5). The coshell facility as a whole is obsolete and
insecure. For a long time now, the statically linked libcoshell
library has been 40+ kilobytes of dead weight in the ksh binary.
Prior discussion (ksh2020): https://github.com/att/ast/issues/619
src/lib/libcoshell/*:
- Removed.
src/cmd/ksh93/*:
- Remove the SHOPT_COSHELL compiler option (which was enabled) and
a lot of code that was conditional upon #ifdef SHOPT_COSHELL.
- init.c: e_version[]: Removing SHOPT_COSHELL changed the "J"
feature identifier in ${.sh.version} to a lowercase "j", which
was conditional upon SHOPT_BGX (background job extensions).
But src/cmd/ksh93/RELEASE documents (at 08-12-04, on line 1188):
| +SHOPT_BGX enables background job extensions. Noted by "J" in
| the version string when enabled. [...]
That is the only available documentation. So change that "j" back
to a "J", leaving the version string unchanged after this commit.
- jobs.c: job_walk(): We need to keep one 'job_waitsafe(SIGCHLD);'
call that was conditional upon SHOPT_COSHELL; removing it caused
a regression test failure in tests/sigchld.sh, 'SIGCHLD blocked
for script at end of pipeline' (which means that until now, a ksh
compiled without libcoshell had broken SIGCHLD handling.)
bin/package, src/cmd/INIT/package.sh:
- Don't export COSHELL variable.
Support for the long-dead 3DFS userland versioning file system was
already removed from ksh93 (although I'd overlooked some minor
things), but libast still supported it. This removes that too.
src/lib/libast/include/fs3d.h,
src/lib/libast/man/fs3d.3,
src/lib/libast/misc/fs3d.c:
- Removed.
bin/package,
src/cmd/INIT/package.sh:
- Remove attempted use of removed vpath builtin.
src/cmd/ksh93/*:
- Remove minor 3dfs vestiges.
src/lib/lib{ast,cmd,coshell}/*:
- Remove code supporting 3dfs.
In <https://github.com/att/ast/commit/d2771913>, GCC version 10 was
specifically special-cased for skipping the -nostartfiles flag
along with versions 7, 8, and 9. It seems more future-proof to
specifically include it for versions up to 6 and remove it for any
version 7 and up.
src/cmd/INIT/make.probe:
- Remove the -nostartfiles for all version of gcc > 7.
Not cleaning up the FIFO broke 'grep -r' in the arch directory,
making it hang forever. So we need a better way of cleaning it up.
bin/package,
src/cmd/INIT/package.sh:
- Unlink the FIFO early after sleeping a second in the background.
This works because the named directory entry is only needed to
establish the pipe, not to keep it going.
Somewhat notable changes in this commit:
- The 'set +r' bugfix (re: 74b41621) is now documented in the
changelog.
- Missing options have been added to the synopsis section of the
ksh man page.
- The minor formatting fix from https://github.com/ksh-community/ksh/pull/5
has been applied to the ksh man page.
- A few fixes from https://github.com/att/ast/commit/5e747cfb
have been applied to the ksh man page.
- The man page fixes from https://github.com/att/ast/pull/353
have been applied, being:
- An addition to document the behavior of 'set -H'.
- A fix for the cd section appending rksh93.
- A fix for some options being indented too far.
- Removal of a duplicate section documenting '-D'.
- Reordering the options for 'set' in alphabetical order.
- A minor fix for the documentation of 'ksh -i'.
This appears to be originating from:
2755 *) if test ! -d $INSTALLROOT
2756 then INSTALLROOT=$PACKAGEROOT;
where INSTALLROOT=PACKAGEROOT and 'clean' deletes everything under
INSTALLROOT thus deleting the entire git repo. This only applies when
there's no arch/$HOSTTYPE directory due to the condition above.
bin/package,
src/cmd/INIT/package.sh:
- Delete arch/$HOSTTYPE as stated in the documentation
for the clean action instead of $INSTALLROOT.
src/cmd/INIT/ratz.c:
- Fix build warning:
src/cmd/INIT/ratz.c:4741:2: warning: case label value exceeds maximum value for
type
4741 | case 0241:
| ^~~~
The character literal in the switch expression was being treated
as a signed char while the case label 0241 is greater than 127,
resulting in this warning.
