If you passed CC=/some/compiler, the build broke on macOS because the
cc.darwin compiler wrapper wasn't used. Among other things, this
wrapper adds a -D_lib_memccpy flag, defining _lib_memccpy as 1
during the build. That was used to override a false negative result
of the lib_memccpy feature test. This commit fixes that feature
test instead, so it correctly returns positive on macOS.
Thanks to Ryan Smith (@ryandesign) for the bug report and for the
fix to the lib_memccpy test.
src/lib/libast/features/lib:
- Fix the lib_memccpy feature test. It was checking the result of
mmap(2) incorrectly, resulting in the test crashing on macOS.
Failure does not return NULL, it returns MAP_FAILED which is
usually -1.
src/cmd/INIT/cc.darwin*:
- Removed. Any other flags in these wrappers are either related to
building dynamic libraries, which is not currently supported, or
were determined to be unnecessary. See the GitHub issue for
discussion. This now makes it possible to pass `CC` to use any
compiler you like on the Mac. Notes:
- Apple's -D_ast_int8_t=int64_t is a no-op; another AST feature
test already defines _ast_int8_t a 64-bit integer type, even on
32-bit systems (on which it is defined as 'long long').
- The -search_paths_first linker flag is the default since 2010.
But even on my museum-grade Power Mac G5 with Mac OS X 10.3
(from 2004), it builds and runs just fine without.
- DCLK_TCK=100 is a no-op as even that ancient Mac system already
defines it as 100. Plus, it's not even actually used.
If a need is found for any of these, please report this in a new
issue so I can special-case it elsewhere in the code.
Resolves: https://github.com/ksh93/ksh/issues/373
List of changes:
- Fixed some -Wuninitialized warnings and removed some unused variables.
- Removed the unused extern for B_login (re: d8eba9d1).
- The libcmd builtins and the vmalloc memfatal function now handle
memory errors with 'ERROR_SYSTEM|ERROR_PANIC' for consistency with how
ksh itself handles out of memory errors.
- Added usage of UNREACHABLE() where it was missing from error handling.
- Extend many variables from short to int to prevent overflows (most
variables involve file descriptors).
- Backported a ksh2020 patch to fix unused value Coverity issues
(https://github.com/att/ast/pull/740).
- Note in src/cmd/ksh93/README that ksh compiles with Cygwin on
Windows 10 and Windows 11, albeit with many test failures.
- Add comments to detail some sections of code. Extensive list of
commits related to this change:
ca2443b5, 7e7f1372, 2db9953a, 7003aba4, 6f50ff64, b1a41311,
222515bf, a0dcdeea, 0aa9e03f, 61437b27, 352e68da, 88e8fa67,
bc8b36fa, 6e515f1d, 017d088c, 035a4cb3, 588a1ff7, 6d63b57d,
a2f13c19, 794d1c86, ab98ec65, 1026006d
- Removed a lot of dead ifdef code.
- edit/emacs.c: Hide an assignment to avoid a -Wunused warning. (See
also https://github.com/att/ast/pull/753, which removed the assignment
because ksh2020 removed the !SHOPT_MULTIBYTE code.)
- sh/nvdisc.c: The sh_newof macro cannot return a null pointer because
it will instead cause the shell to exit if memory cannot be allocated.
That makes the if statement here a no-op, so remove it.
- sh/xec.c: Fixed one unused variable warning in sh_funscope().
- sh/xec.c: Remove a fallthrough comment added in commit ed478ab7
because the TFORK code doesn't fall through (GCC also produces no
-Wimplicit-fallthrough warning here).
- data/builtins.c: The cd and pwd man pages state that these builtins
default to -P if PATH_RESOLVE is 'physical', which isn't accurate:
$ /opt/ast/bin/getconf PATH_RESOLVE
physical
$ mkdir /tmp/dir; ln -s /tmp/dir /tmp/sym
$ cd /tmp/sym
$ pwd
/tmp/sym
$ cd -P /tmp/sym
$ pwd
/tmp/dir
The behavior described by these man pages isn't specified in the ksh
man page or by POSIX, so to avoid changing these builtin's behavior
the inaccurate PATH_RESOLVE information has been removed.
- Mamfiles: Preserve multi-line errors by quoting the $x variable.
This fix was backported from 93v-.
(See also <a7e9cc82>.)
- sh/subshell.c: Remove set but not used sp->errcontext variable.
In iffe tests, some C functions are found in system libraries, but
then are not declared by the system headers on some systems because
the expected standards macros aren't defined, causing the system
headers to hide the function declarations. This may cause warnings
about invalid implicit function declarations in some tests (which
only show up if you export IFFEFLAGS=-d1), but may also cause false
negative test results. The iffe tests should be given the same
environment that their test results are going to be used in.
libast's first-run and most central feature test, 'standards',
figures out what standards macros need to be used on the current
system to get the system headers to declare the expected
functionality. All code that links to libast depends on the header
generated by this feature test. So iffe should use this result for
the tests as well, as soon as it's available (which is early in
libast's compilation cycle).
Concrete example: on Cygwin, in src/cmd/builtin/features/pty, the
'lib ptsname' test detects ptsname(3) in the system library, but
the output{...}end block that uses the _lib_ptsname feature test
result throws an 'implicit function definition' warning because
Cygwin's stdlib.h does not define this function without the
appropriate standards macros being defined first.
src/cmd/INIT/iffe.sh:
- If ${INSTALLROOT}/src/lib/libast/FEATURE/standards is available,
incorporate it directly into iffe's own block of compiler
massaging macros. Do not use #include as the necessary -I flags
are not added for every test.
- Centrally define the iffe version once, not twice.
- Update and tweak getopts self-documentation (iffe --man).
- While we're here, add macOS/Darwin's DYLD_LIBRARY_PATH to the
supported dynamic library search variables. On macOS, this is
normally disabled by System Integrity Protection, plus this
distribution uses static libraries at build time, so this is for
completeness' sake and not much else. This was ported from
@lkujaw's fork: 48ff054429
(thanks to @JohnoKing for pointing it out).
src/lib/libast/features/standards:
- Add a comment. This is to update the file's timestamp, ensuring
that everything will be recompiled after this commit.
Tried to compile on Solaris 10.1 for the first time in a while.
Turns out the obsolete Bourne /bin/sh does not support 'test -e'.
bin/package, src/cmd/INIT/package.sh:
- Use 'test -f' instead.
Commit 92f7ca54 broke compilation with tcc on Linux. The following
error would occur while compiling ksh with tcc:
In file included from /home/johno/GitRepos/KornShell/ksh/src/cmd/ksh93/data/strdata.c:105:
./FEATURE/math:91: error: too many basic types
mamake [cmd/ksh93]: *** exit code 1 making strdata.o
The build failure is fixed by backporting the relevant bugfix from
the 93v- version of iffe.
src/cmd/INIT/iffe.sh:
- Backport the 2013 iffe bugfix for the intrinsic function test to
rule out type names (dated 2013-08-11 in the 93v- changelog).
The build started failing on Solaris Studio cc when 'noreturn' was
introduced, because the wrappers pass the -xc99 flag which sets the
compiler to C99 mode. 'noreturn' is a C11 feature. The
stdnoreturn.h header was correctly included but the compiler still
threw a syntax error (long path abbreviated below):
".../stk.c", line 124: warning: _Noreturn is a keyword in ISO C11
".../stk.c", line 124: warning: old-style declaration or incorrect
type for: _Noreturn
".../stk.c", line 124: syntax error before or at: static
src/cmd/INIT/cc.sol11.*:
- Pass -std=c11 to cc instead of -xc99. At least on i386-64, this
is sufficient to fix the build.
README.md, src/cmd/ksh93/README.md:
- Remove -xc99 from the Solaris build flags example as that is
incompatible with -std=c11 (and was already redundant with the
-xc99 in the wrappers).
src/cmd/ksh93/tests/basic.sh:
- Don't run a newly backported 93v- regression test on Solaris
because it uses the 'join' command with process subsitutions;
Solaris 11.4's join(1) hangs when trying to read from /dev/fd.
This is not ksh's fault. (re: 59bacfd4)
nmake was removed long ago (2940b3f5) and so were the outdated
Makefiles (6cc2f6a0). However, the build system still looked for an
AT&T nmake in $PATH. If a user had it installed, the build would
fail as the system tried to use it.
https://groups.google.com/g/korn-shell/c/2VK8kS0_VhA/m/-Rlnv7PRAgAJ
bin/package, src/cmd/INIT/package.sh:
- Remove all the code supporting nmake.
- Make 'bin/package test' work by simply exec'ing bin/shtests.
src/cmd/INIT/Mamfile:
- Do not install *.mk nmake support files.
lib/package/*.mk, src/cmd/INIT/*.mk:
- nmake support files removed.
This adds a #pragma to disable -Wdeprecated-register* on newer
versions of clang. We could remove all use of the register keyword
instead, as modern compilers ignore it. But it's not harmful, and
for the time being I prefer not to do doing any reformatting or
changing the historic character of this code base.
The #pragmas are removed from src/lib/libast/include/ast.h, because
they're better placed in src/lib/libast/features/common which
generates ast_common.h which is included by everything.
* https://clang.llvm.org/docs/DiagnosticsReference.html#wdeprecated-register
Noteworthy changes:
- The man pages have been updated to fix a ton of instances of
runaway underlining (this was done with `sed -i 's/\\f5/\\f3/g'`
commands). This commit dramatically increased in size because
of this change.
- The documentation for spawnveg(3) has been extended with
information about its usage of posix_spawn(3) and vfork(2).
- The documentation for tmfmt(3) has been updated with the changes
previously made to the man pages for the printf and date builtins
(though the latter builtin is disabled by default).
- The shell's tracked alias tree (hash table) is now documented in
the shell(3) man page.
- Removed the commented out regression test for an ERRNO variable
as the COMPATIBILITY file states it was removed in ksh93.
The changes in this commit allow ksh to be built and run with
ASan[*], although for now it only works under vmalloc. Example
command to build ksh with ASan:
$ bin/package make CCFLAGS='-O0 -g -fsanitize=address'
[*] https://en.wikipedia.org/wiki/AddressSanitizer
src/cmd/INIT/mamake.c:
- Fix a few memory leaks in mamake. This doesn't fix all of the
memory leaks ASan complains about (there is one remaining in the
view() function), but it's enough to get ksh to build under ASan.
src/lib/libast/features/map.c,
src/lib/libast/misc/glob.c:
- Rename the ast globbing functions to _ast_glob() and
_ast_globfree(). Without this change the globbing tests fail
under ASan. See: 2c49eb6e
src/cmd/ksh93/sh/{init,io,nvtree,subshell}.c:
- Fix buffer overflows by using strncmp(3) instead of memcmp(3).
src/cmd/ksh93/sh/name.c:
- Fix another invalid usage of memcmp by using strncmp instead.
This change is also in one of Red Hat's patches:
https://git.centos.org/rpms/ksh/blob/c8s/f/SOURCES/ksh-20120801-nv_open-memcmp.patch
Resolves: https://github.com/ksh93/ksh/issues/230
Many of these changes are minor typo fixes. The other changes
(which are mostly compiler warning fixes) are:
NEWS:
- The --globcasedetect shell option works on older Linux kernels
when used with FAT32/VFAT file systems, so remove the note about
it only working with 5.2+ kernels.
src/cmd/ksh93/COMPATIBILITY:
- Update the documentation on function scoping with an addition
from ksh93v- (this does apply to ksh93u+).
src/cmd/ksh93/edit/emacs.c:
- Check for '_AST_ksh_release', not 'AST_ksh_release'.
src/cmd/INIT/mamake.c,
src/cmd/INIT/ratz.c,
src/cmd/INIT/release.c,
src/cmd/builtin/pty.c:
- Add more uses of UNREACHABLE() and noreturn, this time for the
build system and pty.
src/cmd/builtin/pty.c,
src/cmd/builtin/array.c,
src/cmd/ksh93/sh/name.c,
src/cmd/ksh93/sh/nvtype.c,
src/cmd/ksh93/sh/suid_exec.c:
- Fix six -Wunused-variable warnings (the name.c nv_arrayptr()
fixes are also in ksh93v-).
- Remove the unused 'tableval' function to fix a -Wunused-function
warning.
src/cmd/ksh93/sh/lex.c:
- Remove unused 'SHOPT_DOS' code, which isn't enabled anywhere.
https://github.com/att/ast/issues/272#issuecomment-354363112
src/cmd/ksh93/bltins/misc.c,
src/cmd/ksh93/bltins/trap.c,
src/cmd/ksh93/bltins/typeset.c:
- Add dictionary generator function declarations for former
aliases that are now builtins (re: 1fbbeaa1, ef1621c1, 3ba4900e).
- For consistency with the rest of the codebase, use '(void)'
instead of '()' for print_cpu_times.
src/cmd/ksh93/sh/init.c,
src/lib/libast/path/pathshell.c:
- Move the otherwise unused EXE macro to pathshell() and only
search for 'sh.exe' on Windows.
src/cmd/ksh93/sh/xec.c,
src/lib/libast/include/ast.h:
- Add an empty definition for inline when compiling with C89.
This allows the timeval_to_double() function to be inlined.
src/cmd/ksh93/include/shlex.h:
- Remove the unused 'PIPESYM2' macro.
src/cmd/ksh93/tests/pty.sh:
- Add '# err_exit #' to count the regression test added in
commit 113a9392.
src/lib/libast/disc/sfdcdio.c:
- Move diordwr, dioread, diowrite and dioexcept behind
'#ifdef F_DIOINFO' to fix one -Wunused-variable warning and
multiple -Wunused-function warnings (sfdcdio() only uses these
functions when F_DIOINFO is defined).
src/lib/libast/string/fmtdev.c:
- Fix two -Wimplicit-function-declaration warnings on Linux by
including sys/sysmacros.h in fmtdev().
Renamed: src/cmd/INIT/cc.linux.i386 -> src/cmd/INIT/cc.linux
This ensures that architectures like ARM also use the default Linux
wrapper. This is needed because they may need -D_LARGEFILE64_SOURCE
to compile correctly.
On ARM processors, this fixes at least this regression:
io.sh[243]: long seek not working
Resolves: https://github.com/ksh93/ksh/issues/253
On some systems (such as Ubuntu on ARM), the output of `file`
contains a build hash, such as:
SomeExecutable: ELF 32-bit LSB shared object, ARM, EABI5
version 1 (SYSV), dynamically linked, interpreter
/lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0,
BuildID[sha1]=8934dd61657aac875c190535066466849687a56b,
not stripped
This build hash can contain the string '64', which caused package
to wrongly detect a 64-bit architecture.
bin/package, src/cmd/INIT/package.sh:
- Export LC_ALL=C to ensure 'file' output in English.
- To detect a 64-bit architecture, require the string "64-bit", "64
bit" or "64bit" in 'file' output. The letters 'i' and 't' cannot
occur in a hexadecimal hash, so hopefully that is safe enough. It
is impossible to make this method completely safe, so in the long
term it should be replaced.
Progresses: https://github.com/ksh93/ksh/issues/253
This commit adds '/* FALLTHROUGH */' comments to fix many
GCC warnings when compiling with -Wimplicit-fallthrough.
Additionally, the existing fallthrough comments have been
changed for consistency.
These are minor fixes I've accumulated over time. The following
changes are somewhat notable:
- Added a missing entry for 'typeset -s' to the man page.
- Add strftime(3) to the 'see also' section. This and the date(1)
addition are meant to add onto the documentation for 'printf %T'.
- Removed the man page the entry for ksh reading $PWD/.profile on
login. That feature was removed in commit aa7713c2.
- Added date(1) to the 'see also' section of the man page.
- Note that the 'hash' command can be used instead of 'alias -t' to
workaround one of the caveats listed in the man page.
- Use an 'out of memory' error message rather than 'out of space'
when memory allocation fails.
- Replaced backticks with quotes in some places for consistency.
- Added missing documentation for the %P date format.
- Added missing documentation for the printf %Q and %p formats
(backported from ksh2020: https://github.com/att/ast/pull/1032).
- The comments that show each builtin's options have been updated.
I've had ksh crash one too many times when returning to a previous
build directory as I forgot to restore the previously-used CCFLAGS.
bin/package, src/cmd/INIT/package.sh:
- Save each of CC, CCFLAGS, CCLDFLAGS, LDFLAGS, KSH_RELFLAGS on the
first build run. On subsequent runs, compare and refuse to run if
they changed, issuing an informative error message.
- Allow override by exporting FORCE_FLAGS. Don't tell anyone :)
The package script was not well behaved with these. When you
pressed Ctrl+C, on some shells (including ksh) both the SIGINT (2)
and EXIT (0) traps are activated, showing a double 'make done'
message. The exit status also wasn't > 128 to indicate a signal.
bin/package, src/cmd/INIT/package.sh:
- Be UNIXly well-behaved. Signals should be passed on after
handling, so when one is caught, make the trap handlers print
their message and then unset both itself and EXIT/0 before
resending the signal to self.
Sometimes the shell returned to the prompt before bin/package was
finished writing all of its output. The problem was that 'tee',
which is used to write the output to both the terminal and the log
in arch/*lib/package/gen/make.out, hadn't caught up yet.
bin/package, src/cmd/INIT/package.sh:
- Run the build itself in the background and 'tee' in the
foreground. This way, the script will not terminate until 'tee'
is finished. The build's exit status is obtained with 'wait'.
POSIX warns about "unset PWD" leading to unspecified behavior from
the pwd util, which we then use.
bin/package, src/cmd/INIT/package.sh:
- Determine if the shell has $PWD with a feature test. Only unset
PWD if it does not (the old Bourne shell).
- Clean up 'case' flow.
Co-authored-by: Martijn Dekker <martijn@inlv.org>
Many of the errors fixed in this commit are word repetitions
such as 'the the' and minor spelling errors. One formatting
error in the ksh man page has also been fixed.
Now that the Make Abstract Machine files are maintained manually
and not generated automatically, unused variables are an annoying
distraction -- and there are many.
But the language/format is very simple and very parseable using
shell, awk, etc. -- so this was easy to automate. All variables are
declared with 'setv' and they are used if an expansion of the form
${varname} exists (the braces are mandatory in Mamfiles).
bin/Mamfile_rm_unused_vars:
- Added for reference and future use.
src/*/*/Mamfile:
- Remove all unused 'setv' variable declarations.
src/cmd/INIT/iffe.sh:
- Fix "standard system directories" for the cmd test, which were
hardcoded as bin, /etc, /usr/bin, /usr/etc, /usr/ucb. That's both
unportable and antiquated. Replace this with the path output by
'getconf PATH'.
- Add fixes from modernish for 'getconf PATH' output to compensate
for bugs/shortcomigns in NixOS and AIX. Source:
9e4bf5eb/lib/modernish/aux/defpath.sh
Ref.: https://github.com/NixOS/nixpkgs/issues/65512
src/lib/libast/comp/conf.tab: PATH:
- Add the NixOS and AIX default path fixes here too; this fixes
'command -p' and the builtin 'getconf PATH' on these systems.
bin/package, src/cmd/INIT/package.sh:
- Re-support being launched with just the command name 'package' in
the command line (if the 'package' command is in $PATH). At least
one other script in the build system does this. (re: 6cc2f6a0)
- Go back three levels (../../..) if we were invoked from
arch/*/bin/package, otherwise we won't find src/cmd/ksh93/SHOPT.sh.
iffe feature test that add a -D_LARGEFILE64_SOURCE compiler flag to
detect the presence of 64-bit types like off64_t are very
incorrect; they always find the type even if the rest of the source
is not compiled with that flag, causing an inconsistent compilation
environment. This was the cause of mysterious failures to compile
some feature tests on Linux i386 -- it tried to use an off64_t type
that was wrongly detected.
A flag like -D_LARGEFILE64_SOURCE needs to be added to the compiler
flags consistently so it is used for compiling all files and tests.
src/lib/libast/features/dirent,
src/lib/libast/features/fs,
src/lib/libast/features/lib,
src/lib/libast/features/mmap,
src/cmd/ksh93/features/rlimits:
- Remove the -D_LARGEFILE64_SOURCE flag from all the tests that
used it.
- Fix some preprocessor directives for compiling without
_LARGEFILE64_SOURCE. We cannot rely on the result of the _lib_*64
tests because those functions are still found in glibc even if
_LARGEFILE64_SOURCE is not defined; we have to check for the
existence of the type definitions before using them.
src/cmd/INIT/cc.linux.i386,
src/cmd/INIT/cc.linux.i386-icc:
- Add/update compiler wrappers to hardcode -D_LARGEFILE64_SOURCE
in the flags for the default compiler. If it is overriden with
$CC, then it needs to be added manually if desired.
This takes another small step towards disentangling the build
system from the old AT&T environment. The USAGE_LICENSE macros with
author and copyright information, which was formerly generated
dynamically for each file from a database, are eliminated and the
copyright/author information is instead inserted into the AST
getopt usage strings directly.
Repetitive license/copyright information is also removed from the
getopt strings in the builtin commands (src/lib/libcmd/*.c and
src/cmd/ksh93/data/builtins.c). There's no need to include 55
identical license/copyright strings in the ksh binary; one (in the
main ksh getopt string, shown by ksh --man) ought to be enough!
This makes the ksh binary about 10k smaller.
It does mean that something like 'enum --author', 'typeset
--license' or 'shift --copyright' will now not show those notices
for those builtins, but I doubt anyone will care.
Commit 308696ec caused the build to fail on macOS Catalina.
src/cmd/INIT/iffe.sh:
- Fix a blatantly unportable practice of passing multiple
"|"-separated 'case' patterns through a variable. This was a way
of grepping for some headers including stdio.h, but it only works
this way on ksh93 and possibly the original Bourne shell, and not
on *any* other shell (not even pdksh or mksh) -- and the fact
that it works on ksh93 is arguably a bug. Fix by eliminating the
"noext" variable (which is init'ed once and never changes) and
using the pattern in the relevant 'case' statement directly.
src/cmd/builtin/features/pty:
- No matter what I try, including <stdio.h> causes the build to
fail on Gentoo Linux (i386) with mysterious "invalid identifier:
off64_t" errors -- this is probably some AST preprocessor hackery
gone awry, but I've no idea where to even begin with that. This
works around the problem by using AST sfio instead, which is
built and functional by the time this feature test is run.
- Remove explicit extern declaration for ptsname(2) that was never
used because it depended on an npt_ptsname feature test that
doesn't exist (or no longer exists).
- Add missing <fcntl.h>, <stdlib.h>, and <unistd.h> for open(2),
ptsname(2) and close(2), respectively.
src/lib/libast/features/float,
src/lib/libast/features/sfio,
src/lib/libast/features/stdio:
- Re-include <stdio.h>.
Fixes: https://github.com/ksh93/ksh/issues/164 (I hope)
This makes ksh build at least on AIX 7.1 on RISC (PowerPC).
There are 4 regression test failures:
leaks.sh[159]: memory leak on PATH reset before PATH search
(leaked approx 220 KiB after 16384 iterations)
pty.sh[351]: POSIX sh 104(C): line 364: expected
"^done\r?\n$", got EOF
signal.sh[280]: subshell ignoring signal does not send
signal to parent (expected 'SIGUSR1', got 'done')
signal.sh[282]: parent does not wait for child to complete
before handling signal
src/cmd/INIT/iffe.sh:
- Unset LIBPATH on AIX. The features/pty output{ ... }end will fail
to link to libiconv otherwise, causing a build failure. See:
https://www.ibm.com/support/pages/member-libiconvso2-not-found-archive
src/cmd/builtin/pty.c:
- CMIN is not defined on AIX, so set it to 1 if it's not defined.
src/cmd/ksh93/README:
- Update list of tested OSs.
The only proper documentation of the MAM language is in Glenn
Fowler's paper, which is unfortunately copyrighted so we can't
include it. But we can at least provide a link to it.
src/**/Mamfile:
- Add header comment.
src/cmd/INIT/mamake.c:
- Re-enable clang warnings on unused values (there aren't any).
When building old code for debugging purposes (e.g. when doing 'git
bisect' runs), it's best to use the current build system even with
the old code, because the old build system was very broken. E.g.:
git checkout (some old commit)
git checkout master bin src/cmd/INIT # use new build system
bin/package make
However, that became impossible in 6cc2f6a0 because the new
SHOPT.sh script was unconditionally sourced. The error caused the
script to exit because '.' is a special builtin.
bin/package, src/cmd/INIT/package.sh:
- If src/cmd/ksh93/SHOPT.sh doesn't exist, issue a warning instasd
of trying to source it.
The build system is adapted to make SHOPT_* compile-time options
editable without nmake. We can now easily change ksh's compile-time
options by editing src/cmd/ksh93/SHOPT.sh. The bin/package script
is adapted to turn these into compile flags. This resolves the most
important drawback of not using nmake.
Also, mamake now has support for indented Mam (Make Abstract
Machine) code. Only one type of block (make...done) is supported in
Mamfiles, so they are easy to indent automatically. A script to
(re)do this is included.
Since nmake is not going to be restored (it has too many problems
that no one is interested in fixing), this at least makes mamake
significantly easier to work with.
The Makefiles are deleted. They may still be handy for reference to
understand the Mamfiles, but they haven't actually matched the
Mamfiles for a while -- and you can still look in the git history.
Deleting them requires some adaptations to bin/package and mamake.c
because, even though they do not use those files, they still looked
for them to decide whether to build code in a directory.
Finally, this commit incorporates some #pragmas for clang to
suppress annoying warnings about the coding style used in this
historic code base. (gcc does not complain so much.)
src/cmd/ksh93/SHOPT.sh:
- Added.
bin/package, src/cmd/INIT/package.sh:
- cd into our own directory in case we were run from another dir.
- $makefiles: only look for Mamfiles.
- Add ksh compile-options via KSH_SHOPTFLAGS. Include SHOPT.sh.
- make_recurse(): Do not write a missing Makefile.
- finalize environment: Look for Mamfiles instead of Makefiles.
src/cmd/INIT/mamake.c:
- Tell clang to suppress annoying warnings about coding style.
- Update version string and self-documentation.
- input(): Add support for indented Mam code by skipping initial
whitespace on each input line.
- files[]: Instead of looking for various of Makefiles to decide
where to build, only look for Mamfiles.
src/Makefile, src/cmd/INIT/Makefile, src/cmd/Makefile,
src/cmd/builtin/Makefile, src/cmd/ksh93/Makefile, src/lib/Makefile,
src/lib/libast/Makefile, src/lib/libcmd/Makefile,
src/lib/libdll/Makefile, src/lib/libsum/Makefile:
- Removed.
src/Mamfile, src/cmd/INIT/Mamfile, src/cmd/Mamfile,
src/cmd/builtin/Mamfile, src/cmd/ksh93/Mamfile, src/lib/Mamfile,
src/lib/libast/Mamfile, src/lib/libcmd/Mamfile,
src/lib/libdll/Mamfile, src/lib/libsum/Mamfile:
- Indent the code with tabs.
- In ksh93/Mamfile, add ${KSH_SHOPT_FLAGS} to every $CC command.
- In ksh93/Mamfile, add "prev SHOPT.sh" for every *.o file
so they are rebuilt whenever SHOPT.sh changes.
bin/Mamfile_indent:
- Added, in case someone wants to re-indent a Mamfile.
src/cmd/INIT/proto.c, src/cmd/INIT/ratz.c, src/cmd/INIT/release.c,
src/lib/libast/features/common, src/lib/libast/include/ast.h:
- Tell clang to suppress annoying warnings about coding style that
it disapproves of (mainly concerning the use of parentheses).
src/cmd/INIT/cc.darwin, src/cmd/INIT/cc.freebsd,
src/cmd/INIT/cc.openbsd:
- Remove now-redundant clang warning suppression flags.
Resolves: https://github.com/ksh93/ksh/issues/60
src/cmd/ksh93/bltins/typeset.c:
- The new_argv[] array was one item too short (should be argc+2).
- Use AST stakalloc(3) to allocate it instead of a dynamic array;
this restores compatibility with ISO C90.
src/lib/libast/features/standards, src/cmd/INIT/cc.unixware.i386:
- Add support for UnixWare.
- Do not define any standards macros on this system, as on FreeBSD
and DragonFly BSD.
AIX on ibm.risc comes with a broken version of ksh88 as /bin/sh
where the following causes breakage in the parser (spurious syntax
errors):
(set -o posix) 2>/dev/null && set -o posix
However, prefixing it with 'command' (while keeping the subshell)
circumvents the problem. So, why not.
(command set -o posix) 2>/dev/null && set -o posix
Solaris Studio 12.5 cc seems to produce incorrect code at -O2
(a.k.a. -xO2) optimisation level; integer variables initialise at
random values, and the behaviour of the shell is so incorrect it
can't even run the test scripts. It does not support -Os so that
is skipped for that compiler. At -O it works fine.
src/cmd/INIT/make.probe:
- By default, only try -Os and -O optimisation flags.
If iffe re-ran a test because the source test script changed, but
the result file is unchanged, it didn't update the timestamp of the
result, so the source script remained newer. The build system then
kept pointlessly re-running the test on each rebuild. If a central
test script such as src/lib/libast/features/standards was changed,
this had cascading effects, e.g., causing libast to be rebuilt over
and over as I recompiled small changes elsewhere. Until now, my
workaround was to delete the entire 'arch' directory and start
over. Hopefully that will now no longer be needed.
src/cmd/INIT/iffe.sh:
- If a test output file has not changed after rerunning a test that
has changed, still update the output file's timestamp using
touch(1) to signal that the test has already been run.
Instead, we now link to the libm system math library where needed
by adding -lm to the relevant compile commands in the Mamfiles.
This is not needed on every system but never does any harm.
(This adds more custom edits to the Mamfiles, which were originally
generated from the nmake Makefiles. This takes us further from
restoring nmake, but that already wasn't going to happen anyway,
due to its many problems... the path forward will be to translate
the Mamfiles to some other, current make system such as GNU make.)
bin/package, src/cmd/INIT/package.sh:
- Remove LDFLAGS=-lm hack for DragonFly BSD, NetBSD and Solaris.
src/cmd/builtin/Mamfile,
src/cmd/ksh93/Mamfile,
src/lib/libdll/Mamfile:
- Add -lm where linking failed on any of the three mentioned OSs.
src/lib/libdll/features/dll:
- In the output test program, add missing #include <math.h>, fixing
unknown identifier errors on NetBSD (ldexp, ldexpl).
src/cmd/builtin/features/pty:
- Add missing #include <stdio.h> to make printf work on all systems
(this is just a feature test, no need to bother with sfio here).
src/lib/libast/features/stdio:
- Undef __FILE_T to avoid interference from system headers on QNX.
(There are still other problems preventing a build on QNX 6.5.0.
The shipped version of gcc seems to be broken.)
This now makes ksh build on DragonFly BSD.
bin/package, src/cmd/INIT/package.sh:
- DragonFly also needs the -lm hack for LDFLAGS.
src/cmd/ksh93/sh/main.c, src/cmd/ksh93/tests/basic.sh:
- fixargs() doesn't work on DragonFly either
(re: 9b7c392a, 159fb9ee, cefe087d).
The following are backported from:
https://github.com/att/ast/issues/26#issuecomment-313927854https://github.com/att/ast/pull/19
src/lib/libast/comp/setlocale.c:
- Add missing #include <errno.h> since errno is used.
src/lib/libast/features/standards:
- Do not set any standards macros (_POSIX_SOURCE etc) on FreeBSD or
DragonflyBSD; they disable too much functionality on those.
src/lib/libast/features/wchar:
- Set _STDFILE_DECLARED on DragonFly, too.
src/lib/libast/include/sfio.h, src/lib/libast/include/sfio_t.h,
src/lib/libast/sfio/_sfopen.c, src/lib/libast/sfio/sfclrlock.c,
src/lib/libast/sfio/sfhdr.h, src/lib/libast/sfio/sfnew.c,
src/lib/libast/sfio/sfset.c:
- Rename SF_* macros to SFIO_* to avoid a conflict with system
headers.
src/lib/libast/string/strexpr.c:
- Rename error() to err() to avoid a conflict.
bin/package, src/cmd/INIT/package.sh:
- It can depend on the compiler flags passed whether the compiler
produces code for a 64-bit architecture, so pass $CCFLAGS to
the compiler when testing whether it creates 64-bit object code.
README.md:
- Copy-edit of build instructions.
bin/package, src/cmd/INIT/package.sh:
- CCFLAGS overwrites the autodetected optimisation flags (e.g. -Os)
if set. Unfortunately, that also happened when we added something
to CCFLAGS for a release build or to add an extra flag needed by
Solaris. The fix is to use a new flags variable (KSH_RELFLAGS)
instead. This needs to be done in a different place as it needs
to be added to the mamake command as an assignment argument.
- Remove the Solaris CCFLAGS hack; see features/common below.
src/*/*/Mamfile:
- Add ${KSH_RELFLAGS} to all the compiler commands.
src/lib/libast/features/common:
- Enable POSIX standard on Solaris (i.e.: if __sun is defined) by
defining _XPG6 directly in the feature test that generates
ast_std.h, which is indirectly included by everything. This
removes the need to pass -D_XPG6 via CCFLAGS. (Doing so
automatically with gcc was not otherwise possible.)
src/cmd/INIT/cc.sol11.*:
- No longer pass -D_XPG6, as per above.
bin/package, src/cmd/INIT/package.sh:
- The code for detecting a 64-bit object file was seriously broken:
the temporary file name could contain '64' because it included $$,
the current PID, and 64-bit was detected if the output of 'file'
(which includes the complete file name) contained '64'. Fix by
removing the file name from 'file' output before testing.
- Also refactor that code a bit and remove the nonsensical test if
/bin/sh is a 64-bit binary, which is neither here nor there. It's
what the compiler produces that we need to care about.
src/cmd/INIT/make.probe:
- probe_optimize: Also try -O2 and -O, for compilers (such as
Solaris Studio cc) that do not support -Os.
- Use more robust code to loop through possible optimiser flags.
The versions from the Solaris patch require $CC_EXPLICIT to be set,
which is specific to the internal Solaris build environment.
src/cmd/INIT/cc.sol11.*:
- Cope without $CC_EXPLICIT set in environment; fall back to $CC
and if that is not set either, detect whether to use cc or gcc.
- Set appropriate flags for cc (Solaris Studio) or gcc, including
the necessary -D_XPG6 flag, without which ksh crashes on Solaris.
bin/package, src/cmd/INIT/package.sh:
- Update hack to add the -D_XPG6 flag so it applies to gcc only
(note: the src/cmd/INIT/cc.* scripts are never used for gcc).
This reverts commit 600cb182.
$cc may be a system compiler binary, it is not necessarily a
src/cmd/INIT/cc.* wrapper script; so prefixing 'sh' is wrong.
As of this commit, ksh 93u+m has a standard semantic version number
<https://semver.org/>, beginning with 1.0.0-alpha. This is added to
the version string in a way that should be compatible with scripts
parsing ${.sh.version} or $(ksh --version). This addition does not
replace the release date and does not affect $((.sh.version)).
For non-release builds, the version string will be:
FORK/VERSION+HASH YYYY-MM-DD
e.g.: 93u+m/1.0.0-alpha+41ef7f76 2021-01-03
For release builds, it will be:
FORK/VERSION YYYY-MM-DD
e.g.: 93u+m/1.0.0 2021-01-03
It is now automatically decided by bin/package whether to build a
release or development build. When building from a directory that
is not a git repository, or if the current git branch name starts
with a number (e.g. '1.0'), the release build is enabled; otherwise
a development build is the default. This is arranged by adding -D
flags to $CCFLAGS as described below. These flags are prepended to
$CCFLAGS, so they can be overridden by adding your own -D or -U
flags via the environment.
In addition, AST vmalloc is disabled for release builds as of this
commit, forcing the use of the OS's standard malloc(3). In 2021,
this is generally more reliable, faster, and more economical with
memory than AST vmalloc. Several memory leaks and crashing bugs are
avoided, e.g.: <https://github.com/ksh93/ksh/issues/95>.
For development builds, vmalloc stays enabled (along with its known
bugs) because this allows the use of the vmstat builtin, making it
much more efficient to test for memory leaks. For more info, see
the regression test script: src/cmd/ksh93/tests/leaks.sh
bin/package, src/cmd/INIT/package.sh:
- Add flags for build type. In $CCFLAGS, define _AST_ksh_release if
we're not on any git branch or on a git branch whose name starts
with a number. Otherwise, define _AST_git_commit as the first 8
characters of the current git commit hash.
src/lib/libast/features/vmalloc:
- If _AST_ksh_release is defined, disable vmalloc and force use of
the operating system's malloc. Discussion:
https://github.com/ksh93/ksh/issues/95
src/cmd/ksh93/include/version.h:
- Define new format for version string, adding a semantic version
number as well as (for non-release builds) the git commit hash.
src/cmd/ksh93/sh/init.c: e_version[]:
- Add a 'v' to the ${.sh.version} feature string if ksh was
compiled with vmalloc enabled. This allows scripts, such as
regression tests, to detect this.
src/cmd/ksh93/data/builtins.c: sh_optksh[]:
- Add a copyright line crediting the contributors to ksh 93u+m.
Resolves: https://github.com/ksh93/ksh/issues/95
The package script searches for a good shell to run the build
scripts, preferring a ksh. But it also finds any recently compiled
development version of ksh in arch/*/bin that may be broken, or
have debug code, etc. -- and uses that in preference to anything
else. This is quite capable of breaking the build process.
The way to get around it is to do something like
bin/package make SHELL=/bin/ksh
which is annoying to have to keep doing.
bin/package,
src/cmd/INIT/package.sh:
- When finding a good shell, use the saved user path ($path), not
the current $PATH which includes arch/$HOSTTYPE/bin. Prefix this
temporary path with `getconf PATH`, the system's default path,
so that known-good system shells are found first.
