The more notable ones are:
src/lib/libast/features/standards:
- Do not redefine _GNU_SOURCE and _FILE_OFFSET_BITS if already
defined from $CCFLAGS. Thanks to @hyanias for the heads-up.
(re: 289dd46c)
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/sh/args.c,
src/cmd/ksh93/sh/name.c:
- Remove -T test code activation option. It was basically unused.
The only thing it did was intentionally introduce a memory leak
in table_unset() if the 4th bit in the option argument was set.
A search in ast-open-history reveals a few more trivial test uses
that were later deleted, but nothing interesting.
src/cmd/ksh93/tests/{basic,path}.sh:
- Skip a couple of tests on AIX avoid hangs, at least one of which
is not ksh's fault. Thanks to @HansH111 for the report.
src/cmd/ksh93/tests/builtins.sh:
- Change one awk use to a more portable sed invocation to placate
systems with ancient awk commands, such as AIX. (re: de795e1f)
This commit adds a wrapper for the AIX ar command that uses the
-X64 flag to avoid build errors on that platform.
Resolves: https://github.com/ksh93/ksh/issues/385
src/cmd/INIT/iffe.sh:
- Remove obsolete $posix_noglob/$posix_glob code to disable and
re-enable pathname expansion. Instead, globally disable it using
'set -o noglob'. Except a couple of 'rm' commands, no aspect of
this code uses globbing (note: 'case' patterns do not count), nor
do any of the shell code feature tests that are eval'ed by iffe.
Field splitting is heavily used via unquoted variable expansions;
globbing should not interfere. (Concrete example: one of the test
notes in src/cmd/ksh93/features/options is "SHOPT_* option probe"
and that SHOPT_* is taken as a file name pattern, so could be
expanded.) For the rm commands, globbing is turned back on in a
subshell. If this breaks something that I've missed, hopefully
that will turn up soon.
- Remove $show/$SHOW code to massage different versions of 'echo'
into acting consistently; instead, use a show_test() function
that uses the POSIX-specified printf command.
- For debugging level 2 and 3, use much more extensive PS4 prompts
taken from modernish (which I wrote), depending on the shell.
- Replace the obsolete 'set -' command. In the ancient Bourne
shell, this disables the v and x options, so is equivalent to
'set +v +x'. This still works on almost all modern shells, but
not yash, and POSIX considers it "unspecified".
- Remove a few echo|sed fallbacks I'd missed (re: 215efa15).
src/cmd/ksh93/features/math.sh:
- Disable pathname expansion here as well.
- Pass down an inherited $IFFEFLAGS to recursive iffe invocations.
This commit makes various different improvements to the documentation:
- sh.1: Backported (with changes) mandoc warning fixes from ksh2020
for the ksh93(1) man page: <https://github.com/att/ast/pull/1406>
- Removed unnecessary spaces at the end of lines to fix a few other
mandoc warnings.
- Fixed various typos and capitalization errors in the documentation.
- ANNOUNCE: Document the addition of the ${.sh.pid} variable
(re: 9de65210).
- libast/man/str*: Update the man pages for the libast str* functions
to improve how accurately each function is described.
- ksh93/README: Update regression test/compatibility notes to include
OpenBSD 7.0, FreeBSD 13.0 and WSL running Ubuntu 20.04.
- Change a few places to store the return value from strlen in a
size_t variable rather than signed int.
- comp/setlocale.c: To avoid confusion of two separate variables named
lang, the function local variable has been renamed to langidx.
bin/package, src/cmd/INIT/package.sh:
- Automatically update an existing flat view even if 'flat' wasn't
given for a make action. This stops a flat view becoming
inaccurate if you forget to use 'bin/package flat make'
consistently. If the $PACKAGEROOT/lib/package/gen directory
exists, an existing flat view is assumed.
- Correct symlink fallbacks. We need an absolute path for the
symlink target or it's going to be broken.
.gitignore:
- Update.
In backtick command substitutions '\\' will only print one backslash,
while placing a backslash in front of a different character will
print a backslash followed by that character:
$ echo `echo '\\\\ \3'`
\\ \3
$ echo $(echo '\\\\ \3')
\\\\ \3
This difference in behavior was causing iffe to fail on Haiku after
the changes in aeda3502. This commit applies fixes for backslashes
in quoted strings to fix the Haiku build failure and other possible
bugs introduced by the referenced commit.
bin/package, src/cmd/INIT/package.sh:
- Make the EXIT (0) trap report failure based on $error_status
(see d18469d6), replacing 'done' with 'failed' if it is nonzero.
- Remove extra space before 'at' in that report line.
- If we get a signal, we have to set error_status to nonzero
manually so that the correct exit message is printed.
I've reconsidered excluding build system internals (and also '*.o'
files) from the flat view. Really the only thing that should be
excluded are *.old files.
bin/package, src/cmd/INIT/package.sh:
- Do not exclude anything except *.old files from the flat view.
- This would delete bin/package when cleaning up the flat view,
so explicitly keep bin/package in the clean routine.
- Be much faster about updating an existing flat view by checking
if a link already exists before creating one.
- Add silent cleanup for dozens of orphaned macOS *.dSYM bundles
belonging to deleted temporary feature test executables.
src/cmd/INIT/{iffe,ignore,silent}.sh, bin/{ignore,silent}:
- Remove obsolete Bourne shell fallbacks.
- Modernise command substitutions.
- Remove unused literal() function.
- Update copy() function to use printf.
- Distinguish just two shell types now: ksh and POSIX.
- compile(): Remove obsolete/incorrect grepping for signal messages.
Add a POSIX-compiliant check with 'kill -l' to see if the
compiler was terminated by a signal.
I called the flat view featuritis, but it turns out the dtksh build
system depends on it. But it was broken, so let's make a proper
version instead. This new approach does not symlink directories
before make, but hardlinks files after make. To make performance
tolerable, it requires a modern POSIX 'find' utility with '{} +'.
bin/package, src/cmd/INIT/package.sh:
- We're going to depend on 'test X -ef Y' to check if X is the same
file as Y, so add that to the $min_posix checks even though it is
not technically POSIX. But it's so close to universally available
it doesn't really make a difference.
- After 'make', create a flat view by hardlinking arch/$HOSTTYPE
files, minus build system internals, onto their corresponding
paths in $PACKAGEROOT. Fall back to symlinking if hardlinking
fails (this would happen when crossing device boundaries).
- Clean up flat view when doing clean/clobber. This is done by
using 'find' to loop through the files in arch/ again and
removing any root paths that are the same file as their
corresponding arch/ path (this is where test X -ef Y comes in).
Then delete arch/$HOSTTYPE, then delete empty directories left.
- Check for the 'flat' qualifier when doing clean/clobber. If
given, do not delete arch/$HOSTTYPE but only clean up the flat
view and remove empty directories.
src/cmd/INIT/Mamfile:
- Do not create the lib/package directory. (re: beb3c64a)
.gitignore:
- Update.
After further examining the iffe code I found that fail{ ... }end
as well as pass{ ... }end blocks are executed in iffe's current
environment, using a simple 'eval', with no safeguards whatsoever!
This does of course afford maximum flexibility... for example, a
block can 'exit 1' to abort the iffe run and the whole build with
it. We can use this to abort properly on fatal compilation errors.
src/cmd/INIT/iffe.sh:
- Complete the fail{ and pass{ documentation; it should really be
known that they run in iffe's current environment.
- Make one change: for just the 'eval' command that runs these
blocks, redirect standard error back to the saved $stderr file
descriptor so these blocks can write error messages using the
standard >&2 instead of the undocumented >&$stderr.
src/lib/**/features/*:
- Write error message to standard error and error out properly when
an output{ ... }end block fails to compile, instead of writing an
#error directive to error out later.
I'd forgotten to check for uses of the __STDC__ macro. This is
defined on all C compilers that support C89/C90 or later standards.
So we can remove all fallback code disabled by that macro.
This takes another step towards cleaning up the build system. We
now do not even pretend to be theoretically compatible with
pre-1989 K&R C compilers or with C++ compilers. In practice, this
had already been broken for many years due to bit rot.
Commit 46593a89 already removed the license handling enormity that
depended on proto, so now we can cleanly remove it altogether. But
we do need to leave some backwards compatibility stubs to keep the
build system compatible with older AST code; it should remain
possible to build older ksh versions with the current build system
(the bin/ and src/cmd/INIT/ directories) for testing purposes.
So as of now there is no more __MANGLE__d rubbish in your generated
header files. This is only about a quarter of a century overdue...
This commit also includes a huge amount of code cleanup to remove
thousands of unused K&R C fallbacks and other cruft, particularly
in libast. This code base should now be a little easier to
understand for people who are familiar with a modern(ish) C
standard.
ratz is now also removed; this was a standalone and simplified 2005
version of gunzip. As of 6137b99a, none of our code uses it, even
theoretically. And the real g(un)zip is now everywhere.
src/cmd/INIT/proto.c, src/cmd/INIT/ratz.c:
- Removed.
COPYRIGHT:
- Remove zlib license; this only applied to ratz.
bin/package, src/cmd/INIT/package.sh:
- Related cleanups.
- Unset LC_ALL before invoking a new shell, respecting the user's
locale again and avoiding multibyte character corruption on the
command line.
src/cmd/INIT/proto.sh:
- Add stub for backwards compatibility with Mamfiles that depend on
proto. It does nothing but pass input without modification and is
now installed as the new arch/*/bin/proto by src/cmd/INIT/Mamfile.
src/cmd/INIT/iffe.sh:
- Ignore the proto-related -e (--package) and -p (--prototyped)
options; keep parsing them for backwards compatibility.
- Trim the macros passed to every test to their standard C
versions, removing K&R C and C++ versions. These are now
considered to be for backwards compatibility only.
src/cmd/INIT/iffe.tst:
- Remove proto(1) mangling code.
By the way, iffe can be regression-tested as follows:
$ bin/package use # set up environment in a child shell
$ regress src/cmd/INIT/iffe.tst
$ exit # leave package environment
src/cmd/INIT/make.probe, src/cmd/INIT/probe.win32:
- Remove code to handle C++.
src/lib/libast/features/common:
- As in iffe.sh above, trim macros designed for compatibility with
C++ and ancient C compilers to their standard C versions and
comment that they are for backwards compatibility with AST code.
This is needed to keep all the old ast and ksh code compiling.
src/cmd/ksh93/sh/init.c,
src/cmd/ksh93/sh/name.c:
- Clarify libshell ABI compatibility function versions of macros.
A "proto workaround" comment in the original code mislead me into
thinking this had something to do with the removed proto(1), but
it's unrelated. Call the workaround macro BYPASS_MACRO instead.
src/cmd/ksh93/include/defs.h:
- sh_sigcheck() macro: allow &sh as an argument: parenthesise shp.
src/cmd/ksh93/sh/nvtype.c:
- Remove unused nv_mkstruct() function. (re: d0a5cab1)
**/features/*:
- Remove obsolete iffe 'set prototyped' option.
**/Mamfile:
- Remove all references to the ast/prototyped.h header.
- Remove all use of the proto command. Simply copy instead.
*** 850-ish source files: ***
- Remove all '#pragma prototyped' directives.
- Remove all C++ compat code conditional upon defined(__cplusplus).
- Remove all use of the _ARG_ macro, which on standard C expands to
its argument:
#define _ARG_(x) x
(on K&R C, it expanded to nothing)
- Remove all use of _BEGIN_EXTERNS_ and _END_EXTERNS_ macros (empty
on standard C; this was for C++ compatibility)
- Reduce all #if __STD_C (standard code) #else (K&R code) #endif
blocks to the standard code only, without use of the macro.
- Same for _STD_ macro which seems to have had the same function.
- Change all instances of 'Void_t' to standard 'void'.
If the compiler is called gcc but not cc, the 64-bit detection
didn't work and $HOSTTYPE (and the arch/ subdirectory) did not get
the -64 suffix.
bin/package, src/cmd/INIT/package.sh:
- Run checkcc() before attempting to compile the program. This will
set $cc to the path to gcc if there is no 'cc' command.
- trap: use 'rm -rf' to also delete .dSYM directories (macOS).
- checkcc(): Since we're here, find clang as well.
The goal is to get rid of all compiler/linker wrapper scripts as
they are overridden by passing CC/LD and it should be possible to
select your compiler or linker without breaking the build. The
probing and feature testing system should set the appropriate flags
and macros. This makes some progress towards that.
src/lib/libast/features/standards:
- Eliminate the shotgun approach to standards macros on popular
systems where the macros we we need to set are known and
documented. The following will enable standards compliance plus
all the available extensions:
- Set no macros at all for any BSD system (excluding macOS).
- Set _DARWIN_C_SOURCE on Darwin/macOS.
- Set everything and the kitchen sink for Solaris/illumos in
a way that enables backwards compatibility with older Solaris.
This is unofficial, but following the standards(5) manual
disables a lot of basic functionality that we depend on.
- Set _GNU_SOURCE for GNU (glibc).
- Remove the covered macros from the shotgun approach fallback.
- Add a new heuristic. _POSIX_PATH_MAX and _SC_PAGESIZE are among
the basic macros disabled when you pass recommended standards
macros, killing the build, so it's good to check if they compile.
src/cmd/INIT/ar.freebsd12.amd64,
src/cmd/INIT/ar.linux.i386-64:
- Removed. May cause build failures on some systems as not all 'ar'
implementations support the U option. Plus, I can think of no
good reason to disable deterministic mode (which always creates
identical output) on 'ar' implementations that support it. See:
https://groups.google.com/g/comp.unix.shell/c/LdOD1Ya0C9E/m/U6DhgHVICwAJ
src/cmd/INIT/cc.linux.*-icc,
Removed icc wrappers. These manually source /etc/profile.d/icc.sh
but I don't think that is the build system's job. Profile scripts
should be run at login time and export variables we inherit
through the environment.
src/cmd/INIT.cc.{freebsd,linux,openbsd}*:
- Removed. Should be entirely superfluous now that the standards
feature test sets the appropriate macros.
src/cmd/INIT.cc.sol11.*:
- Removed as the standards feature test now sets the approopriate
macros. Note the Solaris build system should now simply pass CC
as normal instead of passing CC_EXPLICIT.
src/cmd/INIT/package.sh, bin/package:
- Derive the command name from $0 instead of hardcoding it.
- Remove NPROC and related code to support parallel building. This
is not supported with mamake, is unlikely to be reintroduced any
time soon, and if it ever is it will need to be done in a
different way anwyay.
- Invoke 'sed' and 'tr' directly instead of via $SED and $TR
variables. We're not building our own dynamically linked 'sed'
and 'tr' in this distribution so LD_LIBRARY_PATH is irrelevant.
If we ever do again, there are better ways to make sure the OS
standard 'sed' and 'tr' are invoked than this kludge.
- Use note() consistently to print warnings to standard error.
note() is changed to print each argument on a new line prefixed
by the command name, so arguments need to be quoted now if they
are to be shown on a single line.
- Use a new err_out() function to error out, avoiding code
repetition.
Yes, we're finally abandoning the old Bourne shell so we can
use sane $(command substitutions) and the like. POSIX sh (with
tolerance for shell bugs) is very highly portable these days.
Even Solaris 10 has a POSIX shell, though not as /bin/sh.
bin/package, src/cmd/INIT/package.sh:
- Be nice: if we're on an obsolete or broken shell, try hard to
escape to a good one. This should preserve the possibility to
just run 'bin/package make' via ancient /bin/sh on Solaris 10.
- Note: zsh without sh emulation is considered broken because
$path, which we use, is linked to $PATH. You have to run it via
a symlink named sh or via 'zsh --emulate sh' to disable this.
Enabling emulation mode after initialisation will not work.
- More self-documentation cleanups and updates.
- Regenerate the text-only fallback version of the self-doc.
- Remove flat view functionality (no arch directory); it may have
been broken for some time, but quite frankly I could not care
less. It's yet more featuritis. Building in arch/ is fine.
In the referenced commit message I neglected to mention that, when
doing bin/package make, we're now running 'tee' in the background
again and the building job in the foreground, as opposed to the
other way around. Foreground jobs are more reliably interruptable.
But that reintroduced the problem fixed in 5b8d29d3. Now I don't
know what I was thinking then -- the obvious fix is to add a 'wait'
command, allowing 'tee' to catch up before returning to the prompt.
This reduces the bin/package script by more than half!
bin/package, src/cmd/INIT/package.sh:
- Remove obsolete and unused package commands: admin, contents,
license, list, remote, regress, setup, update, verify, write.
- Remove associated documentation.
- Replace install command with a dummy. It'll come back when we
reintroduce the building of dynamic libaries.
- Update the test command to run the regression tests properly
and capture the output in arch/*/lib/package/gen/tests.out, as
documented. Arguments are simply passed to bin/shtests.
src/cmd/INIT/{ditto.sh,hurl.sh,release.c}:
- Removed. These were support scripts for some of the removed
package commands.
src/cmd/ksh93/tests/pty.sh:
- Avoid failure when capturing output via 'bin/package test' by
redirecting standard error to /dev/tty when running the tests.
Running 'bin/package environment' should only show what will happen
during the build process, but in it's current state the feature has
some bugs:
1. Errors can occur relating to the failed creation of files and a
failed attempt to change the directory. Various errors from
'bin/package environment make' and 'bin/package environment make
CC=tcc':
bin/package[5632]: cd: /home/johno/GitRepos/KornShell/ksh/arch/linux.i386-64/lib/package/gen: [No such file or directory]
bin/package: line 5869: /home/johno/GitRepos/KornShell/ksh/arch/linux.i386-64/lib/package/gen/CC: cannot create [No such file or directory]
bin/package: line 5869: /home/johno/GitRepos/KornShell/ksh/arch/linux.i386-64/lib/package/gen/CCFLAGS: cannot create [No such file or directory]
bin/package: line 5869: /home/johno/GitRepos/KornShell/ksh/arch/linux.i386-64/lib/package/gen/CCLDFLAGS: cannot create [No such file or directory]
bin/package: line 5869: /home/johno/GitRepos/KornShell/ksh/arch/linux.i386-64/lib/package/gen/LDFLAGS: cannot create [No such file or directory]
bin/package: line 5869: /home/johno/GitRepos/KornShell/ksh/arch/linux.i386-64/lib/package/gen/KSH_RELFLAGS: cannot create [No such file or directory]
bin/package[5888]: /home/johno/GitRepos/KornShell/ksh/arch/linux.i386-64/lib/package/gen/host: cannot create [No such file or directory]
2. The package script may in some scenarios create a temporary file
at the root of the repository, such as 'pkg77213.c'.
bin/package, src/cmd/INIT/package.sh:
- Avoid creating files or changing the directory while the
environment qualifier is on (this also affects the debug
qualifier). Part of this fix is based on a patch from Marcin
Cieślak[*], with other fixes applied for similar problems the
environment qualifier had.
I'm now taking another small step towards extricating this build
system from the long-dead AT&T AST universe.
This commit modifies/reduces the tool called proto. AT&T used proto
for two purposes:
1. To convert ANSI C code to a form compatible with ancient
(pre-ANSI) K&R C compilers using extremely complex macro
voodo. It was similarly capable of translating to C++.
Theoretically, this entire code base should compile on
anything from a 1980s K&R C compiler to a modern C++ compiler.
In practice, given the massive amount of bit rot we inherited,
I am 99.9% sure that this has been broken for many years.
2. To automagically insert license comments into source files
based on an extremely complicated license database system.
(In all-too-typical AT&T fashion, this second function of
proto is completely unrelated to the first.)
Function 2 has now been removed because, unlike the AT&T legal
department, I don't think it's worth going to unspeakably extreme
lengths to avoid maintaining license information in source code
files by hand.
In the process, proto.c was cleaned up to look halfway like actual
C code, but it's still processed code: most macros have been
expanded to their numeric value, all comments were stripped, etc.
So don't expect to understand this code. The actual source code is
in these two directories in the ast-open-history repo:
https://github.com/ksh93/ast-open-history/tree/master/src/cmd/protohttps://github.com/ksh93/ast-open-history/tree/master/src/lib/libpp
Meanwhile, nobody wants to compile ksh with a pre-ANSI K&R C
compiler in 2021 -- and there's no good reason to be compatible
with C++ because standard C compilers are universally available.
So, proto will go away when I manage to figure out how to pry it
loose from the innards of this build system.
src/lib/libast/port/astlicense.c:
- Removed. This is al the license handling code that was
incorporated in proto.c in stripped form. It was not used
anywhere else, and the environment where it was useful is gone.
src/cmd/INIT/proto.c:
- Cleanup to make this halfway maintainable: indentation, huge
blocks of empty lines, #line directives, etc.
- Delete all the code corresponding to astlicense.c. This was
actually easy as it was in a discrete block.
- proto(), pppopen(): Remove 'license'/'notice' and 'options'
arguments.
- main(): Remove processing of -l (license) and -o (license
options) flags.
**/Mamfile:
- Update all the proto invocations to remove the -l and -o flags.
bin/package, src/cmd/INIT/package.sh:
- Delete the 'copyright' command, which used the -l and -o
options to tell proto to extract copyright information from
*.lic/*.def files in lib/package.
COPYRIGHT:
- Added. This has the information from 'bin/package copyright', with
the copyright years corrected to plausible values as the AST code
used the current year (2021) for all of them. It adds ksh 93u+m
copyright and contributor information at the top as well.
(Yes, some of the lines in the old non-AT&T copyright notices
are clipped. This is the actual output of the 'bin/package
copyright' command as generated by 'proto' in the AST
distribution. For all that extreme complexity, they couldn't even
reproduce the notices correctly. But it's officially sanctioned
by AT&T in exactly this form, so there you have it.)
lib/package/**:
- Removed. All these files are now obsolete and redundant.
If you passed CC=/some/compiler, the build broke on macOS because the
cc.darwin compiler wrapper wasn't used. Among other things, this
wrapper adds a -D_lib_memccpy flag, defining _lib_memccpy as 1
during the build. That was used to override a false negative result
of the lib_memccpy feature test. This commit fixes that feature
test instead, so it correctly returns positive on macOS.
Thanks to Ryan Smith (@ryandesign) for the bug report and for the
fix to the lib_memccpy test.
src/lib/libast/features/lib:
- Fix the lib_memccpy feature test. It was checking the result of
mmap(2) incorrectly, resulting in the test crashing on macOS.
Failure does not return NULL, it returns MAP_FAILED which is
usually -1.
src/cmd/INIT/cc.darwin*:
- Removed. Any other flags in these wrappers are either related to
building dynamic libraries, which is not currently supported, or
were determined to be unnecessary. See the GitHub issue for
discussion. This now makes it possible to pass `CC` to use any
compiler you like on the Mac. Notes:
- Apple's -D_ast_int8_t=int64_t is a no-op; another AST feature
test already defines _ast_int8_t a 64-bit integer type, even on
32-bit systems (on which it is defined as 'long long').
- The -search_paths_first linker flag is the default since 2010.
But even on my museum-grade Power Mac G5 with Mac OS X 10.3
(from 2004), it builds and runs just fine without.
- DCLK_TCK=100 is a no-op as even that ancient Mac system already
defines it as 100. Plus, it's not even actually used.
If a need is found for any of these, please report this in a new
issue so I can special-case it elsewhere in the code.
Resolves: https://github.com/ksh93/ksh/issues/373
List of changes:
- Fixed some -Wuninitialized warnings and removed some unused variables.
- Removed the unused extern for B_login (re: d8eba9d1).
- The libcmd builtins and the vmalloc memfatal function now handle
memory errors with 'ERROR_SYSTEM|ERROR_PANIC' for consistency with how
ksh itself handles out of memory errors.
- Added usage of UNREACHABLE() where it was missing from error handling.
- Extend many variables from short to int to prevent overflows (most
variables involve file descriptors).
- Backported a ksh2020 patch to fix unused value Coverity issues
(https://github.com/att/ast/pull/740).
- Note in src/cmd/ksh93/README that ksh compiles with Cygwin on
Windows 10 and Windows 11, albeit with many test failures.
- Add comments to detail some sections of code. Extensive list of
commits related to this change:
ca2443b5, 7e7f1372, 2db9953a, 7003aba4, 6f50ff64, b1a41311,
222515bf, a0dcdeea, 0aa9e03f, 61437b27, 352e68da, 88e8fa67,
bc8b36fa, 6e515f1d, 017d088c, 035a4cb3, 588a1ff7, 6d63b57d,
a2f13c19, 794d1c86, ab98ec65, 1026006d
- Removed a lot of dead ifdef code.
- edit/emacs.c: Hide an assignment to avoid a -Wunused warning. (See
also https://github.com/att/ast/pull/753, which removed the assignment
because ksh2020 removed the !SHOPT_MULTIBYTE code.)
- sh/nvdisc.c: The sh_newof macro cannot return a null pointer because
it will instead cause the shell to exit if memory cannot be allocated.
That makes the if statement here a no-op, so remove it.
- sh/xec.c: Fixed one unused variable warning in sh_funscope().
- sh/xec.c: Remove a fallthrough comment added in commit ed478ab7
because the TFORK code doesn't fall through (GCC also produces no
-Wimplicit-fallthrough warning here).
- data/builtins.c: The cd and pwd man pages state that these builtins
default to -P if PATH_RESOLVE is 'physical', which isn't accurate:
$ /opt/ast/bin/getconf PATH_RESOLVE
physical
$ mkdir /tmp/dir; ln -s /tmp/dir /tmp/sym
$ cd /tmp/sym
$ pwd
/tmp/sym
$ cd -P /tmp/sym
$ pwd
/tmp/dir
The behavior described by these man pages isn't specified in the ksh
man page or by POSIX, so to avoid changing these builtin's behavior
the inaccurate PATH_RESOLVE information has been removed.
- Mamfiles: Preserve multi-line errors by quoting the $x variable.
This fix was backported from 93v-.
(See also <a7e9cc82>.)
- sh/subshell.c: Remove set but not used sp->errcontext variable.
In iffe tests, some C functions are found in system libraries, but
then are not declared by the system headers on some systems because
the expected standards macros aren't defined, causing the system
headers to hide the function declarations. This may cause warnings
about invalid implicit function declarations in some tests (which
only show up if you export IFFEFLAGS=-d1), but may also cause false
negative test results. The iffe tests should be given the same
environment that their test results are going to be used in.
libast's first-run and most central feature test, 'standards',
figures out what standards macros need to be used on the current
system to get the system headers to declare the expected
functionality. All code that links to libast depends on the header
generated by this feature test. So iffe should use this result for
the tests as well, as soon as it's available (which is early in
libast's compilation cycle).
Concrete example: on Cygwin, in src/cmd/builtin/features/pty, the
'lib ptsname' test detects ptsname(3) in the system library, but
the output{...}end block that uses the _lib_ptsname feature test
result throws an 'implicit function definition' warning because
Cygwin's stdlib.h does not define this function without the
appropriate standards macros being defined first.
src/cmd/INIT/iffe.sh:
- If ${INSTALLROOT}/src/lib/libast/FEATURE/standards is available,
incorporate it directly into iffe's own block of compiler
massaging macros. Do not use #include as the necessary -I flags
are not added for every test.
- Centrally define the iffe version once, not twice.
- Update and tweak getopts self-documentation (iffe --man).
- While we're here, add macOS/Darwin's DYLD_LIBRARY_PATH to the
supported dynamic library search variables. On macOS, this is
normally disabled by System Integrity Protection, plus this
distribution uses static libraries at build time, so this is for
completeness' sake and not much else. This was ported from
@lkujaw's fork: 48ff054429
(thanks to @JohnoKing for pointing it out).
src/lib/libast/features/standards:
- Add a comment. This is to update the file's timestamp, ensuring
that everything will be recompiled after this commit.
Tried to compile on Solaris 10.1 for the first time in a while.
Turns out the obsolete Bourne /bin/sh does not support 'test -e'.
bin/package, src/cmd/INIT/package.sh:
- Use 'test -f' instead.
Commit 92f7ca54 broke compilation with tcc on Linux. The following
error would occur while compiling ksh with tcc:
In file included from /home/johno/GitRepos/KornShell/ksh/src/cmd/ksh93/data/strdata.c:105:
./FEATURE/math:91: error: too many basic types
mamake [cmd/ksh93]: *** exit code 1 making strdata.o
The build failure is fixed by backporting the relevant bugfix from
the 93v- version of iffe.
src/cmd/INIT/iffe.sh:
- Backport the 2013 iffe bugfix for the intrinsic function test to
rule out type names (dated 2013-08-11 in the 93v- changelog).
The build started failing on Solaris Studio cc when 'noreturn' was
introduced, because the wrappers pass the -xc99 flag which sets the
compiler to C99 mode. 'noreturn' is a C11 feature. The
stdnoreturn.h header was correctly included but the compiler still
threw a syntax error (long path abbreviated below):
".../stk.c", line 124: warning: _Noreturn is a keyword in ISO C11
".../stk.c", line 124: warning: old-style declaration or incorrect
type for: _Noreturn
".../stk.c", line 124: syntax error before or at: static
src/cmd/INIT/cc.sol11.*:
- Pass -std=c11 to cc instead of -xc99. At least on i386-64, this
is sufficient to fix the build.
README.md, src/cmd/ksh93/README.md:
- Remove -xc99 from the Solaris build flags example as that is
incompatible with -std=c11 (and was already redundant with the
-xc99 in the wrappers).
src/cmd/ksh93/tests/basic.sh:
- Don't run a newly backported 93v- regression test on Solaris
because it uses the 'join' command with process subsitutions;
Solaris 11.4's join(1) hangs when trying to read from /dev/fd.
This is not ksh's fault. (re: 59bacfd4)
nmake was removed long ago (2940b3f5) and so were the outdated
Makefiles (6cc2f6a0). However, the build system still looked for an
AT&T nmake in $PATH. If a user had it installed, the build would
fail as the system tried to use it.
https://groups.google.com/g/korn-shell/c/2VK8kS0_VhA/m/-Rlnv7PRAgAJ
bin/package, src/cmd/INIT/package.sh:
- Remove all the code supporting nmake.
- Make 'bin/package test' work by simply exec'ing bin/shtests.
src/cmd/INIT/Mamfile:
- Do not install *.mk nmake support files.
lib/package/*.mk, src/cmd/INIT/*.mk:
- nmake support files removed.
This adds a #pragma to disable -Wdeprecated-register* on newer
versions of clang. We could remove all use of the register keyword
instead, as modern compilers ignore it. But it's not harmful, and
for the time being I prefer not to do doing any reformatting or
changing the historic character of this code base.
The #pragmas are removed from src/lib/libast/include/ast.h, because
they're better placed in src/lib/libast/features/common which
generates ast_common.h which is included by everything.
* https://clang.llvm.org/docs/DiagnosticsReference.html#wdeprecated-register
Noteworthy changes:
- The man pages have been updated to fix a ton of instances of
runaway underlining (this was done with `sed -i 's/\\f5/\\f3/g'`
commands). This commit dramatically increased in size because
of this change.
- The documentation for spawnveg(3) has been extended with
information about its usage of posix_spawn(3) and vfork(2).
- The documentation for tmfmt(3) has been updated with the changes
previously made to the man pages for the printf and date builtins
(though the latter builtin is disabled by default).
- The shell's tracked alias tree (hash table) is now documented in
the shell(3) man page.
- Removed the commented out regression test for an ERRNO variable
as the COMPATIBILITY file states it was removed in ksh93.
The changes in this commit allow ksh to be built and run with
ASan[*], although for now it only works under vmalloc. Example
command to build ksh with ASan:
$ bin/package make CCFLAGS='-O0 -g -fsanitize=address'
[*] https://en.wikipedia.org/wiki/AddressSanitizer
src/cmd/INIT/mamake.c:
- Fix a few memory leaks in mamake. This doesn't fix all of the
memory leaks ASan complains about (there is one remaining in the
view() function), but it's enough to get ksh to build under ASan.
src/lib/libast/features/map.c,
src/lib/libast/misc/glob.c:
- Rename the ast globbing functions to _ast_glob() and
_ast_globfree(). Without this change the globbing tests fail
under ASan. See: 2c49eb6e
src/cmd/ksh93/sh/{init,io,nvtree,subshell}.c:
- Fix buffer overflows by using strncmp(3) instead of memcmp(3).
src/cmd/ksh93/sh/name.c:
- Fix another invalid usage of memcmp by using strncmp instead.
This change is also in one of Red Hat's patches:
https://git.centos.org/rpms/ksh/blob/c8s/f/SOURCES/ksh-20120801-nv_open-memcmp.patch
Resolves: https://github.com/ksh93/ksh/issues/230
Many of these changes are minor typo fixes. The other changes
(which are mostly compiler warning fixes) are:
NEWS:
- The --globcasedetect shell option works on older Linux kernels
when used with FAT32/VFAT file systems, so remove the note about
it only working with 5.2+ kernels.
src/cmd/ksh93/COMPATIBILITY:
- Update the documentation on function scoping with an addition
from ksh93v- (this does apply to ksh93u+).
src/cmd/ksh93/edit/emacs.c:
- Check for '_AST_ksh_release', not 'AST_ksh_release'.
src/cmd/INIT/mamake.c,
src/cmd/INIT/ratz.c,
src/cmd/INIT/release.c,
src/cmd/builtin/pty.c:
- Add more uses of UNREACHABLE() and noreturn, this time for the
build system and pty.
src/cmd/builtin/pty.c,
src/cmd/builtin/array.c,
src/cmd/ksh93/sh/name.c,
src/cmd/ksh93/sh/nvtype.c,
src/cmd/ksh93/sh/suid_exec.c:
- Fix six -Wunused-variable warnings (the name.c nv_arrayptr()
fixes are also in ksh93v-).
- Remove the unused 'tableval' function to fix a -Wunused-function
warning.
src/cmd/ksh93/sh/lex.c:
- Remove unused 'SHOPT_DOS' code, which isn't enabled anywhere.
https://github.com/att/ast/issues/272#issuecomment-354363112
src/cmd/ksh93/bltins/misc.c,
src/cmd/ksh93/bltins/trap.c,
src/cmd/ksh93/bltins/typeset.c:
- Add dictionary generator function declarations for former
aliases that are now builtins (re: 1fbbeaa1, ef1621c1, 3ba4900e).
- For consistency with the rest of the codebase, use '(void)'
instead of '()' for print_cpu_times.
src/cmd/ksh93/sh/init.c,
src/lib/libast/path/pathshell.c:
- Move the otherwise unused EXE macro to pathshell() and only
search for 'sh.exe' on Windows.
src/cmd/ksh93/sh/xec.c,
src/lib/libast/include/ast.h:
- Add an empty definition for inline when compiling with C89.
This allows the timeval_to_double() function to be inlined.
src/cmd/ksh93/include/shlex.h:
- Remove the unused 'PIPESYM2' macro.
src/cmd/ksh93/tests/pty.sh:
- Add '# err_exit #' to count the regression test added in
commit 113a9392.
src/lib/libast/disc/sfdcdio.c:
- Move diordwr, dioread, diowrite and dioexcept behind
'#ifdef F_DIOINFO' to fix one -Wunused-variable warning and
multiple -Wunused-function warnings (sfdcdio() only uses these
functions when F_DIOINFO is defined).
src/lib/libast/string/fmtdev.c:
- Fix two -Wimplicit-function-declaration warnings on Linux by
including sys/sysmacros.h in fmtdev().
Renamed: src/cmd/INIT/cc.linux.i386 -> src/cmd/INIT/cc.linux
This ensures that architectures like ARM also use the default Linux
wrapper. This is needed because they may need -D_LARGEFILE64_SOURCE
to compile correctly.
On ARM processors, this fixes at least this regression:
io.sh[243]: long seek not working
Resolves: https://github.com/ksh93/ksh/issues/253
On some systems (such as Ubuntu on ARM), the output of `file`
contains a build hash, such as:
SomeExecutable: ELF 32-bit LSB shared object, ARM, EABI5
version 1 (SYSV), dynamically linked, interpreter
/lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0,
BuildID[sha1]=8934dd61657aac875c190535066466849687a56b,
not stripped
This build hash can contain the string '64', which caused package
to wrongly detect a 64-bit architecture.
bin/package, src/cmd/INIT/package.sh:
- Export LC_ALL=C to ensure 'file' output in English.
- To detect a 64-bit architecture, require the string "64-bit", "64
bit" or "64bit" in 'file' output. The letters 'i' and 't' cannot
occur in a hexadecimal hash, so hopefully that is safe enough. It
is impossible to make this method completely safe, so in the long
term it should be replaced.
Progresses: https://github.com/ksh93/ksh/issues/253
This commit adds '/* FALLTHROUGH */' comments to fix many
GCC warnings when compiling with -Wimplicit-fallthrough.
Additionally, the existing fallthrough comments have been
changed for consistency.
These are minor fixes I've accumulated over time. The following
changes are somewhat notable:
- Added a missing entry for 'typeset -s' to the man page.
- Add strftime(3) to the 'see also' section. This and the date(1)
addition are meant to add onto the documentation for 'printf %T'.
- Removed the man page the entry for ksh reading $PWD/.profile on
login. That feature was removed in commit aa7713c2.
- Added date(1) to the 'see also' section of the man page.
- Note that the 'hash' command can be used instead of 'alias -t' to
workaround one of the caveats listed in the man page.
- Use an 'out of memory' error message rather than 'out of space'
when memory allocation fails.
- Replaced backticks with quotes in some places for consistency.
- Added missing documentation for the %P date format.
- Added missing documentation for the printf %Q and %p formats
(backported from ksh2020: https://github.com/att/ast/pull/1032).
- The comments that show each builtin's options have been updated.
I've had ksh crash one too many times when returning to a previous
build directory as I forgot to restore the previously-used CCFLAGS.
bin/package, src/cmd/INIT/package.sh:
- Save each of CC, CCFLAGS, CCLDFLAGS, LDFLAGS, KSH_RELFLAGS on the
first build run. On subsequent runs, compare and refuse to run if
they changed, issuing an informative error message.
- Allow override by exporting FORCE_FLAGS. Don't tell anyone :)
The package script was not well behaved with these. When you
pressed Ctrl+C, on some shells (including ksh) both the SIGINT (2)
and EXIT (0) traps are activated, showing a double 'make done'
message. The exit status also wasn't > 128 to indicate a signal.
bin/package, src/cmd/INIT/package.sh:
- Be UNIXly well-behaved. Signals should be passed on after
handling, so when one is caught, make the trap handlers print
their message and then unset both itself and EXIT/0 before
resending the signal to self.
Sometimes the shell returned to the prompt before bin/package was
finished writing all of its output. The problem was that 'tee',
which is used to write the output to both the terminal and the log
in arch/*lib/package/gen/make.out, hadn't caught up yet.
bin/package, src/cmd/INIT/package.sh:
- Run the build itself in the background and 'tee' in the
foreground. This way, the script will not terminate until 'tee'
is finished. The build's exit status is obtained with 'wait'.
POSIX warns about "unset PWD" leading to unspecified behavior from
the pwd util, which we then use.
bin/package, src/cmd/INIT/package.sh:
- Determine if the shell has $PWD with a feature test. Only unset
PWD if it does not (the old Bourne shell).
- Clean up 'case' flow.
Co-authored-by: Martijn Dekker <martijn@inlv.org>
Many of the errors fixed in this commit are word repetitions
such as 'the the' and minor spelling errors. One formatting
error in the ksh man page has also been fixed.
Now that the Make Abstract Machine files are maintained manually
and not generated automatically, unused variables are an annoying
distraction -- and there are many.
But the language/format is very simple and very parseable using
shell, awk, etc. -- so this was easy to automate. All variables are
declared with 'setv' and they are used if an expansion of the form
${varname} exists (the braces are mandatory in Mamfiles).
bin/Mamfile_rm_unused_vars:
- Added for reference and future use.
src/*/*/Mamfile:
- Remove all unused 'setv' variable declarations.
src/cmd/INIT/iffe.sh:
- Fix "standard system directories" for the cmd test, which were
hardcoded as bin, /etc, /usr/bin, /usr/etc, /usr/ucb. That's both
unportable and antiquated. Replace this with the path output by
'getconf PATH'.
- Add fixes from modernish for 'getconf PATH' output to compensate
for bugs/shortcomigns in NixOS and AIX. Source:
9e4bf5eb/lib/modernish/aux/defpath.sh
Ref.: https://github.com/NixOS/nixpkgs/issues/65512
src/lib/libast/comp/conf.tab: PATH:
- Add the NixOS and AIX default path fixes here too; this fixes
'command -p' and the builtin 'getconf PATH' on these systems.
bin/package, src/cmd/INIT/package.sh:
- Re-support being launched with just the command name 'package' in
the command line (if the 'package' command is in $PATH). At least
one other script in the build system does this. (re: 6cc2f6a0)
- Go back three levels (../../..) if we were invoked from
arch/*/bin/package, otherwise we won't find src/cmd/ksh93/SHOPT.sh.
iffe feature test that add a -D_LARGEFILE64_SOURCE compiler flag to
detect the presence of 64-bit types like off64_t are very
incorrect; they always find the type even if the rest of the source
is not compiled with that flag, causing an inconsistent compilation
environment. This was the cause of mysterious failures to compile
some feature tests on Linux i386 -- it tried to use an off64_t type
that was wrongly detected.
A flag like -D_LARGEFILE64_SOURCE needs to be added to the compiler
flags consistently so it is used for compiling all files and tests.
src/lib/libast/features/dirent,
src/lib/libast/features/fs,
src/lib/libast/features/lib,
src/lib/libast/features/mmap,
src/cmd/ksh93/features/rlimits:
- Remove the -D_LARGEFILE64_SOURCE flag from all the tests that
used it.
- Fix some preprocessor directives for compiling without
_LARGEFILE64_SOURCE. We cannot rely on the result of the _lib_*64
tests because those functions are still found in glibc even if
_LARGEFILE64_SOURCE is not defined; we have to check for the
existence of the type definitions before using them.
src/cmd/INIT/cc.linux.i386,
src/cmd/INIT/cc.linux.i386-icc:
- Add/update compiler wrappers to hardcode -D_LARGEFILE64_SOURCE
in the flags for the default compiler. If it is overriden with
$CC, then it needs to be added manually if desired.
This takes another small step towards disentangling the build
system from the old AT&T environment. The USAGE_LICENSE macros with
author and copyright information, which was formerly generated
dynamically for each file from a database, are eliminated and the
copyright/author information is instead inserted into the AST
getopt usage strings directly.
Repetitive license/copyright information is also removed from the
getopt strings in the builtin commands (src/lib/libcmd/*.c and
src/cmd/ksh93/data/builtins.c). There's no need to include 55
identical license/copyright strings in the ksh binary; one (in the
main ksh getopt string, shown by ksh --man) ought to be enough!
This makes the ksh binary about 10k smaller.
It does mean that something like 'enum --author', 'typeset
--license' or 'shift --copyright' will now not show those notices
for those builtins, but I doubt anyone will care.
Commit 308696ec caused the build to fail on macOS Catalina.
src/cmd/INIT/iffe.sh:
- Fix a blatantly unportable practice of passing multiple
"|"-separated 'case' patterns through a variable. This was a way
of grepping for some headers including stdio.h, but it only works
this way on ksh93 and possibly the original Bourne shell, and not
on *any* other shell (not even pdksh or mksh) -- and the fact
that it works on ksh93 is arguably a bug. Fix by eliminating the
"noext" variable (which is init'ed once and never changes) and
using the pattern in the relevant 'case' statement directly.
src/cmd/builtin/features/pty:
- No matter what I try, including <stdio.h> causes the build to
fail on Gentoo Linux (i386) with mysterious "invalid identifier:
off64_t" errors -- this is probably some AST preprocessor hackery
gone awry, but I've no idea where to even begin with that. This
works around the problem by using AST sfio instead, which is
built and functional by the time this feature test is run.
- Remove explicit extern declaration for ptsname(2) that was never
used because it depended on an npt_ptsname feature test that
doesn't exist (or no longer exists).
- Add missing <fcntl.h>, <stdlib.h>, and <unistd.h> for open(2),
ptsname(2) and close(2), respectively.
src/lib/libast/features/float,
src/lib/libast/features/sfio,
src/lib/libast/features/stdio:
- Re-include <stdio.h>.
Fixes: https://github.com/ksh93/ksh/issues/164 (I hope)
This makes ksh build at least on AIX 7.1 on RISC (PowerPC).
There are 4 regression test failures:
leaks.sh[159]: memory leak on PATH reset before PATH search
(leaked approx 220 KiB after 16384 iterations)
pty.sh[351]: POSIX sh 104(C): line 364: expected
"^done\r?\n$", got EOF
signal.sh[280]: subshell ignoring signal does not send
signal to parent (expected 'SIGUSR1', got 'done')
signal.sh[282]: parent does not wait for child to complete
before handling signal
src/cmd/INIT/iffe.sh:
- Unset LIBPATH on AIX. The features/pty output{ ... }end will fail
to link to libiconv otherwise, causing a build failure. See:
https://www.ibm.com/support/pages/member-libiconvso2-not-found-archive
src/cmd/builtin/pty.c:
- CMIN is not defined on AIX, so set it to 1 if it's not defined.
src/cmd/ksh93/README:
- Update list of tested OSs.
