Building ksh with the tcc (tinycc) compiler failed as of glibc
commit 5d98a7da. The NEWS file in that commit adds:
+* When _DYNAMIC_STACK_SIZE_SOURCE or _GNU_SOURCE are defined,
+ PTHREAD_STACK_MIN is no longer constant and is redefined to
+ sysconf(_SC_THREAD_STACK_MIN).
The tcc build failure seeminly had nothing to do with that --
however, deleting the PTHREAD_STACK_MIN entry and its dependent
THREAD_STACK_MIN entry from conf.tab fixes the build failure.
For reasons unknown, gcc didn't have a problem with it. However,
these config identifiers aren't used anywhere in the ast codebase
(including the full ast-open-history repo) so it should be fine to
just get rid of them; ksh is not and will not be threaded.
NOTE: To build ksh with tcc, you need to build the latest tcc code
from <https://repo.or.cz/tinycc>. The tcc release packages in OS
distributions are typically too old and will not work.
Resolves: https://github.com/ksh93/ksh/issues/437
Thanks to @JohnoKing for the report.
This commit implements support for the glibc 2.35
posix_spawn_file_actions_addtcsetpgrp_np(3) extension[2][3],
updating spawnveg(3) to use the new function for setting the
terminal group. This was done with the intention of improving
performance in interactive shells without reintroducing previous
race conditions[4][5].
[1]: https://sourceware.org/pipermail/libc-alpha/2022-February/136040.html
[2]: https://sourceware.org/git/?p=glibc.git;a=commit;h=342cc934
[3]: https://sourceware.org/git/?p=glibc.git;a=commit;h=6289d28d
[4]: https://github.com/ksh93/ksh/issues/79
[5]: https://www.mail-archive.com/ast-developers@research.att.com/msg00717.html
src/cmd/ksh93/sh/path.c:
- Tell spawnveg(3) to set the terminal process group when launching
a child process in an interactive shell.
src/cmd/ksh93/sh/xec.c:
- If posix_spawn_file_actions_addtcsetpgrp_np(3) is available,
allow use of spawnveg(3) (via sh_ntfork()) even with job control
active.
- sh_ntfork(): Reimplement most of the SIGTSTP handling code
removed in commit 66c37202.
src/lib/libast/comp/spawnveg.c,
src/lib/libast/misc/procopen.c,
src/lib/libast/features/sys:
- Add support for posix_spawn_file_actions_addtcsetpgrp_np(3).
- Allow spawnveg to set the terminal process group when pgid == 0.
This was necessary to avoid race conditions when using the new
function.
src/lib/libast/features/lib:
- Detect posix_spawn_file_actions_addtcsetpgrp_np(3).
- Do not detect an OS spawnveg(3). With the API changes to spawnveg
in this pull request ksh probably can't use the OS's spawnveg
function anymore. (That's assuming anything else even provides a
spawnveg function to begin with, which is unlikely.)
src/lib/libast/features/api,
src/cmd/ksh93/include/defs.h:
- Bump libast version (20220101 => 20220201) due to the spawnveg(3)
API change.
src/lib/libast/man/spawnveg.3:
- Document the changes to spawnveg(3) in the corresponding man
page. Currently, it will only use the new tcfd argument if
posix_spawn_file_actions_addtcsetpgrp_np(3) is supported. This
could also be implemented for the fork(2) fallback, but for now
I've avoided changing that since actually using it in the fork
code would likely require a lot of hackery to avoid attempting
tcsetpgrp with vfork (the behavior of tcsetpgrp after vfork is
not portable) and would only benefit systems that don't have
posix_spawn and vfork (I can't recall any off the top of my head
that would fall under that category).
- Updated the man page to account for spawnveg's change in
behavior.
Co-authored-by: Martijn Dekker <martijn@inlv.org>
This commit applies various accumulated bugfixes:
- Applied some fixes for compiler warnings based off of the
following pull requests (with whitespace changes excluded to
avoid inflating the size of the diff):
https://github.com/att/ast/pull/281https://github.com/att/ast/pull/283https://github.com/att/ast/pull/303https://github.com/att/ast/pull/304
- clone_type(): Two separate variables in this function share the
same name. A bugfix from ksh93v- 2013-05-24 was backported to
avoid conflict issues.
- Backported a minor error message improvement from ksh2020 for
when the user attempts to run too many coprocesses.
- Backported a ksh2020 bugfix for a file descriptor leak:
58bc8b56
- Backported ksh2020 bugfixes for unused variable and pointless
assignment lint warnings:
47650fe0df209c0d5e417b00
- Applied a few minor improvements to libast from graphviz:
e7c03541969a7cde
- dtmemory(): Mark unused parameters with NOT_USED(). Based on:
6ac3ad99
- Applied a few documentation/comment tweaks for the NEWS file,
printf -v and spawnveg.
- Added a missing regression test for using the rm builtin's -f
option without additional arguments (this was fixed in
ksh93u+ 2012-02-14).
Opening the match stack with the STK_SMALL flag causes the stk code
to allocate memory in blocks of 64*sizeof(char*) instead of
1024*sizeof(char*). This caused a significant slowdown which was
exposed by the extglob.ksh module of shbench. Thanks to @JohnoKing
for noticing and reporting the problem.
src/lib/libast/regex/regcomp.c: regcomp():
- Remove STK_SMALL from the stkopen() option bit flags.
Resolves: https://github.com/ksh93/ksh/issues/440
Notable changes:
src/cmd/ksh93/*.c:
- Get rid of all the dtuserdata(FOO,&sh,1) calls backported in
cc492752. These set pointers to sh in Cdt objects. As of
b590a9f1, the code does not use any pointers to sh, so these are
superfluous.
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/sh.1:
- As of ksh 93l 2001-06-01, the -h/trackall option has no effect at
all, so trim its documentation.
src/lib/libast/man/stk.3,
src/lib/libast/man/stak.3:
- Correct the documentation on what the ST(A)K_SMALL option bit
actually does based on a reading of the code.
- Document the STK_NULL option bit.
README.md,
src/cmd/ksh93/README:
- Add a note that -fdiagnostics-color=always will break the build.
Ref.: https://github.com/ksh93/ksh/issues/379
src/lib/libast/Mamfile:
- Remove a 'rm -f astmath' command -- a file that is never created.
But on Cygwin this removes astmath.exe, which *is* used. As a
result, executing it failed on Cygwin, so the system incorrectly
detected that Cygwin needs -lm for math functions.
This commit implements support for POSIX_SPAWN_SETSID in
spawnveg(3). The fork/vfork fallback for spawnveg already attempts
to use setsid in the manner described by the man page, so the
posix_spawn implementation should also do so.
src/lib/libast/comp/spawnveg.c:
- Add support for POSIX_SPAWN_SETSID to the posix_spawn version of
spawnveg.
- Minor extra: Remove dead code that's never used (the
_lib_posix_spawn < 2 code block is inside of _lib_posix_spawn >
1, plus when it's manually enabled by changing the previous #if
condition you'll find it causes many regression test failures (at
least on OpenBSD)).
src/lib/libast/man/spawnveg.3:
- Document that spawnveg cannot make the new process a session
leader if the operating system doesn't support POSIX_SPAWN_SETSID
and the new process was created using posix_spawn.
On my machine, the build system has been intermittently rebuilding
a sizeable part of libast for no apparent reason. I think I've
finally tracked down the cause: occasionally, the lctab.c file,
generated by port/lcgen.c, randomly changes, triggering said
recompile.
Diff of the latest instance on my system:
--- lctab.c.save 2022-01-28 03:22:47.000000000 +0000
+++ arch/darwin.i386-64/src/lib/libast/lctab.c 2022-01-28 03:30:01.000000000 +0000
@@ -2146,7 +2146,7 @@
#endif
0,0,0,
},
-{"no","norway",LC_primary,
+{"no","norway",0,
#ifdef CTRY_NORWAY
CTRY_NORWAY,
#else
In the port/lc.tab input file, "norway" does *not* have the
"primary" tag, unlike e.g. "sweden" or "united-kingdom". So that
LC_primary value did not belong in the generated code.
A look at the port/lcgen.c code shows that it's using uninitialised
memory. The newof() macro uses malloc, which does not initialise
the memory blocks it allocates:
131:#define newof(p,t,n,x) ((t*)malloc(sizeof(t)*(n)+(x)))
This is then used as follows:
403: case TERRITORY:
404: if (!(tp = newof(0, Territory_t, 1, s - b + 1)))
[...]
444: if (!strcmp(b, "primary"))
445: tp->primary = 1;
The flag is not set to zero if the string does not match, so it's
left uninitialised. Perhaps there are more such problems, but
instead of spending time trying to find them, I'll fix newof().
src/lib/libast/port/lcgen.c:
- In the newof() macro, call calloc(3) instead of malloc(3),
ensuring that all allocated memory is initialised to zero.
posix_spawn(2) was never used as a result of this error as the test
failed to compile, with most systems falling back to vfork(2).
src/lib/libast/features/lib: tst lib_posix_spawn:
- Fix parentheses goof.
Notable changes:
src/cmd/ksh93/include/fault.h:
- Get rid of the superflous sh pointer argument in the
sh_pushcontext() and sh_popcontext() macros. (re: 2d3ec8b6)
src/cmd/ksh93/tests/io.sh:
- Tweak a process substitution test to allow up to a second for
unused process substitution processes to disappear. On the Alpine
Linux console (at least the musl libc version), this is needed to
avoid a test failure as long as no GUI is active. As soon as you
start X11, this phenomenon disappears, even on the console. Very
strange, but also very probably not ksh's fault.
src/cmd/ksh93/tests/shtests:
- Instead of just SIGCONT and SIGPIPE, set all signals to default,
just to be sure to avoid spurious test failures due to signals
that were ignored on entry. (It made no difference to the
aforementioned Alpine Linux test failure, so ignored signals had
nothing to do with that -- but still a good idea.)
.github/workflows/ci.yml:
- On the GitHub CI runs, when testing with SHOPTs disabled, disable
SHOPT_SPAWN as well, which tests that everything still works
correctly with the regular fork(2) method.
COPYRIGHT:
- Remove duplicate of BSD license.
FreeBSD defines an SF_SYNC macro in sys/socket.h that conflicts
with sfio's SF_SYNC discipline, at best rendering it ineffective.
src/lib/libast/sfio/sfhdr.h:
- Temporarily undef __BSD_VISIBLE while including <sys/socket.h>
to hide the BSD extension with the conflicting definition.
src/lib/libast/features/standards:
- Do not emit #defines for the typ u_long test which is only used
as a heuristic in subsequent tests in this file. (Note that 'set'
can set and unset any iffe command-line --option at runtime.)
- Remove definition of _ISOC99_SOURCE macro. This is another old
GNU thing; feature_test_macros(7) says invoking the compiler with
the option -std=c99 has the same effect. But modern GCC has C11
with GNU extensions as the default, which is fine. If a
particular standard is desired, pass a -std=... flag in $CC.
src/cmd/ksh93/features/rlimits:
- Remove overlooked Linux *64* types/functions hackery.
After defining standards macros it caused a build failure
on at least one version of Void Linux (but not 5.15.14_1).
Thanks to @JohnoKing for the report.
src/cmd/ksh93/sh/subshell.c,
src/lib/libdll/dllnext.c:
- Remove now-redundant local definitions of _GNU_SOURCE and
__EXTENSIONS__ macros.
src/cmd/ksh93/tests/builtins.sh:
- Fix broken sed invocation (re: 41829efa).
The more notable ones are:
src/lib/libast/features/standards:
- Do not redefine _GNU_SOURCE and _FILE_OFFSET_BITS if already
defined from $CCFLAGS. Thanks to @hyanias for the heads-up.
(re: 289dd46c)
src/cmd/ksh93/data/builtins.c,
src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/sh/args.c,
src/cmd/ksh93/sh/name.c:
- Remove -T test code activation option. It was basically unused.
The only thing it did was intentionally introduce a memory leak
in table_unset() if the 4th bit in the option argument was set.
A search in ast-open-history reveals a few more trivial test uses
that were later deleted, but nothing interesting.
src/cmd/ksh93/tests/{basic,path}.sh:
- Skip a couple of tests on AIX avoid hangs, at least one of which
is not ksh's fault. Thanks to @HansH111 for the report.
src/cmd/ksh93/tests/builtins.sh:
- Change one awk use to a more portable sed invocation to placate
systems with ancient awk commands, such as AIX. (re: de795e1f)
Turns out that the standards macros set by features/standards (such
as _GNU_SOURCE for Linux or _DARWIN_SOURCE for macOS) were still
*not* included for most C source files! Instead, they were
selectively included for some files only, sometimes via
FEATURE/standards (the output of features/standards), sometimes
via ast_standards.h which is copied from FEATURE/standards.
Consequently, there were still inconsistencies in the system header
interfaces exposed on Linux, macOS, Solaris, et al. It's no wonder
it sometimes took so much hackery to keep everything building.
Of course, making this consistent had to break things somewhere.
Breakage occurred on 32-bit Linux due to a lot of ugly hackery
involving direct use of internal GNU types like off64_t and
functions like fseek64(). This is now all removed and they are
activated by setting the appropriate feature macro instead, so
these types and functions can be used with their standard names
(off_t, fseek, etc.)
Before committing I've tested these changes on the following
i386/x86_64 systems: Linux (glibc 32 and 64 bit, musl libc 64 bit),
Solaris (32 and 64 bit), illumos (32 and 64 bit), FreeBSD (64 bit),
macOS (64 bit), Cygwin (32 bit), and Haiku (64 bit).
(Note: ast_standards.h is copied from FEATURE/standards, whereas
ast_common.h is copied from FEATURE/common.)
src/lib/libast/include/ast_std.h,
src/lib/libast/stdio/stdhdr.h:
- Include <ast_standards.h> first. This should cause all the AST
and dependent code (such as ksh) to get the standards macros.
src/lib/libast/features/standards:
- For GNU (glibc), #define _FILE_OFFSET_BITS 64 to get large file
support with 64-bit offsets.
- Stop GNU and Cygwin <string.h> form defining the GNU version of
basename(3); on Cygwin, that declaration conflicts with the AST
version (and with POSIX) by using a const char* argument instead
of char*. It is deactivated by defining the macro 'basename' (as
'basename'); this causes GNU string.h to consider it to be
already defined by the standard libgen.h header.
All other changed files:
- Remove direct use of *64* types and functions and a lot of
related hackery.
Commit 24fc1bbc broke the build on Cygwin in comp/setlocale.c by no
longer defining _GNU_SOURCE on that system in features/standards.
This caused wcwidth() to be hidden by wchar.h though it was
detected in the libraries.
src/lib/libast/features/standards:
- Detect Cygwin along with GNU as a system on which to define
_GNU_SOURCE.
- Add wcwidth() compilation as an extra heuristic to the BSD,
SunOS, Darwin and GNU/Cygwin tests. (Since it's specified as an
optional (X/Open) feature, it should not be tested for in the
generic fallbacks.)
These are minor things I accumulated over the last month or so.
Notable changes:
src/lib/libast/features/api,
src/lib/libast/misc/state.c,
src/lib/libast/comp/conf.tab,
src/cmd/ksh93/include/defs.h:
- Bump internal libast version to 20220101L. We've made a few
additions to the API, at least pathicase (see 71934570, ca3ec200)
and astconf_long (see c2ac69b2), so this should have been done
already. This also updates '/opt/ast/bin/getconf _AST_VERSION'.
- Use AST_VERSION instead of outdated _AST_VERSION.
- In state.c, use AST_VERSION instead of hardcoding the version.
src/cmd/ksh93/sh/xec.c:
- Remove 'restorefd' variable, unused as of 42becab6.
- Remove 'cmdrecurse' function and SH_RUNPROG macro; this was once
used by a few libcmd commands, but ast-open-archive reveals it's
unused as of ast 1999-12-25.
src/cmd/ksh93/sh/*.c:
- Where available, use e_dot instead of "." for consistency; it is
defined as an extern so we might as well use it.
src/cmd/ksh93/tests/*.sh:
- When reporting signal names in fails, include the SIG prefix.
- Fix a broken process hang test in subshell.sh.
src/lib/libast/man/sfdisc.3:
- Removed. The interfaces described here never made it out of AT&T;
they do not exist in any libast version in ast-open-archive.
Resolves: https://github.com/ksh93/ksh/issues/426
This commit makes various different improvements to the documentation:
- sh.1: Backported (with changes) mandoc warning fixes from ksh2020
for the ksh93(1) man page: <https://github.com/att/ast/pull/1406>
- Removed unnecessary spaces at the end of lines to fix a few other
mandoc warnings.
- Fixed various typos and capitalization errors in the documentation.
- ANNOUNCE: Document the addition of the ${.sh.pid} variable
(re: 9de65210).
- libast/man/str*: Update the man pages for the libast str* functions
to improve how accurately each function is described.
- ksh93/README: Update regression test/compatibility notes to include
OpenBSD 7.0, FreeBSD 13.0 and WSL running Ubuntu 20.04.
- Change a few places to store the return value from strlen in a
size_t variable rather than signed int.
- comp/setlocale.c: To avoid confusion of two separate variables named
lang, the function local variable has been renamed to langidx.
This commit implements the build fixes required to get ksh running on
Haiku. Note that while ksh does compile, it has a ton of regression test
failures on Haiku.
src/cmd/ksh93/data/signals.c,
src/lib/libast/features/signal.c:
- Add support for the SIGKILLTHR signal, which is supported by BeOS and
Haiku.
- SIGINFO was missing an entry in the libast feature test, so add one
(re: 658bba74).
src/cmd/ksh93/RELEASE:
- Add an entry noting that ksh now compiles on Haiku, albeit with many
regression test failures.
src/cmd/ksh93/{include/terminal.h,sh/path.c}:
- Silence compiler warnings on Haiku.
src/lib/libast/features/mmap:
- The mmap feature test freezes on Haiku, so modify the test to fail
immediately on that OS.
src/lib/libast/misc/signal.c:
- Avoid redefining the signal definition on Haiku to fix a compiler
error.
src/lib/libast/features/nl_types:
- For some reason the nl_item typedef on Haiku doesn't work correctly.
Work around that by creating the nl_item type in the libast nl_types
feature test.
This combines 20 cleanup commits from the dev branch.
All changed files:
- Clean up pointer defererences to sh.
- Remove shp arguments from functions.
Other notable changes:
src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/sh/init.c:
- On second thought, get rid of the function version of
sh_getinterp() as libshell ABI compatibility is moot. We've
already been breaking that by reordering the sh struct, so there
is no way it's going to work without recompiling.
src/cmd/ksh93/sh/name.c:
- De-obfuscate the relationship between nv_scan() and scanfilter().
The former just calls the latter as a static function, there's no
need to do that via a function pointer and void* type conversions.
src/cmd/ksh93/bltins/typeset.c,
src/cmd/ksh93/sh/name.c,
src/cmd/ksh93/sh/nvdisc.c:
- 'struct adata' and 'struct tdata', defined as local struct types
in these files, need to have their first three fields in common,
the first being a pointer to sh. This is because scanfilter() in
name.c accesses these fields via a type conversion. So the sh
field needed to be removed in all three at the same time.
TODO: de-obfuscate: good practice definition via a header file.
src/cmd/ksh93/sh/path.c:
- Naming consistency: reserve the path_ function name prefix for
externs and rename statics with that prefix.
- The default path was sometimes referred to as the standard path.
To use one term, rename std_path to defpath and onstdpath() to
ondefpath().
- De-obfuscate SHOPT_PFSH conditional code by only calling
pf_execve() (was path_pfexecve()) if that is compiled in.
src/cmd/ksh93/include/streval.h,
src/cmd/ksh93/sh/streval.c:
- Rename extern strval() to arith_strval() for consistency.
src/cmd/ksh93/sh/string.c:
- Remove outdated/incorrect isxdigit() fallback; '#ifnded isxdigit'
is not a correct test as isxdigit() is specified as a function.
Plus, it's part of C89/C90 which we now require. (re: ac8991e5)
src/cmd/ksh93/sh/suid_exec.c:
- Replace an incorrect reference to shgd->current_pid with
getpid(); it cannot work as (contrary to its misleading directory
placement) suid_exec is an independent libast program with no
link to ksh or libshell at all. However, no one noticed because
this was in fallback code for ancient systems without
setreuid(2). Since that standard function was specified in POSIX
Issue 4 Version 2 from 1994, we should remove that fallback code
sometime as part of another obsolete code cleanup operation to
avoid further bit rot. (re: 843b546c)
src/cmd/ksh93/bltins/print.c: genformat():
- Remove preformat[] which was always empty and had no effect.
src/cmd/ksh93/shell.3:
- Minor copy-edit.
- Remove documentation for nonexistent sh.infile_name. A search
through ast-open-archive[*] reveals this never existed at all.
- Document sh.savexit (== $?).
src/cmd/ksh93/shell.3,
src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/sh/init.c:
- Remove sh.gd/shgd; this is now unused and was never documented
or exposed in the shell.h public interface.
- sh_sigcheck() was documented in shell.3 as taking no arguments
whereas in the actual code it took a shp argument. I decided to
go with the documentation.
- That leaves sh_parse() as the only documented function that still
takes an shp argument. I'm just going to go ahead and remove it
for consistency, reverting sh_parse() to its pre-2003 spec.
- Remove undocumented/unused sh_bltin_tree() function which simply
returned sh.bltin_tree.
- Bump SH_VERSION to 20220106.
(nmake makefiles) defined this macro:
__OBSOLETE__ == $("6 months ago":@F=%(%Y0101)T)
This was used to automatically disable code after a period between
6 and 18 months, on 1st Jan of each year, in preprocessor
directives like:
#if __OBSOLETE__ < 20080101
// obsolete code here
#endif
However, when compiling without nmake (as we do), this __OBSOLETE__
macro is not defined at all. And undefined macros evaluate to zero
in arithmetic comparisons, so all that obsolete code has been
getting compiled. Thankfully it doesn't seem to have done any harm,
but all that code was supposed to expire between 2008 and 2014.
src/lib/libast/disc/sfstrtmp.c:
- Removed. Was supposed to be a stub #if __OBSOLETE__ >= 20070101.
src/lib/libast/include/ast.h:
- Remove unused fmtbasell() macro (/* until 2014-01-01 */).
Other changed files:
- Remove __OBSOLETE__d code.
After further examining the iffe code I found that fail{ ... }end
as well as pass{ ... }end blocks are executed in iffe's current
environment, using a simple 'eval', with no safeguards whatsoever!
This does of course afford maximum flexibility... for example, a
block can 'exit 1' to abort the iffe run and the whole build with
it. We can use this to abort properly on fatal compilation errors.
src/cmd/INIT/iffe.sh:
- Complete the fail{ and pass{ documentation; it should really be
known that they run in iffe's current environment.
- Make one change: for just the 'eval' command that runs these
blocks, redirect standard error back to the saved $stderr file
descriptor so these blocks can write error messages using the
standard >&2 instead of the undocumented >&$stderr.
src/lib/**/features/*:
- Write error message to standard error and error out properly when
an output{ ... }end block fails to compile, instead of writing an
#error directive to error out later.
That intermittent regression test failure in types.sh seems to be
gone. So let's reimport the regex changes into the 1.0 branch to
subject them to wider testing and make sure any failures stay gone.
(re: 48568476, 38aab428, 1aa8f771)
[Original commit message from 1aa8f771 follows]
There are two main changes:
1. The regex code now creates and uses its own stack (env->mst)
instead of using the shared standard stack (stkstd). That seems
likely to be a good thing.
2. Missing mbinit() calls were inserted. The 93v- code uses a
completely different multibyte characters API, so these needed
to be translated back to the older API. But, as mbinit() is no
longer a no-op as of 300cd199, these calls do stop things from
breaking if a previous operation is interrupted mid-character.
I think there might be a couple of off-by-one errors fixed as well,
as there are two instances of this change:
- while ((index += skip[buf[index]]) < mid);
+ while (index < mid)
+ index += skip[buf[index]];
I'd forgotten to check for uses of the __STDC__ macro. This is
defined on all C compilers that support C89/C90 or later standards.
So we can remove all fallback code disabled by that macro.
This takes another step towards cleaning up the build system. We
now do not even pretend to be theoretically compatible with
pre-1989 K&R C compilers or with C++ compilers. In practice, this
had already been broken for many years due to bit rot.
Commit 46593a89 already removed the license handling enormity that
depended on proto, so now we can cleanly remove it altogether. But
we do need to leave some backwards compatibility stubs to keep the
build system compatible with older AST code; it should remain
possible to build older ksh versions with the current build system
(the bin/ and src/cmd/INIT/ directories) for testing purposes.
So as of now there is no more __MANGLE__d rubbish in your generated
header files. This is only about a quarter of a century overdue...
This commit also includes a huge amount of code cleanup to remove
thousands of unused K&R C fallbacks and other cruft, particularly
in libast. This code base should now be a little easier to
understand for people who are familiar with a modern(ish) C
standard.
ratz is now also removed; this was a standalone and simplified 2005
version of gunzip. As of 6137b99a, none of our code uses it, even
theoretically. And the real g(un)zip is now everywhere.
src/cmd/INIT/proto.c, src/cmd/INIT/ratz.c:
- Removed.
COPYRIGHT:
- Remove zlib license; this only applied to ratz.
bin/package, src/cmd/INIT/package.sh:
- Related cleanups.
- Unset LC_ALL before invoking a new shell, respecting the user's
locale again and avoiding multibyte character corruption on the
command line.
src/cmd/INIT/proto.sh:
- Add stub for backwards compatibility with Mamfiles that depend on
proto. It does nothing but pass input without modification and is
now installed as the new arch/*/bin/proto by src/cmd/INIT/Mamfile.
src/cmd/INIT/iffe.sh:
- Ignore the proto-related -e (--package) and -p (--prototyped)
options; keep parsing them for backwards compatibility.
- Trim the macros passed to every test to their standard C
versions, removing K&R C and C++ versions. These are now
considered to be for backwards compatibility only.
src/cmd/INIT/iffe.tst:
- Remove proto(1) mangling code.
By the way, iffe can be regression-tested as follows:
$ bin/package use # set up environment in a child shell
$ regress src/cmd/INIT/iffe.tst
$ exit # leave package environment
src/cmd/INIT/make.probe, src/cmd/INIT/probe.win32:
- Remove code to handle C++.
src/lib/libast/features/common:
- As in iffe.sh above, trim macros designed for compatibility with
C++ and ancient C compilers to their standard C versions and
comment that they are for backwards compatibility with AST code.
This is needed to keep all the old ast and ksh code compiling.
src/cmd/ksh93/sh/init.c,
src/cmd/ksh93/sh/name.c:
- Clarify libshell ABI compatibility function versions of macros.
A "proto workaround" comment in the original code mislead me into
thinking this had something to do with the removed proto(1), but
it's unrelated. Call the workaround macro BYPASS_MACRO instead.
src/cmd/ksh93/include/defs.h:
- sh_sigcheck() macro: allow &sh as an argument: parenthesise shp.
src/cmd/ksh93/sh/nvtype.c:
- Remove unused nv_mkstruct() function. (re: d0a5cab1)
**/features/*:
- Remove obsolete iffe 'set prototyped' option.
**/Mamfile:
- Remove all references to the ast/prototyped.h header.
- Remove all use of the proto command. Simply copy instead.
*** 850-ish source files: ***
- Remove all '#pragma prototyped' directives.
- Remove all C++ compat code conditional upon defined(__cplusplus).
- Remove all use of the _ARG_ macro, which on standard C expands to
its argument:
#define _ARG_(x) x
(on K&R C, it expanded to nothing)
- Remove all use of _BEGIN_EXTERNS_ and _END_EXTERNS_ macros (empty
on standard C; this was for C++ compatibility)
- Reduce all #if __STD_C (standard code) #else (K&R code) #endif
blocks to the standard code only, without use of the macro.
- Same for _STD_ macro which seems to have had the same function.
- Change all instances of 'Void_t' to standard 'void'.
- Redirect error output from the ulimit builtin (re: 3e58851f).
- Fix the test failure for 'cd -eP' on illumos by making a directory
symlink first, then removing the symlink after cd.
- Fix the test failure for 'getconf -l' on illumos by quoting
strings with the -q option.
- astconf.c: Only quote strings if the -q option was passed.
- Improve error messages from intermittently failing types.sh tests
The goal is to get rid of all compiler/linker wrapper scripts as
they are overridden by passing CC/LD and it should be possible to
select your compiler or linker without breaking the build. The
probing and feature testing system should set the appropriate flags
and macros. This makes some progress towards that.
src/lib/libast/features/standards:
- Eliminate the shotgun approach to standards macros on popular
systems where the macros we we need to set are known and
documented. The following will enable standards compliance plus
all the available extensions:
- Set no macros at all for any BSD system (excluding macOS).
- Set _DARWIN_C_SOURCE on Darwin/macOS.
- Set everything and the kitchen sink for Solaris/illumos in
a way that enables backwards compatibility with older Solaris.
This is unofficial, but following the standards(5) manual
disables a lot of basic functionality that we depend on.
- Set _GNU_SOURCE for GNU (glibc).
- Remove the covered macros from the shotgun approach fallback.
- Add a new heuristic. _POSIX_PATH_MAX and _SC_PAGESIZE are among
the basic macros disabled when you pass recommended standards
macros, killing the build, so it's good to check if they compile.
src/cmd/INIT/ar.freebsd12.amd64,
src/cmd/INIT/ar.linux.i386-64:
- Removed. May cause build failures on some systems as not all 'ar'
implementations support the U option. Plus, I can think of no
good reason to disable deterministic mode (which always creates
identical output) on 'ar' implementations that support it. See:
https://groups.google.com/g/comp.unix.shell/c/LdOD1Ya0C9E/m/U6DhgHVICwAJ
src/cmd/INIT/cc.linux.*-icc,
Removed icc wrappers. These manually source /etc/profile.d/icc.sh
but I don't think that is the build system's job. Profile scripts
should be run at login time and export variables we inherit
through the environment.
src/cmd/INIT.cc.{freebsd,linux,openbsd}*:
- Removed. Should be entirely superfluous now that the standards
feature test sets the appropriate macros.
src/cmd/INIT.cc.sol11.*:
- Removed as the standards feature test now sets the approopriate
macros. Note the Solaris build system should now simply pass CC
as normal instead of passing CC_EXPLICIT.
Welcome to AT&T engineering practices in action: a fix in one thing
breaks a completely unreleated thing, but only in very specific
and inscrutable circumstances.
Commit ffe84ee7 introduced a regression test failure in types.sh:
test types begins at 2021-12-14+23:57:35
types.sh[130]: z.r.s should be z.r.x
test types failed at 2021-12-14+23:57:35 with exit code 1 [ 86 tests 1 error ]
test types(C.UTF-8) begins at 2021-12-14+23:57:35
test types(C.UTF-8) passed at 2021-12-14+23:57:35 [ 86 tests 0 errors ]
test types(shcomp) begins at 2021-12-14+23:57:35
test types(shcomp) passed at 2021-12-14+23:57:35 [ 86 tests 0 errors ]
Only enough, I've *only* found this regression on the GitHub CI
runner. I've tried it on three different regular Linux systems and
it occurs on none of them, nor on macOS.
Another odd thing: it only fails on the first of those three test
runs. But my experiments show it fails very consistently.
Through a process of systematic elimination in a test branch, I've
found that the failure is triggered by the change to using a
separate stack in the regex code. All the other changes are fine.
Using a separate stack improves the robustness of the regex code,
but it apparently exposes some breakage in how the very dodgy
'typeset -T' code is handling the stack, which was being masked by
sharing a stack with it. Or at least that seems like the most
plausible explanation to me right now.
So, until that breakage can be traced and fixed, the regex code now
shares the main stack with everything else again for the time being.
_____
Just to record this: by adding a couple of debug lines:
typeset -p z | sed 's/^/[DEBUG] /'
printf '[DEBUG] %s\n' "${z.r.s}" "${z.r.x}"
the symptom reveals itself more clearly on the GitHub runner:
test types begins at 2021-12-15+17:25:57
[DEBUG] Y_t z=(X_t r=(x=foo;y=bam;s=''))
[DEBUG]
[DEBUG] foo
types.sh[132]: z.r.s should be z.r.x
test types failed at 2021-12-15+17:25:57 with exit code 1 [ 86 tests 1 error ]
test types(C.UTF-8) begins at 2021-12-15+17:25:57
[DEBUG] Y_t z=(X_t r=(x=foo;y=bam;s=foo))
[DEBUG] foo
[DEBUG] foo
test types(C.UTF-8) passed at 2021-12-15+17:25:57 [ 86 tests 0 errors ]
test types(shcomp) begins at 2021-12-15+17:25:57
[DEBUG] Y_t z=(X_t r=(x=foo;y=bam;s=foo))
[DEBUG] foo
[DEBUG] foo
test types(shcomp) passed at 2021-12-15+17:25:57 [ 86 tests 0 errors ]
There are two main changes:
1. The regex code now creates and uses its own stack (env->mst)
instead of using the shared standard stack (stkstd). That seems
likely to be a good thing.
2. Missing mbinit() calls were inserted. The 93v- code uses a
completely different multibyte characters API, so these needed
to be translated back to the older API. But, as mbinit() is no
longer a no-op as of 300cd199, these calls do stop things from
breaking if a previous operation is interrupted mid-character.
I think there might be a couple of off-by-one errors fixed as well,
as there are two instances of this change:
- while ((index += skip[buf[index]]) < mid);
+ while (index < mid)
+ index += skip[buf[index]];
This bug was first reported at <https://www.illumos.org/issues/3782>.
The chown builtin when used on illumos can fail with different error
messages after running the same command twice:
$ touch /tmp/x
$ /opt/ast/bin/chown -h 433:434 /tmp/px
chown: /tmp/x: cannot change owner and group [Not owner]
$ /opt/ast/bin/chown -h 433:434 /tmp/px
chown: /tmp/x: cannot change owner and group [Invalid argument]
The error messages differ because the libast struid and strgid
functions will return -2 if the same nonexistent ID is used twice.
The fix for this bug has been ported from here:
4162633a7c
src/lib/libcmd/chgrp.c:
- Remove NOID macro and check for a < 0 error status instead.
This is different from the Illumos fix at
<4162633a7c>
which added another macro.
src/lib/libast/man/{strgid,struid}.3:
- Correct errors in the strgid and struid documentation.
- Document that the strgid and struid functions will return -2 if
the same invalid name is used twice.
Co-authored-by: Martijn Dekker <martijn@inlv.org>
The standards macros consistency fix for iffe exposed breakage on
illumos: the standards flags aren't set properly. Back in 580ff616,
I set _XPG6 from features/common, which is the wrong place; the
correct place is features/standards -- especially now that iffe
uses its results.
In addition, to get header declarations that aren't somehow in
conflict with themselves on illumos, don't result in "implicit
function declaration" warnings, and expose all the functionality,
we need to define *all* the _XPG[4-7] macros *and* __EXTENSIONS__
*and* _XOPEN_SOURCE. Welp. Thankfully, that's just fine with
Solaris, too.
Thanks to @JohnoKing for the heads-up.
I'm now taking another small step towards extricating this build
system from the long-dead AT&T AST universe.
This commit modifies/reduces the tool called proto. AT&T used proto
for two purposes:
1. To convert ANSI C code to a form compatible with ancient
(pre-ANSI) K&R C compilers using extremely complex macro
voodo. It was similarly capable of translating to C++.
Theoretically, this entire code base should compile on
anything from a 1980s K&R C compiler to a modern C++ compiler.
In practice, given the massive amount of bit rot we inherited,
I am 99.9% sure that this has been broken for many years.
2. To automagically insert license comments into source files
based on an extremely complicated license database system.
(In all-too-typical AT&T fashion, this second function of
proto is completely unrelated to the first.)
Function 2 has now been removed because, unlike the AT&T legal
department, I don't think it's worth going to unspeakably extreme
lengths to avoid maintaining license information in source code
files by hand.
In the process, proto.c was cleaned up to look halfway like actual
C code, but it's still processed code: most macros have been
expanded to their numeric value, all comments were stripped, etc.
So don't expect to understand this code. The actual source code is
in these two directories in the ast-open-history repo:
https://github.com/ksh93/ast-open-history/tree/master/src/cmd/protohttps://github.com/ksh93/ast-open-history/tree/master/src/lib/libpp
Meanwhile, nobody wants to compile ksh with a pre-ANSI K&R C
compiler in 2021 -- and there's no good reason to be compatible
with C++ because standard C compilers are universally available.
So, proto will go away when I manage to figure out how to pry it
loose from the innards of this build system.
src/lib/libast/port/astlicense.c:
- Removed. This is al the license handling code that was
incorporated in proto.c in stripped form. It was not used
anywhere else, and the environment where it was useful is gone.
src/cmd/INIT/proto.c:
- Cleanup to make this halfway maintainable: indentation, huge
blocks of empty lines, #line directives, etc.
- Delete all the code corresponding to astlicense.c. This was
actually easy as it was in a discrete block.
- proto(), pppopen(): Remove 'license'/'notice' and 'options'
arguments.
- main(): Remove processing of -l (license) and -o (license
options) flags.
**/Mamfile:
- Update all the proto invocations to remove the -l and -o flags.
bin/package, src/cmd/INIT/package.sh:
- Delete the 'copyright' command, which used the -l and -o
options to tell proto to extract copyright information from
*.lic/*.def files in lib/package.
COPYRIGHT:
- Added. This has the information from 'bin/package copyright', with
the copyright years corrected to plausible values as the AST code
used the current year (2021) for all of them. It adds ksh 93u+m
copyright and contributor information at the top as well.
(Yes, some of the lines in the old non-AT&T copyright notices
are clipped. This is the actual output of the 'bin/package
copyright' command as generated by 'proto' in the AST
distribution. For all that extreme complexity, they couldn't even
reproduce the notices correctly. But it's officially sanctioned
by AT&T in exactly this form, so there you have it.)
lib/package/**:
- Removed. All these files are now obsolete and redundant.
This commit fixes an issue with how ksh was obtaining the value of
NGROUPS_MAX. On some systems this setting can be changed (e.g., on
illumos adding 'set ngroups_max=32' to /etc/system then rebooting
changes NGROUPS_MAX from 16 to 32). Ksh was using NGROUPS_MAX with
the assumption it's a static value, which could cause issues on
systems where it isn't static. This bugfix is inspired by the one
from <b1362c3a5>, although it
has been expanded a bit to account for OPEN_MAX as well.
src/cmd/ksh93/sh/init.c, src/lib/libcmd/fds.c:
- Rename the getconf() macro to astconf_long() and move it to ast.h
to prevent redundancy. Other sections of the code have been
modified to use this macro for astconf() to account for
dynamic settings.
- An equivalent macro for unsigned long values (astconf_ulong) has
been added.
- Prefer sysconf(3) where available. It has better performance as it
returns a numeric value directly instead of via string
conversion.
- The astconf_long and astconf_ulong macros have been documented in
the ast(3) man page.
Turns out there is a bona fide, honest-to-goodness use case for
matching '.' and '..' in globbing after all. It's when globbing is
used as the backend mechanism for file name completion in
interactive shell editors. A tab invisibly adds a * at the end of
the word to the left of your cursor and the resulting pattern is
expanded. In 5312a59d, this broke for '.' and '..'.
Typing '.' followed by two tabs should result in a menu that
includes './' and '../'. Typing '..' followed by a tab should
result in '../', (or a menu that includes it if there are files
with names starting with '..'). This is the behaviour in 93u+ and
we should maintain this.
To restore this functionality without reintroducing the harmful
behaviour fixed in the referenced commits, we should special-case
this, allowing '.' and '..' to match only for file name completion.
src/lib/libast/include/glob.h:
- Fix an inaccurate comment: the GLOB_COMPLETE flag is used for
command completion, not file name completion. This is very clear
from reading the path_expand() function in sh/expand.c.
- Add new GLOB_FCOMPLETE flag for file name completion.
src/lib/libast/misc/glob.c:
- Adapt flags mask to fit the new flag.
- glob_dir(): If GLOB_FCOMPLETE is passed, allow '.' and '..' to
match even if expanded from a pattern.
- Clarify the fix from aad74597 with an extended comment based on
<https://github.com/ksh93/ksh/issues/146#issuecomment-790991990>.
src/cmd/ksh93/sh/expand.c: path_expand():
- If we're in the SH_FCOMPLETE (file name completion) state, then
pass the new GLOB_FCOMPLETE flag to AST glob(3).
Fixes: https://github.com/ksh93/ksh/issues/372
Thanks to @fbrau for the bug report.
The "fd is first arg to poll()" and "fd is second arg to poll()" tests
use write() but don't include the system header in which that function
is declared, leading to "error: implicit declaration of function 'write'
is invalid in C99'" when trying to compile the test. By including the
header, the test can now compile and run as intended.
The "mmap is worth using" test uses memcpy but doesn't include the
system header in which that function is declared, leading to
"error: implicitly declaring library function 'memcpy'" when trying
to compile the test. By including the header, the test can now
compile and run as intended.
On macOS, the result is still negative. The test seems to time
using mmap and another method and only picks mmap if it's faster.
Editing the test to print out timing information on a few runs, I
see that mmap is 2–3✕ slower than the other method:
$ for i in $(seq 1 5); do ./mmap_worth_using; done
/* mmtm=11 rdtm=3 */
/* 4*mmtm=44 3*rdtm=9 */
/* 4*mmtm=44 5*rdtm=15 */
/* mmtm=9 rdtm=5 */
/* 4*mmtm=36 3*rdtm=15 */
/* 4*mmtm=36 5*rdtm=25 */
/* mmtm=10 rdtm=4 */
/* 4*mmtm=40 3*rdtm=12 */
/* 4*mmtm=40 5*rdtm=20 */
/* mmtm=12 rdtm=4 */
/* 4*mmtm=48 3*rdtm=12 */
/* 4*mmtm=48 5*rdtm=20 */
/* mmtm=12 rdtm=4 */
/* 4*mmtm=48 3*rdtm=12 */
/* 4*mmtm=48 5*rdtm=20 */
If you passed CC=/some/compiler, the build broke on macOS because the
cc.darwin compiler wrapper wasn't used. Among other things, this
wrapper adds a -D_lib_memccpy flag, defining _lib_memccpy as 1
during the build. That was used to override a false negative result
of the lib_memccpy feature test. This commit fixes that feature
test instead, so it correctly returns positive on macOS.
Thanks to Ryan Smith (@ryandesign) for the bug report and for the
fix to the lib_memccpy test.
src/lib/libast/features/lib:
- Fix the lib_memccpy feature test. It was checking the result of
mmap(2) incorrectly, resulting in the test crashing on macOS.
Failure does not return NULL, it returns MAP_FAILED which is
usually -1.
src/cmd/INIT/cc.darwin*:
- Removed. Any other flags in these wrappers are either related to
building dynamic libraries, which is not currently supported, or
were determined to be unnecessary. See the GitHub issue for
discussion. This now makes it possible to pass `CC` to use any
compiler you like on the Mac. Notes:
- Apple's -D_ast_int8_t=int64_t is a no-op; another AST feature
test already defines _ast_int8_t a 64-bit integer type, even on
32-bit systems (on which it is defined as 'long long').
- The -search_paths_first linker flag is the default since 2010.
But even on my museum-grade Power Mac G5 with Mac OS X 10.3
(from 2004), it builds and runs just fine without.
- DCLK_TCK=100 is a no-op as even that ancient Mac system already
defines it as 100. Plus, it's not even actually used.
If a need is found for any of these, please report this in a new
issue so I can special-case it elsewhere in the code.
Resolves: https://github.com/ksh93/ksh/issues/373
List of changes:
- Fixed some -Wuninitialized warnings and removed some unused variables.
- Removed the unused extern for B_login (re: d8eba9d1).
- The libcmd builtins and the vmalloc memfatal function now handle
memory errors with 'ERROR_SYSTEM|ERROR_PANIC' for consistency with how
ksh itself handles out of memory errors.
- Added usage of UNREACHABLE() where it was missing from error handling.
- Extend many variables from short to int to prevent overflows (most
variables involve file descriptors).
- Backported a ksh2020 patch to fix unused value Coverity issues
(https://github.com/att/ast/pull/740).
- Note in src/cmd/ksh93/README that ksh compiles with Cygwin on
Windows 10 and Windows 11, albeit with many test failures.
- Add comments to detail some sections of code. Extensive list of
commits related to this change:
ca2443b5, 7e7f1372, 2db9953a, 7003aba4, 6f50ff64, b1a41311,
222515bf, a0dcdeea, 0aa9e03f, 61437b27, 352e68da, 88e8fa67,
bc8b36fa, 6e515f1d, 017d088c, 035a4cb3, 588a1ff7, 6d63b57d,
a2f13c19, 794d1c86, ab98ec65, 1026006d
- Removed a lot of dead ifdef code.
- edit/emacs.c: Hide an assignment to avoid a -Wunused warning. (See
also https://github.com/att/ast/pull/753, which removed the assignment
because ksh2020 removed the !SHOPT_MULTIBYTE code.)
- sh/nvdisc.c: The sh_newof macro cannot return a null pointer because
it will instead cause the shell to exit if memory cannot be allocated.
That makes the if statement here a no-op, so remove it.
- sh/xec.c: Fixed one unused variable warning in sh_funscope().
- sh/xec.c: Remove a fallthrough comment added in commit ed478ab7
because the TFORK code doesn't fall through (GCC also produces no
-Wimplicit-fallthrough warning here).
- data/builtins.c: The cd and pwd man pages state that these builtins
default to -P if PATH_RESOLVE is 'physical', which isn't accurate:
$ /opt/ast/bin/getconf PATH_RESOLVE
physical
$ mkdir /tmp/dir; ln -s /tmp/dir /tmp/sym
$ cd /tmp/sym
$ pwd
/tmp/sym
$ cd -P /tmp/sym
$ pwd
/tmp/dir
The behavior described by these man pages isn't specified in the ksh
man page or by POSIX, so to avoid changing these builtin's behavior
the inaccurate PATH_RESOLVE information has been removed.
- Mamfiles: Preserve multi-line errors by quoting the $x variable.
This fix was backported from 93v-.
(See also <a7e9cc82>.)
- sh/subshell.c: Remove set but not used sp->errcontext variable.
In iffe tests, some C functions are found in system libraries, but
then are not declared by the system headers on some systems because
the expected standards macros aren't defined, causing the system
headers to hide the function declarations. This may cause warnings
about invalid implicit function declarations in some tests (which
only show up if you export IFFEFLAGS=-d1), but may also cause false
negative test results. The iffe tests should be given the same
environment that their test results are going to be used in.
libast's first-run and most central feature test, 'standards',
figures out what standards macros need to be used on the current
system to get the system headers to declare the expected
functionality. All code that links to libast depends on the header
generated by this feature test. So iffe should use this result for
the tests as well, as soon as it's available (which is early in
libast's compilation cycle).
Concrete example: on Cygwin, in src/cmd/builtin/features/pty, the
'lib ptsname' test detects ptsname(3) in the system library, but
the output{...}end block that uses the _lib_ptsname feature test
result throws an 'implicit function definition' warning because
Cygwin's stdlib.h does not define this function without the
appropriate standards macros being defined first.
src/cmd/INIT/iffe.sh:
- If ${INSTALLROOT}/src/lib/libast/FEATURE/standards is available,
incorporate it directly into iffe's own block of compiler
massaging macros. Do not use #include as the necessary -I flags
are not added for every test.
- Centrally define the iffe version once, not twice.
- Update and tweak getopts self-documentation (iffe --man).
- While we're here, add macOS/Darwin's DYLD_LIBRARY_PATH to the
supported dynamic library search variables. On macOS, this is
normally disabled by System Integrity Protection, plus this
distribution uses static libraries at build time, so this is for
completeness' sake and not much else. This was ported from
@lkujaw's fork: 48ff054429
(thanks to @JohnoKing for pointing it out).
src/lib/libast/features/standards:
- Add a comment. This is to update the file's timestamp, ensuring
that everything will be recompiled after this commit.
src/lib/libast/features/standards:
- Add heuristic (u_long availability) for systems that hide rather
than reveal functionality in the presence of _POSIX_SOURCE, etc.
- Define _DARWIN_C_SOURCE, like _GNU_SOURCE, to enable the full
range of definitions on macOS systems.
- Due to the above, remove MACH (macOS)-specific hack.
- These changes ported from https://github.com/att/ast/pull/1492 -
thanks to Lev Kujawski (@lkujaw). His PR indicates that this
fixes the standards macros on UnixWare, too. Therefore, no longer
exclude UnixWare from standards macros (re: ff70c27f).
src/lib/libast/comp/conf.sh:
- Promote the 'op' member in Conf_t (struct Conf_s) from short to
int. This allows some Darwin/macOS values, now exposed, to fit
that would otherwise be truncated, namely:
_CS_DARWIN_USER_CACHE_DIR 65538
_CS_DARWIN_USER_DIR 65536
_CS_DARWIN_USER_TEMP_DIR 65537
Thus, the following AST getconf values are now correct on macOS:
$ /opt/ast/bin/getconf | grep ^DARWIN
DARWIN_USER_CACHE_DIR=/var/folders/nx/(REDACTED)/C/
DARWIN_USER_DIR=/var/folders/nx/(REDACTED)/0/
DARWIN_USER_TEMP_DIR=/var/folders/nx/(REDACTED)/T/
src/lib/libast/features/tty:
- Include <sys/ioctl.h> if available. This silences a compiler
warning in src/lib/libast/misc/procopen.c about an invalid
implicit declaration of ioctl(2).
This commit ports over two of Andy Fiddaman's bugfixes to conf.sh
on illumos:
- The compiler isn't passed on to an invocation of iffe. The bugfix is
from this commit: <63563232>
- The getconf builtin is missing several parameters on illumos.
Reproducer:
$ /opt/ast/bin/getconf ADDRESS_WIDTH
getconf: Invalid argument (ADDRESS_WIDTH) # Should output '64'
This bug occurs because conf.sh expects GNU sed and fails to work
properly with other sed implementations. The bugfix and original bug
report can be found here:
https://www.illumos.org/issues/14044ba443cfd
Vmalloc is incompatible with Cygwin, but the code to disable it on
Cygwin did not work properly, somehow causing the build to freeze
at a seemingly unrelated point (i.e., when iffe feature tests
attempt to write to sfstdout).
Vmalloc has wasted my time for the last time, so now it's getting
disabled by default even on development builds and you'll have to
pass -D_AST_vmalloc in CCFLAGS to enable it for testing purposes.
This commit has a few other build tweaks as well.
src/lib/libast/features/vmalloc:
- tst map_malloc: Remove no-op #if __CYGWIN__ block which was in
the #else clause of another #if __CYGWIN__ block.
- Output block ('cat{'):
- Instead of disabling vmalloc for certain systems, disable it
unless _AST_vmalloc is defined.
- To disable it, set _AST_std_malloc as well as _std_malloc, just
to be sure.
src/lib/libast/vmalloc/malloc.c:
- Remove ineffective Cygwin special-casing.
src/lib/libcmd/vmstate.c:
- This is only useful for vmalloc, so do not pointlessly compile it
if vmalloc is disabled.
src/lib/libast/man/vmalloc.3:
- Add deprecation notice.
Resolves: https://github.com/ksh93/ksh/issues/360
Ksh segfaults on M1 Macs, apparently because Apple's compiler
optimizer can't cope with overriding bzero(3) with libast's
implementation (though it's nothing more than "memset(b, 0, n);").
Apple has disabled libast's bzero function since 2018-12-04:
https://opensource.apple.com/source/ksh/ksh-27/patches/src__lib__libast__comp__omitted.c.diff.auto.html
src/lib/libast/comp/omitted.c:
- Only fall back to the libast bzero function if the OS lacks an
implementation of bzero.
- Remove the bzero undef since this fallback is only reached if the
OS doesn't have bzero.
NEWS:
- Add compat info for macOS on ARM64. This notes that macOS
Monterey can now compile and run ksh, although there is one
regression test failure:
builtins.sh[345]: printf %H: invalid UTF-8 characters
(expected %3F%C2%86%3F%3F%3F; got %3F%C2%86%3F%3Fv%3F%3F)
Thanks to @DesantBucie for the report and the testing.
Resolves: https://github.com/ksh93/ksh/issues/329
I don't expect anyone else to actually use ksh93 on a museum-grade
Power Mac G5 running Mac OS X 10.3.7, but ancient platforms are
great bug and compatibility testing tools. These tweaks restore the
ability to build on that platform.
Also, to avoid a strange path search bug on that platform and
possibly other ancient ones, set SHOPT_DYNAMIC to 0 in SHOPT.sh.
The last commit still failed to build on macOS M1. That va_listval
macro keeps causing trouble. It's an AST thing that is defined in
src/lib/libast/features/common. That looks like some incredibly
opaque attempt to make it compatible with everything, and clearly
it no longer works properly on all systems. I don't dare touch it,
though. That code looks like any minimal change will probably break
the build on some system or other.
src/lib/libast/features/hack:
- Add feature test to check if that macro needs (0) no workaround,
or (1) the workaround from the 93v- beta, or (2) the FreeBSD one.
Whichever version compiles first, it will use. If none does, this
test will not output a value, which will be treated as 0.
src/lib/libast/hash/hashalloc.c,
src/lib/libast/string/tokscan.c:
- Update to use the result of the hack feature test.
src/lib/libast/Mamfile:
- Update for new #include dependencies.
hyenias writes, re the referenced commit:
> This has caused my Ubuntu 18.04 ARMv7 to fail to compile.
>
> /dev/shm/ksh/src/lib/libast/hash/hashalloc.c: In function 'hashalloc':
> /dev/shm/ksh/src/lib/libast/hash/hashalloc.c:156:11: error:
> incompatible types when assigning to type 'va_list * {aka
> __va_list *}' from type 'va_list {aka __va_list}'
> tmpval = va_listval(va_arg(ap, va_listarg));
> ^
> In file included from ./ast_common.h:192:0,
> from /dev/shm/ksh/src/lib/libast/include/ast_std.h:37,
> from /dev/shm/ksh/src/lib/libast/include/ast.h:36,
> from /dev/shm/ksh/src/lib/libast/hash/hashlib.h:34,
> from /dev/shm/ksh/src/lib/libast/hash/hashalloc.c:33:
> /dev/shm/ksh/src/lib/libast/hash/hashalloc.c:157:16: error:
> incompatible type for argument 2 of '__builtin_va_copy'
> va_copy(ap, tmpval);
> ^
> /dev/shm/ksh/src/lib/libast/hash/hashalloc.c:157:16: note: expected
> '__va_list' but argument is of type 'va_list * {aka __va_list *}'
> mamake [lib/libast]: *** exit code 1 making hashalloc.o
> mamake: *** exit code 1 making lib/libast
> mamake: *** exit code 1 making all
> package: make done at Fri May 14 06:10:16 EDT 2021 in
> /dev/shm/ksh/arch/linux.arm
src/lib/libast/hash/hashalloc.c,
src/lib/libast/string/tokscan.c:
- Revert the FreeBSD fix.
- Backport a conditional workaround for clang from ksh 93v- beta.
On some systems, the following won't compile because of the way the
macros are defined in the system headers:
va_copy(ap, va_listval(va_arg(ap, va_listarg)));
The error from clang is something like:
.../hashalloc.c:155:16: error: non-const lvalue reference to type
'__builtin_va_list' cannot bind to a temporary of type 'va_list'
(aka 'char *')
va_copy(ap, va_listval(va_arg(ap, va_listarg)));
~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./ast_common.h:200:23: note: expanded from macro 'va_listval'
#define va_listval(p) (p)
^
.../include/stdarg.h:27:53: note: expanded from macro 'va_copy'
#define va_copy(dest, src) __builtin_va_copy(dest, src)
^~~
1 error generated.
mamake [lib/libast]: *** exit code 1 making hashalloc.o
This commit backports a FreeBSD build fix from:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255308
Thanks to Chase <nicetrynsa@protonmail.ch> for the bug report.
src/lib/libast/hash/hashalloc.c,
src/lib/libast/string/tokscan.c:
- Store va_listval() result in variable and pass that to va_copy().
