1
0
Fork 0
mirror of git://git.code.sf.net/p/cdesktopenv/code synced 2025-02-15 04:32:24 +00:00
Commit graph

136 commits

Author SHA1 Message Date
Johnothan King
f28bce61a7
Fix multiple problems with the getconf builtin (#280)
This commit fixes three problems with getconf pathbound builtin:
1. The -l/--lowercase option did not change all variable names to
   lower case.
2. The -q/--quote option now quotes all string values. Previously,
   it only quoted string values that had a space or other
   non-shellsafe character.
3. The -c/--call, -n/--name and -s/--standard options matched all
   variable names provided by 'getconf -a', even if none were
   actual matches.

Additionally, references to the confstr and sysconf functions have
been updated to reference section 3 of the man pages instead of
section 2.

src/lib/libast/port/astconf.c:
- Previously, only values that had spaces in them were quoted. Change
  that behavior to quote all string values by using the FMT_ALWAYS
  flag. Bug report: https://github.com/att/ast/issues/1173
- Not all variable names were printed in lowercase by 'getconf -l'.
  Fix it by adding a few missing instances of fmtlower.
  Bug report: https://github.com/att/ast/issues/1171
- Add the missing code to the '#if _pth_getconf_a' block to handle
  -c/-n/-s while parsing the OS's native 'getconf -a' output. This
  approach reuses code for name matching from other parts of
  astconflist(). Resolves: https://github.com/ksh93/ksh/issues/279

src/lib/libcmd/getconf.c:
- Update the documentation to note the -q flag only quotes strings.

src/cmd/ksh93/tests/bulitins.sh:
- Add regression tests for the getconf bugs fixed in this commit.

Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-04-21 03:34:54 +01:00
Martijn Dekker
519bb08265
Allow invoking path-bound built-in commands by direct path or preceding PATH assignment (#275)
Path-bound builtins on ksh (such as /opt/ast/bin/cat) break some
basic assumptions about paths in the shell that should hold true,
e.g., that a path output by whence -p or command -v should actually
point to an executable command. This commit should fix the
following:

1. Path-bound built-ins (such as /opt/ast/bin/cat) can now be
   executed by invoking the canonical path (independently of the
   value of $PATH), so the following will now work as expected:

        $ /opt/ast/bin/cat --version
          version         cat (AT&T Research) 2012-05-31
        $ (PATH=/opt/ast/bin:$PATH; "$(whence -p cat)" --version)
          version         cat (AT&T Research) 2012-05-31

   In the event an external command by that path exists, the
   path-bound builtin will now override it when invoked using the
   canonical path. To invoke a possible external command at that
   path, you can still use a non-canonical path, e.g.:
   /opt//ast/bin/cat or /opt/ast/./bin/cat

2. Path-bound built-ins will now also be found on a PATH set
   locally using an assignment preceding the command, so something
   like the following will now work as expected:

        $ PATH=/opt/ast/bin cat --version
          version         cat (AT&T Research) 2012-05-31

   The builtin is not found by sh_exec() because the search for
   builtins happens long before invocation-local preceding
   assignments are processsed. This only happens in sh_ntfork(),
   before forking, or in sh_fork(), after forking. Both sh_ntfork()
   and sh_fork() call path_spawn() to do the actual path search, so
   a check there will cover both cases.

   This does mean the builtin will be run in the forked child if
   sh_fork() is used (which is the case on interactive shells with
   job.jobcontrol set, or always after compiling with SHOPT_SPAWN
   disabled). Searching for it before forking would mean
   fundamentally redesigning that function to be basically like
   sh_ntfork(), so this is hard to avoid.

src/cmd/ksh93/sh/path.c: path_spawn():
- Before doing anything else, check if the passed path appears in
  the builtins tree as a pathbound builtin. If so, run it. Since a
  builtin will only be found if a preceding PATH assignment
  temporarily changed the PATH, and that assignment is currently in
  effect, we can just sh_run() the builtin so a nested sh_exec()
  invocation will find and run it.
- If 'spawn' is not set (i.e. we must return), set errno to 0 and
  return -2. See the change to sh_ntfork() below.

src/cmd/ksh93/sh/xec.c:
- sh_exec(): When searching for built-ins and the restricted option
  isn't active, also search bltin_tree for names beginning with a
  slash.
- sh_ntfork(): Only throw an error if the PID value returned is
  exactly -1. This allows path_spawn() to return -2 after running a
  built-in to tell sh_ntfork() to do the right things to restore
  state.

src/cmd/ksh93/sh/parse.c: simple():
- When searching for built-ins at parse time, only exclude names
  containing a slash if the restricted option is active. This
  allows finding pointers to built-ins invoked by literal path like
  /opt/ast/bin/cat, as long as that does not result from an
  expansion. This is not actually necessary as sh_exec() will also
  cover this case, but it is an optimisation.

src/lib/libcmd/getconf.c:
- Replace convoluted deferral to external command by a simple
  invocation of the path to the native getconf command determined
  at compile time (by src/lib/libast/comp/conf.sh). Based on:
  https://github.com/ksh93/ksh/issues/138#issuecomment-816384871
  If there is ever a system that has /opt/ast/bin/getconf as its
  default native external 'getconf', then there would still be an
  infinite recursion crash, but this seems extremely unlikely.

Resolves: https://github.com/ksh93/ksh/issues/138
2021-04-15 04:08:12 +01:00
Johnothan King
2c38fb93fd
Fix the exit status returned when a command isn't executable (#273)
Previous discussion: https://github.com/att/ast/issues/485

If ksh attempts to execute a non-executable command found in the
PATH, in some instances the error message and return status are
incorrect. In the example below, ksh returns with exit status 126
when using the -c execve(2) optimization or when using fork(2) in
an interactive shell. However, using posix_spawn(3) causes the exit
status to change:
  $ echo 'print cannot execute' > /tmp/x
  # Runs command with spawnveg (i.e., posix_spawn or vfork)
  $ ksh -c 'PATH=/tmp; x; echo $?'
  ksh: x: not found
  127
  # Runs command with execve
  $ ksh -c 'PATH=/tmp; x'; echo $?
  ksh: x: cannot execute [Permission denied]
  126
  # Runs command with fork
  $ ksh -ic 'PATH=/tmp; x; echo $?'
  ksh: x: cannot execute [Permission denied]
  126

Since 'x' is in the PATH but can't be executed, the correct exit
status is 126, not 127. It's worth noting this bug doesn't cause
the regression tests to fail with ksh93u+m, but it does cause one
test to fail when run under dtksh:

    path.sh[706]: Long nonexistent command name: got status 126, ''

This commit backports various fixes for this bug from ksh2020, with
additional fixes applied (since there were still some additional
issues the ksh2020 patch didn't fix). The lacking regression test
for exit status 126 in path.sh has been rewritten to test for more
scenarios where ksh failed to return the correct error message
and/or exit status. I can also confirm with this patch applied the
path.sh regression tests now pass when run under dtksh.

src/cmd/ksh93/sh/path.c:
- Add a comment to path_absolute() describing 'oldpp' is the
  current pointer in the while loop and 'pp' is the next pointer.
  Backported from:
  https://github.com/att/ast/commit/a6cad450

- The patch from ksh2020 didn't fix this bug in the SHOPT_SPAWN
  code (because ksh2020 prefers fork(2)), so issues with the exit
  status could still occur when using spawnveg. To fix this, always
  set 'noexec' to the value of errno if can_execute fails. Before
  this fix, errno was discarded if 'pp' was a null pointer and
  can_execute failed.

- If a command couldn't be executed and the error wasn't ENOENT,
  save errno in a 'not_executable' variable. If an executable
  command couldn't be found in the PATH, exit with status 126 and
  set errno to the saved value. This was based on a ksh2020 bugfix,
  but it has been reworked a little bit to fix a bug that caused a
  mismatch between the error message shown and errno. Example with
  a non-executable file in PATH:
  $ nonexec
  ksh2020: nonexec: cannot execute [No such file or directory]
  The ksh2020 patch: <https://github.com/att/ast/pull/493>

- Backport a ksh2020 bugfix for directories in the PATH when
  running one of the added regression tests on OpenBSD:
  https://github.com/att/ast/pull/767

src/cmd/ksh93/data/msg.c,
src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/sh/{path,xec}.c:
- If a command name is too long (ENAMETOOLONG), then it wasn't
  found in the PATH. For that case return exit status 127, like
  for ENOENT.

src/cmd/ksh93/tests/path.sh:
- Replace the old test with a new set of more extensive tests.
  These tests check the error message and exit status when ksh
  attempts to run a command using any of the following:
   - execve(2), used with the last command run with -c       (*A tests).
   - posix_spawn(3)/vfork(2), used in noninteractive scripts (*B tests).
   - fork(2), used in interactive shells with job control    (*C tests).
   - command -x                                              (*D tests).
   - exec(1)                                                 (*E tests).
- Add a regression test from ksh2020 for attempting to execute a
  directory:
  https://github.com/att/ast/pull/758

src/lib/libast/include/ast.h,
src/lib/libast/include/wait.h:
- Avoid bitshifts in macros for static error codes. The return
  values of command not found and exec related errors are static
  values and should not require any macro magic for calculation.
  Backported from: https://github.com/att/ast/commit/c073b102
- Simplify EXIT_* and W* macros to use 8 bits.
2021-04-15 03:37:57 +01:00
Johnothan King
504cbda269
Fix 'printf %T' ignoring the current locale in LC_TIME (#263)
src/lib/libast/tm/tmlocale.c:
- Load the locale set by LC_TIME or LC_ALL if it hasn't been loaded
  before or if it was loaded previously but isn't the current locale.

src/cmd/ksh93/tests/locale.sh:
- Add a regression test using the nl_NL.UTF-8 and ja_JP.UTF-8 locales.

Fixes: https://github.com/ksh93/ksh/issues/261
2021-04-09 03:49:48 +01:00
Johnothan King
a065558291
Fix more compiler warnings, typos and other minor issues (#260)
Many of these changes are minor typo fixes. The other changes
(which are mostly compiler warning fixes) are:

NEWS:
- The --globcasedetect shell option works on older Linux kernels
  when used with FAT32/VFAT file systems, so remove the note about
  it only working with 5.2+ kernels.

src/cmd/ksh93/COMPATIBILITY:
- Update the documentation on function scoping with an addition
  from ksh93v- (this does apply to ksh93u+).

src/cmd/ksh93/edit/emacs.c:
- Check for '_AST_ksh_release', not 'AST_ksh_release'.

src/cmd/INIT/mamake.c,
src/cmd/INIT/ratz.c,
src/cmd/INIT/release.c,
src/cmd/builtin/pty.c:
- Add more uses of UNREACHABLE() and noreturn, this time for the
  build system and pty.

src/cmd/builtin/pty.c,
src/cmd/builtin/array.c,
src/cmd/ksh93/sh/name.c,
src/cmd/ksh93/sh/nvtype.c,
src/cmd/ksh93/sh/suid_exec.c:
- Fix six -Wunused-variable warnings (the name.c nv_arrayptr()
  fixes are also in ksh93v-).
- Remove the unused 'tableval' function to fix a -Wunused-function
  warning.

src/cmd/ksh93/sh/lex.c:
- Remove unused 'SHOPT_DOS' code, which isn't enabled anywhere.
  https://github.com/att/ast/issues/272#issuecomment-354363112

src/cmd/ksh93/bltins/misc.c,
src/cmd/ksh93/bltins/trap.c,
src/cmd/ksh93/bltins/typeset.c:
- Add dictionary generator function declarations for former
  aliases that are now builtins (re: 1fbbeaa1, ef1621c1, 3ba4900e).
- For consistency with the rest of the codebase, use '(void)'
  instead of '()' for print_cpu_times.

src/cmd/ksh93/sh/init.c,
src/lib/libast/path/pathshell.c:
- Move the otherwise unused EXE macro to pathshell() and only
  search for 'sh.exe' on Windows.

src/cmd/ksh93/sh/xec.c,
src/lib/libast/include/ast.h:
- Add an empty definition for inline when compiling with C89.
  This allows the timeval_to_double() function to be inlined.

src/cmd/ksh93/include/shlex.h:
- Remove the unused 'PIPESYM2' macro.

src/cmd/ksh93/tests/pty.sh:
- Add '# err_exit #' to count the regression test added in
  commit 113a9392.

src/lib/libast/disc/sfdcdio.c:
- Move diordwr, dioread, diowrite and dioexcept behind
  '#ifdef F_DIOINFO' to fix one -Wunused-variable warning and
  multiple -Wunused-function warnings (sfdcdio() only uses these
  functions when F_DIOINFO is defined).

src/lib/libast/string/fmtdev.c:
- Fix two -Wimplicit-function-declaration warnings on Linux by
  including sys/sysmacros.h in fmtdev().
2021-04-08 19:58:07 +01:00
Johnothan King
a28507e0b1
Apply new CentOS fix for strdup null-test bug (re: 7afb30e) (#255)
This is an update to one of Red Hat's patches. The strdup change is
from CentOS:
https://git.centos.org/rpms/ksh/blob/c8s/f/SOURCES/ksh-20120801-annocheck.patch

The reason why gcc (and also clang) optimize out the null check is
because the glibc string.h header gives 's' a nonnull attribute (in
other words, this is a glibc compatibility bug, not a compiler bug).
Clang gives the following informative warning when compiling strdup:

> /home/johno/GitRepos/KornShell/ksh/src/lib/libast/string/strdup.c:66:10: warning: nonnull parameter 's' will evaluate to 'true' on
>         return (s && (t = oldof(0, char, n = strlen(s) + 1, 0))) ? (char*)memcpy(t, s, n) : (char*)0;
>                 ^ ~~
> /usr/include/string.h:172:35: note: declared 'nonnull' here
>      __THROW __attribute_malloc__ __nonnull ((1));
>                                   ^
> /usr/include/sys/cdefs.h:303:44: note: expanded from macro '__nonnull'
> # define __nonnull(params) __attribute__ ((__nonnull__ params))

The proper fix is to rename the function in strdup.c to
'_ast_strdup'. This avoids the string.h conflict and fixes the Red
Hat bug. I've also made a similar change to getopt.c, since clang
was throwing a nonnull warning there as well.

src/lib/libast/features/map.c (which generates FEATURE/map which is
indirectly included by everything) is updated to always map getopt
to _ast_getopt and strdup to _ast_strdup.
2021-04-08 05:26:16 +01:00
Johnothan King
c4f980eb29
Introduce usage of __builtin_unreachable() and noreturn (#248)
This commit adds an UNREACHABLE() macro that expands to either the
__builtin_unreachable() compiler builtin (for release builds) or
abort(3) (for development builds). This is used to mark code paths
that are never to be reached.

It also adds the 'noreturn' attribute to functions that never
return: path_exec(), sh_done() and sh_syntax(). The UNREACHABLE()
macro is not added after calling these.

The purpose of these is:
* to slightly improve GCC/Clang compiler optimizations;
* to fix a few compiler warnings;
* to add code clarity.

Changes of note:

src/cmd/ksh93/sh/io.c: outexcept():
- Avoid using __builtin_unreachable() here since errormsg can
  return despite using ERROR_system(1), as shp->jmplist->mode is
  temporarily set to 0. See: https://github.com/att/ast/issues/1336

src/cmd/ksh93/tests/io.sh:
- Add a regression test for the ksh2020 bug referenced above.

src/lib/libast/features/common:
- Detect the existence of either the C11 stdnoreturn.h header or
  the GCC noreturn attribute, preferring the former when available.
- Test for the existence of __builtin_unreachable(). Use it for
  release builds. On development builds, use abort() instead, which
  crahses reliably for debugging when unreachable code is reached.

Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-04-05 00:28:24 +01:00
Johnothan King
56913f8c2a
Fix bugs related to 'uname -d' in the 'uname' builtin (#251)
This commit fixes a bug in the ksh uname builtin's -d option that could
change the output of -o (I was only able to reproduce this on Linux):
    $ builtin uname
    $ uname -o
    GNU/Linux
    $ uname -d
    (none)
    $ uname -o
    (none)
I identified this patch from ksh2020 as a fix for this bug:
<https://github.com/att/ast/pull/1187>
The linked patch was meant to fix a crash in 'uname -d', although I've
had no luck reproducing it: <https://github.com/att/ast/issues/1184>

src/lib/libcmd/uname.c:
- Pass correct buffer to getdomainname() while executing uname -d.

src/cmd/ksh93/tests/builtins.sh:
- Add a regression test for the reported 'uname -d' crash.
- Add a regression test for the output of 'uname -o' after 'uname -d'.
- To handle potential crashes when running the regression tests in older
  versions of ksh, fork the command substitutions that run 'uname -d'.
2021-04-04 22:18:43 +01:00
Johnothan King
ed478ab7e3
Fix many GCC -Wimplicit-fallthrough warnings (#243)
This commit adds '/* FALLTHROUGH */' comments to fix many
GCC warnings when compiling with -Wimplicit-fallthrough.
Additionally, the existing fallthrough comments have been
changed for consistency.
2021-03-30 21:49:20 +01:00
Johnothan King
767d23b3fe
Fix FreeBSD timezone name determination again (re: 9f43f8d1, d7c94707) (#244)
src/lib/libast/tm/tminit.c:
- Commit 9f43f8d1, in addition to backporting fixes from ksh93v-, also
  backported this bug:
      $ printf '%(%Z)T' now
      PPT  # Should be PDT
  Reapply the ksh2020 bugfix to fix the %Z time
  format again.

src/cmd/ksh93/tests/builtins.sh:
- Add a regression test so this bug (hopefully) isn't backported from
  ksh93v- again).
2021-03-26 19:36:13 +00:00
Martijn Dekker
181e87d228 Update #include-related dependencies in Mamfiles
I grepped for #include changes in all the commits and compared
that to the changes in the Mamfiles. I found 7 commits that don't
update the Mamfiles with the appropriate dependencies while
adding #includes, as I only learned how this works after having
worked with this code for some time.

This commit adds the missing Mamfile updates for the
corresponding #include changes in the following commits:
06e721c3, 65d363fd, 70fc1da7, 79d19458, b1a41311, bb4d6a2e,
db71b3ad, and this commit.

Additionally:

src/lib/libast/comp/setlocale.c:
- Change include errno.h to error.h to use EILSEQ fallback if
  needed; remove corresponding #ifdef (re: 4dcf5c50, 71bfe028).

src/cmd/ksh93/Mamfile:
- Fix a broken dependency on libast FEATURE/float (re: 72968eae).
  We can't use 'prev' for a file that was not mentioned before in
  the same Mamfile, we have to use a 'make'...'done' on the first
  mention. Add subdependencies matching those in libast/Mamfile.
2021-03-25 01:55:59 +00:00
Johnothan King
22e044c339
Fix compile when using tcc >0.9.27 (#238)
This allows ksh to be compiled with versions of tcc that define
__dso_handle in libtcc1.a, i.e., versions as of this commit:
https://repo.or.cz/tinycc.git/commit/dd60b20c

Older versions of tcc still fail to compile ksh, although now they
fail after reaching the libdll feature test. I'm not sure if fixing
that is feasible since even if I hack out the failing libdll
feature test, ksh fails to link with a '__dso_handle' error.

src/lib/libast/comp/atexit.c,
src/lib/libast/features/lib,
src/lib/libast/vmalloc/vmexit.c:
- From what I've been able to gather the only OSes with support
  for on_exit are Linux and SunOS 4. However, on_exit takes two
  arguments, so the macro that defines it as taking one argument
  is incorrect. Since Solaris (SunOS 5) no longer has this call
  and the macro breaks on Linux, the clean fix is to remove it
  (atexit(3) is used instead).

src/lib/libast/include/ast.h:
- When compiling with tcc on FreeBSD, pretend to be gcc 2.95.3
  instead of gcc 9.3.0. This stops /usr/include/math.h from
  activating gcc 3.0+ math compiler builtins that don't exist on
  tcc, while still identifying as gcc which is needed to avoid
  other FreeBSD system header breakage.

src/cmd/builtin/Mamfile,
src/cmd/builtin/features/pty,
src/lib/libdll/Mamfile,
src/lib/libdll/features/dll:
- tcc forbids combining the -c compiler flag with -l* linker flags.
  Use the -lm flag in the iffe feature tests instead of the
  Mamfiles. This avoids iffe combining -lm with the -c flag.

src/lib/libast/vmalloc/malloc.c:
- Fix failure to compile with -D_std_malloc.
  This patch is from OpenSUSE:
  https://build.opensuse.org/package/view_file/shells/ksh/ksh93-malloc-hook.dif
  As it turns out tcc needs this change to build ksh with
  -D_std_malloc, so it has been applied.

Co-authored-by: Martijn Dekker <martijn@inlv.org>
Resolves: https://github.com/ksh93/ksh/issues/232
2021-03-23 14:46:58 +00:00
Johnothan King
ca3ec2000c
Linux bugfixes for globcasedetect (re: 71934570) (#240)
src/lib/libast/features/lib,
src/lib/libast/path/pathicase.c:
- FAT32 file systems on Linux don't support FS_CASEFOLD_FL, which
  caused globbing to break. Reproducer using a UEFI boot partition:
      $ echo /boot/eF*
      /boot/eF*
  This is fixed by checking for FAT attributes with ioctl, then
  checking for FS_CASEFOLD_FL if that fails.
- The check for FS_CASEFOLD_FL didn't work correctly; I still wasn't
  able to get --globcasedetect to work on a case-insensitive ext4
  folder. Fix that by adding missing parentheses.
2021-03-23 13:59:02 +00:00
Martijn Dekker
88d7a62b4d cleanup: fix redundant NOT_USED macro redefinition (re: 733f70e9)
The NOT_USED() macro is already defined in ast.h (which is included
by shell.h) as an alias of NoP(). So it's better to apply the fix
to NoP() so it takes effect for both verrsions, for libast and ksh.
2021-03-23 02:44:01 +00:00
Martijn Dekker
71934570bf Add --globcasedetect shell option for globbing and completion
One of the best-kept secrets of libast/ksh93 is that the code
includes support for case-insensitive file name generation (a.k.a.
pathname expansion, a.k.a. globbing) as well as case-insensitive
file name completion on interactive shells, depending on whether
the file system is case-insensitive or not. This is transparently
determined for each directory, so a path pattern that spans
multiple file systems can be part case-sensitive and part case-
insensitive. In more precise terms, each slash-separated path name
component pattern P is treated as ~(i:P) if its parent directory
exists on a case-insensitive file system. I recently discovered
this while dealing with <https://github.com/ksh93/ksh/issues/223>.

However, that support is dead code on almost all current systems.
It depends on pathconf(2) having a _PC_PATH_ATTRIBUTES selector.
The 'c' attribute is supposedly returned if the given directory is
on a case insensitive file system. There are other attributes as
well (at least 'l', see src/lib/libcmd/rm.c). However, I have been
unable to find any system, current or otherwise, that has
_PC_PATH_ATTRIBUTES. Google and mailing list searches yield no
relevant results at all. If anyone knows of such a system, please
add a comment to this commit on GitHub, or email me.

An exception is Cygwin/Windows, on which the "c" attribute was
simply hardcoded, so globbing/completion is always case-
insensitive. As of Windows 10, that is wrong, as it added the
possibility to mount case-sensitive file systems.

On the other hand, this was never activated on the Mac, even
though macOS has always used a case-insensitive file like Windows.
But, being UNIX, it can also mount case-sensitive file systems.

Finally, Linux added the possibility to create individual case-
insensitive ext4 directories fairly recently, in version 5.2.
https://www.collabora.com/news-and-blog/blog/2020/08/27/using-the-linux-kernel-case-insensitive-feature-in-ext4/

So, since this functionality latently exists in the code base, and
three popular OSs now have relevant file system support, we might
as well make it usable on those systems. It's a nice idea, as it
intuitively makes sense for globbing and completion behaviour to
auto-adapt to file system case insensitivity on a per-directory
basis. No other shell does this, so it's a nice selling point, too.

However, the way it is coded, this is activated unconditionally on
supported systems. That is not a good idea. It will surprise users.
Since globbing is used with commands like 'rm', we do not want
surprises. So this commit makes it conditional upon a new shell
option called 'globcasedetect'. This option is only compiled into
ksh on systems where we can actually detect FS case insensitivity.

To implement this, libast needs some public API additions first.

*** libast changes ***

src/lib/libast/features/lib:
- Add probes for the linux/fs.h and sys/ioctl.h headers.
  Linux needs these to use ioctl(2) in pathicase(3) (see below).

src/lib/libast/path/pathicase.c,
src/lib/libast/include/ast.h,
src/lib/libast/man/path.3,
src/lib/libast/Mamfile:
- Add new pathicase(3) public API function. This uses whatever
  OS-specific method it can detect at compile time to determine if
  a particular path is on a case-insensitive file system. If no
  method is available, it only sets errno to ENOSYS and returns -1.
  Currently known to work on: macOS, Cygwin, Linux 5.2+, QNX 7.0+.
- On systems (if any) that have the mysterious _PC_PATH_ATTRIBUTES
  selector for pathconf(2), call astconf(3) and check for the 'c'
  attribute to determine case insensitivity. This should preserve
  compatibility with any such system.

src/lib/libast/port/astconf.c:
- dynamic[]: As case-insensitive globbing is now optional on all
  systems, do not set the 'c' attribute by default on _WINIX
  (Cygwin/Windows) systems.
- format(): On systems that do not have _PC_PATH_ATTRIBUTES, call
  pathicase(3) to determine the value for the "c" (case
  insensitive) attribute only. This is for compatibility as it is
  more efficient to call pathicase(3) directly.

src/lib/libast/misc/glob.c,
src/lib/libast/include/glob.h:
- Add new GLOB_DCASE public API flag to glob(3). This is like
  GLOB_ICASE (case-insensitive matching) except it only makes the
  match case-insensitive if the file system for the current
  pathname component is determined to be case-insensitive.
- gl_attr(): For efficiency, call pathicase(3) directly instead of
  via astconf(3).
- glob_dir(): Only call gl_attr() to determine file system case
  insensitivity if the GLOB_DCASE flag was passed. This makes case
  insensitive globbing optional on all systems.
- glob(): The options bitmask needs to be widened to fit the new
  GLOB_DCASE option. Define this centrally in a new GLOB_FLAGMASK
  macro so it is easy to change it along with GLOB_MAGIC (which
  uses the remaining bits for a sanity check bit pattern).

src/lib/libast/path/pathexists.c:
- For efficiency, call pathicase(3) directly instead of via
  astconf(3).

*** ksh changes ***

src/cmd/ksh93/features/options,
src/cmd/ksh93/SHOPT.sh:
- Add new SHOPT_GLOBCASEDET compile-time option. Set it to probe
  (empty) by default so that the shell option is compiled in on
  supported systems only, which is determined by new iffe feature
  test that checks if pathicase(3) returns an ENOSYS error.

src/cmd/ksh93/data/options.c,
src/cmd/ksh93/include/shell.h:
- Add -o globcasedetect shell option if compiling with
  SHOPT_GLOBCASEDET.

src/cmd/ksh93/sh/expand.c: path_expand():
- Pass the new GLOB_DCASE flag to glob(3) if the
  globcasedetect/SH_GLOBCASEDET shell option is set.

src/cmd/ksh93/edit/completion.c:
- While file listing/completion is based on globbing and
  automatically becomes case-insensitive when globbing does, it
  needs some additional handling to make a string comparison
  case-insensitive in corresponding cases. Otherwise, partial
  completions may be deleted from the command line upon pressing
  tab. This code was already in ksh 93u+ and just needs to be
  made conditional upon SHOPT_GLOBCASEDET and globcasedetect.
- For efficiency, call pathicase(3) directly instead of via
  astconf(3).

src/cmd/ksh93/sh.1:
- Document the new globcasedetect shell option.
2021-03-22 18:45:19 +00:00
Martijn Dekker
71bfe0283d libast: consolidate errno ID fallbacks into error.h
In various places in libast and libcmd there are preprocessor
fallbacks like this, for systems that don't define all the commonly
used errno value IDs:

    #ifndef ENOSYS
    #define ENOSYS	EINVAL
    #endif

and many others. It is better to have these all in one place so
they are not duplicated and we don't risk inconsistencies when
adding new code.

src/lib/libast/include/error.h includes the OS's <errno.h>, so it
is the logical file to move all these fallbacks into.

Quite possibly there is no remotely current system that needs any
of these, but they won't do any harm either.

Most files already use <error.h> directly or indirectly. Four
needed new #include <error.h> directives to use the fallbacks if
needed. The libast Mamfile is updated to make those files depend on
that header.
2021-03-22 14:55:16 +00:00
Johnothan King
814b5c6890
Fix various minor problems and update the documentation (#237)
These are minor fixes I've accumulated over time. The following
changes are somewhat notable:

- Added a missing entry for 'typeset -s' to the man page.
- Add strftime(3) to the 'see also' section. This and the date(1)
  addition are meant to add onto the documentation for 'printf %T'.
- Removed the man page the entry for ksh reading $PWD/.profile on
  login. That feature was removed in commit aa7713c2.
- Added date(1) to the 'see also' section of the man page.
- Note that the 'hash' command can be used instead of 'alias -t' to
  workaround one of the caveats listed in the man page.
- Use an 'out of memory' error message rather than 'out of space'
  when memory allocation fails.
- Replaced backticks with quotes in some places for consistency.
- Added missing documentation for the %P date format.
- Added missing documentation for the printf %Q and %p formats
  (backported from ksh2020: https://github.com/att/ast/pull/1032).
- The comments that show each builtin's options have been updated.
2021-03-21 14:39:03 +00:00
Johnothan King
2d7e9a0d6d
Fix the CI build by removing _c99_in_the_wild code (re: 38f2b94f) (#236)
src/lib/libast/sfio/sfcvt.c:
- The C99 code formerly behind '#if _c99_in_the_wild' broke the CI
  build:
  https://github.com/ksh93/ksh/runs/2158627969 (failing build)
  https://github.com/JohnoKing/ksh/runs/2158860590 (build after revert)
  Since this code wasn't used before that commit, it has been removed
  to fix the CI build.
2021-03-21 12:53:19 +00:00
Martijn Dekker
7b0e0776e2 cleanup: remove legacy code for systems without fork(2)
In 2021, it seems like it's about time to join the 21st century
and officially require fork(2). In practice this was already the
case as the legacy code was unmaintained and didn't compile.
2021-03-21 06:39:32 +00:00
Martijn Dekker
38f2b94f55 Some more #ifdef cleanups
src/cmd/ksh93/edit/edit.c,
src/cmd/ksh93/edit/history.c,
src/cmd/ksh93/sh/deparse.c:
- Remove experimental code protected by '#ifdef future'.
  No one is going to do anything with this, it's just clutter.

src/lib/libast/sfio/sfcvt.c:
- In 2021, it might be time to actually start using some C99
  features were available. Change two checks for a _c99_in_the_wild
  macro to actual checks for C99, enabling the use of fpclassify().

Resolves: https://github.com/ksh93/ksh/issues/219
2021-03-21 06:39:32 +00:00
Martijn Dekker
aad74597f7 Fixes for -G/--globstar (re: 5312a59d)
The fix for '.' and '..' in regular globbing broke '.' and '..' in
globstar. No globstar pattern that contains '.' or '..' as any
pathname component still matched. This commit fixes that.

This commit also makes symlink/** mostly work, which it never has
done in any ksh93 version. It is correct and expected that symlinks
found by patterns are not resolved, but symlinks were not resolved
even when specified as explicit non-pattern pathname components.
For example, /tmp/** breaks if /tmp is a symlink (e.g. on macOS),
which looks like a bug.

src/lib/libast/include/glob.h,
src/lib/libast/misc/glob.c: glob_dir():
- Make symlink/** work. we can check if the string pointed to by
  pat is exactly equal to *. If so, we are doing regular globbing
  for that particular pathname element, and it's okay to resolve
  symlinks. If not (if it's **), we're doing globstar and we should
  not be matching symlinks.
- Let's also introduce proper identification of symlinks (GLOB_SYM)
  and not lump them in with other special files (GLOB_DEV).
- Fix the bug with literal '.' and '..' components in globstar
  patterns. In preceding code, the matchdir pointer gets set to the
  complete glob pattern if we're doing globstar for the current
  pathname element, null if not. The pat pointer gets set to the
  elements of the pattern that are still left to be processed;
  already-done elements are trimmed from it by increasing the
  pointer. So, to do the right thing, we need to make sure that '.'
  or '..' is skipped if, and only if, it is the final element in
  the pattern (i.e., if pat does not contain a slash) and is not
  specified literally as '.' or '..', i.e., only if '.' or '..' was
  actually resolved from a glob pattern. After this change,
  '**/.*', '**/../.*', etc. do the right thing, showing all your
  hidden files and directories without undesirable '.' and '..'
  results; '.' and '..' are skipped as final elements, unless you
  literally specify '**/.', '**/..', '**/foo/bar/..', etc.

src/cmd/ksh93/COMPATIBILITY:
- Note the symlink/** globstar change.

src/cmd/ksh93/sh.1:
- Try to document the current globstar behaviour more exhausively.

src/cmd/ksh93/tests/glob.sh:
- Add tests. Try to cover all the corner cases.

src/cmd/ksh93/tests/shtests:
- Since tests in glob.sh do not use err_exit, they were not
  counted. Special-case glob.sh for counting the tests: count the
  lines starting with a test_* function call.

Resolves: https://github.com/ksh93/ksh/issues/146
2021-03-07 01:57:21 +00:00
Johnothan King
c1986c4e1a
Fix Ctrl+D after ksh receives SIGWINCH (#208)
src/cmd/ksh93/edit/edit.c: ed_read():
- The loop that handles SIGWINCH assumes sfpkrd will return and
  set errno to EINTR if ksh is sent SIGWINCH. This only occurs
  when select(2) is used to wait for input, so tell sfpkrd to
  use select if possible. This is only done if the last argument
  given to sfpkrd is '2', which should avoid regressions.

src/lib/libast/sfio/sfpkrd.c: sfpkrd():
- Always use select if the last argument is 2. This allows
  sfpkrd() to intercept SIGWINCH when necessary.

Fixes: https://github.com/ksh93/ksh/issues/202
2021-03-06 06:43:38 +00:00
Martijn Dekker
7a0934a8d6 libast: remove antiquated macOS bug workaround
That Mac OS X bug workaround is now 23 days shy of the age of
majority, and that bug (symlinks testing as regular files) is
pretty basic, so I'm betting it's fixed by now.

src/lib/libast/include/ast_dir.h:
- Do not disable D_TYPE on macOS.
2021-03-04 23:46:20 +00:00
Martijn Dekker
e3882fe71b Only run pty tests on systems where pty is known to be good
On some systems (AIX, HP-UX, OpenBSD) the pty tests may hang.

On all systems except Darwin/macOS, FreeBSD and Linux, the pty
tests show one or more regressions. But when I try out the failing
tests manually in a real session, it seems to work fine. So I
suspect pty is broken and not ksh.

src/cmd/ksh93/tests/pty.sh:
- For now, only run the pty tests on Darwin, FreeBSD and Linux.

src/lib/libast/Mamfile:
- tvsleep.c: Add missing error.h dependency (re: 2f7918de).
  (unrelated, but just wasn't worth its own commit)
2021-02-23 10:54:56 +00:00
Johnothan King
733f70e94b
Fix many compiler warnings and remove unused variables (#191)
Most of these changes remove unused variables, functions and labels
to fix -Wunused compiler warnings. Somewhat notable changes:

src/cmd/ksh93/bltins/print.c:
- Removed the unused 'neg' variable.
  Patch from ksh2020: https://github.com/att/ast/pull/725

src/cmd/ksh93/bltins/sleep.c:
- Initialized ns to fix three -Wsometimes-uninitialized warnings.

src/cmd/ksh93/edit/{emacs,vi}.c:
- Adjust strncpy size to fix two -Wstringop-truncation warnings.

src/cmd/ksh93/include/shell.h:
- The NOT_USED macro caused many -Wunused-value warnings,
  so it has been replaced with ksh2020's macro:
  https://github.com/att/ast/commit/19d0620a

src/cmd/ksh93/sh/expand.c:
- Removed an unnecessary 'ap = ' since 'ap' is never read
  between stakseek and stakfreeze.

src/cmd/ksh93/edit/vi.c: refresh():
- Undef this function's 'w' macro at the end of it to stop it
  potentially interfering with future code changes.

src/cmd/ksh93/sh/nvdisc.c,
src/lib/libast/misc/magic.c,
src/lib/libast/regex/regsubexec.c,
src/lib/libast/sfio/sfpool.c,
src/lib/libast/vmalloc/vmbest.c:
- Fixed some indentation to silence -Wmisleading-indentation
  warnings.

src/lib/libast/include/ast.h:
- For clang, now only suppress hundreds of -Wparentheses warnings
  as well as a few -Wstring-plus-int warnings.
  Clang's -Wparentheses warns about things like
  	if(foo = bar())
  which assigns to foo and checks the assigned value.
  Clang wants us to change this into
  	if((foo = bar()))
  Clang's -Wstring-plus-int warns about things like
  	"string"+x
  where x is an integer, e.g. "string"+3 represents the string
  "ing". Clang wants us to change that to
  	"string"[3]
  The original versions represent a perfectly valid coding style
  that was common in the 1980s and 1990s and is not going to change
  in this historic code base. (gcc does not complain about these.)

Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-02-22 22:16:32 +00:00
Martijn Dekker
18529b88c6 Add lots of checks for out of memory (re: 0ce0b671)
Huge typeset -L/-R adjustment length values were still causing
crashses on sytems with not enough memory. They should error out
gracefully instead of crashing.

This commit adds out of memory checks to all malloc/calloc/realloc
calls that didn't have them (which is all but two or three).

The stkalloc/stakalloc calls don't need the checks; it has
automatic checking, which is done by passing a pointer to the
outofspace() function to the stakinstall() call in init.c.

src/lib/libast/include/error.h:
- Change the ERROR_PANIC exit status value from ERROR_LEVEL (255)
  to 77, which is what it is supposed to be according to the libast
  error.3 manual page. Exit statuses > 128 for anything else than
  signals are not POSIX compliant and may cause misbehaviour.

src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/init.c:
- To facilitate consistency, add a simple extern sh_outofmemory()
  function that throws an ERROR_PANIC "out of memory".

src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/data/builtins.c:
- Remove now-redundant e_nospace[] extern message; it is now only
  used in one place so it might as well be a string literal in
  sh_outofmemory().

All other changed files:
- Verify the result of all malloc/calloc/realloc calls and call
  sh_outofmemory() if they fail.
2021-02-21 22:27:28 +00:00
Johnothan King
2b805f7f1c
Fix many spelling errors and word repetitions (#188)
Many of the errors fixed in this commit are word repetitions
such as 'the the' and minor spelling errors. One formatting
error in the ksh man page has also been fixed.
2021-02-20 03:22:24 +00:00
Lev Kujawski
4e47f89b06
libast: Improve and harden tvsleep (re: 72968eae) (#186)
On SVR4 platforms, select is a sometimes erroneous wrapper around
the poll system call, so call poll directly for efficiency purposes if
it implies no loss in precision.

See, e.g., http://bugs.motifzone.net/long_list.cgi?buglist=129 .

src/lib/libast/features/tvlib:
- For systems lacking nanosleep, test whether select is truly more
  precise than poll.

src/lib/libast/tm/tvsleep.c:
- Verify that tv argument is not null.
- Immediately return if asked to sleep for a duration of zero.
- Immediately initialize timespec in the nanosleep case.
- Revise variable names to use Apps Hungarian.
- Use poll instead of select when there is no loss in precision.
- Check for overflow in the poll case.
- Improve comments.
- Revise arithmetic to work with unsigned types, rather than
  casting to long.
- Do not return non-zero if we have slept for a sufficient
  time.
2021-02-19 22:58:04 +00:00
Lev Kujawski
72968eaed6
Revise the pow(1,inf) IEEE feature test to defeat clever compilers (#184)
Compilers like GCC are capable of optimizing away calls like
pow(1,inf), which caused the IEEE compliance feature test within
libast to incorrectly succeed on platforms with non-IEEE behavior.

This is arguably a bug within GCC, as floating point optimizations
should never alter the behavior of code unless IEEE compliance is
explicitly disabled via a flag like -ffast-math. Programs in which
only some calls to pow are optimized away are liable to severely
malfunction.

Thanks to Martijn Dekker for pointing this issue out and the kind
operators of polarhome.com for permitting me gratis use of their
Unix systems.

src/lib/libast/comp/omitted.c:
- Add IEEE compliant functions that wrap powf, pow, and powl.

src/lib/libast/features/float:
- Look for powf, pow, and powl in the C library.
- For compilers that do the right thing, like the native toolchains
  of Solaris and UnixWare, use lightweight macros to wrap the pow
  functions.
- Use a volatile function pointer through which to access the C
  library's pow function in an attempt to defeat code optimization.
- For these overzealous compilers, define pow to _ast_pow so that
  the same technique can be used within the above functions.
2021-02-18 18:45:27 +00:00
Martijn Dekker
af5f7acf99 Fix bugs related to --posix shell option (re: 921bbcae, f45a0f16)
This fixes the following:
1. 'set --posix' now works as an equivalent of 'set -o posix'.
2. The posix option turns off braceexpand and turns on letoctal.
   Any attempt to override that in a single command such as 'set -o
   posix +o letoctal' was quietly ignored. This now works as long
   as the overriding option follows the posix option in the command.
3. The --default option to 'set' now stops the 'posix' option, if
   set or unset in the same 'set' command, from changing other
   options. This allows the command output by 'set +o' to correctly
   restore the current options.

src/cmd/ksh93/data/builtins.c:
- To make 'set --posix' work, we must explicitly list it in
  sh_set[] as a supported option so that AST optget(3) recognises
  it and won't override it with its own default --posix option,
  which converts the optget(3) string to at POSIX getopt(3) string.
    This means it will appear as a separate entry in --man output,
  whether we want it to or not. So we might as well use it as an
  example to document how --optionname == -o optionname, replacing
  the original documentation that was part of the '-o' description.

src/cmd/ksh93/sh/args.c: sh_argopts():
- Add handling for explitit --posix option in data/builtins.c.
- Move SH_POSIX syncing SH_BRACEEXPAND and SH_LETOCTAL from
  sh_applyopts() into the option parsing loop here. This fixes
  the bug that letoctal was ignored in 'set -o posix +o letoctal'.
- Remember if --default was used in a flag, and do not sync options
  with SH_POSIX if the flag is set. This makes 'set +o' work.

src/cmd/ksh93/include/argnod.h,
src/cmd/ksh93/data/msg.c,
src/cmd/ksh93/sh/args.c: sh_printopts():
- Do not potentially translate the 'on' and 'off' labels in 'set
  -o' output. No other shell does, and some scripts parse these.

src/cmd/ksh93/sh/init.c: sh_init():
- Turn on SH_LETOCTAL early along with SH_POSIX if the shell was
  invoked as sh; this makes 'sh -o' and 'sh +o' show expected
  options (not that anyone does this, but correctness is good).

src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/include/shell.h:
- The state flags were in defs.h and most (but not all) of the
  shell options were in shell.h. Gather all the shell state and
  option flag definitions into one place in shell.h for clarity.
- Remove unused SH_NOPROFILE and SH_XARGS option flags.

src/cmd/ksh93/tests/options.sh:
- Add tests for these bugs.

src/lib/libast/misc/optget.c: styles[]:
- Edit default optget(3) option self-documentation for clarity.

Several changed files:
- Some SHOPT_PFSH fixes to avoid compiling dead code.
2021-02-14 23:51:19 +00:00
Lev Kujawski
e2d54b7169
sleep: guarantee sleeping specified time at minimum (#174)
With this patch, the Korn shell can now guarantee that calls to
sleep on systems using the select or poll method always result in
the system clock advancing by that much time, assuming no
interruptions. This compensates for deficiencies in certain
systems, including SCO UnixWare.

Discussion: https://github.com/ksh93/ksh/pull/174

src/lib/libast/tm/tvsleep.c:
- Ensure that at least the time requested to sleep has elapsed
  for the select and poll methods.
- Simplify the logic of calculating the time remaining to
  sleep and handle the case of an argument of greater than
  10e9 nanoseconds being passed to tvsleep.

src/cmd/ksh93/bltins/sleep.c:
- Eliminate the check for EINTR to handle other cases wherein
  we have not slept enough.

src/cmd/ksh93/tests/variables.sh:
- Improve the diagnostic message when the sleep test fails.
- Revise the SECONDS function test to expect that we always
  sleep for at least the time specified.

src/cmd/ksh93/tests/functions.h:
- Redirect ps stderr to /dev/null. UnixWare ps prints an error
  message about not being able to find the controlling terminal
  when shtests output is piped, but we are only using ps to find
  the PID.
2021-02-14 07:27:04 +00:00
Martijn Dekker
21d00ffb6c Fix build on QNX
This makes ksh 93u+m build on the following system:

	$ uname -a
	QNX qnx 6.5.0 2010/07/09-14:44:03EDT x86pc x86

Thanks to polarhome.com for providing the QNX shell account.

There are a number of regressions left to work out:

arrays.sh[636]: copying a large array fails
bracket.sh[129]: /tmp/ksh93.shtests.1753215026.6923/bracket.C/original should be older than /tmp/ksh93.shtests.1753215026.6923/bracket.C/newer
bracket.sh[132]: /tmp/ksh93.shtests.1753215026.6923/bracket.C/newer should be newer than /tmp/ksh93.shtests.1753215026.6923/bracket.C/original
builtins.sh[683]: real_t1 not found after parent directory renamed in subshell
functions.sh[1023]: cannot handle comsub depth > 256 in function
io.sh[252]: <# not working for pipes
io.sh[337]: read -n3 from pipe not working
io.sh[346]: read -n3 from fifo failed -- expected 'a', got 'abc'
io.sh[349]: read -n1 from fifo failed -- expected 'b', got 'd'
io.sh[379]: should have timed out
io.sh[380]: line1 should be 'prompt1: '
io.sh[381]: line2 should be line2
io.sh[382]: line3 should be 'prompt2: '
io.sh[406]: LC_ALL=C read -n2 from pipe 'a bcd' failed -- expected 'a bcd', got 'ab cd'
io.sh[406]: LC_ALL=C.UTF-8 read -n2 from pipe 'a bcd' failed -- expected 'a bcd', got 'ab cd'
jobs.sh[86]: warning: skipping subshell job control test due to non-compliant 'ps'
pty.sh[105]: POSIX sh 026(C): line 120: expected "(Stopped|Suspended)", got EOF
pty.sh[128]: POSIX sh 028(C): line 143: expected "(Stopped|Suspended) \(SIGTTIN\)", got EOF
pty.sh[151]: POSIX sh 029(C): line 166: expected "(Stopped|Suspended) \(SIGTTOU\)", got EOF
signal.sh[310]: kill -TERM $$ failed, required termination by signal 'EXIT'
signal.sh[310]: kill -VTALRM $$ failed, required termination by signal 'EXIT'
signal.sh[310]: kill -PIPE $$ failed, required termination by signal 'EXIT'

(The io.sh failures mean libast sfpkrd() is not working.)

src/lib/libast/obsolete/spawn.c:
- Removed. Didn't compile due to wrong number of arguments to
  spawnve(2), but is obsolete and unused.

src/lib/libast/comp/localeconv.c:
- The initialisation of two static 'struct lconv' variables was
  done in a way that depended on OS headers declaring the struct
  members in a certain order. This holds on most systems, but not
  on QNX, and POSIX does not actually specify the order at all:
	https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/locale.h.html
  So each member must be initialised by name. But C89 does not
  support initialising struct members by name, so we have to do it
  using an initialiser function that simply assigns the values.

src/lib/libast/comp/spawnveg.c:
- Fix for systems without either P_DETACH or _P_DETACH.

src/lib/libast/features/vmalloc,
src/lib/libast/vmalloc/vmmopen.c,
src/lib/libast/Mamfile:
- Add test for sys/shm.h header. If it doesn't exist, as it doesn't
  on QNX, use the stub vmmapopen() as the real one won't compile.
  (Mamfile: Add dependency on FEATURE/vmalloc to vmmopen.c.)

src/lib/libast/vmalloc/malloc.c:
- Remove superfluous externs that are already provided by either
  AST or system headers. The 'void cfree' extern caused a build
  failure on QNX because cfree() is of type int on QNX.

src/lib/libast/comp/conf.tab:
- Remove check for _map_spawnve; src/lib/libast/RELEASE says it was
  removed.
2021-02-14 01:28:35 +00:00
Lev Kujawski
a9e7012dfd Fix a potential read of uninitialized memory through a pointer
src/lib/libast/misc/magic.c:
- Use strncpy instead of memcpy to avoid reading past the null
  terminator of the string pointed to by p.
2021-02-12 13:46:07 +00:00
Lev Kujawski
d5a94b3722 Produce IEEE compliant output from pow() despite platform deviations
src/cmd/ksh93/features/math.sh:
- Specify ast_float.h within iffehdrs instead of math.h, so that iffe
  will pick up on macro substitutions within libast. This should make
  any future efforts to remedy floating point behavior easier as well.
- Always include ast_float.h within the generated math header file,
  not just on IA64 platforms.

src/cmd/ksh93/tests/arith.sh:
- Test pow(1.0,-Inf) and pow(1.0,NaN) for IEEE compliance as well.
- Test the exponentiation operator (**) in addition, as streval.c,
  which processes the same, calls pow() separately.

src/lib/libast/features/float:
- Test the IEEE compliance of the underlying math library's pow()
  function and substitute macros producing compliant behavior if
  necessary.
2021-02-12 13:23:16 +00:00
Martijn Dekker
a9d77bba40 libast: optget(): emphasis: add screen/tmux, dtterm (re: ec79563b)
src/lib/libast/misc/optget.c:
- Add screen* (which includes tmux) and dtterm* (CDE terminal) to
  the glob pattern deciding whether to use ANSI boldface sequences.
- Don't bother parsing the env var if stderr is not on a terminal.

src/cmd/ksh93/sh.1:
- Extend self-documentation documentation; document how optget(3)
  uses the ERROR_OPTIONS env var to control boldface output.
- Tweaks and minor edits.
2021-02-07 20:42:52 +00:00
Martijn Dekker
ec79563b8f libast: optget(): fix emphasis state initialisation
This fixes a bug in libast optget()'s use of emphasis in the
display of --man(uals) via standard error on a terminal.

Symptom:

$ printf --man 2>&1 | more
(ok; emphasis disabled, no escape codes shown)
$ printf --man
(ok; emphasis correctly displayed)
$ printf --man 2>&1 | more
(whoops; emphasis not disabled; escape codes garble 'more' output)

The problem was that the state.emphasis variable was not
initialised and, when set to one, was never reset again
(except through the use of the --api, --html or --nroff option).

The source code also reveals an undocumented feature: if the
environment variable $ERROR_OPTIONS contains 'noemphasi', emphasis
is forced off, else if it contains 'emphasi', it's forced on.
Other characters (such as the final 's' of emphasis) are ignored.
This was also broken (forcing off didn't work) and is now fixed.

src/lib/libast/misc/optget.c:
- Do not assume that enabling emphasis is forever; re-initialise
  the state on every relevant getopts invocation.
- Increase the number of terminals on which emphasis is displayed
  using ANSI escape codes. (This is a hack and we should ask the OS
  for the correct codes, but never mind -- ANSI is now universal.)
2021-02-07 05:53:48 +00:00
Martijn Dekker
43dfe3c8fa build system: rm unused variable declarations from Mamfiles
Now that the Make Abstract Machine files are maintained manually
and not generated automatically, unused variables are an annoying
distraction -- and there are many.

But the language/format is very simple and very parseable using
shell, awk, etc. -- so this was easy to automate. All variables are
declared with 'setv' and they are used if an expansion of the form
${varname} exists (the braces are mandatory in Mamfiles).

bin/Mamfile_rm_unused_vars:
- Added for reference and future use.

src/*/*/Mamfile:
- Remove all unused 'setv' variable declarations.
2021-02-05 15:39:31 +00:00
Martijn Dekker
e220445265 Re-allow disabling vmalloc for release builds (re: 399886da)
Well, that commit was based on a silly oversight: of course it's
necessary to pass ${KSH_RELFLAGS} to the feature tests too as they
use this flag to determine whether to enable or disable vmalloc.

On further analysis I think the annoying warnings can be solved in
a different way. Quotes (single or double) in 'exec -' commands
don't seem to be special to mamake at all; it looks like they are
passed on to the shell as is. So Mamfile variables are expanded and
the expansions backslash-escaped the same way regardless of quotes.
Which means we can make the shell remove the unwanted level of
backslashes by using double instead of single quotes.

src/*/*/Mamfile:
- On iffe commands, restore ${KSH_RELFLAGS}, using double quotes to
  group the compiler command as one argument to iffe.
2021-02-04 01:40:53 +00:00
Martijn Dekker
7dbcbce407 Revert 7a6980a3, restoring build on Alpine Linux
This reverts an OpenSUSE patch ("libast/comp/conf.sh: apply limits
detection fixes for Linux"). It broke the build on Alpine Linux
with the musl C library (see also e245856f).
2021-02-03 21:44:55 +00:00
Martijn Dekker
f5eaf217ed build system: fix detection of default system directories
src/cmd/INIT/iffe.sh:
- Fix "standard system directories" for the cmd test, which were
  hardcoded as bin, /etc, /usr/bin, /usr/etc, /usr/ucb. That's both
  unportable and antiquated. Replace this with the path output by
  'getconf PATH'.
- Add fixes from modernish for 'getconf PATH' output to compensate
  for bugs/shortcomigns in NixOS and AIX. Source:
  https://github.com/modernish/modernish/blob/9e4bf5eb/lib/modernish/aux/defpath.sh
  Ref.: https://github.com/NixOS/nixpkgs/issues/65512

src/lib/libast/comp/conf.tab: PATH:
- Add the NixOS and AIX default path fixes here too; this fixes
  'command -p' and the builtin 'getconf PATH' on these systems.

bin/package, src/cmd/INIT/package.sh:
- Re-support being launched with just the command name 'package' in
  the command line (if the 'package' command is in $PATH). At least
  one other script in the build system does this. (re: 6cc2f6a0)
- Go back three levels (../../..) if we were invoked from
  arch/*/bin/package, otherwise we won't find src/cmd/ksh93/SHOPT.sh.
2021-02-03 17:49:00 +00:00
Martijn Dekker
be33942415 Some more build fixes from OpenSUSE
These update some outdated library path searches as well as
a couple of gcc-specific improvements.

Original patch:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-compat.dif
2021-02-02 21:03:01 +00:00
Martijn Dekker
1bd0620708 libast: sfio(3) fixes from ksh 93v- beta
These fixes were backported by OpenSUSE. Original patch:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-sfio.dif
2021-02-02 16:20:59 +00:00
Martijn Dekker
db71b3ad43 libast: robustness fixes for temporary files and streams
From an OpenSUSE patch:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-pathtemp.dif

See src/lib/libast/man/path.3 for pathtemp()
and src/lib/libast/man/sfio.3 for sftmp()

src/lib/libast/path/pathtemp.c:
- Error check fix: add an access check wrapper function that checks
  if a path was given and if there is enough free space on the
  device, setting errno appropriately in case of trouble.

src/lib/libast/sfio/sftmp.c:
- On Linux, use the /dev/shm shared memory objects for the new
  temporary file descriptor -- that is, do not access HD or SSD but
  only the memory based tmpfs of the POSIX SHM.
2021-02-02 14:44:34 +00:00
Martijn Dekker
3002c4113e vmalloc: better detect default region size on Linux
This is a Linux-specific fix for vmalloc from OpenSUSE, for what
that's still worth; we're not going to keep using vmalloc long term.
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-vm.dif
2021-02-02 11:32:37 +00:00
Martijn Dekker
e2c2753ee8 libast: Fix detection of GCC 4.1+ 64/32-bit atomic operations model
This fix to the atomic scalar operations feature tests is from OpenSUSE:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-aso.dif
2021-02-02 05:27:08 +00:00
Martijn Dekker
627df2b19a libast: optget(3): fix some uninitialised variables
This is a part of this OpenSUSE patch:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-gcc.dif
2021-02-02 05:17:52 +00:00
Martijn Dekker
0684806d47 feature tests: signbit detection fix for ia64
An Intel Itanium fix from OpenSUSE:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-ia64.dif
2021-02-02 04:40:09 +00:00
Martijn Dekker
7a6980a380 libast/comp/conf.sh: apply limits detection fixes for Linux
This is another patch from OpenSUSE:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-limits.dif
2021-02-02 04:35:54 +00:00
Martijn Dekker
8a58166bc4 Add OpenSUSE fixes for the uname builtin on Linux
Original patches:
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-uname.dif
https://build.opensuse.org/package/view_file/shells/ksh/ksh93-no-sysctl.dif
2021-02-02 03:31:21 +00:00
Martijn Dekker
f37098f177 Build fix for Linux i386
iffe feature test that add a -D_LARGEFILE64_SOURCE compiler flag to
detect the presence of 64-bit types like off64_t are very
incorrect; they always find the type even if the rest of the source
is not compiled with that flag, causing an inconsistent compilation
environment. This was the cause of mysterious failures to compile
some feature tests on Linux i386 -- it tried to use an off64_t type
that was wrongly detected.

A flag like -D_LARGEFILE64_SOURCE needs to be added to the compiler
flags consistently so it is used for compiling all files and tests.

src/lib/libast/features/dirent,
src/lib/libast/features/fs,
src/lib/libast/features/lib,
src/lib/libast/features/mmap,
src/cmd/ksh93/features/rlimits:
- Remove the -D_LARGEFILE64_SOURCE flag from all the tests that
  used it.
- Fix some preprocessor directives for compiling without
  _LARGEFILE64_SOURCE. We cannot rely on the result of the _lib_*64
  tests because those functions are still found in glibc even if
  _LARGEFILE64_SOURCE is not defined; we have to check for the
  existence of the type definitions before using them.

src/cmd/INIT/cc.linux.i386,
src/cmd/INIT/cc.linux.i386-icc:
- Add/update compiler wrappers to hardcode -D_LARGEFILE64_SOURCE
  in the flags for the default compiler. If it is overriden with
  $CC, then it needs to be added manually if desired.
2021-01-31 23:47:43 +00:00