1
0
Fork 0
mirror of git://git.code.sf.net/p/cdesktopenv/code synced 2025-02-13 11:42:21 +00:00
Commit graph

767 commits

Author SHA1 Message Date
hyenias
a61430f1b5
Readonly attribute size fix (#201)
Corrected the size of attribute(s) being overwritten with 0 when
'readonly' or 'typeset -r' was applied to an existing variable. Since
one cannot set any attributes with the 'readonly' command, its function
call to setall() needs to be adjusted to acquire the current size from
the old size or existing size of the variable. A plain 'typeset -r' is
the same as 'readonly' in that it needs to load the old size as its
current size for use in the subsequent to call to nv_newattr().

src/cmd/ksh93/bltins/typeset.c: setall():
- Both 'readonly' and 'typeset -r' end up calling setall(). setall()
  has full visibility into all user supplied values and existing
  values that are needed to differentiate whereas name.c newattr()
  acquires combined state flags.
- Added a conditional check if the readonly flag was requested by
  user then meets the criteria of having present size of 0, cannot
  be a numeric nor binary string, and is void of presence of any of
  the justified string attributes.
- -L/R/Z justified string attributes if not given a value default
  to a size of 0 which means to autosize. A binary string can have
  a fixed field size, e.g. -bZ. The present of any of the -L/R/Z
  attribules means that current size is valid and should be used
  even if it is zero.

src/cmd/ksh93/tests/attributes.sh:
- Added various tests to capture and reiterate that 'readonly' should
  be equivalent to 'typeset -r' and applying them should not alter the
  previous existing size unless additional attributes are set along
  with typeset command.
2021-03-03 03:26:39 +00:00
Martijn Dekker
6146848693 Fix compiling with SHOPT_REGRESS and SHOPT_P_SUID
src/cmd/ksh93/Mamfile:
- regress.c: add missing SH_DICT define for getopt self-doc string,
  needed after USAGE_LICENSE macros were removed. (re: ede47996)

src/cmd/ksh93/init.c: sh_init():
- Do not set error_info.exit early in init. This is the function
  that is called when an error exits the shell. It defaults to
  exit(3). Setting it to sh_exit() early on can cause a crash if an
  error is thrown before shell initialisation is fully finished.
  So set it at the end of sh_init() instead.
- __regress__: Remove error_info.exit workaround. (re: 506bd2b2)
- Fix SHOPT_P_SUID directive. This is not actually a 0/1 value, so
  we should use #ifdef and not #if. If SHOPT_REGRESS is on, it it
  set to a function call. (re: 2182ecfa)

src/cmd/ksh93/SHOPT.sh:
- Document that SHOPT_P_SUID cannot be set to 0 to be turned off.
2021-02-28 23:24:58 +00:00
Martijn Dekker
5d82004426 Misc regression test fixes
src/cmd/ksh93/tests/basic.sh:
- Fix syntax error (unbalanced single quote) in two -c script
  invocations. It only failed to throw a syntax error due to a
  problematic hack in ksh that may be removed soon.
  See: https://github.com/ksh93/ksh/issues/199

src/cmd/ksh93/tests/builtins.sh,
src/cmd/ksh93/tests/io.sh:
- Redirect standard error on two ksh -i invocations to /dev/null
  to work around the test hanging on AIX.

src/cmd/ksh93/tests/comvario.sh:
- Remove duplicate copyright header.
- Fix warning format.

src/cmd/ksh93/tests/functions.sh:
- Fix the 'TERM signal sent to last process of function kills the
  script' test so that it works on AIX. We cannot rely on grepping
  'ps' output as the external 'sleep' command does not show the
  command name on AIX. Instead, find it by its parent PID.

src/cmd/ksh93/tests/locale.sh,
src/cmd/ksh93/tests/substring.sh:
- Rewrite the very broken multibyte locale tests (two outright
  syntax errors due to unbalanced quotes, and none of the tests
  actually worked).
- Since they set LC_ALL, move them to locale.sh.

src/cmd/ksh93/tests/variables.sh:
- Redirect stderr on some 'ulimit -t unlimited' invocations (which
  fork subshells as the intended side effect) to /dev/null in case
  that exceeds a system-defined limit.
2021-02-28 21:57:38 +00:00
Johnothan King
7ad274f8b6
Add more out of memory checks (re: 18529b88) (#192)
The referenced commit neglected to add checks for strdup() calls.
That calls malloc() as well, and is used a lot.

This commit switches to another strategy: it adds wrapper functions
for all the allocation macros that check if the allocation
succeeded, so those checks don't need to be done manually.

src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/init.c:
- Add sh_malloc(), sh_realloc(), sh_calloc(), sh_strdup(),
  sh_memdup() wrapper functions with success checks. Call nospace()
  to error out if allocation fails.
- Update new_of() macro to use sh_malloc().
- Define new sh_newof() macro to replace newof(); it uses
  sh_realloc().

All other changed files:
- Replace the relevant calls with the wrappers.
- Remove now-redundant success checks from 18529b88.
- The ERROR_PANIC error message calls are updated to inclusive-or
  ERROR_SYSTEM into the exit code argument, so libast's error()
  appends the human-readable version of errno in square brackets.
  See src/lib/libast/man/error.3

src/cmd/ksh93/edit/history.c:
- Include "defs.h" to get access to the wrappers even if KSHELL is
  not defined.
- Since we're here, fix a compile error that occurred with KSHELL
  undefined by updating the type definition of hist_fname[] to
  match that of history.h.

src/cmd/ksh93/bltins/enum.c:
- To get access to sh_newof(), include "defs.h" instead of
  <shell.h> (note that "defs.h" includes <shell.h> itself).

src/cmd/ksh93/Mamfile:
- enum.c: depend on defs.h instead of shell.h.
- enum.o: add an -I. flag in the compiler invocation so that defs.h
  can find its subsequent includes.

src/cmd/builtin/pty.c:
- Define one outofmemory() function and call that instead of
  repeating the error message call.
- outofmemory() never returns, so remove superfluous exit handling.

Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-02-27 21:21:58 +00:00
Martijn Dekker
c928046aa9 Fix ${.sh.fun} leaking out of DEBUG trap
The value of the ${.sh.fun} variable, which is supposed to contain
the name of the function currently being executed, leaks out of the
DEBUG trap if it executes a function. Reproducer:

$ fn() { echo "executing the function"; }
$ trap fn DEBUG
$ trap - DEBUG
executing the function
$ echo ${.sh.fun}
fn

${.sh.fun} should be empty outside the function.

Annalysis:

The sh_debug() function in xec.c, which executes the DEBUG trap
action, contains these lines, which are part of restoring the state
after running the trap action with sh_trap():

	nv_putval(SH_PATHNAMENOD,shp->st.filename,NV_NOFREE);
	nv_putval(SH_FUNNAMENOD,shp->st.funname,NV_NOFREE);
 	shp->st = savst;

First the SH_PATHNAMENOD (${.sh.file}) and SH_FUNNAMENOD
(${.sh.fun}) variables get restored from the values in the shell's
scoped information struct (shp->st), but that is done *before*
restoring the parent scope with 'shp->st = savst;'. It should be
done after. Fixing the order is sufficient to fix the bug.

However, I am not convinced that these nv_putval() calls are good
for anything at all. Setting, unsetting, restoring, etc. the
${.sh.fun} and ${.sh.file} variables is already being handled
perfectly well elsewhere in the code for executing functions and
sourcing dot scripts. The DEBUG trap is neither here nor there.
There's no reason for it to get involved with these variables.

I was unable to break anything after simply removing those two
lines. So I strongly suspect this is another case, out of many now,
where a bug in ksh93 is properly fixed by removing some code.

I couldn't get ${.sh.file} to leak similarly -- I think this is
because SH_PATHNAMENOD (and not SH_FUNNOD) is set explicitly in
exfile() in main.c, masking this incorrect restore. It is the only
place where SH_PATHNAMENOD and SH_FUNNOD are not both set.

src/cmd/ksh93/sh/xec.c:
- Remove these two spurious nv_putval() calls.

src/cmd/ksh93/tests/variables.sh:
- Add regression test for leaking ${.sh.fun}.
2021-02-27 01:25:59 +00:00
Martijn Dekker
ef8b80cfd7 edit.c: make tput invocation work in restricted mode (re: 7ff6b73b)
At init, and then whenever the TERM variable changes, ed_setup()
uses sh_trap() to run the external 'tput' command to get the
current terminal escape sequence for moving up the cursor one line.

A sh_trap() call executes a shell command as if a shell script's
trap action had executed it, so is subject to modes like the
restricted mode. As of 7ff6b73b, we execute tput using its absolute
path (found and hardcoded at compile time) for better
robustness/security. This fails in restricted mode as it does not
allow executing commands by absolute path. But in C, nothing stops
us from turning that off.

src/cmd/ksh93/edit/edit.c: ed_setup():

- Block SIGINT while doing all of the following, so the user can't
  interrupt it and escape from restricted mode. Even without that,
  it's probably a good idea to do this, so an interrupt doesn't
  cause an inconsistent state.
      Note that sigblock() and sigrelease() are macros defined in
  features/sigfeatures. To get those, we need to include <fault.h>.

- Temporarily turn off SH_RESTRICTED before sh_trap()ping tput to
  get the terminal command to move the cursor up one position.

- Avoid potentially using a sequence that was cut off. Only use the
  resulting string if its length does not exceed the space reserved
  for CURSOR_UP. Otherwise, empty it.

src/cmd/ksh93/Mamfile:
- Add fault.h dependency to edit.c.

src/cmd/ksh93/edit/history.c:
- Fix typos in introductory comment.
2021-02-26 12:58:40 +00:00
Martijn Dekker
d9865ceae1 emacs: Fix three tab completion bugs
1. The editor accepted literal tabs without escaping in certain
   cases, causing buggy and inconsistent completion behaviour.
   https://github.com/ksh93/ksh/issues/71#issuecomment-656970959
   https://github.com/ksh93/ksh/issues/71#issuecomment-657216472

2. After completing a filename by choosing from a file completion
   menu, the terminal cursor was placed one position too far to the
   right, corrupting command line display. This happened with
   multiline active.
   https://github.com/ksh93/ksh/issues/71#issue-655093805

3. A completion menu was displayed if the file name to be completed
   was at the point where the rest of it started with a number,
   even if that part uniquely identified it so the menu had 1 item.
   https://www.mail-archive.com/ast-users@lists.research.att.com/msg00436.html

src/cmd/ksh93/edit/emacs.c:

- Cosmetic consistency: change two instances of cntl('[') to ESC.

- ed_emacsread(): Fix number 1 by refusing to continue into default
  processing if a tab character was not used for tab completion.
  Instead, beep and continue to the next read loop iteration. This
  behaviour is consistent with most other shells, so I doubt there
  will be objections. To enter a literal tab it's simple enough to
  escape it with ^V (the 'stty lnext' character) or \.

- draw(): Fix number 2 by correcting an off-by-one error in the
  ed_setcursor() call that updates the terminal's cursor display
  in multiline mode. The 'old' and 'new' parameters need to have
  identical values in this particular call to avoid the cursor
  position being off by one to the right. This change makes it
  match the corresponding ed_setcursor() call in vi.c. See below*
  for details. Thanks to Lev Kujawski for the help in analysing.

src/cmd/ksh93/edit/completion.c: ed_expand():

- Fix number 3 by changing from '=' mode (menu-based completion) to
  '\' mode (ordinary filename completion) if the menu would only
  show one option, which was pointless and annoying. This never
  happened in vi mode, so possibly the ed_expand() call in emacs.c
  could have been improved instead. But I'm comfortable with fixing
  it here and not in emacs.c, because this fixes it at a more
  fundamental level, plus it's straightforward and obvious here.

Resolves: https://github.com/ksh93/ksh/issues/71
____
* Further details on bug number 2:

At https://github.com/ksh93/ksh/issues/71#issuecomment-786391565
Martijn Dekker wrote:
> I'm back to my original hypothesis that there is somehow an
> off-by-one error related to the ed_setcursor() call that gets
> executed when in multiline mode. I cannot confirm whether that
> off-by-one error is actually in the call itself, or occurs
> sometime earlier on one of the many possible occasions where
> ep->cursor is changed. But everything else appears to work
> correctly, so it's not unlikely that the problem is in the call
> itself.
>
> For reference, this is the original version of that call in
> emacs.c:
>
> ksh/src/cmd/ksh93/edit/emacs.c
> Lines 1556 to 1557 in df2b9bf
>  if(ep->ed->e_multiline && option == REFRESH)
>  	ed_setcursor(ep->ed, ep->screen, ep->cursor-ep->screen, ep->ed->e_peol, -1);
>
> There is a corresponding call in the vi.c refresh() function
> (which does the same thing as draw() in emacs.c), where the third
> (old) and fourth (new) arguments are actually identical:
>
> ksh/src/cmd/ksh93/edit/vi.c
>
> Lines 2086 to 2087 in df2b9bf
>  if(vp->ed->e_multiline && vp->ofirst_wind==INVALID)
>  	ed_setcursor(vp->ed, physical, last_phys+1, last_phys+1, -1);
>
> The expectation for this particular call is in fact that they
> should be identical, so that a delta of zero is calculated in
> that function. Delta not being zero is what causes the cursor to
> be positioned wrong.
>
> In vi.c, last_phys is a macro that is defined as editb.e_peol,
> and editb is a macro that is defined as (*vp->ed). Which means
> last_phys means vp->ed->e_peol, which is the same as
> ep->ed->e_peol in emacs.c. (These editors were originally
> separate programs by different authors, and I suppose this is how
> it shows. Korn didn't want to change all the variable names to
> integrate them, so made macros instead.)
>
> That leaves the question of why vi.c adds 1 to both last_phys
> a.k.a. e_peol arguments, and emacs.c uses e_peol for new without
> adding anything. Analysing the ed_setcursor() code could answer
> that question.
>
> So, this patch makes emacs.c do it the same way vi.c does. Let's
> make the third argument identical to the fourth. My brief testing
> shows the bug is fixed, and the regression tests yield no
> failures. This fix is also the most specific change possible, so
> there are few opportunities for side effects (I hope).

At https://github.com/ksh93/ksh/issues/71#issuecomment-786466652
Lev Kujawski wrote:
> I did a bit of research on this, and I think the fix to have the
> Emacs editing mode do the same as Vi is correct.
>
> From RELEASE:
> 08-05-01 In multiline edit mode, the refresh operation will now clear
> the remaining portion of the last line.
>
> Here's a fragment from the completion.c of the venerable but
> dated CDE DtKsh:
>
>                 else
>                         while (*com)
>                         {
>                                 *out++  = ' ';
>                                 out = strcopy(out,*com++);
>                         }
>                 *cur = (out-outbuff);
>                 /* restore rest of buffer */
>                 out = strcopy(out,stakptr(0));
>                 *eol = (out-outbuff);
>
> Noticeably missing is the code to add a space after file name
> completions. So, it seems plausible that if multiline editing
> mode was added beforehand,the ep->ed->p_eol !=
> ep->cursor-ep->screen case might never have occurred during
> testing.
>
> Setting the 'first' parameter to -1 seems to be a pretty explicit
> indicator that the author(s) intended the line clearing code to
> run, hence the entry in RELASE.
>
> The real issue is that if we update the cursor by calling
> ed_setcursor on line 1554 with old != new, the later call to
> setcursor on line 1583, here:
>
> 	I = (ncursor-nscreen) - ep->offset;
> 	setcursor(ep,i,0);
>
> will use outdated screen information to call setcursor, which,
> coincidentally, calls ed_setcursor.
2021-02-26 11:20:58 +00:00
Martijn Dekker
df2b9bf67f vi: fix buffer corruption after filename completion (re: 4cecde1d)
This bug was backported along with a fix from 93v-. An inconsistent
state occurred if you caused a file name completion menu to appear
with two TABs (which also puts you in command mode) but then
re-enter insert mode (e.g. with 'a') instead of entering a number.

    $ set -o vi
    $ cd /
    $ bin/p    [press TAB twice]
    1) pax
    2) ps
    3) pwd     [now type 'a', 'wd', return]
    $ bin/pwd
    >          [PS2 prompt wrongly appears; press return]
    /
    $

Here's another reproducer, suggesting the problem is a write past
the end of the screen buffer:

    $ set -o vi
    $ cd /
    $ bin/p    [press TAB twice]
    1) pax
    2) ps
    3) pwd     [press '0', then '$']
    $ bin/p    [cursor is one too far to the right, past the 'p'!]
    [Further operations show random evidence of memory corruption]

Harald van Dijk found the cause (thanks!):
> In vi.c's textmod there is
>
> case '=':               /** list file name expansions **/
> ...
>         ++last_virt;
> ...
>         if(ed_expand(vp->ed,(char*)virtual, &cur_virt, &last_virt, ch, vp->repeat_set?vp->repeat:-1)<0)
>         {
> ...
>                 last_virt = i;
> ...
>         }
>         else if((c=='=' || (c=='\\'&&virtual[last_virt]=='/')) && !vp->repeat_set)
>         {
> ...
>         }
>         else
>         {
> ...
>                 --last_virt;
> ...
>         }
>         break;
>
> That middle block does not restore last_virt, and everything goes
> wrong after that. That function used to restore last_virt until
> commit 4cecde1 (#41). The commit message says it was taken from
> ksh93v- and indeed this bug is also present in that version too.
> If I restore the last_virt = i; that was there originally, like
> below, then this bug seems to be fixed. I do not know why it was
> taken out, taking it out does not seem to be necessary to fix the
> original bug.

src/cmd/ksh93/edit/vi.c: textmod():
- Restore the missing restore of last_virt.

src/cmd/ksh93/tests/pty.sh:
- Add test that checks basic completion menu functionality works
  and runs modified versions of the two reproducers above.

Resolves: https://github.com/ksh93/ksh/issues/195
2021-02-26 02:01:09 +00:00
Martijn Dekker
82d6272733 manual: invocation options: edits for clarity
src/cmd/ksh93/data/builtins.c:
- sh_optksh[]: Edit descriptions of -c and -s options for clarity.
- sh_set[]: The --rc long name equivalent for -E was documented
  wrong, but in any case it does not belong in sh_set[], because
  that also shows up in 'set --man' and this invocation-only option
  cannot be used with 'set'. Remove it. (Note that all other
  invocation options already don't have inline documentation of
  their long equivalents. This may or may not be fixed at some
  point. It is problematic because they should not be documented in
  sh_set[] but there is no other good place for them.)

src/cmd/ksh93/sh.1:
- Generally edit the Invocation section for clarity.
- Document the long invocation option equivalents.
- Remove some nonsense from the -s description: "Shell output,
  except for the output of the Special Commands listed above, is
  written to file descriptor 2" (which is standard error).
  In fact, this option has no influence at all on what is written
  to standard error or standard output.
2021-02-25 17:22:53 +00:00
Martijn Dekker
caf7ab6c71 Make PATH properly survive a shared-state ${ comsub; }
Reproducer:

$ ksh -c 'v=${ PATH=/dev/null; }; echo $PATH; whence ls'
/dev/null
/bin/ls

The PATH=/dev/null assignment should survive the shared-state
command substitution, and does, yet 'ls' is still found.
The variable became inconsistent with the internal pathlist.

This bugfix is from the 93v- beta.

src/cmd/ksh93/sh/subshell.c: sh_subshell():
- Do not save and restore pathlist for a subshare.
- A few other subshell tweaks from 93v- that made sense:
  . reset shp->subdup (bitmask for dups of 1) after saving it
  . use e_dot instead of "." for consistency
  . retry close(1) if it was interrupted

src/cmd/ksh93/tests/path.sh:
- Add test for this bug.
2021-02-23 22:16:06 +00:00
Martijn Dekker
e3882fe71b Only run pty tests on systems where pty is known to be good
On some systems (AIX, HP-UX, OpenBSD) the pty tests may hang.

On all systems except Darwin/macOS, FreeBSD and Linux, the pty
tests show one or more regressions. But when I try out the failing
tests manually in a real session, it seems to work fine. So I
suspect pty is broken and not ksh.

src/cmd/ksh93/tests/pty.sh:
- For now, only run the pty tests on Darwin, FreeBSD and Linux.

src/lib/libast/Mamfile:
- tvsleep.c: Add missing error.h dependency (re: 2f7918de).
  (unrelated, but just wasn't worth its own commit)
2021-02-23 10:54:56 +00:00
Johnothan King
733f70e94b
Fix many compiler warnings and remove unused variables (#191)
Most of these changes remove unused variables, functions and labels
to fix -Wunused compiler warnings. Somewhat notable changes:

src/cmd/ksh93/bltins/print.c:
- Removed the unused 'neg' variable.
  Patch from ksh2020: https://github.com/att/ast/pull/725

src/cmd/ksh93/bltins/sleep.c:
- Initialized ns to fix three -Wsometimes-uninitialized warnings.

src/cmd/ksh93/edit/{emacs,vi}.c:
- Adjust strncpy size to fix two -Wstringop-truncation warnings.

src/cmd/ksh93/include/shell.h:
- The NOT_USED macro caused many -Wunused-value warnings,
  so it has been replaced with ksh2020's macro:
  https://github.com/att/ast/commit/19d0620a

src/cmd/ksh93/sh/expand.c:
- Removed an unnecessary 'ap = ' since 'ap' is never read
  between stakseek and stakfreeze.

src/cmd/ksh93/edit/vi.c: refresh():
- Undef this function's 'w' macro at the end of it to stop it
  potentially interfering with future code changes.

src/cmd/ksh93/sh/nvdisc.c,
src/lib/libast/misc/magic.c,
src/lib/libast/regex/regsubexec.c,
src/lib/libast/sfio/sfpool.c,
src/lib/libast/vmalloc/vmbest.c:
- Fixed some indentation to silence -Wmisleading-indentation
  warnings.

src/lib/libast/include/ast.h:
- For clang, now only suppress hundreds of -Wparentheses warnings
  as well as a few -Wstring-plus-int warnings.
  Clang's -Wparentheses warns about things like
  	if(foo = bar())
  which assigns to foo and checks the assigned value.
  Clang wants us to change this into
  	if((foo = bar()))
  Clang's -Wstring-plus-int warns about things like
  	"string"+x
  where x is an integer, e.g. "string"+3 represents the string
  "ing". Clang wants us to change that to
  	"string"[3]
  The original versions represent a perfectly valid coding style
  that was common in the 1980s and 1990s and is not going to change
  in this historic code base. (gcc does not complain about these.)

Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-02-22 22:16:32 +00:00
Martijn Dekker
83630f9d1c editors: fix broken SIGWINCH handling
In the emacs editor:
 1.  press ESC
 2.  change the size of your terminal window
and your screen is mysteriously cleared. (Until recent fixes, the
shell probably also crashed somewhere in the job control code.)

The cause is the way SIGWINCH is handled by ed_read() in edit.c.
For the emacs editor, it sends a Ctrl+L character to the input
buffer. The Ctrl+L command refreshes the command line. And it so
happens that ESC plus Ctrl+L is a command to clear the screen in
the emacs editor.

With the exeption of vi insert/command mode for which it uses a
shared flag, edit.c does not know the state of the editor, because
their data are internal to emacs.c and vi.c. So it doesn't know
whether you're in some mode that treats keyboard input specially.
Which means this way of dealing with SIGWINCH is fundamentally
misdesigned and is not worth fixing.

It gets sillier: in addition to sending keyboard commands, edit.c
was also communicating directly with emacs.c and vi.c via a flag,
e_nocrnl, which means 'please don't make Ctrl+L emit a linefeed'
(it normally refreshes on a new line but that is undesirable for
SIGWINCH). So there is already a hack that breaks the barrier
between edit.c and emacs.c/vi.c. Let's do that properly instead.

As of this commit, ed_read() does not send any fake keystrokes.
Instead, two extern functions, emacs_redraw() and vi_redraw(), are
defined for redrawing the command line. These are put in emacs.c
and vi.c so they have access to relevant static data and functions.
Then, instead of sending keyboard commands to the editor and
returning, ed_read() simply calls the redraw function for the
active editor, then continues and waits for input. Much cleaner.

src/cmd/ksh93/include/edit.h:
- Remove e_nocrnl flag from Edit_t struct.
- Define externs emacs_redraw() and vi_redraw(). Since Emacs_t and
  Vi_t types are not known here, we have to declare void* pointers
  and the functions will have to use typecasts.

src/cmd/ksh93/edit/edit.c:
- ed_read(): Call emacs_redraw() or vi_redraw() as per above.
- ed_getchar(): Remove comment about a nonexistent while loop.

src/cmd/ksh93/edit/emacs.c:
- Updates corresponding to removal of e_nocrnl flag.
- Add emacs_redraw(). This one is pretty simple. Refresh the
  command line, then ed_flush() to update the cursor display.

src/cmd/ksh93/edit/vi.c:
- Updates corresponding to removal of e_nocrnl flag. Also remove a
  similar internal 'nonewline' flag which is now equally redundant.
- Move the Ctrl+L handling code (minus writing the newline) into
  the vi_redraw() function.
- Change two cases where vi set nonewline and sent Ctrl+L to itself
  into simple vi_redraw() calls.
- Add vi_redraw(). This is more complicated as it incorporates the
  previous Ctrl+L code. It needs an added refresh() call with a
  check whether we're currently in command or insert mode, as those
  use different refresh methods. Luckily edit.c already maintains
  an *e_vi_insert flag in ed_getchar() that we can use. Since vi's
  refresh() already calls ed_flush(), we don't need to add that.
2021-02-22 00:11:59 +00:00
Martijn Dekker
18529b88c6 Add lots of checks for out of memory (re: 0ce0b671)
Huge typeset -L/-R adjustment length values were still causing
crashses on sytems with not enough memory. They should error out
gracefully instead of crashing.

This commit adds out of memory checks to all malloc/calloc/realloc
calls that didn't have them (which is all but two or three).

The stkalloc/stakalloc calls don't need the checks; it has
automatic checking, which is done by passing a pointer to the
outofspace() function to the stakinstall() call in init.c.

src/lib/libast/include/error.h:
- Change the ERROR_PANIC exit status value from ERROR_LEVEL (255)
  to 77, which is what it is supposed to be according to the libast
  error.3 manual page. Exit statuses > 128 for anything else than
  signals are not POSIX compliant and may cause misbehaviour.

src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/sh/init.c:
- To facilitate consistency, add a simple extern sh_outofmemory()
  function that throws an ERROR_PANIC "out of memory".

src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/data/builtins.c:
- Remove now-redundant e_nospace[] extern message; it is now only
  used in one place so it might as well be a string literal in
  sh_outofmemory().

All other changed files:
- Verify the result of all malloc/calloc/realloc calls and call
  sh_outofmemory() if they fail.
2021-02-21 22:27:28 +00:00
hyenias
0ce0b67149
Fix segmentation fault for justified strings (re: bdb99741) (#190)
Additional adjustments to previous commit bdb9974 to correct
crashes when the max size of a justified string is requested.
This commit corrects the following:

Before (Ubuntu 64bit):
$ typeset -L $(((1<<31)-1)) s=h; typeset +p s
Segmentation fault (core dumped)

After:
$ typeset -L $(((1<<31)-1)) s=h; typeset +p s
typeset -L 2147483647 s

src/cmd/ksh93/sh/name.c: nv_putval():
- Alter the variables size, dot, and append from int to unsigned
  int to prevent unwanted negative values from being expressed.
- By creating size, dot, and append as unsigned ints; (unsigned)
  type casting is avoided.
2021-02-21 09:34:18 +00:00
Martijn Dekker
51b2e360fa job_reap(): fix use of unitialised pointer
This solves another intermittent crash that happened upon
processing SIGWINCH in the emacs editor. See also: 7ff6b73b

I found this bug while testing ksh 93u+m on OpenBSD. Due to its
pervasive security hardening, this system crashes a program
reliably where others crash it intermittently, which is invaluable.

src/cmd/ksh93/sh/jobs.c: job_reap():
- The pw pointer is not ever given a value if the loop breaks on
  line 318-319, but it is used unconditionally on lines 464-470,
  Initialise the pointer to null on function entry and do not call
  job_list() and job_unpost() if the pointer is still null.
2021-02-20 23:40:00 +00:00
Martijn Dekker
797adc39cc Top-level documentation tweaks
LICENSE.md:
- Fix Markdown formatting of headers.

README.md:
- Add coding style note about opening braces.

TODO:
- Add note about OpenBSD regress test failures.
2021-02-20 23:19:50 +00:00
Martijn Dekker
f57f7b7a19 emacs: more correct fix (re: 9ccb9572)
sptr is set to drawbuff, but may change; drawbuff itself will not
change and reading before the start of that was the actual cause of
the crash.
2021-02-20 22:12:23 +00:00
Martijn Dekker
9ccb9572f3 emacs: fix crash due to read before start of buffer
It's amazing what can happen when you compile ksh using standard
malloc (i.e. with AST vmalloc disabled) on OpenBSD. Its security
hardening provokes crashes that reveal decades-old unsolved bugs.
This one is an attempt to access one byte before the beginning of
the command line buffer when the cursor is at the beginning of it.
On this system configuration, it provoked an instant crash whenever
you moved the cursor back to the beginning of the command line,
e.g. with ^A or the cursor keys.

src/cmd/ksh93/edit/emacs.c: draw():
- Check that the cursor is actually past the first position of the
  command line buffer before trying to read the position
  immediately before it. If not, zero the value.
2021-02-20 21:09:01 +00:00
Martijn Dekker
500757d78b Error out on 'redirect >foo' inside ${ shared-state comsub; }
The following caused an infinite loop:

        v=${ exec >/dev/tty; }
        v=${ redirect >/dev/tty; }

Even the original authors didn't figure out how to 'exec >foo' or
'redirect >foo' inside a non-forking command substitution, so they
fork it by calling sh_subfork(). If we delete that call, even
normal command substitutions enter into that infinite loop. But of
course a shared-state comsub can never fork as it would no longer
share its state. Without a solution to make this work without
forking, an error message is the only sensible thing left to do.

src/cmd/ksh93/sh/io.c: sh_redirect():
- If we're redirecting standard output (1), the redirection is
  permanent as in 'exec'/'redirect' (flag==2), and we're in a
  subshare, then error out.

Resolves: https://github.com/ksh93/ksh/issues/128
2021-02-20 19:52:08 +00:00
sterlingjensen
e1690f61ff
package: Fix unspecified behavior after "unset PWD" (#176)
POSIX warns about "unset PWD" leading to unspecified behavior from
the pwd util, which we then use.

bin/package, src/cmd/INIT/package.sh:
- Determine if the shell has $PWD with a feature test. Only unset
  PWD if it does not (the old Bourne shell).
- Clean up 'case' flow.

Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-02-20 13:58:52 +00:00
Martijn Dekker
bdb997415d Fix multiple buffer overflows with justified strings (-L/-R/-Z)
ksh crashed in various different and operating system-dependent
ways when attempting to create or apply justification strings
using typeset -L/-R/-Z, especially if large sizes are used.

The crashes had two immediate causes:
- In nv_newattr(), when applying justification attributes, a buffer
  was allocated for the justified string that was exactly 8 bytes
  longer than the original string. Any larger justification string
  caused a buffer overflow (!!!).
- In nv_putval(), when applying existing attributes to a new value,
  the corresponding memmove() either did not zero-terminate the
  justified string (if the original string was longer than the
  justified string) or could read memory past the original string
  (if the original string was shorter than the justified string).
  Both scenarios can cause a crash.

This commit fixes other minor issues as well, such as a mysterious
8 extra bytes allocated by several malloc/realloc calls. This may
have been some naive attempt to paper over the above bugs. It seems
no one can make any other kind of sense of it.

A readjustment bug with zero-filling was also fixed.

src/cmd/ksh93/sh/name.c:
- nv_putval():
  . Get rid of the magical +8 bytes for malloc and realloc. Just
    allocate one extra byte for the terminating zero.
  . Fix the memmove operation to use strncpy instead, so that
    buffer overflows are avoided in both scenarios described above.
    Also make it conditional upon a size adjustment actually
    happening (i.e. if 'dot' is nonzero).
  . Mild refactoring: combine two 'if(sp)' blocks into one;
    declare variables only used there locally for legibility.
- nv_newattr():
  * Replace the fatally broken "let's allocate string length + 8
    bytes no matter the size of the adjustment" routine with a new
    one based on work by @hyenias (see comments in #142). It is
    efficient with memory, taking into account numeric types,
    growing strings, and shrinking strings.
  * Fix zero-filling in readjustment after changing the initial
    size of a -Z attribute. If the number was zero, all zeros were
    still skipped, leaving an empty string.

Thanks to @hyenias for originally identifying this breakage and
laying the groundwork for fixing nv_newattr(), and to @lijog for
the crash analysis that revealed the key to the nv_putval() fix.

Resolves: https://github.com/ksh93/ksh/issues/142
Resolves: https://github.com/ksh93/ksh/issues/181
2021-02-20 13:05:38 +00:00
Martijn Dekker
a959a35291 DEBUG trap: restore status 2 trigger to skip command (re: d00b4b39)
So now we know what that faulty check for shp->indebug in sh_trap()
was meant to do: it was meant to pass down the trap handler's exit
status, via sh_debug(), down to sh_exec() (xec.c) so that it could
then skip the execution of the next command if the trap's exit
status is 2, as documented in the manual page. As of d00b4b39, exit
status 2 was not passed down, so this stopped working.

This commit reinstates that functionality, but without the exit
status bug in command substitutions caused by the old way.

src/cmd/ksh93/sh/fault.c: sh_trap():
- Save the trap's exit status before restoring the parent
  envionment's exit status. Make this saved exit status the return
  value of the function. (This does not break anything, AFAICT; the
  majority of sh_trap() calls ignore the return value, and the few
  that don't ignore it seem to expect it to return exactly this.)

src/cmd/ksh93/sh/xec.c: sh_exec():
- The sh_trap() fix has one side effect: whereas the exit status of
  a skipped command was always 2 (as per the trap handler), now it
  is always 0, because it gets reset in sh_exec() but no command is
  executed. That is probably not a desirable change in behaviour,
  so let's fix that here instead: set sh.exitval to 2 when skipping
  commands.

src/cmd/ksh93/sh.1:
- Document that ${.sh.command} shell-quotes its arguments for use
  by 'eval' and such. This fact was not documented anywhere, AFAIK.

src/cmd/ksh93/shell.3:
- Document that $? (exit status) is made local to trap handlers.
- Document that sh_trap() returns the trap handler's exit status.

src/cmd/ksh93/tests/basic.sh:
- Add test for this bug.
- Add a missing test for the exit status 255 functionality (if a
  DEBUG trap handler yields this exit status and we're executing a
  function or dot script, a return is triggered).

Fixes: https://github.com/ksh93/ksh/issues/187
2021-02-20 05:13:51 +00:00
Johnothan King
2b805f7f1c
Fix many spelling errors and word repetitions (#188)
Many of the errors fixed in this commit are word repetitions
such as 'the the' and minor spelling errors. One formatting
error in the ksh man page has also been fixed.
2021-02-20 03:22:24 +00:00
Lev Kujawski
4e47f89b06
libast: Improve and harden tvsleep (re: 72968eae) (#186)
On SVR4 platforms, select is a sometimes erroneous wrapper around
the poll system call, so call poll directly for efficiency purposes if
it implies no loss in precision.

See, e.g., http://bugs.motifzone.net/long_list.cgi?buglist=129 .

src/lib/libast/features/tvlib:
- For systems lacking nanosleep, test whether select is truly more
  precise than poll.

src/lib/libast/tm/tvsleep.c:
- Verify that tv argument is not null.
- Immediately return if asked to sleep for a duration of zero.
- Immediately initialize timespec in the nanosleep case.
- Revise variable names to use Apps Hungarian.
- Use poll instead of select when there is no loss in precision.
- Check for overflow in the poll case.
- Improve comments.
- Revise arithmetic to work with unsigned types, rather than
  casting to long.
- Do not return non-zero if we have slept for a sufficient
  time.
2021-02-19 22:58:04 +00:00
Martijn Dekker
350b52ea4e Update comsub-with-alias anti-leak hack (re: fe20311f)
In the 93v- beta, they add a newline instead of a space.
This has fewer side effects as final newlines get stripped.
It's still a hack and it would still be nice to have a real fix,
but it seems even the AT&T guys couldn't come up with one.

src/cmd/ksh93/sh/macro.c:
- To somehow avoid a memory leak involving alias substitution,
  append a linefeed instead of a space to the comsub buffer.

src/cmd/ksh93/tests/subshell.sh:
- Add test for minor regression caused by the RedHat version.
2021-02-18 23:35:20 +00:00
Lev Kujawski
72968eaed6
Revise the pow(1,inf) IEEE feature test to defeat clever compilers (#184)
Compilers like GCC are capable of optimizing away calls like
pow(1,inf), which caused the IEEE compliance feature test within
libast to incorrectly succeed on platforms with non-IEEE behavior.

This is arguably a bug within GCC, as floating point optimizations
should never alter the behavior of code unless IEEE compliance is
explicitly disabled via a flag like -ffast-math. Programs in which
only some calls to pow are optimized away are liable to severely
malfunction.

Thanks to Martijn Dekker for pointing this issue out and the kind
operators of polarhome.com for permitting me gratis use of their
Unix systems.

src/lib/libast/comp/omitted.c:
- Add IEEE compliant functions that wrap powf, pow, and powl.

src/lib/libast/features/float:
- Look for powf, pow, and powl in the C library.
- For compilers that do the right thing, like the native toolchains
  of Solaris and UnixWare, use lightweight macros to wrap the pow
  functions.
- Use a volatile function pointer through which to access the C
  library's pow function in an attempt to defeat code optimization.
- For these overzealous compilers, define pow to _ast_pow so that
  the same technique can be used within the above functions.
2021-02-18 18:45:27 +00:00
Martijn Dekker
c2cb0eae19 Make 'read' compatible with Shift-JIS
This commit fixes a bug in the 'read' built-in: it did not properly
skip over multibyte characters. The bug never affects UTF-8 locales
because all UTF-8 bytes have the high-order bit set. But Shift-JIS
characters may include a byte corresponding to the ASCII backslash
character, which cauased buggy behaviour when using 'read' without
the '-r' option that disables backslash escape processing.

It also makes the regression tests compatible with Shift-JIS
locales. They failed with syntax errors.

src/cmd/ksh93/bltins/read.c:
- Use the multibyte macros when skipping over word characters.
  Based on a patch from the old ast-developers mailing list:
  https://www.mail-archive.com/ast-developers@lists.research.att.com/msg01848.html

src/cmd/ksh93/include/defs.h:
- Be a bit smarter about causing the compiler to optimise out
  multibyte code when SHOPT_MULTIBYTE is disabled. See the updated
  comment for details.

src/cmd/ksh93/tests/locale.sh:
- Put all the supported locales in an array for future tests.
- Add test for the 'read' bug. Include it in a loop that tests
  64 SHIFT-JIS character combinations. Only one fails on old ksh:
  the one where the final byte corresponds to the ASCII backslash.
  It doesn't hurt to test all the others anyway.

src/cmd/ksh93/tests/basic.sh,
src/cmd/ksh93/tests/builtins.sh,
src/cmd/ksh93/tests/quoting2.sh:
- Fix syntax errors that occurred in SHIFT-JIS locales as the
  parser was processing literal UTF-8 characters. Not executing
  that code is not enough; we need to make sure it never gets
  parsed as well. This is done by wrapping the commands containing
  literal UTF-8 strings in an 'eval' command as a single-quoted
  operand.

.github/workflows/ci.yml:
- Run the tests in the ja_JP.SJIS locale instead of ja_JP.UTF-8.
  UTF-8 is already covered by the nl_NL.UTF-8 test run; that should
  be good enough.
2021-02-18 16:07:12 +00:00
Martijn Dekker
8c2d8e5f46 Revert 096f46ee ("Fix for memory mgmt in variable expansion")
This reverts a Solaris patch (105-CR7032068) with no documentation
on what it fixes or how or why. There are reports about it causing
a crash and/or a regression:

https://github.com/ksh93/ksh/issues/180#issuecomment-780980442
2021-02-18 02:05:39 +00:00
Martijn Dekker
50b665b1ed Revert 0e4c4d61 ("Fix minor typeset attribute regressions")
This commit introduced the following bug, which is worse than the
one that commit fixed: it became impossible to alter the size of an
existing justified string attribute.

Thanks to @hyenias for catching this bug:
https://github.com/ksh93/ksh/issues/142#issuecomment-780931533

$ unset s; typeset -L 100 s=h; typeset +p s; typeset -L 5 s; typeset +p s
typeset -L 100 s
typeset -L 100 s

Expected output:
typeset -L 100 s
typeset -L 5 s

src/cmd/ksh93/sh/name.c:
- Revert.

src/cmd/ksh93/tests/attributes.sh:
- Revert: re-disable tests for minor attribute output regressions.
- Add a test for this bug and potential similar bugs.
2021-02-18 01:11:53 +00:00
Martijn Dekker
241b5a4af5 tests/variables.sh: now also test PATH (re: 222515bf) 2021-02-17 18:06:15 +00:00
Martijn Dekker
911d6b066f Fix subshell scoping of changes in shared command substitution
A ${ shared-state command substitution; } (internally called
subshare) is documented to share its state with the parent shell
environment, so all changes made within the command substitution
survive outside of it. However, when it is run within a
virtual/non-forked subshell, variables that are not already local
to that subshell will leak out of it into the grandparent state.
Reproducer:

	$ ksh -c '( v=${ bug=BAD; } ); echo "$bug"'
	BAD

If the variable pre-exists in the subshell, the bug does not occur:

	$ ksh -c '( bug=BAD1; v=${ bug=BAD2; } ); echo "$bug"'
	(empty line, as expected)

The problem is that the sh_assignok() function, which is
responsible for variable scoping in virtual subshells, does not
ever bother to create a virtual subshell scope for a subshare.
That is an error if a subshare's parent (or higher-up ancestor)
environment is a virtual subshell, because a scope needs to be
created in that parent environment if none exists.

To make this bugfix possible, first we need to get something out of
the way. nv_restore() temporarily sets the subshell's pointer to
the preesnt working directory, shpwd, to null. This causes
sh_assignok() to assume that the subshell is a subshare (because
subshares don't store their own PWD) and refuse to create a scope.
However, nv_restore() sets it to null for a different purpose: to
temporarily disable scoping for *all* virtual subshells, making
restoring possible. This is a good illustration of why it's often
not a good idea to use the same variable for unrelated purposes.

src/cmd/ksh93/sh/subshell.c:
- Add a global static subshell_noscope flag variable to replace the
  misuse of sh.shpwd described above.
- sh_assignok():
  . Check subshell_noscope instead of shpwd to see if scope
    creation is disabled. This makes it possible to distinguish
    between restoring scope and handling subshares.
  . If the current environment is a subshare that is in a virtual
    subshell, create a scope in the parent subshell. This is done
    by temporarily making the parent virtual subshell the current
    subshell (by setting the global subshell_data pointer to it)
    and calling sh_assignok() again, recursively.
- nv_restore(): To disable subshell scope creation while restoring,
  set subshell_noscope instead of saving and unsetting sh.shpwd.

src/cmd/ksh93/tests/subshell.sh:
- Add tests. I like tests. Tests are good.

Fixes: https://github.com/ksh93/ksh/issues/143
2021-02-17 15:33:48 +00:00
Johnothan King
a282ebc8fe
Fix emacs backslash escaping behavior (#179)
This commit fixes the following:

1. Emacs mode ignores --nobackslashctrl (re: 24598fed) when in
   reverse search.

2. When entering more than one backslash, emacs reverse search mode
   deletes multiple backslashes after pressing backspace once.
   Reproducer:
   $ set --emacs --nobackslashctrl
   $ <Ctrl+R> \\\\<Backspace>

3. Except when in reverse search, the backslash fails to escape a
   subsequent interrupt character (^C). Reproducer:
   $ set --emacs --backslashctrl
   $ teststring \<Ctrl+C>

src/cmd/ksh93/edit/emacs.c:
- Disable escaping backslashes in emacs reverse search if
  'nobackslashctrl' is enabled.
- Fix the buggy behavior of backslashes in emacs reverse
  search by processing backslashes in a loop.

src/cmd/ksh93/tests/pty.sh:
- Add regression tests.

src/cmd/ksh93/sh.1:
- Fix a minor documentation error (^C is the usual interrupt
  character, not ^?).

Co-authored-by: Martijn Dekker <martijn@inlv.org>
2021-02-17 14:29:12 +00:00
Martijn Dekker
fe74702766 Fix miscellaneous typos 2021-02-16 16:45:06 +00:00
Martijn Dekker
3a315f58f6 More 'case' regression tests (re: e37aa358)
Ref.: https://github.com/ksh93/ksh/issues/177
2021-02-16 13:36:50 +00:00
Martijn Dekker
e37aa358bf Fix BUG_CASEEMPT: empty 'case' list was syntax error
'case x in esac' should be syntactically correct, but was an error:

	$ ksh -c 'case x in esac'
	ksh: syntax error at line 1: `case' unmatched

Inserting a newline was a workaround:

	$ ksh -c $'case x in\nesac'
	(no output)

The problem was that the 'esac' reserved word was not being
recognised if it immediately followed the 'in' reserved word.

src/cmd/ksh93/sh/lex.c: sh_lex():
- Do not turn off recognition of reserved words after 'in' if we're
  in a 'case' construct; only do this for 'for' and 'select'.

src/cmd/ksh93/tests/case.sh:
- Add seven regression test for correct recognition of 'esac'.
  Only two failed on ksh93. The rest is to catch future bugs.

Fixes: https://github.com/ksh93/ksh/issues/177
2021-02-16 06:50:12 +00:00
Johnothan King
29b11bba3a
Fix the Alt+D and Alt+H keyboard shortcuts in emacs mode (#178)
This commit fixes the functionality of Alt+D and Alt+H in emacs mode.
These keyboard shortcuts are intended to work on whole words, but
after commit 13c3fb21 their functionality was reduced to deleting only
singular letters:

$ Test word <Alt+H>    # This should delete 'word', not just 'd'.
$ Foo <Alt+B> <Alt+D>  # This should delete 'Foo', not just 'F'.

Man page entries for reference:
  M-d       Delete current word.
  M-^H      (Meta-backspace) Delete previous word.
  M-h       Delete previous word.

src/cmd/ksh93/edit/emacs.c:
- 'count' cannot be overridden when handling Alt+D or Alt+H,
  so add the total number of repetitions to count (the number of
  repetitions can't be negative).
- If 'count' is a negative number, set it to one before adding the
  number of repetitions.
2021-02-16 01:47:15 +00:00
Martijn Dekker
24598fed7c Add new 'nobackslashctrl' shell option; other minor editor tweaks
The following emacs editor 'feature' kept making me want to go back
to bash. I forget a backslash escape in a command somewhere. So I
go back to insert it. I type the \, then want to go forward. My
right arrow key, instead of moving the cursor, then replaces my
backslash with garbage. Why? The backslash escapes the following
control character at the editor level and inserts it literally.

The vi editor has a variant of this which is much less harmful. It
only works in insert mode and the backslash only escapes the next
kill or erase character.

In both editors, this feature is completely redundant with the
'stty lnext' character which is ^V by default -- and works better
as well because it also escapes ^C, ^J (linefeed) and ^M (Return).

 [In fact, you could even issue 'stty lnext \\' and get a much more
 consistent version of this feature on any shell. You have to type
 two backslashes to enter one, but it won't kill your cursor keys.]

If it were up to me alone, I'd simply remove this misfeature from
both editors. However, it is long-standing documented behaviour.
It's in the 1995 book. Plus, POSIX specifies the vi variant of it.

So, this adds a shell option instead. It was quite trivial to do.
Now I can 'set --nobackslashctrl' in my ~/.kshrc. What a relief!

Note: To keep .kshrc compatibile with older ksh versions, use:
    command set --nobackslashctrl 2>/dev/null

src/cmd/ksh93/include/shell.h,
src/cmd/ksh93/data/options.c:
- Add new SH_NOBACKSLCTRL/"nobackslashctrl" long-form option. The
  "no" prefix shows it to the user as "backslashctrl" which is on
  by default. This avoids unexpectedly changing historic behaviour.

src/cmd/ksh93/edit/emacs.c: ed_emacsread(),
src/cmd/ksh93/edit/vi.c: getline():
- Only set the flag for special backslash handling if
  SH_NOBACKSLCTRL is off.

src/cmd/ksh93/sh.1,
src/cmd/ksh93/data/builtins.c:
- Document the new option (as "backslashctrl", on by default).

Other minor tweaks:

src/cmd/ksh93/edit/edit.c:
- ed_setup(): Add fallback #error if no tput method is set. This
  should never be triggered; it's to catch future editing mistakes.
- escape(): cntl('\t') is nonsense as '\t' is already a control
  character, so change this to just '\t'.
- xcommands(): Let's enable the ^X^D command for debugging
  information on non-release builds.

src/cmd/ksh93/features/cmds:
- The tput feature tests assumed a functioning terminal in $TERM.
  However, for all we know we might be compiling with no tty and
  TERM=dumb. The tput commands won't work then. So set TERM=ansi
  to use a standard default.
2021-02-16 01:29:00 +00:00
Martijn Dekker
786a549e49 test/jobs.sh: use slightly more widely supported ps -o format 2021-02-15 15:41:31 +00:00
Martijn Dekker
5be585f45b tests/options.sh: add forgotten SHOPT_BRACEPAT check (re: af5f7acf) 2021-02-15 01:57:17 +00:00
Martijn Dekker
af5f7acf99 Fix bugs related to --posix shell option (re: 921bbcae, f45a0f16)
This fixes the following:
1. 'set --posix' now works as an equivalent of 'set -o posix'.
2. The posix option turns off braceexpand and turns on letoctal.
   Any attempt to override that in a single command such as 'set -o
   posix +o letoctal' was quietly ignored. This now works as long
   as the overriding option follows the posix option in the command.
3. The --default option to 'set' now stops the 'posix' option, if
   set or unset in the same 'set' command, from changing other
   options. This allows the command output by 'set +o' to correctly
   restore the current options.

src/cmd/ksh93/data/builtins.c:
- To make 'set --posix' work, we must explicitly list it in
  sh_set[] as a supported option so that AST optget(3) recognises
  it and won't override it with its own default --posix option,
  which converts the optget(3) string to at POSIX getopt(3) string.
    This means it will appear as a separate entry in --man output,
  whether we want it to or not. So we might as well use it as an
  example to document how --optionname == -o optionname, replacing
  the original documentation that was part of the '-o' description.

src/cmd/ksh93/sh/args.c: sh_argopts():
- Add handling for explitit --posix option in data/builtins.c.
- Move SH_POSIX syncing SH_BRACEEXPAND and SH_LETOCTAL from
  sh_applyopts() into the option parsing loop here. This fixes
  the bug that letoctal was ignored in 'set -o posix +o letoctal'.
- Remember if --default was used in a flag, and do not sync options
  with SH_POSIX if the flag is set. This makes 'set +o' work.

src/cmd/ksh93/include/argnod.h,
src/cmd/ksh93/data/msg.c,
src/cmd/ksh93/sh/args.c: sh_printopts():
- Do not potentially translate the 'on' and 'off' labels in 'set
  -o' output. No other shell does, and some scripts parse these.

src/cmd/ksh93/sh/init.c: sh_init():
- Turn on SH_LETOCTAL early along with SH_POSIX if the shell was
  invoked as sh; this makes 'sh -o' and 'sh +o' show expected
  options (not that anyone does this, but correctness is good).

src/cmd/ksh93/include/defs.h,
src/cmd/ksh93/include/shell.h:
- The state flags were in defs.h and most (but not all) of the
  shell options were in shell.h. Gather all the shell state and
  option flag definitions into one place in shell.h for clarity.
- Remove unused SH_NOPROFILE and SH_XARGS option flags.

src/cmd/ksh93/tests/options.sh:
- Add tests for these bugs.

src/lib/libast/misc/optget.c: styles[]:
- Edit default optget(3) option self-documentation for clarity.

Several changed files:
- Some SHOPT_PFSH fixes to avoid compiling dead code.
2021-02-14 23:51:19 +00:00
Martijn Dekker
cd1cd9c5da add NEWS entry for e2d54b71 2021-02-14 22:26:02 +00:00
Lev Kujawski
e2d54b7169
sleep: guarantee sleeping specified time at minimum (#174)
With this patch, the Korn shell can now guarantee that calls to
sleep on systems using the select or poll method always result in
the system clock advancing by that much time, assuming no
interruptions. This compensates for deficiencies in certain
systems, including SCO UnixWare.

Discussion: https://github.com/ksh93/ksh/pull/174

src/lib/libast/tm/tvsleep.c:
- Ensure that at least the time requested to sleep has elapsed
  for the select and poll methods.
- Simplify the logic of calculating the time remaining to
  sleep and handle the case of an argument of greater than
  10e9 nanoseconds being passed to tvsleep.

src/cmd/ksh93/bltins/sleep.c:
- Eliminate the check for EINTR to handle other cases wherein
  we have not slept enough.

src/cmd/ksh93/tests/variables.sh:
- Improve the diagnostic message when the sleep test fails.
- Revise the SECONDS function test to expect that we always
  sleep for at least the time specified.

src/cmd/ksh93/tests/functions.h:
- Redirect ps stderr to /dev/null. UnixWare ps prints an error
  message about not being able to find the controlling terminal
  when shtests output is piped, but we are only using ps to find
  the PID.
2021-02-14 07:27:04 +00:00
Martijn Dekker
0e9aaf1635 move exit/return tests from basic.sh to return.sh (re: 092b90da) 2021-02-14 06:32:57 +00:00
Martijn Dekker
21d00ffb6c Fix build on QNX
This makes ksh 93u+m build on the following system:

	$ uname -a
	QNX qnx 6.5.0 2010/07/09-14:44:03EDT x86pc x86

Thanks to polarhome.com for providing the QNX shell account.

There are a number of regressions left to work out:

arrays.sh[636]: copying a large array fails
bracket.sh[129]: /tmp/ksh93.shtests.1753215026.6923/bracket.C/original should be older than /tmp/ksh93.shtests.1753215026.6923/bracket.C/newer
bracket.sh[132]: /tmp/ksh93.shtests.1753215026.6923/bracket.C/newer should be newer than /tmp/ksh93.shtests.1753215026.6923/bracket.C/original
builtins.sh[683]: real_t1 not found after parent directory renamed in subshell
functions.sh[1023]: cannot handle comsub depth > 256 in function
io.sh[252]: <# not working for pipes
io.sh[337]: read -n3 from pipe not working
io.sh[346]: read -n3 from fifo failed -- expected 'a', got 'abc'
io.sh[349]: read -n1 from fifo failed -- expected 'b', got 'd'
io.sh[379]: should have timed out
io.sh[380]: line1 should be 'prompt1: '
io.sh[381]: line2 should be line2
io.sh[382]: line3 should be 'prompt2: '
io.sh[406]: LC_ALL=C read -n2 from pipe 'a bcd' failed -- expected 'a bcd', got 'ab cd'
io.sh[406]: LC_ALL=C.UTF-8 read -n2 from pipe 'a bcd' failed -- expected 'a bcd', got 'ab cd'
jobs.sh[86]: warning: skipping subshell job control test due to non-compliant 'ps'
pty.sh[105]: POSIX sh 026(C): line 120: expected "(Stopped|Suspended)", got EOF
pty.sh[128]: POSIX sh 028(C): line 143: expected "(Stopped|Suspended) \(SIGTTIN\)", got EOF
pty.sh[151]: POSIX sh 029(C): line 166: expected "(Stopped|Suspended) \(SIGTTOU\)", got EOF
signal.sh[310]: kill -TERM $$ failed, required termination by signal 'EXIT'
signal.sh[310]: kill -VTALRM $$ failed, required termination by signal 'EXIT'
signal.sh[310]: kill -PIPE $$ failed, required termination by signal 'EXIT'

(The io.sh failures mean libast sfpkrd() is not working.)

src/lib/libast/obsolete/spawn.c:
- Removed. Didn't compile due to wrong number of arguments to
  spawnve(2), but is obsolete and unused.

src/lib/libast/comp/localeconv.c:
- The initialisation of two static 'struct lconv' variables was
  done in a way that depended on OS headers declaring the struct
  members in a certain order. This holds on most systems, but not
  on QNX, and POSIX does not actually specify the order at all:
	https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/locale.h.html
  So each member must be initialised by name. But C89 does not
  support initialising struct members by name, so we have to do it
  using an initialiser function that simply assigns the values.

src/lib/libast/comp/spawnveg.c:
- Fix for systems without either P_DETACH or _P_DETACH.

src/lib/libast/features/vmalloc,
src/lib/libast/vmalloc/vmmopen.c,
src/lib/libast/Mamfile:
- Add test for sys/shm.h header. If it doesn't exist, as it doesn't
  on QNX, use the stub vmmapopen() as the real one won't compile.
  (Mamfile: Add dependency on FEATURE/vmalloc to vmmopen.c.)

src/lib/libast/vmalloc/malloc.c:
- Remove superfluous externs that are already provided by either
  AST or system headers. The 'void cfree' extern caused a build
  failure on QNX because cfree() is of type int on QNX.

src/lib/libast/comp/conf.tab:
- Remove check for _map_spawnve; src/lib/libast/RELEASE says it was
  removed.
2021-02-14 01:28:35 +00:00
Martijn Dekker
cd35ec6048 tests/builtins.sh: fix fail due to translated system messages
(re: f7ffaaba, d1483150)
2021-02-13 23:27:43 +00:00
Martijn Dekker
fb7551634a Fix exec/redirect (fd != 1) in shared-state comsub (re: db72f41f)
That OpenSUSE patch introduced a bug: file descriptors other than 1
that were globally redirected using 'exec' or 'redirect' no longer
survived a ${ shared-state; } command substitution.

Related: https://github.com/ksh93/ksh/issues/128

src/cmd/ksh93/sh/io.c:
- Add check for shp->subshare to the OpenSUSE patch.

src/cmd/ksh93/tests/io.sh:
- Add test.
2021-02-13 16:01:42 +00:00
Martijn Dekker
5529d13d28 tests/pty.sh: avoid typeahead: add 10ms delay (re: 5c389035)
We can't actually check for the typeahead effects as they are
timing-dependent and do not work on the slower GitHub runners.
2021-02-13 14:30:40 +00:00
Martijn Dekker
d790731a80 github: Revert failed experiment (re: 2e6c56df, 9ad9a1de, 5c389035)
.github/workflows/ci.yml:
- Go back to wrapping the regression tests in script(1).

src/cmd/ksh93/data/builtins.c:
- Never mind about the stty builtin.

src/cmd/ksh93/tests/pty.sh:
- Refuse to run if there isn't a functioning tty.
- Make sure stty(1) works on /dev/tty by redirecting stdin.
2021-02-13 13:52:14 +00:00
Martijn Dekker
2e6c56df82 github: Try to make the tests pass (re: 9ad9a1de, 5c389035)
So, the pty regression tests on the Linux GitHub runner all failed.

Let's test an assumption: the reason is that we need the stty
builtin to properly set the pty state, because the OS-provided stty
command does not work if there is no real tty.

src/cmd/ksh93/data/builtins.c:
- Compile in the stty built-in. This adds about 20k to the binary
  for a command that most users rarely need and even more rarely
  need to be built in, so only compile it in on non-release builds.

src/cmd/ksh93/tests/pty.sh:
- Skip the tests if we cannot either use the stty builtin or change
  the state of the real terminal to be compatible with the tests.
2021-02-13 13:15:46 +00:00