Reset lexer state on syntax error and on SIGINT (Ctrl+C)

ksh crashed if you pressed Ctrl+C or Ctrl+D on a PS2 prompt while
you haven't finished entering a $(command substitution). It
corrupts subsequent command substitutions. Sometimes the situation
recovers, sometimes the shell crashes.

Simple crash reproducer:

  $ PS1="\$(echo foo) \$(echo bar) \$(echo baz) > "
  foo bar baz > echo $(                        <-- now press Ctrl+D
  > ksh: syntax error: `(' unmatched
  Memory fault

The same happens with Ctrl+C, minus the syntax error message.

The problem is that the lexer state becomes inconsistent when the
lexer is interrupted in the middle of reading a command
substitution of the form $( ... ). This is tracked in the
'lexd.dolparen' variable in the lexer state struct.

Resetting that variable is sufficient to fix this issue. However,
in this commit I prefer to just reinitialise the lexer state
completely to pre-empt any other possible issues. Whether there was
a syntax error or the user pressed Ctrl+C, we just interrupted all
lexing and parsing, so the lexer *should* restart from scratch.

src/cmd/ksh93/sh/fault.c: sh_fault():
- If the shell is in an interactive state (e.g. not a subshell) and
  SIGINT was received, reinitialise the lexer state. This fixes the
  crash with Ctrl+C.

src/cmd/ksh93/sh/lex.c: sh_syntax():
- When handling a syntax error, reset the lexer state. This fixes
  the crash with Ctrl+D.

NEWS:
- Also add the forgotten item for the previous fix (re: 2322f939).

NEWS

@@ -3,6 +3,16 @@ For full details, see the git log at: https://github.com/ksh93/ksh
 
 Any uppercase BUG_* names are modernish shell bug IDs.
 
+2021-12-09:
+
+- Increased the general robustness of discipline function handling, fixing
+  crashing bugs with PS2.get() and .sh.tilde.get() disciplines, among others.
+
+- Fixed a crash that occurred on the interactive shell if the PS1 prompt
+  contains multiple command substitutions and the user interrupts input
+  while the shell is on a PS2 prompt waiting for the user to complete a
+  command substitution of the form $( ... ).
+
 2021-12-08:
 
 - Fixed: if a function returned with a status > 256 using the 'return' command

src/cmd/ksh93/include/version.h

@@ -21,7 +21,7 @@
 
 #define SH_RELEASE_FORK	"93u+m"		/* only change if you develop a new ksh93 fork */
 #define SH_RELEASE_SVER	"1.0.0-beta.2"	/* semantic version number: https://semver.org */
-#define SH_RELEASE_DATE	"2021-12-08"	/* must be in this format for $((.sh.version)) */
+#define SH_RELEASE_DATE	"2021-12-09"	/* must be in this format for $((.sh.version)) */
 #define SH_RELEASE_CPYR	"(c) 2020-2021 Contributors to ksh " SH_RELEASE_FORK
 
 /* Scripts sometimes field-split ${.sh.version}, so don't change amount of whitespace. */

src/cmd/ksh93/sh/fault.c

@@ -68,6 +68,9 @@ void	sh_fault(register int sig)
 	register char		*trap;
 	register struct checkpt	*pp = (struct checkpt*)shp->jmplist;
 	int	action=0;
+	/* reset lexer state on Ctrl+C */
+	if(sh_isstate(SH_INTERACTIVE) && sig==SIGINT)
+		sh_lexopen(sh.lex_context, &sh, 0);
 	/* reset handler */
 	if(!(sig&SH_TRAP))
 		signal(sig, sh_fault);

src/cmd/ksh93/sh/lex.c

@@ -2106,6 +2106,8 @@ noreturn void sh_syntax(Lex_t *lp)
 		fcclose();
 	shp->inlineno = lp->inlineno;
 	shp->st.firstline = lp->firstline;
+	/* reset lexer state */
+	sh_lexopen(lp, &sh, 0);
 	if(!sh_isstate(SH_INTERACTIVE) && !sh_isstate(SH_PROFILE))
 		errormsg(SH_DICT,ERROR_exit(SYNBAD),e_lexsyntax1,lp->lastline,tokstr,cp);
 	else