From 9bed28c3f9e7d27f8c87f3070d0909d8c2899987 Mon Sep 17 00:00:00 2001 From: atheik <14833674+atheik@users.noreply.github.com> Date: Sat, 21 May 2022 23:51:50 +0100 Subject: [PATCH] Fix line continuation within command substitutions In command substitutions of the $(standard) and ${ shared state; } form, backslash line continuation is broken. Reproducer: echo $( echo one two\ three ) Actual output (ksh93, all versions): one two\ three Expected output (every other shell, POSIX spec): one twothree src/cmd/ksh93/sh/lex.c: sh_lex(): case S_REG: - Do not skip new-line joining if we're currently processing a command substitution of one of these forms (i.e., if the lp->lexd.dolparen level is > 0). Background info/analysis: comsub() is called from sh_lex() when S_PAR is the current state. In src/cmd/ksh93/data/lexstates.c, we see that S_PAR is reached in the ST_DOL state table at index 40. Decimal 40 is ( in ASCII. So, the previous skipping of characters was done according to the ST_DOL state table, and the character that stopped it was (. This means we have $(. Alternatively, comsub() may be called from sh_lex() by jumping to the do_comsub label. In brief, that is the case when we have ${. Regardless of which it is from the two, comsub() is now called from sh_lex(). In comsub(), lp->lexd.dolparen is incremented at the beginning and decremented at the end. Between them, we see that sh_lex() is called. So, lp->lexd.dolparen in sh_lex() indicates the depth of nesting $( or ${ statements we're in. Thus, it is also the number of comsub() invocations seen in a backtrace taken in sh_lex(). The codepath for `...` is different (and never had this bug). Co-authored by: Martijn Dekker Resolves: https://github.com/ksh93/ksh/issues/367 --- NEWS | 8 +++++++- src/cmd/ksh93/include/version.h | 2 +- src/cmd/ksh93/sh/lex.c | 4 ++-- src/cmd/ksh93/tests/quoting2.sh | 23 +++++++++++++++++++++++ 4 files changed, 33 insertions(+), 4 deletions(-) diff --git a/NEWS b/NEWS index e79fe4712..66d203682 100644 --- a/NEWS +++ b/NEWS @@ -3,11 +3,17 @@ For full details, see the git log at: https://github.com/ksh93/ksh/tree/1.0 Any uppercase BUG_* names are modernish shell bug IDs. +2022-05-21: + +- Fixed a bug, present since the beginning of ksh93, that broke backslash line + continuation within $(standard) and ${ shared-state; } command substitutions. + Thanks to atheik for the analysis and the fix. (BUG_CSUBLNCONT) + 2022-05-20: - Fixed an ancient bug that caused a spurious syntax error when using double quotes within a `backtick` command substitution within double quotes. - Thanks to atheik for the analysis and the fix. + Thanks to atheik for the analysis and the fix. (BUG_CSUBBTQUOT) 2022-03-10: diff --git a/src/cmd/ksh93/include/version.h b/src/cmd/ksh93/include/version.h index 63515c6c3..c04cdcf6b 100644 --- a/src/cmd/ksh93/include/version.h +++ b/src/cmd/ksh93/include/version.h @@ -21,7 +21,7 @@ #define SH_RELEASE_FORK "93u+m" /* only change if you develop a new ksh93 fork */ #define SH_RELEASE_SVER "1.0.0-beta.2" /* semantic version number: https://semver.org */ -#define SH_RELEASE_DATE "2022-05-20" /* must be in this format for $((.sh.version)) */ +#define SH_RELEASE_DATE "2022-05-21" /* must be in this format for $((.sh.version)) */ #define SH_RELEASE_CPYR "(c) 2020-2022 Contributors to ksh " SH_RELEASE_FORK /* Scripts sometimes field-split ${.sh.version}, so don't change amount of whitespace. */ diff --git a/src/cmd/ksh93/sh/lex.c b/src/cmd/ksh93/sh/lex.c index 969b930da..b86f933b4 100644 --- a/src/cmd/ksh93/sh/lex.c +++ b/src/cmd/ksh93/sh/lex.c @@ -671,8 +671,8 @@ int sh_lex(Lex_t* lp) if(mode==ST_BEGIN) { do_reg: - /* skip new-line joining */ - if(c=='\\' && fcpeek(0)=='\n') + /* skip new-line joining if not called from comsub() */ + if(c=='\\' && fcpeek(0)=='\n' && !lp->lexd.dolparen) { sh.inlineno++; fcseek(1); diff --git a/src/cmd/ksh93/tests/quoting2.sh b/src/cmd/ksh93/tests/quoting2.sh index 59ab3268d..d55cd7182 100755 --- a/src/cmd/ksh93/tests/quoting2.sh +++ b/src/cmd/ksh93/tests/quoting2.sh @@ -295,5 +295,28 @@ got="${var:+'}text between expansions${var:+'}" got=$(eval 'foo="`: "^Exec(\[[^]=]*])?="`"' 2>&1) || err_exit "Backtick command substitutions can't nest double quotes" \ "(got $(printf %q "$got"))" +# ====== +# https://github.com/ksh93/ksh/issues/367 +exp='one twothree' +got=$( +echo one two\ +three +) +[[ $got == "$exp" ]] || err_exit "Line continuation broken within standard command substitution" \ + "(expected $(printf %q "$exp"), got $(printf %q "$got"))" +got=${ +echo one two\ +three +} +[[ $got == "$exp" ]] || err_exit "Line continuation broken within shared-state command substitution" \ + "(expected $(printf %q "$exp"), got $(printf %q "$got"))" +# backticks did not have this bug but let's test them anyway +got=` +echo one two\ +three +` +[[ $got == "$exp" ]] || err_exit "Line continuation broken within backtick command substitution" \ + "(expected $(printf %q "$exp"), got $(printf %q "$got"))" + # ====== exit $((Errors<125?Errors:125))