Fix expansion of multibyte character after $1 - $9, $?, etc (#102)

A multibyte character immediately following an expansion of a
single-character name, e.g. $1 through $9, $?, $-, etc. was
corrupted when in a UTF-8 locale, e.g.:

    $ set -- foo; echo "$1テスト"
    foo?スト

Prior discussion:
https://www.mail-archive.com/ast-users@lists.research.att.com/msg01060.html
https://bugzilla.redhat.com/show_bug.cgi?id=1256495

src/cmd/ksh93/sh/macro.c:
- Apply a Red Hat patch by Paulo Andrade that avoids calling
  fcmbget() if backtracking more than one byte might be required.

src/cmd/ksh93/tests/basic.c:
- Test "テスト" following expansion of "$1", "$?" and "$#".

Co-authored-by: Martijn Dekker <martijn@inlv.org>

NEWS

@@ -8,6 +8,10 @@ Any uppercase BUG_* names are modernish shell bug IDs.
 - Fixed a bug that caused multidimensional associative arrays to be created
   with an extra array member.
 
+- Fixed a bug that caused the expansions of positional parameters $1 - $9,
+  as well as special parameters such as $? and $-, to corrupt any multibyte
+  characters immediately following the expansion if a UTF-8 locale is active.
+
 2020-07-29:
 
 - On a ksh compiled to use fork(2) to run external commands, a bug has been

src/cmd/ksh93/sh/macro.c

@@ -1506,7 +1506,10 @@ retry1:
 	    default:
 		goto nosub;
 	}
-	c = fcmbget(&LEN);
+	if(type)
+		c = fcmbget(&LEN);
+	else
+		c = fcget();
 	if(type>M_TREE)
 	{
 		if(c!=RBRACE)

src/cmd/ksh93/tests/basic.sh

@@ -615,5 +615,18 @@ then	actual=$(
 		"(expected $(printf %q "$expect"), got $(printf %q "$actual"))"
 fi
 
+#"======
+# Expansion of multibyte characters after expansion of single-character names $1..$9, $?, $!, $-, etc.
+function exptest
+{
+	print -r "$1テスト"
+	print -r "$?テスト"
+	print -r "$#テスト"
+}
+expect=$'fooテスト\n0テスト\n1テスト'
+actual=$(exptest foo)
+[[ $actual == "$expect" ]] || err_exit 'Corruption of multibyte char following expansion of single-char name' \
+	"(expected $(printf %q "$expect"), got $(printf %q "$actual"))"
+
 # ======
 exit $((Errors<125?Errors:125))