external/cde
1
0
Fork 0
mirror of git://git.code.sf.net/p/cdesktopenv/code synced 2025-02-13 03:32:24 +00:00
Code Issues Releases Wiki Activity

Fix multiple bugs in .sh.match (#455)

Browse source 
This commit backports all of the relevant .sh.match bugfixes from
ksh93v-. Most of the .sh.match rewrite is from versions 2012-08-24
and 2012-10-04, with patches from later releases of 93v- and
ksh2020 also applied. Note that there are still some remaining bugs
in .sh.match, although now the total count of .sh.match bugs should
be less that before.

These are the relevant changes in the ksh93v- changelog that were
backported:
12-08-07  .sh.match no longer gets set for patterns in PS4 during
          set -x.
12-08-10  Rewrote .sh.match expansions fixing several bugs and
          improving performance.
12-08-22  .sh.match now handles subpatterns that had no matches with
          ${var//pattern} correctly.
12-08-21  A bug in setting .sh.match after ${var//pattern/string}
          when string is empty has been fixed.
12-08-21  A bug in setting .sh.match after [[ string == pattern ]]
          has been fixed.
12-08-31  A bug that could cause a core dump after
          typeset -m var=.sh.match has been fixed.
12-09-10  Fixed a bug in typeset -m the .sh.match is being renamed.
12-09-07  Fixed a bug in .sh.match code that coud cause the shell
          to quitely
13-02-21  The 12-01-16 bug fix prevented .sh.match from being used
          in the replacement string. The previous code was restored
          and a different fix which prevented .sh.match from being
          computed for nested replacement has been used instead.
13-05-28  Fixed two bug for typeset -c and typeset -m for variable
          .sh.match.

Changes:
- The SHOPT_2DMATCH option has been removed. This was already the
  default behavior previously, and now it's documented in the man
  page.
- init.c: Backported the sh_setmatch() rewrite from 93v- 2012-08-24
  and 2012-10-04.
- Backported the libast 93v- strngrpmatch() function, as the
  .sh.match rewrite requires this API.
- Backported the sh_match regression tests from ksh93v-, with many
  other sh_match tests backported from ksh2020. Much of the sh_match
  script is based on code from Roland Mainz:
  https://marc.info/?l=ast-developers&m=134606574109162&w=2
  https://marc.info/?l=ast-developers&m=134490505607093
- tests/{substring,treemove}.sh: Backported other relevant .sh.match
  fixes, with tests added to the substring and treemove test scripts.
- tests/types.sh: One of the (now reverted) memory leak bugfixes
  introduced a CI test failure in this script, so for that test the
  error message has been improved.
- string/strmatch.c: The original ksh93v- code for the strngrpmatch()
  changes introduced a crash that could occur because strlen would
  be used on a null pointer. This has been fixed by avoiding strlen
  if the string is null.

One nice side effect of these changes is a considerable performance
improvement in the shbench[1] gsub benchmark (results from 20
iterations with CCFLAGS=-Os):
--------------------------------------------------
name      /tmp/ksh-current     /tmp/ksh-matchfixes
--------------------------------------------------
gsub.ksh  0.883 [0.822-0.959]  0.457 [0.442-0.505]
--------------------------------------------------

Despite all of the many fixes and improvements in the backported
93v- .sh.match code, there are a few remaining bugs:

- .sh.match is printed with a default [0] subscript (see also
  https://github.com/ksh93/ksh/issues/308#issuecomment-1025016088):
     $ arch/*/bin/ksh -c 'echo ${!.sh.match}'
       .sh.match[0]
  This bug appears to have been introduced by the changes from
  ksh93v- 2012-08-24.
- The wrong variable name is given for 'parameter not set' errors
  (from https://marc.info/?l=ast-developers&m=134489094602596):
     $ arch/*/bin/ksh -u
     $ x=1234
     $ true "${x//~(X)([012])|([345])/}"
     $ compound co
     $ typeset -m co.array=.sh.match
     $ printf "%q\n" "${co.array[2][0]}"
     arch/linux.i386-64/bin/ksh: co.array[2][(null)]: parameter not set
- .sh.match leaks out of subshells. Further information and a
  reproducer can be found here:
  https://marc.info/?l=ast-developers&m=136292897330187

[1]: https://github.com/ksh-community/shbench
This commit is contained in:
Johnothan King 2022-02-08 16:01:40 -08:00 committed by Martijn Dekker
parent 232b7bff30
commit f38494ea1d
24 changed files with 1355 additions and 143 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.github/workflows/ci.yml vendored
View file

@ -25,7 +25,7 @@ jobs:
LANG=ja_JP.SJIS script -q -e -c "bin/shtests --locale --nocompile" &&
: disable most SHOPTs, rebuild ksh &&
sed --regexp-extended --in-place=.orig \
'/^SHOPT (2DMATCH|AUDIT|BGX|BRACEPAT|DEVFD|DYNAMIC|EDPREDICT|ESH|FIXEDARRAY|HISTEXPAND|MULTIBYTE|NAMESPACE|OPTIMIZE|SPAWN|STATS|SUID_EXEC|VSH)=/ s/=1?/=0/' \
'/^SHOPT (AUDIT|BGX|BRACEPAT|DEVFD|DYNAMIC|EDPREDICT|ESH|FIXEDARRAY|HISTEXPAND|MULTIBYTE|NAMESPACE|OPTIMIZE|SPAWN|STATS|SUID_EXEC|VSH)=/ s/=1?/=0/' \
src/cmd/ksh93/SHOPT.sh &&
bin/package make &&
: default regression tests with SHOPTs disabled &&

3
NEWS
View file

@ -27,6 +27,9 @@ Any uppercase BUG_* names are modernish shell bug IDs.
associative array when read back in by the shell. Elements that are sparse
indexed arrays are now prefixed with "typeset -a".
- The rewritten .sh.match code from ksh93v- has been backported to ksh93u+m,
fixing many bugs and improving performance by a considerable amount.
2022-02-05:
- Fixed: for indexed arrays, given an unset array member a[i] with i > 0,

2
src/cmd/ksh93/README
View file

@ -20,8 +20,6 @@ options where no feature probe is available, probe is the same as off.
The options have the following defaults and meanings:
2DMATCH on Two-dimensional ${.sh.match} for ${var//pat/str}.
ACCT off Shell accounting.
Noted by "L" in the version string when enabled.
See README-AUDIT.md.

1
src/cmd/ksh93/SHOPT.sh
View file

@ -5,7 +5,6 @@
# For a more complete description of the options, see src/cmd/ksh93/README.
#
SHOPT 2DMATCH=1 # two dimensional ${.sh.match} for ${var//pat/str}
SHOPT ACCT=0 # accounting
SHOPT ACCTFILE=0 # per-user accounting info
SHOPT AUDIT=1 # enable auditing per SHOPT_AUDITFILE

6
src/cmd/ksh93/bltins/test.c
View file

@ -83,7 +83,7 @@ static int e3(struct test*);
static int test_strmatch(const char *str, const char *pat)
{
regoff_t match[2*(MATCH_MAX+1)],n;
int match[2*(MATCH_MAX+1)],n;
register int c, m=0;
register const char *cp=pat;
while(c = *cp++)
@ -99,9 +99,9 @@ static int test_strmatch(const char *str, const char *pat)
match[0] = 0;
if(m > elementsof(match)/2)
m = elementsof(match)/2;
n = strgrpmatch(str, pat, match, m, STR_GROUP|STR_MAXIMAL|STR_LEFT|STR_RIGHT);
n = strgrpmatch(str, pat, (ssize_t*)match, m, STR_GROUP|STR_MAXIMAL|STR_LEFT|STR_RIGHT|STR_INT);
if(m==0 && n==1)
match[1] = strlen(str);
match[1] = (int)strlen(str);
if(n)
sh_setmatch(str, -1, n, match, 0);
return(n);

4
src/cmd/ksh93/include/defs.h
View file

@ -29,8 +29,8 @@
#define defs_h_defined
#include <ast.h>
#if !defined(AST_VERSION) || AST_VERSION < 20220201
#error libast version 20220201 or later is required
#if !defined(AST_VERSION) || AST_VERSION < 20220208
#error libast version 20220208 or later is required
#endif
#if !_lib_fork
#error In 2021, ksh joined the 21st century and started requiring fork(2).

1
src/cmd/ksh93/include/shell.h
View file

@ -327,6 +327,7 @@ struct Shell_s
char instance; /* in set_instance */
char decomma; /* decimal_point=',' */
char redir0; /* redirect of 0 */
char intrace; /* set when trace expands PS4 */
char *readscript; /* set before reading a script */
int subdup; /* bitmask for dups of 1 */
int *inpipe; /* input pipe pointer */

7
src/cmd/ksh93/sh.1
View file

@ -1695,6 +1695,13 @@ element stores the complete match and the
element stores the
.IR i\^ -th
submatch.
For
.B //
the array is two dimensional with the first subscript indicating the
most recent match and subpattern match and the second script indicating
which match with
.B 0
representing the first match.
The
.B .sh.match
variable

6
src/cmd/ksh93/sh/array.c
View file

@ -1176,7 +1176,7 @@ Namval_t *nv_putsub(Namval_t *np,register char *sp,register long mode)
if(!ap || !ap->header.fun)
#endif /* SHOPT_FIXEDARRAY */
{
if(sp)
if(sp && sp!=Empty)
{
if(ap && ap->xp && !strmatch(sp,"+([0-9])"))
{
@ -1185,7 +1185,11 @@ Namval_t *nv_putsub(Namval_t *np,register char *sp,register long mode)
size = nv_getnum(mp);
}
else
{
Dt_t *root = sh.last_root;
size = (int)sh_arith((char*)sp);
sh.last_root = root;
}
}
if(size <0 && ap)
size += array_maxindex(np);

1
src/cmd/ksh93/sh/fault.c
View file

@ -585,6 +585,7 @@ void sh_exit(register int xno)
if(!pp)
sh_done(sig);
sh.arithrecursion = 0;
sh.intrace = 0;
sh.prefix = 0;
#if SHOPT_TYPEDEF
sh.mktype = 0;

185
src/cmd/ksh93/sh/init.c
View file

@ -176,10 +176,13 @@ struct match
const char *v;
char *val;
char *rval[2];
regoff_t *match;
char node[NV_MINSZ+sizeof(char*)+sizeof(Dtlink_t)];
regoff_t first;
int *match;
char *nodes;
char *names;
int first;
int vsize;
int vlen;
int msize;
int nmatch;
int index;
int lastsub[2];
@ -775,76 +778,118 @@ static void put_lastarg(Namval_t* np,const char *val,int flags,Namfun_t *fp)
np->nvenv = 0;
}
static void match2d(struct match *mp)
{
Namval_t *np;
int i;
Namarr_t *ap;
nv_disc(SH_MATCHNOD, &mp->hdr, NV_POP);
np = nv_namptr(mp->nodes, 0);
for(i=0; i < mp->nmatch; i++)
{
np->nvname = mp->names + 3 * i;
if(i > 9)
{
*np->nvname = '0' + i / 10;
np->nvname[1] = '0' + (i % 10);
}
else
*np->nvname = '0' + i;
nv_putsub(np, (char*)0, 1);
nv_putsub(np, (char*)0, 0);
nv_putsub(SH_MATCHNOD, (char*)0, i);
nv_arraychild(SH_MATCHNOD, np, 0);
np = nv_namptr(np + 1, 0);
}
if(ap = nv_arrayptr(SH_MATCHNOD))
ap->nelem = mp->nmatch;
}
/*
* store the most recent value for use in .sh.match
* treat .sh.match as a two dimensional array
*/
void sh_setmatch(const char *v, int vsize, int nmatch, regoff_t match[],int index)
void sh_setmatch(const char *v, int vsize, int nmatch, int match[], int index)
{
struct match *mp = &((Init_t*)sh.init_context)->SH_MATCH_init;
Namval_t *np = (Namval_t*)(&(mp->node[0]));
register int i,n,x;
unsigned int savesub = sh.subshell;
Init_t *ip = sh.init_context;
struct match *mp = &ip->SH_MATCH_init;
register int i,n,x, savesub=sh.subshell;
Namarr_t *ap = nv_arrayptr(SH_MATCHNOD);
Namarr_t *ap_save = ap;
/* do not crash if .sh.match is unset */
if(!ap)
Namval_t *np;
if(sh.intrace)
return;
sh.subshell = 0;
#if !SHOPT_2DMATCH
index = 0;
#else
if(index==0)
#endif /* !SHOPT_2DMATCH */
if(index<0)
{
if(ap->hdr.next != &mp->hdr)
np = nv_namptr(mp->nodes,0);
if(mp->index==0)
match2d(mp);
for(i=0; i < mp->nmatch; i++)
{
free((void*)ap);
ap = nv_arrayptr(np);
SH_MATCHNOD->nvfun = &ap->hdr;
nv_disc(np,&mp->hdr,NV_LAST);
nv_putsub(np,(char*)0,mp->index);
for(x=mp->index; x >=0; x--)
{
n = i + x*mp->nmatch;
if(mp->match[2*n+1]>mp->match[2*n])
nv_putsub(np,Empty,ARRAY_ADD|x);
}
if(ap)
if((ap=nv_arrayptr(np)) && array_elem(ap)==0)
{
ap->nelem &= ~ARRAY_SCAN;
i = array_elem(ap);
ap->nelem++;
while(--i>= 0)
{
nv_putsub(SH_MATCHNOD, (char*)0,i);
nv_putsub(SH_MATCHNOD,(char*)0,i);
_nv_unset(SH_MATCHNOD,NV_RDONLY);
}
ap->nelem--;
np = nv_namptr(np+1,0);
}
if(!nv_hasdisc(SH_MATCHNOD,mp->hdr.disc))
sh.subshell = savesub;
return;
}
mp->index = index;
if(index==0)
{
if(mp->nodes)
{
np = nv_namptr(mp->nodes,0);
for(i=0; i < mp->nmatch; i++)
{
if(np->nvfun && np->nvfun != &mp->hdr)
{
free((void*)np->nvfun);
np->nvfun = 0;
}
np = nv_namptr(np+1,0);
}
free((void*)mp->nodes);
mp->nodes = 0;
}
mp->vlen = 0;
if(ap && ap->hdr.next != &mp->hdr)
free((void*)ap);
SH_MATCHNOD->nvalue.cp = 0;
SH_MATCHNOD->nvfun = 0;
if(!(mp->nmatch=nmatch) && !v)
{
sh.subshell = savesub;
return;
}
mp->nodes = sh_calloc(mp->nmatch*(NV_MINSZ+sizeof(void*)+3),1);
mp->names = mp->nodes + mp->nmatch*(NV_MINSZ+sizeof(void*));
np = nv_namptr(mp->nodes,0);
nv_disc(SH_MATCHNOD,&mp->hdr,NV_LAST);
if(nmatch)
nv_putsub(SH_MATCHNOD, NIL(char*), (nmatch-1)|ARRAY_FILL|ARRAY_SETSUB);
ap_save->nelem = mp->nmatch = nmatch;
for(i=nmatch; --i>=0;)
{
if(match[2*i]>=0)
nv_putsub(SH_MATCHNOD,Empty,ARRAY_ADD|i);
}
mp->v = v;
mp->first = match[0];
}
#if SHOPT_2DMATCH
else
{
if(index==1)
{
np->nvalue.cp = Empty;
np->nvfun = SH_MATCHNOD->nvfun;
nv_onattr(np,NV_NOFREE|NV_ARRAY);
SH_MATCHNOD->nvfun = 0;
for(i=0; i < mp->nmatch; i++)
{
nv_putsub(SH_MATCHNOD, (char*)0, i);
nv_arraychild(SH_MATCHNOD, np,0);
match2d(mp);
}
ap_save->nelem = mp->nmatch;
}
ap = nv_arrayptr(np);
nv_putsub(np, NIL(char*), index|ARRAY_FILL|ARRAY_SETSUB);
}
#endif /* SHOPT_2DMATCH */
sh.subshell = savesub;
index *= 2*mp->nmatch;
if(mp->nmatch)
{
for(n=mp->first+(mp->v-v),vsize=0,i=0; i < 2*nmatch; i++)
@ -852,30 +897,36 @@ void sh_setmatch(const char *v, int vsize, int nmatch, regoff_t match[],int inde
if(match[i]>=0 && (match[i] - n) > vsize)
vsize = match[i] -n;
}
index *= 2*mp->nmatch;
i = (index+2*mp->nmatch)*sizeof(match[0]);
if((i+vsize) >= mp->vsize)
if(i >= mp->msize)
{
if(mp->msize)
mp->match = (int*)sh_realloc(mp->match,2*i);
else
mp->match = (int*)sh_malloc(2*i);
mp->msize = 2*i;
}
if(vsize >= mp->vsize)
{
if(mp->vsize)
mp->match = (int*)sh_realloc(mp->match,i+vsize+1);
mp->val = (char*)sh_realloc(mp->val,x=2*vsize);
else
mp->match = (int*)sh_malloc(i+vsize+1);
mp->vsize = i+vsize+1;
mp->val = (char*)sh_malloc(x=vsize+1);
mp->vsize = x;
}
mp->val = ((char*)mp->match)+i;
memcpy(mp->match+index,match,nmatch*2*sizeof(match[0]));
for(x=0,i=0; i < 2*nmatch; i++)
for(i=0; i < 2*nmatch; i++)
{
if(match[i]>=0)
mp->match[index+i] -= n;
else
x=1;
}
ap_save->nelem -= x;
while(i < 2*mp->nmatch)
mp->match[index+i++] = -1;
memcpy(mp->val,v+n,vsize);
mp->val[vsize] = 0;
if(index==0)
v+= mp->first;
memcpy(mp->val+mp->vlen,v,vsize-mp->vlen);
mp->val[mp->vlen=vsize] = 0;
mp->lastsub[0] = mp->lastsub[1] = -1;
}
}
@ -886,6 +937,8 @@ static char* get_match(register Namval_t* np, Namfun_t *fp)
int sub,sub2=0,n,i =!mp->index;
char *val;
sub = nv_aindex(SH_MATCHNOD);
if(sub<0)
sub = 0;
if(np!=SH_MATCHNOD)
sub2 = nv_aindex(np);
if(sub>=mp->nmatch)
@ -915,7 +968,15 @@ static char* get_match(register Namval_t* np, Namfun_t *fp)
return(mp->rval[i]);
}
static const Namdisc_t SH_MATCH_disc = { sizeof(struct match), 0, get_match };
static char *name_match(Namval_t *np, Namfun_t *fp)
{
int sub = nv_aindex(SH_MATCHNOD);
sfprintf(sh.strbuf,".sh.match[%d]",sub);
return(sfstruse(sh.strbuf));
}
static const Namdisc_t SH_MATCH_disc = { sizeof(struct match), 0, get_match,
0,0,0,0,name_match };
static char* get_version(register Namval_t* np, Namfun_t *fp)
{

79
src/cmd/ksh93/sh/macro.c
View file

@ -46,11 +46,6 @@
#include "national.h"
#include "streval.h"
#undef STR_GROUP
#ifndef STR_GROUP
# define STR_GROUP 0
#endif
#if _WINIX
static int Skip;
#endif /* _WINIX */
@ -74,6 +69,7 @@ typedef struct _mac_
char arith; /* set for ((...)) */
char arrayok; /* $x[] ok for arrays */
char subcopy; /* set when copying subscript */
char macsub; /* set to 1 when running mac_substitute */
int dotdot; /* set for .. in subscript */
void *nvwalk; /* for name space walking */
} Mac_t;
@ -96,7 +92,7 @@ typedef struct _mac_
#define M_TYPE 8 /* ${@var} */
static noreturn void mac_error(Namval_t*);
static int substring(const char*, const char*, int[], int);
static int substring(const char*, size_t, const char*, int[], int);
static void copyto(Mac_t*, int, int);
static void comsubst(Mac_t*, Shnode_t*, int);
static int varsub(Mac_t*);
@ -862,7 +858,22 @@ done:
static void mac_substitute(Mac_t *mp, register char *cp,char *str,register int subexp[],int subsize)
{
register int c,n;
register char *first=cp;
register char *first=fcseek(0);
char *ptr;
Mac_t savemac;
n = stktell(sh.stk);
savemac = *mp;
mp->pattern = 3;
mp->split = 0;
mp->macsub++;
fcsopen(cp);
copyto(mp,0,0);
sfputc(sh.stk,0);
ptr = cp = sh_strdup(stkptr(sh.stk,n));
stkseek(sh.stk,n);
*mp = savemac;
fcsopen(first);
first = cp;
while(1)
{
while((c= *cp++) && c!=ESCAPE);
@ -890,6 +901,7 @@ static void mac_substitute(Mac_t *mp, register char *cp,char *str,register int s
}
if(n=cp-first-1)
mac_copy(mp,first,n);
free(ptr);
}
#if SHOPT_FILESCAN
@ -1362,7 +1374,7 @@ retry1:
ap = nv_arrayptr(np=nq);
if(ap)
{
nv_putsub(np,v,ARRAY_SCAN);
np = nv_putsub(np,v,ARRAY_SCAN);
v = stkptr(stkp,mp->dotdot);
dolmax =1;
if(array_assoc(ap))
@ -1796,22 +1808,7 @@ retry1:
}
pattern = sh_strdup(argp);
if((type=='/' || c=='/') && (repstr = mac_getstring(pattern)))
{
Mac_t savemac;
char *first = fcseek(0);
int n = stktell(stkp);
savemac = *mp;
fcsopen(repstr);
mp->pattern = 3;
mp->split = 0;
copyto(mp,0,0);
sfputc(stkp,0);
repstr = sh_strdup(stkptr(stkp,n));
replen = strlen(repstr);
stkseek(stkp,n);
*mp = savemac;
fcsopen(first);
}
if(v || c=='/' && offset>=0)
stkseek(stkp,offset);
}
@ -1822,29 +1819,30 @@ retry2:
if(v && (!nulflg || *v ) && c!='+')
{
int ofs_size = 0;
regoff_t match[2*(MATCH_MAX+1)];
int nmatch, nmatch_prev, vsize_last;
char *vlast = NIL(char*);
int match[2*(MATCH_MAX+1)],index;
int nmatch, nmatch_prev, vsize_last, tsize;
char *vlast = NIL(char*), *oldv;
while(1)
{
if(!v)
v= "";
if(c=='/' || c=='#' || c== '%')
{
int index = 0;
flag = (type || c=='/')?(STR_GROUP|STR_MAXIMAL):STR_GROUP;
if(c!='/')
flag |= STR_LEFT;
nmatch = 0;
index = nmatch = 0;
tsize = (int)strlen(v);
while(1)
{
vsize = strlen(v);
vsize = tsize;
oldv = v;
nmatch_prev = nmatch;
if(c=='%')
nmatch=substring(v,pattern,match,flag&STR_MAXIMAL);
nmatch=substring(v,tsize,pattern,match,flag&STR_MAXIMAL);
else
nmatch=strgrpmatch(v,pattern,match,elementsof(match)/2,flag);
if(nmatch && replen>0)
nmatch=strngrpmatch(v,vsize,pattern,(ssize_t*)match,elementsof(match)/2,flag|STR_INT);
if(nmatch && repstr && !mp->macsub)
sh_setmatch(v,vsize,nmatch,match,index++);
if(nmatch)
{
@ -1871,13 +1869,16 @@ retry2:
mac_copy(mp,v,1);
v++;
}
tsize -= v-oldv;
continue;
}
vsize = -1;
break;
}
if(replen==0)
if(!mp->macsub && (!repstr || (nmatch==0 && index==0)))
sh_setmatch(vlast,vsize_last,nmatch,match,index++);
if(!mp->macsub && index>0 && c=='/' && type)
sh_setmatch(0,0,nmatch,0,-1);
}
if(vsize)
mac_copy(mp,v,vsize>0?vsize:strlen(v));
@ -2044,8 +2045,6 @@ retry2:
nv_close(np);
if(pattern)
free(pattern);
if(repstr)
free(repstr);
if(idx)
free(idx);
return(1);
@ -2534,27 +2533,27 @@ static void endfield(register Mac_t *mp,int split)
* Finds the right substring of STRING using the expression PAT
* the longest substring is found when FLAG is set.
*/
static int substring(register const char *string,const char *pat,int match[], int flag)
static int substring(register const char *string,size_t len,const char *pat,int match[], int flag)
{
register const char *sp=string;
register int size,len,nmatch,n;
register int size,nmatch,n;
int smatch[2*(MATCH_MAX+1)];
if(flag)
{
if(n=strgrpmatch(sp,pat,smatch,elementsof(smatch)/2,STR_RIGHT|STR_MAXIMAL))
if(n=strngrpmatch(sp,len,pat,(ssize_t*)smatch,elementsof(smatch)/2,STR_RIGHT|STR_MAXIMAL|STR_INT))
{
memcpy(match,smatch,n*2*sizeof(smatch[0]));
return(n);
}
return(0);
}
size = len = strlen(sp);
size = (int)len;
sp += size;
while(sp>=string)
{
if(mbwide())
sp = lastchar(string,sp);
if(n=strgrpmatch(sp,pat,smatch,elementsof(smatch)/2,STR_RIGHT|STR_LEFT|STR_MAXIMAL))
if(n=strgrpmatch(sp,pat,(ssize_t*)smatch,elementsof(smatch)/2,STR_RIGHT|STR_LEFT|STR_MAXIMAL|STR_INT))
{
nmatch = n;
memcpy(match,smatch,n*2*sizeof(smatch[0]));

24
src/cmd/ksh93/sh/name.c
View file

@ -3355,6 +3355,27 @@ int nv_rename(register Namval_t *np, int flags)
}
else
mp = np;
if(nr==SH_MATCHNOD)
{
Sfio_t *iop;
Dt_t *save_root = sh.var_tree;
int trace = sh_isoption(SH_XTRACE);
sfprintf(sh.strbuf,"typeset -a %s=",nv_name(mp));
nv_outnode(nr,sh.strbuf,-1,0);
sfwrite(sh.strbuf,")\n",2);
cp = sfstruse(sh.strbuf);
iop = sfopen((Sfio_t*)0,cp,"s");
if(trace)
sh_offoption(SH_XTRACE);
sh.var_tree = last_root;
sh_eval(iop,SH_READEVAL);
sh.var_tree = save_root;
if(trace)
sh_onoption(SH_XTRACE);
if(flags&NV_MOVE)
sh_setmatch(0,0,0,0,0);
}
else
nv_clone(nr,mp,(flags&NV_MOVE)|NV_COMVAR);
mp->nvenv = nvenv;
if(flags&NV_MOVE)
@ -3632,6 +3653,8 @@ char *nv_name(register Namval_t *np)
#endif /* SHOPT_NAMESPACE */
return(np->nvname);
}
if(!np->nvname)
goto skip;
#if SHOPT_FIXEDARRAY
ap = nv_arrayptr(np);
#endif /* SHOPT_FIXEDARRAY */
@ -3651,6 +3674,7 @@ char *nv_name(register Namval_t *np)
sh.last_table = nv_parent(np);
else if(!nv_isref(np))
{
skip:
for(fp= np->nvfun ; fp; fp=fp->next)
if(fp->disc && fp->disc->namef)
{

3
src/cmd/ksh93/sh/nvtree.c
View file

@ -672,9 +672,10 @@ static void outval(char *name, const char *vname, struct Walk *wp)
register Namval_t *np, *nq=0, *last_table=sh.last_table;
register Namfun_t *fp;
int isarray=0, special=0,mode=0;
Dt_t *root = wp->root?wp->root:sh.var_base;
if(*name!='.' || vname[strlen(vname)-1]==']')
mode = NV_ARRAY;
if(!(np=nv_open(vname,wp->root,mode|NV_VARNAME|NV_NOADD|NV_NOASSIGN|NV_NOFAIL|wp->noscope)))
if(!(np=nv_open(vname,root,mode|NV_VARNAME|NV_NOADD|NV_NOASSIGN|NV_NOFAIL|wp->noscope)))
{
sh.last_table = last_table;
return;

2
src/cmd/ksh93/sh/xec.c
View file

@ -2816,9 +2816,11 @@ int sh_trace(register char *argv[], register int nl)
cp = "+ ";
else
{
sh.intrace = 1;
sh_offoption(SH_XTRACE);
cp = sh_mactry(cp);
sh_onoption(SH_XTRACE);
sh.intrace = 0;
}
if(*cp)
sfputr(sfstderr,cp,-1);

1076
src/cmd/ksh93/tests/sh_match.sh Executable file
View file

File diff suppressed because it is too large Load diff

21
src/cmd/ksh93/tests/substring.sh
View file

@ -252,7 +252,7 @@ then err_exit '${var//+(\S)/Q} not workding'
fi
var=$($SHELL -c 'v=/vin:/usr/vin r=vin; : ${v//vin/${r//v/b}};typeset -p .sh.match') 2> /dev/null
((SHOPT_2DMATCH)) && exp='typeset -a .sh.match=((vin vin) )' || exp='typeset -a .sh.match=(vin)'
exp='typeset -a .sh.match=((vin vin) )'
[[ $var == "$exp" ]] || err_exit '.sh.match not correct when replacement pattern contains a substring match' \
"(expected $(printf %q "$exp"), got $(printf %q "$var"))"
@ -683,5 +683,24 @@ Errors=$?
# Test for a crash after unsetting ${.sh.match} then matching a pattern
$SHELL -c 'unset .sh.match; [[ bar == ba* ]]' || err_exit 'crash after unsetting .sh.match then trying to match a pattern'
# Tests for ${.sh.match} backported from ksh93v-
unset v d
v=abbbc
d="${v/~(E)b{2,4}/dummy}"
[[ ${.sh.match} == bbb ]] || err_exit '.sh.match wrong after ${s/~(E)b{2,4}/dummy}'
[[ $d == adummyc ]] || err_exit '${s/~(E)b{2,4}/dummy} not working'
x=1234
: "${x//~(X)([012])|([345])/}"
[[ ${.sh.match[1][600..602]} ]] && err_exit '${.sh.match[0][600..602]} is not the empty string'
: "${x//~(X)([012])|([345])/}"
x=$(print -v .sh.match)
compound co
typeset -m co.array=.sh.match
[[ $x == "$(print -v co.array)" ]] || err_exit 'typeset -m for .sh.match to compound variable not working (1)'
: "${x//~(X)([345])|([012])/}"
[[ $x == "$(print -v co.array)" ]] || err_exit 'typeset -m for .sh.match to compound variable not working (2)'
# ======
exit $((Errors<125?Errors:125))

19
src/cmd/ksh93/tests/treemove.sh
View file

@ -2,7 +2,7 @@
# #
# This software is part of the ast package #
# Copyright (c) 1982-2011 AT&T Intellectual Property #
# Copyright (c) 2020-2021 Contributors to ksh 93u+m #
# Copyright (c) 2020-2022 Contributors to ksh 93u+m #
# and is licensed under the #
# Eclipse Public License, Version 1.0 #
# by AT&T Intellectual Property #
@ -152,4 +152,21 @@ then if [[ $(kill -l $exitval) == SEGV ]]
else err_exit 'typeset -m "c.board[1][i]=el" gives wrong value'
fi
fi
function f2
{
nameref mar=$1 exp=$2
typeset dummy x="-1a2b3c4d9u"
dummy="${x//~(E)([[:digit:]])|([[:alpha:]])/D}"
exp=${ print -v .sh.match;}
typeset -m "mar=.sh.match"
}
function f1
{
typeset matchar exp
f2 matchar exp
[[ ${ print -v matchar;} == "$exp" ]] || err_exit 'moving .sh.match to a function local variable using a name reference fails'
}
f1
exit $((Errors<125?Errors:125))

3
src/cmd/ksh93/tests/types.sh
View file

@ -552,7 +552,8 @@ compound -a b.ca
b_t b.ca[4].b
exp='typeset -C b=(typeset -C -a ca=( [4]=(b_t b=(a_t b=(a=hello))));)'
got=$(typeset -p b)
[[ $got == "$exp" ]] || err_exit 'typeset -p of nested type not correct'
[[ $got == "$exp" ]] || err_exit 'typeset -p of nested type not correct' \
"(expected $(printf %q "$exp"), got $(printf %q "$got"))"
typeset -T u_t=(
integer dummy

4
src/lib/libast/features/api
View file

@ -1,8 +1,8 @@
iff AST_API
ver ast 20220201
ver ast 20220208
api ast 20120528 regexec regnexec regrexec regsubexec strgrpmatch
api ast 20120528 regexec regnexec regrexec regsubexec
api ast 20120411 cmdopen

4
src/lib/libast/features/map.c
View file

@ -2,7 +2,7 @@
* *
* This software is part of the ast package *
* Copyright (c) 1985-2011 AT&T Intellectual Property *
* Copyright (c) 2020-2021 Contributors to ksh 93u+m *
* Copyright (c) 2020-2022 Contributors to ksh 93u+m *
* and is licensed under the *
* Eclipse Public License, Version 1.0 *
* by AT&T Intellectual Property *
@ -322,6 +322,8 @@ main()
printf("#define strgid _ast_strgid\n");
printf("#undef strgrpmatch\n");
printf("#define strgrpmatch _ast_strgrpmatch\n");
printf("#undef strngrpmatch\n");
printf("#define strngrpmatch _ast_strngrpmatch\n");
printf("#undef strhash\n");
printf("#define strhash _ast_strhash\n");
printf("#undef strkey\n");

15
src/lib/libast/include/ast.h
View file

@ -169,11 +169,12 @@ typedef struct
* strgrpmatch() flags
*/
#define STR_MAXIMAL 01 /* maximal match */
#define STR_LEFT 02 /* implicit left anchor */
#define STR_RIGHT 04 /* implicit right anchor */
#define STR_ICASE 010 /* ignore case */
#define STR_GROUP 020 /* (|&) inside [@|&](...) only */
#define STR_MAXIMAL 0x01 /* maximal match */
#define STR_LEFT 0x02 /* implicit left anchor */
#define STR_RIGHT 0x04 /* implicit right anchor */
#define STR_ICASE 0x08 /* ignore case */
#define STR_GROUP 0x10 /* (|&) inside [@|&](...) only */
#define STR_INT 0x20 /* int* match array */
/*
* fmtquote() flags
@ -371,8 +372,8 @@ extern int strexp(char*, int);
extern long streval(const char*, char**, long(*)(const char*, char**));
extern long strexpr(const char*, char**, long(*)(const char*, char**, void*), void*);
extern int strgid(const char*);
extern int strgrpmatch(const char*, const char*, int*, int, int);
extern int strgrpmatch_20120528(const char*, const char*, ssize_t*, int, int);
extern int strgrpmatch(const char*, const char*, ssize_t*, int, int);
extern int strngrpmatch(const char*, size_t, const char*, ssize_t*, int, int);
extern unsigned int strhash(const char*);
extern void* strlook(const void*, size_t, const char*);
extern int strmatch(const char*, const char*);

4
src/lib/libast/man/astsa.3
View file

@ -124,7 +124,7 @@ Returns the error message text corresponding to the
\fIerrno\fP.
.Ss "char* strerror(int \fIerrno\fP)"
Equivalent to fmterror(\fIerrno\fP).
.Ss "int strgrpmatch(const char* \fIstring\fP, const char* \fIpattern\fP, int* \fIsub\fP, int \fInsub\fP, int \fIflags\fP)"
.Ss "int strgrpmatch(const char* \fIstring\fP, const char* \fIpattern\fP, ssize_t* \fIsub\fP, int \fInsub\fP, int \fIflags\fP)"
Matches the null-terminated \fIstring\fP against the null-terminated
.BR ksh (1)
augmented \fIpattern\fP.
@ -148,8 +148,6 @@ Ignore case.
.Tp
\f3STR_GROUP\fP:
(|&) inside [@|*|+{n,m}](...) only.
.Ss "int strmatch(const char* \fIstring\fP, const char* \fIpattern\fP, int* \fIsub\fP, int \fInsub\fP, int \fIflags\fP)"
Equivalent to strgrpmatch(\fIstring\fP,\fIpattern\fP,0,0,STR_MAXIMAL|STR_LEFT|STR_RIGHT).
.SH "SEE ALSO"
.BR ast (3),
.BR ccode (3),

16
src/lib/libast/string/strmatch.c
View file

@ -2,7 +2,7 @@
* *
* This software is part of the ast package *
* Copyright (c) 1985-2012 AT&T Intellectual Property *
* Copyright (c) 2020-2021 Contributors to ksh 93u+m *
* Copyright (c) 2020-2022 Contributors to ksh 93u+m *
* and is licensed under the *
* Eclipse Public License, Version 1.0 *
* by AT&T Intellectual Property *
@ -64,8 +64,6 @@ static struct State_s
int nmatch;
} matchstate;
#define STR_INT 040000
/*
* subgroup match
* 0 returned if no match
@ -77,7 +75,7 @@ static struct State_s
*/
int
strgrpmatch(const char* b, const char* p, ssize_t* sub, int n, register int flags)
strngrpmatch(const char* b, size_t z, const char* p, ssize_t* sub, int n, register int flags)
{
register regex_t* re;
register ssize_t* end;
@ -140,7 +138,7 @@ strgrpmatch(const char* b, const char* p, ssize_t* sub, int n, register int flag
return 0;
matchstate.nmatch = n;
}
if (regexec(re, b, n, matchstate.match, reflags & ~(REG_MINIMAL|REG_SHELL_GROUP|REG_LEFT|REG_RIGHT|REG_ICASE)))
if (regnexec(re, b, z, n, matchstate.match, reflags & ~(REG_MINIMAL|REG_SHELL_GROUP|REG_LEFT|REG_RIGHT|REG_ICASE)))
return 0;
if (!sub || n <= 0)
return 1;
@ -176,7 +174,7 @@ strgrpmatch(const char* b, const char* p, ssize_t* sub, int n, register int flag
int
strmatch(const char* s, const char* p)
{
return strgrpmatch(s, p, NiL, 0, STR_MAXIMAL|STR_LEFT|STR_RIGHT);
return strngrpmatch(s, s ? strlen(s) : 0, p, NiL, 0, STR_MAXIMAL|STR_LEFT|STR_RIGHT);
}
/*
@ -192,7 +190,7 @@ strsubmatch(const char* s, const char* p, int flags)
{
ssize_t match[2];
return strgrpmatch(s, p, match, 1, (flags ? STR_MAXIMAL : 0)|STR_LEFT) ? (char*)s + match[1] : (char*)0;
return strngrpmatch(s, s ? strlen(s) : 0, p, match, 1, (flags ? STR_MAXIMAL : 0)|STR_LEFT) ? (char*)s + match[1] : (char*)0;
}
#undef strgrpmatch
@ -201,7 +199,7 @@ strsubmatch(const char* s, const char* p, int flags)
#endif
int
strgrpmatch(const char* b, const char* p, int* sub, int n, int flags)
strgrpmatch(const char* b, const char* p, ssize_t* sub, int n, int flags)
{
return strgrpmatch_20120528(b, p, (ssize_t*)sub, n, flags|STR_INT);
return strngrpmatch(b, b ? strlen(b) : 0, p, sub, n, flags|STR_INT);
}