ksh: .sh.match index array result issues, crash after unset
While researching, I became aware of some issues with .sh.match index array results: parent array having both -C and -a types, invalid script names of NULL, concealed/null array elements expressed as spaces, these null elements count towards size and subscript listing, and finally invalid size count on parent array. All of these issues have been fixed in ksh93v-.
ksh93u+m/1.1.0-alpha+1fc28dc2 2021-05-13: extra compound variable attribute, phantom array elements, and a static number of 4194303 for the size for parent array, invalid subscript names.
$ ./ksh -c 'x=1234; true ${x//~(X)([012])|([345])/ }; typeset -p .sh.match .sh.match[1] .sh.match[2]; echo ${#.sh.match[@]} ${#.sh.match[1][@]} ${#.sh.match[2][@]}; echo ${!.sh.match[2][@]};'
typeset -C -a .sh.match=((1 2 3 4) (1 2 ) ( 3 4) )
typeset -a .sh.match[(null)]=(1 2 )
typeset -a .sh.match[(null)]=( 3 4)
4194303 4 4
0 1 2 3
ksh93u+ 2012-08-01: sparse interval array listing for arrays and subarrays when most are not, wrong subarray sizes, invalid subscript names, phantom array elements.
$ ./ksh-att -c 'x=1234; true ${x//~(X)([012])|([345])/ }; typeset -p .sh.match .sh.match[1] .sh.match[2]; echo ${#.sh.match[@]} ${#.sh.match[1][@]} ${#.sh.match[2][@]}; echo ${!.sh.match[2][@]};'
typeset -a .sh.match=(([0]=1 [1]=2 [2]=3 [3]=4) ([0]=1 [1]=2) ([2]=3 [3]=4) )
typeset -a .sh.match[(null)]=([0]=1 [1]=2)
typeset -a .sh.match[(null)]=([2]=3 [3]=4)
3 3 3
2 3
ksh93v- 2014-12-24: all good. even the sparse indexed match[2] array is correct. 😃
$ ./ksh93v- -c 'x=1234; true ${x//~(X)([012])|([345])/ }; typeset -p .sh.match .sh.match[1] .sh.match[2]; echo ${#.sh.match[@]} ${#.sh.match[1][@]} ${#.sh.match[2][@]}; echo ${!.sh.match[2][@]};'
typeset -a .sh.match=((1 2 3 4) (1 2) ([2]=3 [3]=4) )
typeset -a .sh.match[1]=(1 2)
typeset -a .sh.match[2]=([2]=3 [3]=4)
3 2 2
2 3
ksh93u+m/1.1.0-alpha+dev 2021-05-13 (patch provided below): same as first but provided patch fixes -C issue.
$ ./ksh-p1 -c 'x=1234; true ${x//~(X)([012])|([345])/ }; typeset -p .sh.match .sh.match[1] .sh.match[2]; echo ${#.sh.match[@]} ${#.sh.match[1][@]} ${#.sh.match[2][@]}; echo ${!.sh.match[2][@]};'
typeset -a .sh.match=((1 2 3 4) (1 2 ) ( 3 4) )
typeset -a .sh.match[(null)]=(1 2 )
typeset -a .sh.match[(null)]=( 3 4)
4194303 4 4
0 1 2 3
diff --git a/src/cmd/ksh93/sh/init.c b/src/cmd/ksh93/sh/init.c
index 5b7dcdc..a4839ea 100644
--- a/src/cmd/ksh93/sh/init.c
+++ b/src/cmd/ksh93/sh/init.c
@@ -801,7 +801,10 @@ void sh_setmatch(Shell_t *shp,const char *v, int vsize, int nmatch, regoff_t mat
if(!nv_hasdisc(SH_MATCHNOD,mp->hdr.disc))
nv_disc(SH_MATCHNOD,&mp->hdr,NV_LAST);
if(nmatch)
+ {
+ nv_onattr(SH_MATCHNOD,NV_NOFREE|NV_ARRAY);
nv_putsub(SH_MATCHNOD, NIL(char*), (nmatch-1)|ARRAY_FILL|ARRAY_SETSUB);
+ }
ap_save->nelem = mp->nmatch = nmatch;
mp->v = v;
mp->first = match[0];
@@ -813,7 +816,6 @@ void sh_setmatch(Shell_t *shp,const char *v, int vsize, int nmatch, regoff_t mat
{
np->nvalue.cp = Empty;
np->nvfun = SH_MATCHNOD->nvfun;
- nv_onattr(np,NV_NOFREE|NV_ARRAY);
SH_MATCHNOD->nvfun = 0;
for(i=0; i < mp->nmatch; i++)
{
RESEARCH
- https://github.com/att/ast/blob/2016-01-10-beta/src/cmd/ksh93/RELEASE: 12-08-10 Rewrote .sh.match expansions fixing several bugs and improving performance.
- https://marc.info/?l=ast-users&m=134461525925663&w=2
- https://marc.info/?l=ast-developers&m=134462636529838&w=2
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 26
Commits related to this issue
- Add .sh.match reproducer from https://github.com/ksh93/ksh/issues/308 — committed to JohnoKing/ksh by JohnoKing 2 years ago
- Fix sh_match unit test portability From https://github.com/ksh93/ksh/issues/308#issuecomment-1024992776 — committed to JohnoKing/ksh by JohnoKing 2 years ago
- Fix multiple bugs in .sh.match This pull request backports all of the relevant .sh.match bugfixes from ksh93v-. Most of the .sh.match rewrite is from versions 2012-08-24 and 2012-10-04, with patches ... — committed to JohnoKing/ksh by JohnoKing 2 years ago
- Fix multiple bugs in .sh.match This pull request backports all of the relevant .sh.match bugfixes from ksh93v-. Most of the .sh.match rewrite is from versions 2012-08-24 and 2012-10-04, with patches ... — committed to JohnoKing/ksh by JohnoKing 2 years ago
- Fix multiple bugs in .sh.match This pull request backports all of the relevant .sh.match bugfixes from ksh93v-. Most of the .sh.match rewrite is from versions 2012-08-24 and 2012-10-04, with patches ... — committed to JohnoKing/ksh by JohnoKing 2 years ago
- Fix multiple bugs in .sh.match This pull request backports all of the relevant .sh.match bugfixes from ksh93v-. Most of the .sh.match rewrite is from versions 2012-08-24 and 2012-10-04, with patches ... — committed to JohnoKing/ksh by JohnoKing 2 years ago
- Fix multiple bugs in .sh.match This pull request backports all of the relevant .sh.match bugfixes from ksh93v-. Most of the .sh.match rewrite is from versions 2012-08-24 and 2012-10-04, with patches ... — committed to JohnoKing/ksh by JohnoKing 2 years ago
- Fix multiple bugs in .sh.match (#455) This commit backports all of the relevant .sh.match bugfixes from ksh93v-. Most of the .sh.match rewrite is from versions 2012-08-24 and 2012-10-04, with patch... — committed to ksh93/ksh by JohnoKing 2 years ago
- Fix multiple bugs in .sh.match (#455) This commit backports all of the relevant .sh.match bugfixes from ksh93v-. Most of the .sh.match rewrite is from versions 2012-08-24 and 2012-10-04, with patch... — committed to ksh93/ksh by JohnoKing 2 years ago
- Avoid printing a [0] subscript for .sh.match This long overdue bugfix prevents name_match() in init.c from printing a spurious zero element when using ${!.sh.match}. I haven't been able to break this... — committed to JohnoKing/ksh by JohnoKing 5 months ago
- Avoid printing a [0] element for .sh.match This long overdue bugfix prevents name_match() in init.c from printing a spurious zero element when using ${!.sh.match}. I haven't been able to break this b... — committed to JohnoKing/ksh by JohnoKing 5 months ago
- Avoid printing a [0] element for .sh.match This long overdue bugfix prevents name_match() in init.c from printing a spurious zero element when using ${!.sh.match}. I haven't been able to break this b... — committed to JohnoKing/ksh by JohnoKing 5 months ago
- Avoid printing a [0] element for `${!.sh.match}` (#709) This fix prevents name_match() in init.c from printing a spurious extra subscript when using ${!.sh.match}. src/cmd/ksh93/sh/init.c: - Rem... — committed to ksh93/ksh by JohnoKing 5 months ago
- Avoid printing a [0] element for `${!.sh.match}` (#709) This fix prevents name_match() in init.c from printing a spurious extra subscript when using ${!.sh.match}. src/cmd/ksh93/sh/init.c: - Rem... — committed to JohnoKing/ksh by JohnoKing 5 months ago
The sh_match.sh tests don’t run on my system because of the unportable way it creates a temporary directory. But that code is also superfluous as the 93u+m version of shtests gives each test script its own temporary directory already, and makes it the PWD. This patch allows the tests to run (and pass) on my system.
I found some additional issues with the matchfixes branch which I will post later after gathering my notes.
Separately, I wanted to make you all aware of some potential bug fixes from another ksh fork that may benefit ksh93u+m including this issue. Please peruse https://github.com/l0stman/ksh/commits/master with special relevance to the following:
I’ve managed to backport all of the fixes from ksh93v- required for the reproducer to print the expected results (primarily from the 2012-08-24 and 2012-10-04 versions), although there are bound to be regressions also backported from ksh93v- (one of the
.sh.match
regression tests fails). As the patch grew to a rather large size, I’ve put my work thus far into a separate branch: https://github.com/JohnoKing/ksh/commits/matchfixes