ksh: .sh.match index array result issues, crash after unset

While researching, I became aware of some issues with .sh.match index array results: parent array having both -C and -a types, invalid script names of NULL, concealed/null array elements expressed as spaces, these null elements count towards size and subscript listing, and finally invalid size count on parent array. All of these issues have been fixed in ksh93v-.

ksh93u+m/1.1.0-alpha+1fc28dc2 2021-05-13: extra compound variable attribute, phantom array elements, and a static number of 4194303 for the size for parent array, invalid subscript names.

$ ./ksh -c 'x=1234; true ${x//~(X)([012])|([345])/ }; typeset -p .sh.match .sh.match[1] .sh.match[2]; echo ${#.sh.match[@]} ${#.sh.match[1][@]} ${#.sh.match[2][@]}; echo ${!.sh.match[2][@]};'
typeset -C -a .sh.match=((1 2 3 4) (1 2  ) (  3 4) )
typeset -a .sh.match[(null)]=(1 2  )
typeset -a .sh.match[(null)]=(  3 4)
4194303 4 4
0 1 2 3

ksh93u+ 2012-08-01: sparse interval array listing for arrays and subarrays when most are not, wrong subarray sizes, invalid subscript names, phantom array elements.

$ ./ksh-att -c 'x=1234; true ${x//~(X)([012])|([345])/ }; typeset -p .sh.match .sh.match[1] .sh.match[2]; echo ${#.sh.match[@]} ${#.sh.match[1][@]} ${#.sh.match[2][@]}; echo ${!.sh.match[2][@]};'
typeset -a .sh.match=(([0]=1 [1]=2 [2]=3 [3]=4) ([0]=1 [1]=2) ([2]=3 [3]=4) )
typeset -a .sh.match[(null)]=([0]=1 [1]=2)
typeset -a .sh.match[(null)]=([2]=3 [3]=4)
3 3 3
2 3

ksh93v- 2014-12-24: all good. even the sparse indexed match[2] array is correct. 😃

$ ./ksh93v- -c 'x=1234; true ${x//~(X)([012])|([345])/ }; typeset -p .sh.match .sh.match[1] .sh.match[2]; echo ${#.sh.match[@]} ${#.sh.match[1][@]} ${#.sh.match[2][@]}; echo ${!.sh.match[2][@]};'
typeset -a .sh.match=((1 2 3 4) (1 2) ([2]=3 [3]=4) )
typeset -a .sh.match[1]=(1 2)
typeset -a .sh.match[2]=([2]=3 [3]=4)
3 2 2
2 3

ksh93u+m/1.1.0-alpha+dev 2021-05-13 (patch provided below): same as first but provided patch fixes -C issue.

$ ./ksh-p1 -c 'x=1234; true ${x//~(X)([012])|([345])/ }; typeset -p .sh.match .sh.match[1] .sh.match[2]; echo ${#.sh.match[@]} ${#.sh.match[1][@]} ${#.sh.match[2][@]}; echo ${!.sh.match[2][@]};'
typeset -a .sh.match=((1 2 3 4) (1 2  ) (  3 4) )
typeset -a .sh.match[(null)]=(1 2  )
typeset -a .sh.match[(null)]=(  3 4)
4194303 4 4
0 1 2 3
diff --git a/src/cmd/ksh93/sh/init.c b/src/cmd/ksh93/sh/init.c
index 5b7dcdc..a4839ea 100644
--- a/src/cmd/ksh93/sh/init.c
+++ b/src/cmd/ksh93/sh/init.c
@@ -801,7 +801,10 @@ void sh_setmatch(Shell_t *shp,const char *v, int vsize, int nmatch, regoff_t mat
 		if(!nv_hasdisc(SH_MATCHNOD,mp->hdr.disc))
 			nv_disc(SH_MATCHNOD,&mp->hdr,NV_LAST);
 		if(nmatch)
+		{
+			nv_onattr(SH_MATCHNOD,NV_NOFREE|NV_ARRAY);
 			nv_putsub(SH_MATCHNOD, NIL(char*), (nmatch-1)|ARRAY_FILL|ARRAY_SETSUB);
+		}
 		ap_save->nelem = mp->nmatch = nmatch;
 		mp->v = v;
 		mp->first = match[0];
@@ -813,7 +816,6 @@ void sh_setmatch(Shell_t *shp,const char *v, int vsize, int nmatch, regoff_t mat
 		{
 			np->nvalue.cp = Empty;
 			np->nvfun = SH_MATCHNOD->nvfun;
-			nv_onattr(np,NV_NOFREE|NV_ARRAY);
 			SH_MATCHNOD->nvfun = 0;
 			for(i=0; i < mp->nmatch; i++)
 			{

RESEARCH

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 26

Commits related to this issue

Most upvoted comments

The sh_match.sh tests don’t run on my system because of the unportable way it creates a temporary directory. But that code is also superfluous as the 93u+m version of shtests gives each test script its own temporary directory already, and makes it the PWD. This patch allows the tests to run (and pass) on my system.

diff --git a/src/cmd/ksh93/tests/sh_match.sh b/src/cmd/ksh93/tests/sh_match.sh
index 8f2584e6c..04dc9e304 100644
--- a/src/cmd/ksh93/tests/sh_match.sh
+++ b/src/cmd/ksh93/tests/sh_match.sh
@@ -27,14 +27,6 @@

 # test setup
 . "${SHTESTS_COMMON:-${0%/*}/_common}"
-typeset ocwd
-typeset tmpdir
-
-# create temporary test directory
-ocwd="${PWD}"
-tmpdir="${ mktemp -t -d 'test_sh_match.XXXXXXXX' ; }" || err_exit 'Cannot create temporary directory.'
-
-cd "${tmpdir}" || { err_exit "cd ${tmpdir} failed." ; exit $((Errors<125?Errors:125)) ; }

 # tests
 function test_xmlfragment1
@@ -101,7 +93,7 @@ cat >"${testscript}" <<-TEST1SCRIPT
 		# rebuild the original text from "ar" (copy of ".sh.match")
 		# and compare it to the content of "xtext"
 		#
-		tmpfile=\$(mktemp)
+		tmpfile=rebuild_xml_and_verify.\$\$

 		{
 			# rebuild the original text, based on our matches
@@ -659,11 +651,7 @@ test_testop_v2
 test_num_elements1
 test_nomatch

-
-# cleanup
-cd "${ocwd}"
-rmdir "${tmpdir}" || err_exit "Cannot remove temporary directory ${tmpdir}."
-
+# ======
 set +u
 x=1234
 compound co

I found some additional issues with the matchfixes branch which I will post later after gathering my notes.

Separately, I wanted to make you all aware of some potential bug fixes from another ksh fork that may benefit ksh93u+m including this issue. Please peruse https://github.com/l0stman/ksh/commits/master with special relevance to the following:

I’ve managed to backport all of the fixes from ksh93v- required for the reproducer to print the expected results (primarily from the 2012-08-24 and 2012-10-04 versions), although there are bound to be regressions also backported from ksh93v- (one of the .sh.match regression tests fails). As the patch grew to a rather large size, I’ve put my work thus far into a separate branch: https://github.com/JohnoKing/ksh/commits/matchfixes