ksh: Memory leak when initialising associative array in subshell

When doing this in a virtual/non-forked subshell:

(typeset -A foo=([a]=1 [b]=2 [c]=3))

a memory leak occurs. The memory occupied by the array is not freed when exiting the subshell.

A reproducer with ps can confirm this bug exists on 93u+ 2012-08-01:

for ((i=1; i<=10; i++)); do
        for ((n=1; n<=100000; n++)); do
                (typeset -A foo=([a]=1 [b]=2 [c]=3))
        done
        ps -p "$$" -o rss=,vsz=
done

Output on macOS:

 15900  4285776
 29996  4299856
 44092  4313968
 58188  4328048
 72288  4342160
 86380  4356240
100480  4370352
114572  4384432
128672  4398544
142772  4412656

Reproducer with vmstate, ~~which is compiled into 93u+m by default~~ (edit: as of f9364b17098ca05dc6ab886d8906ee55bfb2126c, you have to pass -D_AST_vmalloc to enable it):

builtin vmstate || exit
for ((i=1; i<=10; i++)); do
        (typeset -A foo=([a]=1 [b]=2 [c]=3))
        vmstate
done

Output (note increasing busy and decreasing free):

region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(282256,170,65552) free=(107776,9,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283280,185,65552) free=(106560,6,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283408,188,65552) free=(106368,7,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283440,191,65552) free=(106128,17,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283664,194,65552) free=(105952,11,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283712,197,65552) free=(105728,19,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283888,200,65552) free=(105600,13,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283920,203,65552) free=(105392,21,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(284096,206,65552) free=(105392,7,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(284208,209,65552) free=(105120,14,34288)

About this issue

Original URL
State: open
Created 4 years ago
Comments: 17

Commits related to this issue

nv_restore(): Fix leaking of associative array members src/cmd/ksh93/sh/subshell.c: nv_restore(): - Before calling _nv_unset for the to-be-restored variable, turn off the NV_NOFREE attribute for th... — committed to atheik/ksh by atheik 2 years ago
nv_restore(): Fix leaking of associative array members src/cmd/ksh93/sh/subshell.c: nv_restore(): - src/cmd/ksh93/bltins/typeset.c: unall(): - src/cmd/ksh93/sh/name.c: nv_newattr(); - Fixes: #94 — committed to atheik/ksh by atheik 2 years ago

Most upvoted comments

I’ve found that this change in ksh93s- is what introduced the memory leak for associative arrays in subshells:

--- b/src/cmd/ksh93/sh/name.c
+++ a/src/cmd/ksh93/sh/name.c
@@ -291,8 +291,7 @@ void nv_setlist(register struct argnod *arg,register int flags)
 				sh.prefix = prefix;
 				if(nv_isarray(np) && (mp=nv_opensub(np)))
 					np = mp;
-				if(nv_isnull(np))
-					nv_setvtree(np);
+				nv_setvtree(np);
 				continue;
 			}
 			cp = arg->argval;

Reverting that change fixes the leak in 93s-, but the code appears to have changed substantially since then (the diff doesn’t apply cleanly to 93u+m). Regardless, it may help provide clues for fixing the memory leak.

JohnoKing on Jan 29, 2022