ksh: $RANDOM does not get re-seeded within a subshell that redirects stdout

The ksh man page says RANDOM variable is supposed to return “a random integer, uniformly distributed between 0 and 32767” each time it’s referenced. However, if the value of $RANDOM is used in a subshell that previously redirected the stdout of any command, $RANDOM has the same value if it’s read multiple times within the same command. Here’s an example of the bizarre behavior.

$ ksh --version
  version         sh (AT&T Research) 93u+ 2012-08-01
$ function rand {
>  print "this redirect should not matter" >/dev/null
>  print $RANDOM
> }
# It returns the same value every time within the loop,
$ for i in {1..3}; do x=$(rand); print $x; done
16720
16720
16720
# but a different value in each loop.
$ for i in {1..3}; do x=$(rand); print $x; done
30698
30698
30698
# When called outside the subshell, each value is different, as expected.
$ for i in {1..3}; do rand; done 
4135
21706
5128
# Without the redirect, it works as intended.
$ function rand_without_redirect { 
> print $RANDOM
}
$ for i in {1..3}; do x=$(rand_without_redirect); print $x; done
4612
23899
10106

zsh has a similar behavior that’s not quite the same. man zshparam says “The values of RANDOM form an intentionally-repeatable pseudo-random sequence; subshells that reference RANDOM will result in identical pseudo-random values unless the value of RANDOM is referenced or seeded in the parent shell in between subshell invocations.” But this isn’t zsh, and the redirection of a previous command shouldn’t matter.

This random number manipulation makes $RANDOM even more cryptographically unsecure than it already was. If anyone was depending on $RANDOM for security - which they shouldn’t have been - then they’re in for a bad time.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 20

Commits related to this issue

Set a different pseudorandom seed after forking a subshell This fixes the issue detailed in bug #285. The main problem is that ksh doesn't set a new pseudorandom seed after forking a subshell. The re... — committed to JohnoKing/ksh by JohnoKing 3 years ago
Set a different pseudorandom seed after forking a subshell This fixes the issue detailed in bug #285. The main problem is that ksh doesn't set a new pseudorandom seed after forking a subshell. The re... — committed to JohnoKing/ksh by JohnoKing 3 years ago
Fix $RANDOM to act consistently in subshells This fixes the following: 1. Using $RANDOM in a virtual/non-forked subshell no longer influences the reproducible $RANDOM sequence in the parent env... — committed to McDutchie/ksh by McDutchie 3 years ago
Fix $RANDOM to act consistently in subshells (#294) This fixes the following: 1. Using $RANDOM in a virtual/non-forked subshell no longer influences the reproducible $RANDOM sequence in the pare... — committed to ksh93/ksh by McDutchie 3 years ago

Most upvoted comments

Here is a thought. If you want more randomness in a subshell as compared to its parent,

We actually do not, though. See this comment.

McDutchie on Apr 26, 2021

That’s clever: since you can’t restore the state of the pseudorandom generator, you recreate it instead.

But I see a problem with this approach: the more $RANDOM iterations have been done in the main shell, the slower a subshell is going to be, as it has to redo at least that many rand() calls. Even though a rand() call is very fast, at some point you’ll reach the point where forking is faster.

Here is a test on my system with subshells after 100 million $RANDOM iterations in the main shell.

print "Moment..." >&2
integer i
for ((i=0; i<1000000; i++))
do	: $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM \
	$RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM \
	$RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM \
	$RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM \
	$RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM \
	$RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM \
	$RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM \
	$RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM \
	$RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM \
	$RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM
done
print "Timing virtual subshells with \$RANDOM..." >&2
TIMEFORMAT=$'real\t%3lR'
time (: $RANDOM)
time (true $RANDOM)
print
print "Timing forked subshells with \$RANDOM..." >&2
time (ulimit -t unlimited; : $RANDOM)
time (ulimit -t unlimited; true $RANDOM)

Output:

$ arch/*/bin/ksh test.sh
Moment...
Timing virtual subshells with $RANDOM...
real	0m00,688s
real	0m00,737s

Timing forked subshells with $RANDOM...
real	0m00,001s
real	0m00,001s

IMO this is a clever idea, but possibly too clever. It introduces complexity and gradual performance degradation. I’d prefer just forking the subshell upon expanding $RANDOM.

McDutchie on Apr 26, 2021