magento2: Jmeter error

Using benchmark.jmx from CE setup/performance-toolkit folder

jmeter.bat -n -t benchmark.jmx -j jmeter.log -l jmeter-results.jtl -Jhost=XXXXXX -Jbase_path=/ -Jadmin_path=backend -JsetupAndTearDownThread=10 -Jramp_period=300 -Jusers=1000 -Jorders=100000 -Jadmin_user=admin -Jadmin_password=XXXXXX -Jrequest_protocol=https

After small time test fails.

On jmeter.log I can see error:

WARN o.a.j.p.j.s.BeanShellSampler: Exception executing script. org.apache.jorphan.util.JMeterException: Error invoking bsh method: eval Sourced file: inline evaluation of: ``import java.util.ArrayList; ArrayList categoryUrlKeysList; // If it is first ite . . .

Small investigation, error is here (and on other interation lists)

// If it is first iteration of cycle then recreate category url key list if (1 == Integer.parseInt(vars.get("category_url_key_counter"))) { categoryUrlKeysList = new ArrayList(); props.put("category_url_keys_list", categoryUrlKeysList); props.put("category_url_key", vars.get("category_url_key")); } else { categoryUrlKeysList = props.get("category_url_keys_list"); }

It seems, that if condition is logically correct, but in execution - it fails. (I’m not java developer, but error says, that categoryUrlKeysList isn’t created)

If I add ArrayList categoryUrlKeysList; before if condition I’m do not get errors, but get test exit (due to no categories in benchmark workflow)

Same error, if using EE setup/performance-toolkit folder

Using 2.2.3 CE, but it’s error/bug in benchmark.jmx Using Jmeter 2/3/4