magento2: Performance toolkit exits before running any scenarios
The Magento Performance Toolkit is unable to run against a vanilla Magento 2 instance installed from scratch with the required fixture data. It successfully completes the preliminary preparation steps, but never gets to execute any frontend scenarios. JMeter exits without any errors.
Preconditions (*)
- Magento 2.2.7 w/ Redis session storage
mysql -e "CREATE DATABASE magento2"
git clone https://github.com/magento/magento2.git
cd magento2/
git checkout -b v2.2.7 2.2.7
composer install --no-dev
chmod -R ug+rwX app/etc/ var/ pub/media/
php bin/magento setup:install \
--base-url="http://127.0.0.1/" \
--backend-frontname="admin" \
--use-rewrites=1 \
--db-host="localhost" --db-name="magento2" \
--admin-user="admin" --admin-password="123123q" \
--admin-firstname="Admin" --admin-lastname="Admin" --admin-email="admin@example.com" \
--language=en_US --currency=USD --timezone=America/Los_Angeles \
--session-save=redis --session-save-redis-host=/var/run/redis.sock --session-save-redis-port=0 \
--session-save-redis-db=0 --session-save-redis-disable-locking=1 --session-save-redis-log-level=3
php bin/magento config:set dev/static/sign 1
php bin/magento config:set dev/css/minify_files 1
php bin/magento config:set dev/js/minify_files 1
php bin/magento config:set dev/js/enable_js_bundling 1
php bin/magento config:set admin/security/admin_account_sharing 1
php bin/magento config:set admin/security/use_form_key 0
php bin/magento deploy:mode:set production
composer dump-autoload --no-dev --optimize
php bin/magento indexer:set-mode schedule
php bin/magento indexer:reindex
php bin/magento cache:flush
- JMeter 3.1 w/ JSON plugin
sudo yum install java -y
curl -L -O https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-3.1.tgz
tar -xzf apache-jmeter-3.1.tgz
curl -L -O https://jmeter-plugins.org/files/packages/jpgc-json-2.5.zip
unzip jpgc-json-2.5.zip -d ./apache-jmeter-3.1/
Steps to reproduce (*)
- Generate fixture data
php bin/magento setup:performance:generate-fixtures setup/performance-toolkit/profiles/ce/small.xml
- Run JMeter test plan
jmeter -n \
-t magento2/setup/performance-toolkit/benchmark.jmx \
-j jmeter.log \
-l jmeter-results.jtl \
-Jhost=127.0.0.1 \
-Jbase_path=/ \
-Jadmin_path=admin \
-Jadmin_user=admin \
-Jadmin_password=123123q \
-Jfiles_folder=magento2/setup/performance-toolkit/files/ \
-Jloops=1 \
-JfrontendPoolUsers=1 \
-JadminPoolUsers=0 \
-JcsrPoolUsers=0 \
-JothersPoolUsers=0
Expected result (*)
- JMeter successfully runs all frontend scenarios
Actual result (*)
- JMeter successfully completes the preparation (log in to admin, load products, categories, customers, etc.)
- JMeter exits before executing any frontend scenarios
Writing log file to: /home/ec2-user/jmeter.log
Creating summariser <summary>
Created the tree successfully using /home/ec2-user/magento2/setup/performance-toolkit/benchmark.jmx
Starting the test @ Sun Feb 10 05:51:07 UTC 2019 (1549777867357)
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary = 224 in 00:00:05 = 45.0/s Avg: 15 Min: 0 Max: 676 Err: 0 (0.00%)
Tidying up ... @ Sun Feb 10 05:51:12 UTC 2019 (1549777872774)
... end of run
- Nginx access logs shows only preparation requests
3.81.150.213 - - [10/Feb/2019:07:33:39 +0000] "GET /admin HTTP/1.1" 302 5 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:39 +0000] "GET /admin/admin/ HTTP/1.1" 200 2358 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:39 +0000] "POST /admin/admin/dashboard/ HTTP/1.1" 302 5 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:40 +0000] "GET /admin/admin/dashboard/ HTTP/1.1" 200 9246 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:40 +0000] "POST /rest/V1/integration/admin/token HTTP/1.1" 200 65 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:40 +0000] "GET /rest/V1/categories/list?searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=path&searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=1%2F2%2F%25&searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5BconditionType%5D=like&searchCriteria%5BfilterGroups%5D%5B1%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=level&searchCriteria%5BfilterGroups%5D%5B1%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=2&searchCriteria%5BpageSize%5D=100 HTTP/1.1" 200 536 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:40 +0000] "POST /rest/V1/integration/admin/token HTTP/1.1" 200 65 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:40 +0000] "GET /rest/default/V1/categories/list?searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=children_count&searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=0&searchCriteria%5BfilterGroups%5D%5B1%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=level&searchCriteria%5BfilterGroups%5D%5B1%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=2&searchCriteria%5BfilterGroups%5D%5B1%5D%5Bfilters%5D%5B0%5D%5BconditionType%5D=gt&searchCriteria%5BpageSize%5D=0 HTTP/1.1" 200 892 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:40 +0000] "POST /rest/V1/integration/admin/token HTTP/1.1" 200 65 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:40 +0000] "GET /rest/V1/products?searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=type_id&searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=configurable&searchCriteria%5BpageSize%5D=15 HTTP/1.1" 200 8107 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:41 +0000] "POST /rest/V1/integration/admin/token HTTP/1.1" 200 65 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:41 +0000] "GET /rest/V1/products?searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=type_id&searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=configurable&searchCriteria%5BpageSize%5D=15&searchCriteria%5BcurrentPage%5D=2 HTTP/1.1" 200 1271 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:41 +0000] "POST /rest/V1/integration/admin/token HTTP/1.1" 200 65 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:41 +0000] "GET /rest/V1/products?searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=type_id&searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=simple&searchCriteria%5BpageSize%5D=30&searchCriteria%5BcurrentPage%5D=1&searchCriteria%5BfilterGroups%5D%5B1%5D%5Bfilters%5D%5B1%5D%5Bfield%5D=attribute_set_id&searchCriteria%5BfilterGroups%5D%5B1%5D%5Bfilters%5D%5B1%5D%5Bvalue%5D=4 HTTP/1.1" 200 5550 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:41 +0000] "POST /rest/V1/integration/admin/token HTTP/1.1" 200 65 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:41 +0000] "GET /rest/V1/products?searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=type_id&searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=simple&searchCriteria%5BpageSize%5D=30&searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B1%5D%5Bfield%5D!=attribute_set_id&searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B1%5D%5Bvalue%5D=4 HTTP/1.1" 200 5528 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:41 +0000] "POST /admin/admin/user/roleGrid/limit/200/?ajax=true&isAjax=true HTTP/1.1" 200 3209 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:42 +0000] "GET /admin/customer/index/ HTTP/1.1" 200 7914 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:42 +0000] "GET /admin/mui/index/render/?namespace=customer_listing&sorting%5Bfield%5D=entity_id&sorting%5Bdirection%5D=asc&isAjax=true&filters%5Bplaceholder%5D=customer_since%5Blocale%5D%3Den_US&filters%5Bgroup_id%5D=1&filters%5Bwebsite_id%5D=1&paging%5BpageSize%5D=100&paging%5Bcurrent%5D=1&sorting%5Bfield%5D=entity_id&sorting%5Bdirection%5D=asc&isAjax=true HTTP/1.1" 200 1932 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:42 +0000] "GET /admin/directory/json/countryRegion/?parent=US HTTP/1.1" 200 969 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:43 +0000] "POST /checkout/cart/add HTTP/1.1" 302 5 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:43 +0000] "GET / HTTP/1.1" 200 7050 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
3.81.150.213 - - [10/Feb/2019:07:33:43 +0000] "GET /page_cache/block/render/?blocks=[\x22customer_form_login\x22]&handles=[\x22default\x22,\x22customer_account_login\x22]&originalRequest={\x22route\x22:\x22customer\x22,\x22controller\x22:\x22account\x22,\x22action\x22:\x22login\x22,\x22uri\x22:\x22/customer/account/login/\x22}&ajax=true&isAjax=true HTTP/1.1" 200 754 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" "-"
- JMeter log shows no errors jmeter.log jmeter-results.jtl
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 29 (6 by maintainers)
Hello, The problem is that with a loop of 1, and none of the ThroughputController having 100%, you can end up executing nothing. In the long run (more loops) the percentage will be matched.
This is expected behavior. If you want a guarantee of running use one of those 2 approaches:
@kandy , if I can help on the test plan let me know where to make a kind of review.
I had reported following issue but AFAICU ,nothing was taken into account:
https://github.com/magento/magento2/issues/723
@kandy I can confirm that the workaround of
-Jloops=2
does work!