locust: GRPC compatibility : Locust load test throws greenlet.GreenletExit exception on reaching test time limit

Describe the bug

Locust load test throws greenlet.GreenletExit exception on reaching test time limit while testing GRPC based services. To monkey patch GRPC with gevent I am using grpc’s:

import grpc.experimental.gevent as grpc_gevent
grpc_gevent.init_gevent()

but no luck.

Expected behavior

Should gracefully stop all greenlets and generate the necessary outputs.

Actual behavior

Test keeps on generating load, does not stop, has to be stopped via CTRL+C Keyboard Interrupt. Thrown exception:

[2021-01-18 13:40:47,715] C02CD0BNMD6N/INFO/locust.main: Run time limit set to 30 seconds
[2021-01-18 13:40:47,715] C02CD0BNMD6N/INFO/locust.main: Starting Locust 1.4.1
[2021-01-18 13:40:47,716] C02CD0BNMD6N/DEBUG/locust.runners: Updating state to 'spawning', old state was 'ready'
[2021-01-18 13:40:47,716] C02CD0BNMD6N/INFO/locust.runners: Spawning 5 users at the rate 3 users/s (0 users already running)...

 a bunch of related stats generated .......

[2021-01-18 13:41:17,717] C02CD0BNMD6N/INFO/locust.main: Time limit reached. Stopping Locust.
[2021-01-18 13:41:17,717] C02CD0BNMD6N/DEBUG/locust.runners: Stopping all users
[2021-01-18 13:41:17,718] C02CD0BNMD6N/DEBUG/locust.runners: Updating state to 'cleanup', old state was 'running'
[2021-01-18 13:41:17,718] C02CD0BNMD6N/INFO/locust.runners: Stopping 5 users
[2021-01-18 13:41:17,718] C02CD0BNMD6N/DEBUG/locust.runners: Stopping Greenlet-0
[2021-01-18 13:41:17,718] C02CD0BNMD6N/DEBUG/locust.runners: Stopping Greenlet-1
[2021-01-18 13:41:17,718] C02CD0BNMD6N/DEBUG/locust.runners: Stopping Greenlet-2
[2021-01-18 13:41:17,718] C02CD0BNMD6N/DEBUG/locust.runners: Stopping Greenlet-3
[2021-01-18 13:41:17,718] C02CD0BNMD6N/DEBUG/locust.runners: Stopping Greenlet-4

Traceback (most recent call last):
  File "src/gevent/event.py", line 159, in gevent._gevent_cevent.Event.wait
  File "src/gevent/_abstract_linkable.py", line 437, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait
  File "src/gevent/_abstract_linkable.py", line 403, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
  File "src/gevent/_abstract_linkable.py", line 406, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
  File "src/gevent/_abstract_linkable.py", line 358, in gevent._gevent_c_abstract_linkable.AbstractLinkable._AbstractLinkable__wait_to_be_notified
  File "src/gevent/_abstract_linkable.py", line 367, in gevent._gevent_c_abstract_linkable.AbstractLinkable._switch_to_hub
  File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
greenlet.GreenletExit
Exception ignored in: 'grpc._cython.cygrpc.run_loop'

Steps to reproduce

I am using locust to test bunch of GRPC services and tasks involve bunch of GRPC calls.

Environment

  • OS: MacOS Catalina 10.15.7
  • Python version: 3.8
  • Locust version: 1.4.1, grpcio version : 1.30.0
  • Locust command line that you ran: locust --host myhost -f locustfile --headless -u 5 -r 3 -t 30s MyLocustClass

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 1
  • Comments: 19 (1 by maintainers)

Most upvoted comments

Thanks @beandrad, it does work 👍

def on_stop(self, force=False): should be def stop(self, force=False):