websocket-client: ssl.SSLEOFError: EOF occurred in violation of protocol

Hello,

I am getting an exception when subscribing to a high throughput websocket endpoint, wss://fstream.binance.com/stream?streams=btcusdt@bookTicker/btcusdt@depth20@100ms/btcusdt@aggTrade/btcusdt@markPrice@1s/btcusdt@forceOrder. To be precise, it is the btcusdt@bookTicker that is very busy, and even if I switch off any computation completely and just run websocket-client and rel, eventually I run into the below exception.

File "...", line 179, in __init__
    rel.dispatch()
  File ".../python3.10/site-packages/rel/rel.py", line 228, in dispatch
    registrar.dispatch()
  File ".../python3.10/site-packages/rel/registrar.py", line 125, in dispatch
    if not self.loop():
  File ".../python3.10/site-packages/rel/registrar.py", line 134, in loop
    e = self.check_events()
  File ".../python3.10/site-packages/rel/registrar.py", line 285, in check_events
    self.callback('read', fd)
  File ".../python3.10/site-packages/rel/registrar.py", line 178, in callback
    self.events[etype][fd].callback()
  File ".../python3.10/site-packages/rel/listener.py", line 127, in callback
    if not self.cb(*self.args) and not self.persist and self.active:
  File ".../python3.10/site-packages/websocket/_app.py", line 437, in read
    op_code, frame = self.sock.recv_data_frame(True)
  File ".../python3.10/site-packages/websocket/_core.py", line 427, in recv_data_frame
    self.pong(frame.data)
  File ".../python3.10/site-packages/websocket/_core.py", line 351, in pong
    self.send(payload, ABNF.OPCODE_PONG)
  File ".../python3.10/site-packages/websocket/_core.py", line 283, in send
    return self.send_frame(frame)
  File ".../python3.10/site-packages/websocket/_core.py", line 311, in send_frame
    l = self._send(data)
  File ".../python3.10/site-packages/websocket/_core.py", line 525, in _send
    return send(self.sock, data)
  File ".../python3.10/site-packages/websocket/_socket.py", line 170, in send
    return _send()
  File ".../python3.10/site-packages/websocket/_socket.py", line 147, in _send
    return sock.send(data)
  File ".../python3.10/ssl.py", line 1206, in send
    return self._sslobj.write(data)
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2426)

Do we expect with websocket-client and rel to cope with very busy websocket streams?

Can I stick rel.dispatch in some kind of a while loop, catch the exception and run rel.dispatch again?

My code looks like this:

for ws in self.ws_apps:
    ws.close()
self.ws_apps = []
for socket in [self.user_data_socket, self.mkt_data_socket]:
    ws = websocket.WebSocketApp(
        socket, on_open=self.on_open, on_message=self.on_message,
        on_error=self.on_error, on_close=self.on_close
    )
    self.ws_apps.append(ws)
    # noinspection PyTypeChecker
    ws.run_forever(
        dispatcher=rel, skip_utf8_validation=True, reconnect=3,
        ping_interval=180, ping_timeout=60
    )
rel.dispatch()

thanks Gabriel

About this issue

Most upvoted comments

#983 fixed it yes, thank you !

@gborrageiro-ld @andrea-mucci @engn33r

I put together a websocket throughput test in dez, which is an async network stack built on rel. The new test is in the latest version, 0.10.10.1, which can be installed thusly:

pip install dez

To run the test, first run this:

dez_test wsecho

That starts up a pretty basic websocket echo server (code) with a couple extra features.

Then, in a different terminal, run:

dez_test wsc

That (code) will prompt you to select a test configuration (high throughput in upstream/downstream/both/neither directions). It will then start up a WebSocketApp (with rel dispatcher) and proceed to direct heavy traffic in the specified direction. Anyway, it runs fine for me in every configuration.

Is anyone able to reproduce the issue using this test? Feel free to modify the code. Or is there a different websocket server that has this issue? For whatever reason, it doesn’t seem to be a problem with dez.

Anyway, this shouldn’t be hard to fix, if anyone can come up with a reliable repro - thoughts and test cases welcome!

I think SSL error might be a red herring. I think that rel crashes when any of the WS streams are very busy / return a lot of data quickly.

Dr Gabriel Borrageiro Senior Quantitative Portfolio Manager Laser Digital


From: engn33r @.> Sent: Sunday, September 10, 2023 10:42:14 PM To: websocket-client/websocket-client @.> Cc: Gabriel Borrageiro @.>; Mention @.> Subject: Re: [websocket-client/websocket-client] ssl.SSLEOFError: EOF occurred in violation of protocol (Issue #942)

Checking any search engine for “python EOF occurred in violation of protocol” shows this a common error message and often related to the SSL library. I suggest sharing the OS that you are using (Windows, Mac, nix) and the output of these commands NkdkJdXPPEBannerStart External Sender - From: (engn33r @.**>)https://spambrella.cloud-protect.net/email-details/?k=k1&payload=53616c7465645f5f08b5bebb288f6c82050ba47dc706c640491676361fdbfebe870235497ff17e2d625e0d1bb885d2bc84362ff4d68126208494c2b3f49994d73e127e014b7ec09005cfbfe7c9d0ce4f8954bf801efd12e74be0f9acf06ac56c1bd536557cec982a96910cacd6638226b31a3ad68c7bf734e01f60d1cbbed945168bee6641a3b4fb6380ccf34843dc31580a27d34afee617b32b4b06d4e181a9c3ff8d45c56745ccb3206d71863e05236e71ce12d0a890d3239b60d177c34d3200129fdf785c6fffabe9fa68cf74a6983c5ad65e3c7659f6376fd7a38170a6eb22eae7eb73098329242f6710334ce832 This message came from outside your organisation. Learn Morehttps://spambrella.cloud-protect.net/email-details/?k=k1&payload=53616c7465645f5f08b5bebb288f6c82050ba47dc706c640491676361fdbfebe870235497ff17e2d625e0d1bb885d2bc84362ff4d68126208494c2b3f49994d73e127e014b7ec09005cfbfe7c9d0ce4f8954bf801efd12e74be0f9acf06ac56c1bd536557cec982a96910cacd6638226b31a3ad68c7bf734e01f60d1cbbed945168bee6641a3b4fb6380ccf34843dc31580a27d34afee617b32b4b06d4e181a9c3ff8d45c56745ccb3206d71863e05236e71ce12d0a890d3239b60d177c34d3200129fdf785c6fffabe9fa68cf74a6983c5ad65e3c7659f6376fd7a38170a6eb22eae7eb73098329242f6710334ce832 Do not click any links, open attachments, or reply if you do not believe that the email is legitimate. If in doubt, report using the report email button NkdkJdXPPEBannerEnd

Checking any search engine for “python EOF occurred in violation of protocol” shows this a common error message and often related to the SSL library. I suggest sharing the OS that you are using (Windows, Mac, *nix) and the output of these commands when such an SSL error is encountered:

openssl version python3 -c “import ssl; print(ssl.OPENSSL_VERSION)” python3 -V

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_websocket-2Dclient_websocket-2Dclient_issues_942-23issuecomment-2D1712908531&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=jHSk2w3iI_5gQOIDQ6L3stXnXofiJhZ43_UzHLOdT71DfGoMStAgy-CWLDRnfG0F&m=qeezOWT66tnJ_zU1nMrfOnRIR-Wx4UxLwCdf_CPE-uRGJe0TjGUJgQ23qKkfnBMS&s=puPJ7Iw8BC63gyzHP63bDkizLXunvr-wi0YV4_JEiKI&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_BBUWXOSPFLPDG2GGNXVSZRTXZYCYNANCNFSM6AAAAAA3N3IBRU&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=jHSk2w3iI_5gQOIDQ6L3stXnXofiJhZ43_UzHLOdT71DfGoMStAgy-CWLDRnfG0F&m=qeezOWT66tnJ_zU1nMrfOnRIR-Wx4UxLwCdf_CPE-uRGJe0TjGUJgQ23qKkfnBMS&s=fcKTaWE24qfVR_IblDmlezwbfIZrEj0ajnN1lqDAMVQ&e=. You are receiving this because you were mentioned.Message ID: @.***>


This email has been scanned for spam and viruses. Click herehttps://spambrella.cloud-protect.net/index01.php?mod_id=11&mod_option=logitem&report=1&type=easyspam&k=k1&payload=53616c7465645f5fcf3902304698b850c506a635de29d9eabe2be8aa3ca3141a76e3a475c4459967c81634555c7d345c7a116d8b267bd8b135d6f19f9b8a0b614d5df62b68fdb3021ec75525c0ad65083521332fa9facd72bc09ade37a74d6e0bb486c78af1f0ee9d4144dd69f753aab113fe318341f0e3c4ec1fc7df2d92b7d78508de6e548608fa274af9dc3821d181c9dfea7dca6ba2c7483390f5036a965971501bccd96745a275da8f00f16ccc7 to report this email as spam.

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.