websocket-client: ssl.SSLEOFError: EOF occurred in violation of protocol
Hello,
I am getting an exception when subscribing to a high throughput websocket endpoint, wss://fstream.binance.com/stream?streams=btcusdt@bookTicker/btcusdt@depth20@100ms/btcusdt@aggTrade/btcusdt@markPrice@1s/btcusdt@forceOrder.
To be precise, it is the btcusdt@bookTicker that is very busy, and even if I switch off any computation completely and just run websocket-client and rel, eventually I run into the below exception.
File "...", line 179, in __init__
rel.dispatch()
File ".../python3.10/site-packages/rel/rel.py", line 228, in dispatch
registrar.dispatch()
File ".../python3.10/site-packages/rel/registrar.py", line 125, in dispatch
if not self.loop():
File ".../python3.10/site-packages/rel/registrar.py", line 134, in loop
e = self.check_events()
File ".../python3.10/site-packages/rel/registrar.py", line 285, in check_events
self.callback('read', fd)
File ".../python3.10/site-packages/rel/registrar.py", line 178, in callback
self.events[etype][fd].callback()
File ".../python3.10/site-packages/rel/listener.py", line 127, in callback
if not self.cb(*self.args) and not self.persist and self.active:
File ".../python3.10/site-packages/websocket/_app.py", line 437, in read
op_code, frame = self.sock.recv_data_frame(True)
File ".../python3.10/site-packages/websocket/_core.py", line 427, in recv_data_frame
self.pong(frame.data)
File ".../python3.10/site-packages/websocket/_core.py", line 351, in pong
self.send(payload, ABNF.OPCODE_PONG)
File ".../python3.10/site-packages/websocket/_core.py", line 283, in send
return self.send_frame(frame)
File ".../python3.10/site-packages/websocket/_core.py", line 311, in send_frame
l = self._send(data)
File ".../python3.10/site-packages/websocket/_core.py", line 525, in _send
return send(self.sock, data)
File ".../python3.10/site-packages/websocket/_socket.py", line 170, in send
return _send()
File ".../python3.10/site-packages/websocket/_socket.py", line 147, in _send
return sock.send(data)
File ".../python3.10/ssl.py", line 1206, in send
return self._sslobj.write(data)
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2426)
Do we expect with websocket-client and rel to cope with very busy websocket streams?
Can I stick rel.dispatch in some kind of a while loop, catch the exception and run rel.dispatch again?
My code looks like this:
for ws in self.ws_apps:
ws.close()
self.ws_apps = []
for socket in [self.user_data_socket, self.mkt_data_socket]:
ws = websocket.WebSocketApp(
socket, on_open=self.on_open, on_message=self.on_message,
on_error=self.on_error, on_close=self.on_close
)
self.ws_apps.append(ws)
# noinspection PyTypeChecker
ws.run_forever(
dispatcher=rel, skip_utf8_validation=True, reconnect=3,
ping_interval=180, ping_timeout=60
)
rel.dispatch()
thanks Gabriel
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 22
#983 fixed it yes, thank you !
@gborrageiro-ld @andrea-mucci @engn33r
I put together a websocket throughput test in dez, which is an async network stack built on rel. The new test is in the latest version, 0.10.10.1, which can be installed thusly:
To run the test, first run this:
That starts up a pretty basic websocket echo server (code) with a couple extra features.
Then, in a different terminal, run:
That (code) will prompt you to select a test configuration (high throughput in upstream/downstream/both/neither directions). It will then start up a WebSocketApp (with rel dispatcher) and proceed to direct heavy traffic in the specified direction. Anyway, it runs fine for me in every configuration.
Is anyone able to reproduce the issue using this test? Feel free to modify the code. Or is there a different websocket server that has this issue? For whatever reason, it doesn’t seem to be a problem with dez.
Anyway, this shouldn’t be hard to fix, if anyone can come up with a reliable repro - thoughts and test cases welcome!
I think SSL error might be a red herring. I think that rel crashes when any of the WS streams are very busy / return a lot of data quickly.
Dr Gabriel Borrageiro Senior Quantitative Portfolio Manager Laser Digital
From: engn33r @.> Sent: Sunday, September 10, 2023 10:42:14 PM To: websocket-client/websocket-client @.> Cc: Gabriel Borrageiro @.>; Mention @.> Subject: Re: [websocket-client/websocket-client] ssl.SSLEOFError: EOF occurred in violation of protocol (Issue #942)
Checking any search engine for “python EOF occurred in violation of protocol” shows this a common error message and often related to the SSL library. I suggest sharing the OS that you are using (Windows, Mac, nix) and the output of these commands NkdkJdXPPEBannerStart External Sender - From: (engn33r @.**>)https://spambrella.cloud-protect.net/email-details/?k=k1&payload=53616c7465645f5f08b5bebb288f6c82050ba47dc706c640491676361fdbfebe870235497ff17e2d625e0d1bb885d2bc84362ff4d68126208494c2b3f49994d73e127e014b7ec09005cfbfe7c9d0ce4f8954bf801efd12e74be0f9acf06ac56c1bd536557cec982a96910cacd6638226b31a3ad68c7bf734e01f60d1cbbed945168bee6641a3b4fb6380ccf34843dc31580a27d34afee617b32b4b06d4e181a9c3ff8d45c56745ccb3206d71863e05236e71ce12d0a890d3239b60d177c34d3200129fdf785c6fffabe9fa68cf74a6983c5ad65e3c7659f6376fd7a38170a6eb22eae7eb73098329242f6710334ce832 This message came from outside your organisation. Learn Morehttps://spambrella.cloud-protect.net/email-details/?k=k1&payload=53616c7465645f5f08b5bebb288f6c82050ba47dc706c640491676361fdbfebe870235497ff17e2d625e0d1bb885d2bc84362ff4d68126208494c2b3f49994d73e127e014b7ec09005cfbfe7c9d0ce4f8954bf801efd12e74be0f9acf06ac56c1bd536557cec982a96910cacd6638226b31a3ad68c7bf734e01f60d1cbbed945168bee6641a3b4fb6380ccf34843dc31580a27d34afee617b32b4b06d4e181a9c3ff8d45c56745ccb3206d71863e05236e71ce12d0a890d3239b60d177c34d3200129fdf785c6fffabe9fa68cf74a6983c5ad65e3c7659f6376fd7a38170a6eb22eae7eb73098329242f6710334ce832 Do not click any links, open attachments, or reply if you do not believe that the email is legitimate. If in doubt, report using the report email button NkdkJdXPPEBannerEnd
Checking any search engine for “python EOF occurred in violation of protocol” shows this a common error message and often related to the SSL library. I suggest sharing the OS that you are using (Windows, Mac, *nix) and the output of these commands when such an SSL error is encountered:
openssl version python3 -c “import ssl; print(ssl.OPENSSL_VERSION)” python3 -V
— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_websocket-2Dclient_websocket-2Dclient_issues_942-23issuecomment-2D1712908531&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=jHSk2w3iI_5gQOIDQ6L3stXnXofiJhZ43_UzHLOdT71DfGoMStAgy-CWLDRnfG0F&m=qeezOWT66tnJ_zU1nMrfOnRIR-Wx4UxLwCdf_CPE-uRGJe0TjGUJgQ23qKkfnBMS&s=puPJ7Iw8BC63gyzHP63bDkizLXunvr-wi0YV4_JEiKI&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_BBUWXOSPFLPDG2GGNXVSZRTXZYCYNANCNFSM6AAAAAA3N3IBRU&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=jHSk2w3iI_5gQOIDQ6L3stXnXofiJhZ43_UzHLOdT71DfGoMStAgy-CWLDRnfG0F&m=qeezOWT66tnJ_zU1nMrfOnRIR-Wx4UxLwCdf_CPE-uRGJe0TjGUJgQ23qKkfnBMS&s=fcKTaWE24qfVR_IblDmlezwbfIZrEj0ajnN1lqDAMVQ&e=. You are receiving this because you were mentioned.Message ID: @.***>
This email has been scanned for spam and viruses. Click herehttps://spambrella.cloud-protect.net/index01.php?mod_id=11&mod_option=logitem&report=1&type=easyspam&k=k1&payload=53616c7465645f5fcf3902304698b850c506a635de29d9eabe2be8aa3ca3141a76e3a475c4459967c81634555c7d345c7a116d8b267bd8b135d6f19f9b8a0b614d5df62b68fdb3021ec75525c0ad65083521332fa9facd72bc09ade37a74d6e0bb486c78af1f0ee9d4144dd69f753aab113fe318341f0e3c4ec1fc7df2d92b7d78508de6e548608fa274af9dc3821d181c9dfea7dca6ba2c7483390f5036a965971501bccd96745a275da8f00f16ccc7 to report this email as spam.
Disclaimer
The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.