Flask-SocketIO: JSON Data is serialized incorrectly.
Here’s a fun one.
Basically, the Python JSON library generates invalid JSON by default, by serializing NaN, infinity and -infinity as NaN, infinity and -infinity. JSON does not allow any of these symbols (see here).
The Socket.IO JSON deserialization appears to silently ignore any messages that fail to deserialize, such as messages containing any of the above (see the bug report for that).
Basically, wherever json.dumps(x) is being called in the codebase needs to be replaced with json.dumps(x, allow_nan=False).
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Reactions: 1
- Comments: 15 (4 by maintainers)
Wow! As a newcomer to Python, I cannot believe that anyone would even bother acquiescing to this bad choice by the Flask devs! As I understand it, Flask is used as a server to serve to clients, maybe /even/ remote clients who might operate on, y’know, global standards, rather than bending their code to fit somone’s unilateral decision that the JSON spec is not wide enough.
It’s not possible to abstain on the question of sticking to a standard or not. Flask-SocketIO is actively supporting this choice, which breaks frontends. (As an example, Firefox recognises Nan is JSON as a Syntax Error, so if it is “usually included in JSON payloads” then whoever is consuming those payloads has not managed to touch Firefox in the development process, which makes me suspect “usually” or even “common” are overstatements).
Flask devs should be pushed into compatibility with the JSON standard by default. The alternative is to try to push the entire rest of the web into allowing a de facto extension to the JSON standard, which I somehow don’t think will be successful. (And would just create headaches for the world’s devs in the future if it were. I cannot believe that developers of such a major library can be so irresponsible. If this were Javascript, Flask devs would be (rightly) flamed and a new library would already be taking Flask’s market share.
As for Flask-SocketIO, I guess you can ignore the problem if you want, but the more libraries acquiesce to this nonsense, the more people are angered and take an entrenched position before the war actually starts. Nip it the bud, I beg you!
I know that. The point is that by default, it’s broken.
I shouldn’t have to write a wrapper around the built-in
jsonjust to make it work with the remote client it’s designed to interoperate with.I’m not saying override the JSON encoder by default. I’m saying just pass
allow_nan=Falseby default.As it is, it fails entirely to work with socket.io in any case where you have a
nanorinfinityanywhere in your message data structure. The resulting JSON data just gets swallowed silently by socket.io.Yes, you should because Mongo uses BSON and extended JSON, not JSON.
You might have luck here: https://github.com/pallets/flask/pull/2832
Though the decision there too is left to the end user to implement as correct solution.
@morkeltry Your rant doesn’t belong here. It literally has nothing to do with flask-socketio. If you don’t like the default json module you can change it as I mentioned above. It would be weird, unexpected, and unpythonic to override the defaults just for this library.
@fake-name if you pass
allow_nan=Falsethe problem isn’t solved, you will just get aValueErrorexception, so basically you are moving the error from one place to another. The real problem is that your application is trying to send a JSON payload with values that aren’t supported by JSON.A better solution would be for you to walk your data and replace any unsupported values with
None, but this is application specific. It can be made into a custom JSON exporter.Alternatively, you can send the data as binary, then decode it with your custom code in the client.