streamlink: How to avoid "Input is not padded or padding is corrupt" error?

Bug Report

This is a bug report and I have read the contribution guidelines.

Description

Sometimes, on streams that involve some kind of decryption, I get a streamlink.exceptions.StreamError: Input is not padded or padding is corrupt, got padding size of xxx

Whenever this happens, streamlink completely stalls until it reaches the timeout value, and then it just quits. This is very frustrating as this is not reproducible, and I have no idea how it is going to happen or when it is going to happen. In some cases, I can be recording the same exact same stream in two different commands running at the same time, and one will quit and the other will keep recording. In the case of downloading a replay of a stream, I can just restart the download and streamlink always gets past the point where it crashed previously with no issue.

What I would like to know is how I can avoid this error. For example if the error comes up, how can I force streamlink to try redownload/redecrypt the segment?

Expected / Actual behavior

Streamlink downloads the stream with no issues.

Reproduction steps / Explicit stream URLs to test

NA. The stream needs to have some sort of encryption. I have received this error across Fox Sports Go streams, ESPN+ streams, and live streams of Turner websites (TNT/TBS).

Log output

streamlink 'hlsvariant://https://turnerlive.akamaized.net/hls/live/572100/true/aes_slate/de/master.m3u8?hdnts=exp=1553034983~acl=/hls/live/572100/true/*!/hls/live/572100-b/true/*!/hls/live/630682/true/*!/hls/live/630682-b/true/*~id=c5c9ce531e437434c07dc7ff65b0ab94~hmac=112e16086c033e16ffcb3b487ff24f3f4f6626ccc2c57f0be36a60f30d269aee' best -o trutv.ts
[cli][info] Found matching plugin hls for URL hlsvariant://https://turnerlive.akamaized.net/hls/live/572100/true/aes_slate/de/master.m3u8?hdnts=exp=1553034983~acl=/hls/live/572100/true/*!/hls/live/572100-b/true/*!/hls/live/630682/true/*!/hls/live/630682-b/true/*~id=c5c9ce531e437434c07dc7ff65b0ab94~hmac=112e16086c033e16ffcb3b487ff24f3f4f6626ccc2c57f0be36a60f30d269aee
[cli][info] Available streams: 360p_alt2 (worst), 360p_alt, 360p, 480p_alt, 480p, 540p_alt, 540p, 720p_alt, 720p (best)
[cli][info] Opening stream: 720p (hls)                                                                                                                                              [download][trutv.ts] Written 763.8 MB (28m49s @ 424.4 KB/s)                                                                                                                                               Exception in thread Thread-HLSStreamWriter:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/streamlink/stream/segmented.py", line 167, in run
    self.write(segment, result)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/streamlink/stream/hls.py", line 140, in write
    self.reader.buffer.write(pkcs7_decode(decrypted_chunk))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/streamlink/stream/hls.py", line 31, in pkcs7_decode
    raise StreamError("Input is not padded or padding is corrupt, got padding size of {0}".format(val))
streamlink.exceptions.StreamError: Input is not padded or padding is corrupt, got padding size of 156

error: Error when reading from stream: Read timeout, exiting
[cli][info] Stream ended
[cli][info] Closing currently open stream...

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 27 (7 by maintainers)

Most upvoted comments

Let’s reopen for visibility. However, it is a bug/issue in Streamlink’s dependencies, and unfortunately there’s no clean way of fixing this here.

https://github.com/psf/requests/issues/4956#issuecomment-573325001

The fix I think urllib3 should just make enforce_content_length=True by default. … If urllib3 makes that change, then Requests can simply bump their minimum required urllib3 version to whatever release includes the change, and be done. Failing that, Requests should pass enforce_content_length=True explicitly.

Since requests.adapters.HTTPAdapter.send doesn’t allow for custom parameters to be set for urllib3.response.HTTPResponse, this means all that could be done in Streamlink is patching out request’s import of urllib3’s HTTPResponse. This needs to be done in streamlink.plugin.api.http_session. Since I haven’t run into this issue yet, I can’t test this myself.

bastimeyer on May 31, 2021

This has apparently been reported in psf/requests#4956 years ago but there is no developer response in that issue.

maxb on Feb 28, 2021

I think I may have stumbled across the cause of this (or one of them, anyway): it seems that Python’s requests/urllib3 libraries do not signal any kind of error if the server closes the connection mid-way through a file (i.e. it does not verify that it received as many bytes as it should have, using the Content-Length header).

The padding is corrupt because it’s not padding, it’s arbitrary data where the download was truncated.

urllib3 does have an optional keyword argument enforce_content_length=True that can be specified. However, I can’t see any way in requests to get that passed on to urllib3.

maxb on Feb 27, 2021

Would this solution just remove the offending segment from the stream

Yes, it would remove these segments and would thus get rid of the error, but that’s not the solution we’re looking for here.

ValueError: Padding is incorrect.

https://github.com/Legrandin/pycryptodome/blob/v3.9.9/lib/Crypto/Util/Padding.py#L87-L90

bastimeyer on Jan 22, 2021