pytube: [BUG] Download speed getting slower and slower when downloading many videos

When downloading multiple videos continuously, the download speed will become slower and slower.

System information Please provide the following information:

Python version (run python --version) Python 3.8.0
Pytube version (run print(pytube.__version__) in python) 10.8.5
Command used to install pytube python3.8 -m pip install pytube

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 2
Comments: 51

Most upvoted comments

Slow progress today; two more javascript functions left to emulate, then I can try to put together the last piece of this and hopefully get it working

tfdahlin on Jun 30, 2021

Finished writing the code that will calculate the new value of n. Feel free to test out the experimental branch here, and let me know if it fixes the slowdown problem for you. One thing that I’m unclear about is whether or not we will need to disable caching the base.js file.

tfdahlin on Jul 3, 2021

I’m about halfway done w/ the next step. Have to stop here for now

tfdahlin on Jun 29, 2021

The transform plan seems to be successfully extracting. Next step is to create the array of values + functions so that the plan can be applied.

tfdahlin on Jun 29, 2021

For documentation purposes, an example of what this function looks like (after adding indentation to make it more readable):

Function for computing n

cha=function(a){
    var b=a.split(""),
        c=[
            142389785,
            -1062889771,
            function(d,e){
                for(
                    e=(e%d.length+d.length)%d.length;
                    e--;
                    
                )
                    d.unshift(d.pop())
            },
            b,
            1966130824,
            -388933535,
            952987009,
            -152323794,
            -646774959,
            -1835069559,
            null,
            -1891128459,
            2069343819,
            264462757,
            null,
            -1827754172,
            function(d,e){
                e=(e%d.length+d.length)%d.length;
                d.splice(
                    0,
                    1,
                    d.splice(
                        e,
                        1,
                        d[0]
                    )[0]
                )
            },
            null,
            b,
            function(d){
                d.reverse()
            },
            1468427690,
            b,
            "pop",
            416624305,
            86884804,
            function(d,e){
                d.push(e)
            },
            1815184233,
            1319603020,
            function(d,e){
                for(
                    var f=64,h=[];
                    ++f-h.length-32;
                    
                ){
                    switch(f){
                        case 58:
                            f-=14;
                        case 91:
                        case 92:
                        case 93:
                            continue;
                        case 123:
                            f=47;
                        case 94:
                        case 95:
                        case 96:
                            continue;
                        case 46:
                            f=95
                    }
                    h.push(String.fromCharCode(f))
                }
                d.forEach(
                    function(l,m,n){
                        this.push(
                            n[m]=h[
                                (h.indexOf(l)-h.indexOf(this[m])+m-32+f--)%h.length
                            ]
                        )
                    },
                    e.split("")
                )
            },
            1823083268,
            1134033916,
            -646774959,
            -2126127395,
            function(d,e){
                e=(e%d.length+d.length)%d.length;
                d.splice(-e).reverse().forEach(
                    function(f){
                        d.unshift(f)
                    }
                )
            },
            1845050767,
            1965266158,
            1132026013,
            -294807870,
            -508492140,
            -2000244778,
            52788620,
            2012640407,
            function(d,e){
                for(
                    e=(e%d.length+d.length)%d.length;
                    e--;
                    
                )
                    d.unshift(d.pop())
            },
            -2021094533,
            1012922155,
            -1870404083,
            -161415485,
            1496545176,
            function(d,e){
                e=(e%d.length+d.length)%d.length;
                var f=d[0];
                d[0]=d[e];
                d[e]=f
            },
            1247337884,
            1584781061,
            function(d){
                for(
                    var e=d.length;
                    e;
                )
                    d.push(d.splice(--e,1)[0])
            },
            416358396,
            -410621770,
            function(d,e){
                e=(e%d.length+d.length)%d.length;
                d.splice(e,1)
            }
        ];
        c[10]=c;
        c[14]=c;
        c[17]=c;
        try{
            c[16](c[3],c[24]),
            c[16](c[18],c[35]),
            c[16](c[21],c[43]),
            c[16](c[14],c[7]),
            c[54](c[3],c[8]),
            c[16](c[18],c[53]),
            c[16](c[3],c[5]),
            c[33](c[3],c[0]),
            c[33](c[17],c[39]),
            c[40](c[21],c[22]),
            c[53](c[8],c[41]),
            c[38](c[22],c[28]),
            c[27](c[42],c[37]),
            c[22](c[50],c[15]),
            c[54](c[46],c[14]),
            c[28](c[32],c[12]),
            c[22](c[46],c[23]),
            c[22](c[39],c[42]),
            c[2](c[50],c[51]),
            c[28](c[47],c[35]),
            c[28](c[47],c[44]),
            c[31](c[50],c[26]),
            c[45](c[50],c[3]),
            c[22](c[39],c[24]),
            c[28](c[47],c[49]),
            c[16](c[46],c[11]),
            c[42](c[25],c[43]),
            c[14](c[0],c[5]),
            c[2](c[25],c[26]),
            c[55](c[53],c[38]),
            c[46](c[53],c[3]),
            c[11](c[30],c[4]),
            c[8](c[22]),
            c[24](c[26]),
            c[39](c[26],c[1]),
            c[46](c[29],c[16]),
            c[55](c[26],c[13]),
            c[43](c[55],c[50]),
            c[13](c[43],c[1]),
            c[49](c[23],c[39]),
            c[29](c[41],c[6])
        }
        catch(d){
            return"enhanced_except_8pIBje3-_w8_"+a
        }
    return b.join("")
}

tfdahlin on Jun 29, 2021

Current progress: I have the functionality to find and extract the relevant function for computing n implemented. I have not yet implemented the functionality for performing the relevant operations on n. That will be the hard part.

tfdahlin on Jun 28, 2021

@tfdahlin I am using the latest build 10.9.1. It except by

except IncompleteRead:
        logger.error("IncompleteRead: %s", url)

2021-07-07 01:56:26,836 - tid 140216156170048 - ERROR: IncompleteRead: https://www.youtube.com/watch?v=yTl0qi70sMU

joon612 on Jul 7, 2021

Ok, thank you very much!

From: Taylor Fox Dahlin @.> Sent: Tuesday, July 6, 2021 9:02:32 AM To: pytube/pytube @.> Cc: Joon612 @.>; Mention @.> Subject: Re: [pytube/pytube] [BUG] Download speed getting slower and slower when downloading many videos (#1033)

Sometimes it takes a while for that icon to update, it’s showing up correctly for me

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/pytube/pytube/issues/1033#issuecomment-874389514, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIGM4R3H2K2FGEGASSPTENDTWJI2RANCNFSM47JKZYRA.

joon612 on Jul 6, 2021

https://github.com/ytdl-org/youtube-dl/issues/29326 looks like this is a new youtube issue, with a possible solution, but it sounds like it might be a bit hard to write. I’ll try to investigate and implement it this week if I can, just might take me a while

tfdahlin on Jun 28, 2021