YoutubeExplode: Playlist response is broken (search & normal playlists)

Might be related to this one, but since this afternoon I can’t seem to fetch any playlists at all. In my code, it crashes right here;

LockControls(); var client = new YoutubeClient(); var playlist = await client.Playlists.GetVideosAsync(SUBSCRIPTION);

This bit’s from the subscriptions function I have, but the same rings true for importing a playlist on its own.

Looks like they removed that endpoint altogether.

If it helps at all, I have updated our codebase (which has some parts YouTubeExplode code in it) to work without this endpoint.

Inspired by @YazeedAlKhalaf we ended up using the following method:

• get INNERTUBE_API_KEY from sw.js
• load channel page HTML and grab the video browse endpoint params from ytInitialData
• load paginated videos from youtubei/v1/browse

https://github.com/markledwich2/Recfluence/blob/master/App/YtReader/YtWebsite/YtWeb.cs#L191

It gives us quite a bit less info than before and uses more data 😢. We used to load most videos for all 7K+ channels every day, so we have to get a bit more stingy with this change, but it works. Check out https://transparency.tube/ if you are interested in what we are doing with it.

Yes, looks like playlists are down too. They seem to have removed the endpoint we were using. Embedded player now retrieves playlist videos using a different endpoint. Example: https://www.youtube.com/youtubei/v1/embedded_player?key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8 (not sure how the key is generated). The data it returns is exactly what is shown on the screen which is less than what Video encapsulates currently.

I found some info about reverse engineering the internal Youtube API and decoding the protobuf data here. The instructions are for YouTube Music and the code is written Objective-C, but the process for YouTube should be similar.

This is interesting. I poked around, but still can’t find what kind of data it returns. If it’s less or around the same as the raw HTML approach you found in #502 then it’s probably easier to just rely on HTML parsing.

It seems we’d have drop a bunch of data from playlist videos, separate them in its own class (i.e. Video -> PlaylistVideo) and release a new major version because this is a breaking change. Among the fields that are missing, I think only upload date is important, but it’s better to not have it at all rather than an approximation (e.g. “6 months ago”).

The key is INNERTUBE_API_KEY, below is more explanation The key is not changing now but you can get it from this URL: https://www.youtube.com/sw.js_data Simple GET request and no need for any edits. there is also https://www.youtube.com/sw.js which I found out about through trial and error. it made me know that the key is INNERTUBE_API_KEY

https://github.com/Tyrrrz/YoutubeExplode/blob/master/YoutubeExplode/ReverseEngineering/Responses/PlaylistResponse.cs#L145

here is the problem, this URL does not work anymore, I mean the one mentioned in the file above

@Montegro

So it seems the full date is still available as an endpoint attached to the videos themselves, does the playlist parse result in a different video listing missing this endpoint?

Yes. The only currently available replacement features date in a form of “3 days ago”.

Are you incorporating Mark’s update to code into an update from Nuget?

Yes, looks like playlists are down too. They seem to have removed the endpoint we were using. Embedded player now retrieves playlist videos using a different endpoint. Example: https://www.youtube.com/youtubei/v1/embedded_player?key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8 (not sure how the key is generated). The data it returns is exactly what is shown on the screen which is less than what Video encapsulates currently.

I found some info about reverse engineering the internal Youtube API and decoding the protobuf data here.

The instructions are for YouTube Music and the code is written Objective-C, but the process for YouTube should be similar.

This is interesting. I poked around, but still can’t find what kind of data it returns. If it’s less or around the same as the raw HTML approach you found in #502 then it’s probably easier to just rely on HTML parsing.

It seems we’d have drop a bunch of data from playlist videos, separate them in its own class (i.e. Video -> PlaylistVideo) and release a new major version because this is a breaking change. Among the fields that are missing, I think only upload date is important, but it’s better to not have it at all rather than an approximation (e.g. “6 months ago”).