MusicBot: [Youtube Issue] Error when loading from YouTube (429/captcha form)
Describe the bug YouTube has recently starting enforcing ratelimits. This is likely because streaming the data directly harms their service (as they receive no ad revenue when videos are streamed in this way). When your IP is ratelimited, no data from YouTube can be loaded, and attempting to load anything from YouTube will result in an error.
Does this affect you?
- If you see
Invalid status code for video page response: 429in error messages, it means that your IP has been ratelimited. - If you see
Sorry for the interruption. We have been receiving a large volume of requests from your network.in error messages, it means that YouTube is is requesting a captcha for your requests.
Solution There is no solid solution to this at the moment. Your best options are:
- Turn off the bot for a few days (and don’t attempt to use anything that connects to YouTube from the same IP address)
- Host from a different IP address
Version Info This affects all connections to YouTube, regardless of which version of JMusicBot is used.
Additional Info Here’s the information that YouTube/Google provides when a music bot (or any service really) gets ratelimited:
Our systems have detected unusual traffic from your computer network. Please try your request again later. Why did this happen? This page appears when Google automatically detects requests coming from your computer network which appear to be in violation of the Terms of Service. The block will expire shortly after those requests stop. This traffic may have been sent by malicious software, a browser plug-in, or a script that sends automated requests. If you share your network connection, ask your administrator for help — a different computer using the same IP address may be responsible. Learn more Sometimes you may see this page if you are using advanced terms that robots are known to use, or sending requests very quickly.
About this issue
- Original URL
- State: open
- Created 5 years ago
- Comments: 32 (8 by maintainers)
Add the option for a local persistant music cache so music from youtube has to only be streamed (and saved) once. Should reduce the load of the autoplaylist and frequently requested tracks. Mind licenses.
This is true, I have been hit with a rate limit but my friend that works for google’s YouTube department said that if you fake a browser agent it will make it look like a normal person. This will not initiate a rate limit as soon or at all in some cases. Maybe something we should look into. I have connected my system to a squid proxy that fakes the browser agent and have not been hit since. The bot has been running on YouTube every day since May.
I downloaded my whole default playlist using YouTube-dl, working great. Super easy to do with https://github.com/MrS0m30n3/youtube-dl-gui because I’m not too experienced with terminal programs.
That is not related to this issue, that just means you aren’t using 64-bit java
I think that’s a bit out of the scope of a personal music bot. If anything, I think the solution to captchas would be to somehow forward them to the bot owner or the person trying to play the song, but even that could be difficult. Also, that would only solve a subset of the issues that people are facing with YouTube (it wouldn’t solve ratelimits or other video blockages).
@adamhradil those logs are not related to this issue; those logs are the result of the bot attempting to put the “now playing” info in a channel topic (which itself is the result of the bot having the Manage Channels permission in the channel in which you’ve used settc).
Is there a reason the bot can’t cache http requests? I set this up in python the other day and it was extremely easy.
Java solution (I haven’t looked into it extensively)
Obviously this would get pretty big, so you can make it optional, set a size limit, and allow configuring expiration periods, but it would help heavy bot users at least.
That’s an issue with discord not this plugin you can receive that using discordsrv and a number of other plugins that push data to discord.
A cache of songs is something I’ve thought about before (and will continue to think about), but there are several issues.
Lavaplayer (the internal library used for loading youtube and other sources) is strictly a streaming tool, and thus the songs that are streamed never have associated files. To cache music, all of the streamed packets would need to be collected, encoded, and saved to a valid format. This would technically be possible, but would break down when streaming live sources or when seeking within tracks. Therefore, the songs would need to be downloaded separate from the stream, which would mean potentially worsening the ratelimit issue (depending on when things were downloaded).
This would additionally require a lot of work to write such a downloading & encoding system, or adding new dependencies like python & youtube-dl to handle the downloading. This would also require a new searching mechanism to handle caching the results of searches (as even searching - not even playing - can affect ratelimiting).
Other minor issues include disk space and search results changing over time, both of which could potentially be solved by having a time- or disk-size-based expiry system for the cache. Licenses and copyrighted content would be another issue.
Overall, I think the easiest solution for the majority of people would be to just play more local files; use local files or mp3 urls in playlists, and keep other local files under simple and easy-to-access filenames. Adding play shortcuts (key-value pairs of keyword to song uri) might be a feature to assist with this as well.