spotify-downloader: HTTP Error 429: Too Many Requests

What is the purpose of your issue?

  • Bug
  • Feature Request
  • Question
  • Other

Description

Rate limit will occure while fetching lyrics(shown in the log), while youtube-dl is working with same IP. Is there any solution except changing IP address, e.g, make lyrics optional and skippable

Log

spotdl -s https://open.spotify.com/track/25AHHw85uL4TI6ZDKkmG15?si=p1nFwd32TFaPpmsUwuhXOg -f . --log-level=DEBUG
DEBUG: Input song is a Spotify URL
DEBUG: Fetching metadata for given track URL
DEBUG: Fetching lyrics
DEBUG: {'album': {'album_type': 'album',
               'artists': [{'external_urls': {'spotify': 'https://open.spotify.com/artist/2w9zwq3AktTeYYMuhMjju8'},
                            'href': 'https://api.spotify.com/v1/artists/2w9zwq3AktTeYYMuhMjju8',
                            'id': '2w9zwq3AktTeYYMuhMjju8',
                            'name': 'INNA',
                            'type': 'artist',
                            'uri': 'spotify:artist:2w9zwq3AktTeYYMuhMjju8'}],
               'external_urls': {'spotify': 'https://open.spotify.com/album/1zOyANSJozlJ21CPIwIXzq'},
               'href': 'https://api.spotify.com/v1/albums/1zOyANSJozlJ21CPIwIXzq',
               'id': '1zOyANSJozlJ21CPIwIXzq',
               'images': [{'height': 640,
                           'url': 'https://i.scdn.co/image/8b7de8621a8c8667a838cebdf0af7aad10279aff',
                           'width': 640},
                          {'height': 300,
                           'url': 'https://i.scdn.co/image/c662b63f4585410a33ab35aa7d1db8e1fae1a099',
                           'width': 300},
                          {'height': 64,
                           'url': 'https://i.scdn.co/image/8de9278fec0bf1b62452206bffd33a9b4da4a816',
                           'width': 64}],
               'name': 'Nirvana',
               'release_date': '2017-12-11',
               'release_date_precision': 'day',
               'total_tracks': 12,
               'type': 'album',
               'uri': 'spotify:album:1zOyANSJozlJ21CPIwIXzq'},
     'artists': [{'external_urls': {'spotify': 'https://open.spotify.com/artist/2w9zwq3AktTeYYMuhMjju8'},
                  'href': 'https://api.spotify.com/v1/artists/2w9zwq3AktTeYYMuhMjju8',
                  'id': '2w9zwq3AktTeYYMuhMjju8',
                  'name': 'INNA',
                  'type': 'artist',
                  'uri': 'spotify:artist:2w9zwq3AktTeYYMuhMjju8'}],
     'copyright': 'Global Records',
     'disc_number': 1,
     'duration': 194.886,
     'explicit': False,
     'external_ids': {'isrc': 'FRX201795585'},
     'external_urls': {'spotify': 'https://open.spotify.com/track/25AHHw85uL4TI6ZDKkmG15'},
     'genre': 'Dance Pop',
     'href': 'https://api.spotify.com/v1/tracks/25AHHw85uL4TI6ZDKkmG15',
     'id': '25AHHw85uL4TI6ZDKkmG15',
     'is_local': False,
     'lyrics': None,
     'name': 'Tropical',
     'popularity': 38,
     'preview_url': 'https://p.scdn.co/mp3-preview/b1b4996e3691a8bf076ae8cfb4a60633cd692f3b?cid=4fe3fecfe5334023a1472516cc99d805',
     'publisher': 'Global Records',
     'release_date': '2017-12-11',
     'spotify_metadata': True,
     'total_tracks': 12,
     'track_number': 4,
     'type': 'track',
     'uri': 'spotify:track:25AHHw85uL4TI6ZDKkmG15',
     'year': '2017'}
DEBUG: Opening URL: https://www.youtube.com/results?sp=EgIQAQ%253D%253D&q=INNA%20-%20Tropical%20lyrics
Traceback (most recent call last):
  File "/usr/local/bin/spotdl", line 11, in <module>
    load_entry_point('spotdl', 'console_scripts', 'spotdl')()
  File "/opt/spotify-downloader/spotdl/spotdl.py", line 63, in main
    match_args()
  File "/opt/spotify-downloader/spotdl/spotdl.py", line 27, in match_args
    track_dl = downloader.Downloader(raw_song=track)
  File "/opt/spotify-downloader/spotdl/downloader.py", line 98, in __init__
    self.content, self.meta_tags = youtube_tools.match_video_and_metadata(raw_song)
  File "/opt/spotify-downloader/spotdl/youtube_tools.py", line 80, in match_video_and_metadata
    content = go_pafy(track, meta_tags)
  File "/opt/spotify-downloader/spotdl/youtube_tools.py", line 41, in go_pafy
    track_url = generate_youtube_url(raw_song, meta_tags)
  File "/opt/spotify-downloader/spotdl/youtube_tools.py", line 217, in generate_youtube_url
    url = url_fetch.scrape()
  File "/opt/spotify-downloader/spotdl/youtube_tools.py", line 311, in scrape
    item = self._fetch_response(search_url).read()
  File "/opt/spotify-downloader/spotdl/youtube_tools.py", line 397, in _fetch_response
    return urllib.request.urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 504, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 429: Too Many Requests

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 17 (6 by maintainers)

Most upvoted comments

downloading more than 1k tracks per day

Ritiek is a bit busy, he might take a while to reply (just giving you the heads up)

@linusg and also with 15 Failure Attempts πŸ˜‚ really why.

queue
      .create("download", {
        link: req.body.link,
        requestId: req.body.requestId,
        type: "track"
      })
      .priority("high")
      .attempts(15)
      .backoff({ type: "exponential" })
      .save(err => {
        if (!err) res.json(job);
      });

honestly i’m trying to clone Spotify db by daily crawling, and at the end ( if something bad doesn’t happen 😐 ) make an open source music-api to resolve spotify links faster than youtube fetching, converting, e.t.c. πŸ˜„ you know. something to help community and tools to be more robust πŸ˜„