vision: Downloading MNIST dataset with torchvision gives HTTP Error 403

@eduardo4jesus You could patch your model script at the top using:

from six.moves import urllib
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
urllib.request.install_opener(opener)

It will use that user agent for the entire script assuming the opener does not get overwritten somewhere else.

copy this snippet at the top of your notebook, run it, and then just load your datasets as usual…

from six.moves import urllib opener = urllib.request.build_opener() opener.addheaders = [('User-agent', 'Mozilla/5.0')] urllib.request.install_opener(opener)

I’ve just got the same problem. Waiting for the answer without changing codes… (ROOKIE ALERT)

Clone this to your working dir: https://github.com/knamdar/data

Did anyone got urllib.error.HTTPError: HTTP Error 503: Service Unavailable last night?

FYI we changed torchvision to try to fix this, and the fix should be present in the latest release (from yesterday), with a fix in https://github.com/pytorch/vision/pull/3499

Same on Colab:

import torchvision
import torchvision.transforms as transforms
root_dir = './data/MNIST/'
torchvision.datasets.MNIST(root=root_dir,download=True)

I get: HTTPError: HTTP Error 403: Forbidden

torchvision.__version__ -> 0.8.2+cu101

@BernardoOlisan @ChengguiSun the solution from @mlelarge works great, the following worked for me in a notebook

!wget www.di.ens.fr/~lelarge/MNIST.tar.gz
!tar -zxvf MNIST.tar.gz

from torchvision.datasets import MNIST
from torchvision import transforms

mnist_train = MNIST('./', download=False,
                    transform=transforms.Compose([
                        transforms.ToTensor(),
                    ]), train=True)

@eduardo4jesus You can explicitly add headers as stated above, something alike:

opener = urllib.request.URLopener()
opener.addheader('User-Agent', some_user_agent)
opener.retrieve(
    url, fpath,
    reporthook=gen_bar_updater()
)

(line 81 and onwards in vision/torchvision/datasets/utils.py). Seems to be a quick workaround that works.

Thank you ! @mlelarge and @alisterburt

You can download it from my webpage: https://gist.github.com/mlelarge/60ddefa9e16bc06f7f4fc7bff769bdb1

still happens

This should have been fixed now, there is no need to update torchvision.

All should be working as before, without any change on the user side.

This was fixed on the server hosting the original dataset (thanks @soumith !).

As such, I’m closing this issue but let us know if you still face this issue.

This should be fixed (again) in the next torchvision nightly, and the fix will be present in the next minor release of torchvision, which should be out soon.

See https://github.com/pytorch/vision/pull/3544 for more details

I think perhaps we should start a new issue (that links back to this one) as this one is closed and may get no attention. So far as I can see no one has made a new one yet, so I will try to create one now, referring back here.

Is there any way to have a quick fix without using the master? I am concerned about the potential changes I have to do in my code for going from the version I am using (1.4.0) and the master.

this is because the download links for mnist at https://github.com/pytorch/vision/blob/master/torchvision/datasets/mnist.py#L33-L36 are hosted on yann.lecun.com and that server has moved under CloudFlare protection.

@fmassa we need to maybe mirror and change the URLs to maybe the PyTorch S3 bucket or something

copy this snippet at the top of your notebook, run it, and then just load your datasets as usual…

from six.moves import urllib opener = urllib.request.build_opener() opener.addheaders = [('User-agent', 'Mozilla/5.0')] urllib.request.install_opener(opener)

Dude this is insane you save my day, thank you a lot bro!!

Thanks @kesterlester I did put MNIST on my webpage: wget www.di.ens.fr/~lelarge/MNIST.tar.gz if you need it!

so could we make a hot-fix somehow?

Thanks for reporting! I can reproduce the issue locally, and downloading from the browser works.

I don’t yet know what the root cause is though.