scikit-learn: Getting HTTPError: HTTP Error 403: Forbidden when trying to load California Housing dataset
Describe the bug
When trying to load the dataset I get an error.
Steps/Code to Reproduce
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
housing = fetch_california_housing()
X_train_full, X_test, y_train_full, y_test = train_test_split(
housing.data, housing.target)
X_train, X_valid, y_train, y_valid = train_test_split(X_train_full, y_train_full)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_valid_scaled = scaler.transform(X_valid)
X_test_scaled = scaler.transform(X_test)
Expected Results
Dataset loads
Actual Results
HTTPError Traceback (most recent call last)
[/var/folders/wx/mz49j6yd5514yjn5k60sk6900000gn/T/ipykernel_16344/1379907178.py](https://file+.vscode-resource.vscode-cdn.net/var/folders/wx/mz49j6yd5514yjn5k60sk6900000gn/T/ipykernel_16344/1379907178.py) in <module>
3 from sklearn.preprocessing import StandardScaler
4
----> 5 housing = fetch_california_housing()
6
7 X_train_full, X_test, y_train_full, y_test = train_test_split(
[~/opt/anaconda3/lib/python3.9/site-packages/sklearn/datasets/_california_housing.py](https://file+.vscode-resource.vscode-cdn.net/Users/aryamanbhatia/neural%20network%20for%20practice/~/opt/anaconda3/lib/python3.9/site-packages/sklearn/datasets/_california_housing.py) in fetch_california_housing(data_home, download_if_missing, return_X_y, as_frame)
133 This dataset consists of 20,640 samples and 9 features.
134 """
--> 135 data_home = get_data_home(data_home=data_home)
136 if not exists(data_home):
137 makedirs(data_home)
[~/opt/anaconda3/lib/python3.9/site-packages/sklearn/datasets/_base.py](https://file+.vscode-resource.vscode-cdn.net/Users/aryamanbhatia/neural%20network%20for%20practice/~/opt/anaconda3/lib/python3.9/site-packages/sklearn/datasets/_base.py) in _fetch_remote(remote, dirname)
[~/opt/anaconda3/lib/python3.9/urllib/request.py](https://file+.vscode-resource.vscode-cdn.net/Users/aryamanbhatia/neural%20network%20for%20practice/~/opt/anaconda3/lib/python3.9/urllib/request.py) in urlretrieve(url, filename, reporthook, data)
237 url_type, path = _splittype(url)
238
--> 239 with contextlib.closing(urlopen(url, data)) as fp:
240 headers = fp.info()
241
...
--> 641 raise HTTPError(req.full_url, code, msg, hdrs, fp)
642
643 class HTTPRedirectHandler(BaseHandler):
HTTPError: HTTP Error 403: Forbidden
Versions
System:
python: 3.9.13 (main, Aug 25 2022, 18:29:29) [Clang 12.0.0 ]
executable: /Users/aryamanbhatia/opt/anaconda3/bin/python
machine: macOS-10.16-x86_64-i386-64bit
Python dependencies:
pip: 22.2.2
setuptools: 63.4.1
sklearn: 1.0.2
numpy: 1.26.3
scipy: 1.9.1
Cython: 0.29.32
pandas: 1.4.4
matplotlib: 3.5.2
joblib: 1.1.0
threadpoolctl: 2.2.0
Built with OpenMP: True
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 2
- Comments: 36 (21 by maintainers)
I cannot reproduce. Either it was something transient or it is on your network side.
Some kind of summary comment:
Indeed, it’s easy to disconnect the runtime (and then reconnect by reexecuting the cells from the start of the notebook) using the following drop down menu:
You have approximately a 50%/50% chance to get rebalanced to a node where the HTTP requests go through.
I got the same error on Google Colab too. Someone said this bug occurs very frequent. Sometime it works, sometime it doesn’t. Is there any solution to it?
@lesteve Great support - thank you!!!
I got an answer from Figshare support saying that they fixed the issue. I tried roughly 10 times on Colab and the same on Kaggle notebooks and I was not able to reproduce the issue 🎉.
I am going to close this one, if you encounter the issue again please comment in this issue!
You can also try the documented work-arounds in https://github.com/scikit-learn/scikit-learn/issues/28297#issuecomment-2178930890 and mention in case that does not fix the issue.
@lesteve Thanks for your comment. Yes, that’s exactly what I did. I just reported to possibly help troubleshoot this issue 😃
I tried on Kaggle notebooks and it seems like indeed the problem happens from time to time. The work-around is to click on the power button (Stop session), reexecute the code (you will get a different IP address) and cross your fingers 🤞, make sure you have internet enabled as well (just in case).
The debugging command I used is the following one, which downloads the file needed for California housing:
Here is a screenshot where it did not work (HTTP 403)
Here is a screenshot where it did work (HTTP 200):
I inquiry the support from figshare to see if they are aware of some limitations on their side.
Same here! (Based in France FYI, but using Colab)
I tried yo change the account and it worked. If you change the account that you are working with and try the same code it should be fine.