pyodide: open_url not working?

I am starting a Firefox browser (version 100.0.2) on Ubuntu linux 22.04, using a new profile/

Navigating to https://pyodide.org/en/stable/console.html showing banner:

Welcome to the Pyodide terminal emulator 🐍
Python 3.10.2 (main, Apr  9 2022 20:52:01) on WebAssembly VM
Type "help", "copyright", "credits" or "license" for more information.

In the console I run and get an unexpected exception:

>>> import pyodide
>>> pyodide.http.open_url("https://www.w3.org/Protocols/HTTP/")
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/lib/python3.10/site-packages/pyodide/http.py", line 41, in open_url
    req.send(None)
JsException: NetworkError: A network error occurred.

Another attempt:

>>> url = "https://www.ofai.at/about"
>>> with open_url(url) as infp:
...     text = infp.read()
... 
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/lib/python3.10/site-packages/pyodide/http.py", line 41, in open_url
    req.send(None)
  File "https://pyodide.org/en/stable/console.html", line 1, in XMLHttpRequest.prototype.send<
JsException: NetworkError: A network error occurred.

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 1
Comments: 15 (8 by maintainers)

Most upvoted comments

On the contrary, even the rare URL which does work with pyodide open_url before, does produce a network error when prepending https://crossorigin.me/

Oh it seems like crossorigin.me service is currently dead… Probably you should find some other services or run your own proxy server.

so you could add req.setRequestHeader("Origin", "whatever):

Well I think this is not true. origin header is automatically set by the browser and the user cannot modify it.

ryanking13 on Jun 6, 2022

I guess this completely out of my control then

One possible option is to use the CORS proxy server (e.g. https://corsproxy.github.io/).

ryanking13 on Jun 6, 2022

I’m surprised you are getting so many of these errors. If I go to localhost:8000/console.html and use open_url("http://localhost:8000/console.html") it works fine…

hoodmane on Jun 12, 2022

As an afterthought, it may be useful to add information about this limitation to the FAQ and/or the open_url docstring to make it easier for people like me to not consider this a bug? 😃

johann-petrak on Jun 7, 2022

For your testing purposes you can run Chrome without CORS with --disable-web-security (more in mdns link from @rth). I tested your link https://www.ofai.at/about with pyodide.open_url and it works.

loosak on Jun 4, 2022