pyodide: Failed to import skimage in a web-worker

After closing https://github.com/iodide-project/pyodide/issues/253 with https://github.com/iodide-project/pyodide/pull/866, we should be able to use scikit-image, however, when I try to import it (from cdn.jsdelivr.net/pyodide/dev/full/) it fails.

Specifically, I can use micropip to install the pacakge:

Loaded pytest, setuptools, pyparsing, py, pluggy, more-itertools, attrs, atomicwrites, scikit-image, pywavelets, scipy, numpy, matplotlib, pytz, python-dateutil, pillow, kiwisolver, cycler, networkx, decorator

But importing the module in Python results the following error:

pyodide.asm.js:8 Uncaught (in promise) Error: Traceback (most recent call last):
  File "<exec>", line 5, in try_import_skimage
  File "/lib/python3.8/site-packages/skimage/__init__.py", line 127, in <module>
    from .util.dtype import (img_as_float32,
  File "/lib/python3.8/site-packages/skimage/util/__init__.py", line 1, in <module>
    from .dtype import (img_as_float32, img_as_float64, img_as_float,
  File "/lib/python3.8/site-packages/skimage/util/dtype.py", line 1, in <module>
    import numpy as np
  File "/lib/python3.8/site-packages/numpy/__init__.py", line 142, in <module>
    from . import add_newdocs
  File "/lib/python3.8/site-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/lib/python3.8/site-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/lib/python3.8/site-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/lib/python3.8/site-packages/numpy/core/__init__.py", line 103, in <module>
    from numpy.testing._private.pytesttester import PytestTester
  File "/lib/python3.8/site-packages/numpy/testing/__init__.py", line 10, in <module>
    from unittest import TestCase
  File "/lib/python3.8/unittest/__init__.py", line 60, in <module>
    from .async_case import IsolatedAsyncioTestCase
  File "/lib/python3.8/unittest/async_case.py", line 1, in <module>
    import asyncio
  File "/lib/python3.8/asyncio/__init__.py", line 8, in <module>
    from .base_events import *
  File "/lib/python3.8/asyncio/base_events.py", line 39, in <module>
    from . import coroutines
  File "/lib/python3.8/asyncio/coroutines.py", line 5, in <module>
    import inspect
  File "/lib/python3.8/inspect.py", line 35, in <module>
    import dis
RecursionError: maximum recursion depth exceeded during compilation

    at _hiwire_throw_error (pyodide.asm.js:8)
    at _pythonexc2js (cdn.jsdelivr.net/pyodide/dev/full/pyodide.asm.wasm:wasm-function[372]:0x1078d8)
    at __pyproxy_apply (cdn.jsdelivr.net/pyodide/dev/full/pyodide.asm.wasm:wasm-function[370]:0x1075d1)
    at Module.__pyproxy_apply (pyodide.asm.js:8)
    at Object.apply (pyodide.asm.js:8)

I am running this in a webworker:

self.languagePluginUrl = "https://cdn.jsdelivr.net/pyodide/dev/full/";
importScripts("https://cdn.jsdelivr.net/pyodide/dev/full/pyodide.js");

const startupScript = `
import micropip

def try_import_skimage(_):
    import skimage

micropip.install(["imjoy-rpc>=0.2.59", "scikit-image"]).then(try_import_skimage)
`;

languagePluginLoader.then(() => {
  self.pyodide.loadPackage(["micropip"]).then(async () => {
    self.pyodide.runPython(startupScript);
  });
});

@rth @nopid Could you please take a look?

About this issue

Original URL
State: open
Created 4 years ago
Comments: 33 (33 by maintainers)

Most upvoted comments

Another useful tip: using console.error("some message", an_object) gives a stack trace but doesn’t throw an error. This is very helpful.

hoodmane on Dec 30, 2020

I can tell you what the stuff in that stack trace is, but it’s not very helpful.

__PyObject_CallMethodIdObjArgs // This is calling "pyodide.eval_code"
_object_vacall  // A python internal: "variable arguments call"
(anonymous) 
__PyFunction_Vectorcall // this is the new python fast dispatch method added in 3.8
_function_code_fastcall 
_PyEval_EvalFrameEx // This is a hook for JITs that controls the evaluation of python functions
(anonymous) 
__PyEval_EvalFrameDefault // This is the ordinary non-jit value for the FrameEx hook.
_call_function // This is calling eval (_base.py line 88)
(anonymous)  
_cfunction_vectorcall_FASTCALL 
(anonymous) 
_builtin_eval // finally got to eval's body
_PyEval_EvalCode 
__PyEval_EvalCodeWithName 
(anonymous) 
__PyEval_EvalFrameDefault // Okay now we finally made it to executing the evaled code. Apparently eval skips the JIT hook.
_call_function 
__PyObject_MakeTpCall  // Someone called a callable object
(anonymous) 
_JsBoundMethod_Call // that callable object was a `JsBoundMethod`. Now it's in our pyodide code
_hiwire_call_member // hiwire is a transfer layer between C and js, it's going to call the actual js object for us
Promise.then (async) // Okay we were calling Promise.then.
apply // Called `.then` with a pyproxy, so we're heading back into pyodide
Module.__pyproxy_apply // Call a py proxy, the js wrapper
__pyproxy_apply // Call a py proxy the C code
_pythonexc2js // `PyObject_Call` raised an error, now pyodide is formatting the error method. 
_hiwire_throw_error // tell javascript to throw an error.

So the error was raised in PyProxy_Call, reported back to us three lines down on line 111.

hoodmane on Dec 30, 2020

Thanks for the link! The main way I debug things is by using console.log, printf, etc. Put lots of these into the code and then you can narrow down what code paths are getting executed and where the error occurred. Once you’ve located the region where the error happens, you can start printing out program state.

I will try again.

hoodmane on Dec 30, 2020

With Firefox inside a pyodide console pointing to cdn.jsdelivr.net/pyodide/dev/full/, the packages load without trouble:

>>> import micropip
>>> micropip.install(["imjoy-rpc>=0.2.59", "scikit-image"])
Installed scikit-image, numpy, pytest, werkzeug, imjoy-rpc
>>> import skimage
>>> 
>>> dir(skimage)
_INPLACE_MSG,_STANDARD_MSG,__SKIMAGE_SETUP__,__builtins__,__cached__,__doc__,__file__,__loader__,__name__,__package__,__path__,__spec__,__version__,_raise_build_error,_shared,color,data,data_dir,dr
aw,dtype_limits,ensure_python_version,exposure,external,filters,img_as_bool,img_as_float,img_as_float32,img_as_float64,img_as_int,img_as_ubyte,img_as_uint,io,lookfor,measure,restoration,transform,u
til
>>> import imjoy_rpc
>>> dir(imjoy_rpc)
ApiWrapper,ContextLocal,Local,LocalProxy,__all__,__builtins__,__cached__,__doc__,__file__,__loader__,__name__,__package__,__path__,__spec__,_rpc_context,api,default_config,dotdict,logger,logging,os
,register_default_codecs,setup_connection,sys,type_of_script,utils

nopid on Dec 23, 2020