ploomber: Cannot load notebooks: ModuleNotFoundError: Error processing dotted path

I have a notebook folder in ploomber project and ploomber tries to run something so I can’t even load the notebook (which BTW is not referenced in pipeline.yaml in any way)

What happened? How is it possible that ploomber breaks jupyter?

Error message

[I 11:59:22.671 NotebookApp] [Ploomber] Requested model: work/wwf/notebooks/Enrich_Test.ipynb. Looking for DAG with root dir: /home/kuba/Projects                            
[E 11:59:22.786 NotebookApp] Uncaught exception GET /api/contents/work/wwf/notebooks/Enrich_Test.ipynb?type=notebook&_=1656064762442 (127.0.0.1)                             
    HTTPServerRequest(protocol='http', host='localhost:8888', method='GET', uri='/api/contents/work/wwf/notebooks/Enrich_Test.ipynb?type=notebook&_=1656064762442', version='
HTTP/1.1', remote_ip='127.0.0.1')                                                                                                                                            
    Traceback (most recent call last):                                                                                                                                       
      File "/home/kuba/.local/lib/python3.8/site-packages/tornado/web.py", line 1704, in _execute                                                                            
        result = await result                                                                                                                                                
      File "/home/kuba/.local/lib/python3.8/site-packages/tornado/gen.py", line 234, in wrapper                                                                              
        yielded = ctx_run(next, result)                                                                                                                                      
      File "/home/kuba/.local/lib/python3.8/site-packages/notebook/services/contents/handlers.py", line 118, in get                                                          
        model = yield maybe_future(self.contents_manager.get(                  
      File "/home/kuba/.local/lib/python3.8/site-packages/ploomber/jupyter/manager.py", line 267, in get                                                                     
        self.load_dag(                                                                                                                                                       
      File "/home/kuba/.local/lib/python3.8/site-packages/ploomber/jupyter/manager.py", line 193, in load_dag                                                                
        pairs = [(resolve_path(                                                       
      File "/home/kuba/.local/lib/python3.8/site-packages/ploomber/jupyter/manager.py", line 196, in <listcomp>                                                                      if t.source.loc is not None]                                                  
      File "/home/kuba/.local/lib/python3.8/site-packages/ploomber/sources/pythoncallablesource.py", line 130, in loc                                                        
        self._loc = self._callable_loader.get_loc()                                                                                                                          
      File "/home/kuba/.local/lib/python3.8/site-packages/ploomber/sources/pythoncallablesource.py", line 58, in get_loc                                                     
        loc, _ = lazily_locate_dotted_path(self._primitive)                                                                                                                        File "/home/kuba/.local/lib/python3.8/site-packages/ploomber/util/dotted_path.py", line 343, in lazily_locate_dotted_path                                              
        raise ModuleNotFoundError('Error processing dotted '
    ModuleNotFoundError: Error processing dotted path 'deepsense_wwf.data_utils.copy_zipped_data': 'deepsense_wwf' appears to be a namespace package, which are not supported
[W 11:59:22.790 NotebookApp] Unhandled error          
[E 11:59:22.791 NotebookApp] {                                                                                                                                               
      "Host": "localhost:8888",                                                                                                                                              
      "Accept": "application/json, text/javascript, */*; q=0.01",                                                                                                            
      "Referer": "http://localhost:8888/notebooks/work/wwf/notebooks/Enrich_Test.ipynb",                                                                                     
      "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:101.0) Gecko/20100101 Firefox/101.0"                                                                         
    }

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 16 (12 by maintainers)

Most upvoted comments

I can confirm this is fixed. Thanks!

lambdaofgod on Aug 2, 2022

click here to tweet something nice

edublancas on Aug 2, 2022

@94rain before you start working on this, ensure you get the latest version from master since I recently pushed changes to this file.

I couldn’t reproduce @lambdaofgod error, but the overall problem is that we should catch any exceptions raised by load_dag.

https://github.com/ploomber/ploomber/blob/48b5ac0ff4faae1ba90f8c141bc305edaee1de6f/src/ploomber/jupyter/manager.py#L268

so on a second thought, the solution should be more general (as opposed to only covering the pairs = ... statement.

something like this should work:

def _load_dag(self, ...):
   # actual implpementation

def load_dag(self, ...):
   try:
     self._load_dag(...)
   except Exception as e:
      # log exception along with the message 
     # a problem happened when loading your pipeline

I’m unsure if this would case any side-effects so let’s get this change done and then see if any tests break

edublancas on Jul 18, 2022

Ok, thanks for the feedback. The ploomber extension activates when you run pip install ploomber. So if you uninstall ploomber, it should remove it as well.

It’s weird that disabling it didn’t turn it off on jupyter, but many users have experienced issues with this in the past (not being able to activate it, the extension doesn’t show up, etc). So who knows what’s happening.

this is a pretty bad bug so we’ll get to it. I’ll work on it this week.

edublancas on Jun 28, 2022