nbconvert: "KeyError: 'state'" when trying to convert a notebook from Google Colab

Hello,

I seem to every so often get the following error:

File "/opt/hostedtoolcache/Python/3.10.2/x64/lib/python3.10/site-packages/nbconvert/filters/widgetsdatatypefilter.py", line 56, in __call__
[95](https://github.com/mrdbourke/pytorch-deep-learning/runs/5597148101?check_suite_focus=true#step:8:95)
    metadata['widgets'][WIDGET_STATE_MIMETYPE]['state'] if
[96](https://github.com/mrdbourke/pytorch-deep-learning/runs/5597148101?check_suite_focus=true#step:8:96)
KeyError: 'state'
[97](https://github.com/mrdbourke/pytorch-deep-learning/runs/5597148101?check_suite_focus=true#step:8:97)
Error: Process completed with exit code 1.

It seems to be most often when I’m saving a notebook from Google Colab into GitHub and then running a GitHub Action to convert it directly into documentation.

The GitHub Action traceback can be seen here: https://github.com/mrdbourke/pytorch-deep-learning/runs/5597148101?check_suite_focus=true

The notebook in question can be found here: https://github.com/mrdbourke/pytorch-deep-learning/blob/main/03_pytorch_computer_vision.ipynb

The issue doesn’t occur on other notebooks converted in the same manner.

Things I’ve tried:

Make a copy of the notebook, resave under same name (in Google Colab)
Clone the notebook locally, rerun, save, commit

Both of these result in the same result as above.

Any help would be greatly appreciated, I’m not quite sure where this error is coming from nor why it happens to some notebooks and not others.

To the best of my knowledge, the notebook it happens on “03_pytorch_computer_vision.ipynb” uses the exact same style as other notebooks (Python code and markdown).

About this issue

Original URL
State: open
Created 2 years ago
Reactions: 10
Comments: 15 (1 by maintainers)

Most upvoted comments

Remove metadata/widgets in notebook file.

Thanks

“Widgets” --> “Clear Notebook Widgets State” helped me in jupyter

+22

Pqlet on Oct 26, 2022

Dumb workaround for without access to CLI jq and without reuploading to Colab:

Open .ipynb file in notepad
Copy file JSON contents
Go to https://jqplay.org/
Paste contents into JSON field
Copy/paste del(.metadata.widgets) into filter pane (Thanks @hs211216)
Copy result from right Result field
Paste into original .ipynb file
Save file and exit

Works now.

+10

ndjhartman on Mar 30, 2023

Just hit with https://colab.research.google.com/github/kstathou/vector_engine/blob/master/notebooks/001_vector_search.ipynb

The workaround recommended by @pplonski worked for me. Here is what I did

Download 001_vector_search.ipynb with Colab gui
Fail to convert it with nbconvert

% jupyter nbconvert --to html 001_vector_search.ipynb
  [NbConvertApp] Converting notebook 001_vector_search.ipynb to html
  Traceback (most recent call last):
    File "/usr/local/bin/jupyter-nbconvert", line 8, in <module>
      sys.exit(main())
    File "/usr/local/lib/python3.9/site-packages/jupyter_core/application.py", line 264, in launch_instance
      return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
    ...
    File "/usr/local/share/jupyter/nbconvert/templates/base/display_priority.j2", line 7, in block 'data_priority'
      {%- for type in output.data | filter_data_type -%}
    File "/usr/local/lib/python3.9/site-packages/nbconvert/filters/widgetsdatatypefilter.py", line 57, in __call__
      metadata["widgets"][WIDGET_STATE_MIMETYPE]["state"]
  KeyError: 'state'

Remove .metadata.widgets with Jq

% jq -M 'del(.metadata.widgets)' 001_vector_search.ipynb > 001_vector_search.fixed.ipynb
% jupyter nbconvert --to html 001_vector_search.fixed.ipynb
  [NbConvertApp] Converting notebook 001_vector_search.fixed.ipynb to html
  [NbConvertApp] Writing 629781 bytes to 001_vector_search.fixed.html

% open 001_vector_search.fixed.html

  ... output looks good

For reference I’m on OSX 10.15.7 and here are my jupyter versions

% jupyter --version
  Selected Jupyter core packages...
  IPython          : 8.0.1
  ipykernel        : 6.9.0
  ipywidgets       : not installed
  jupyter_client   : 7.1.2
  jupyter_core     : 4.9.1
  Jupyter_server   : not installed
  jupyterlab       : not installed
  nbclient         : 0.6.4
  nbconvert        : 6.5.0
  nbformat         : 5.4.0
  notebook         : not installed
  qtconsole        : not installed
  traitlets        : 5.2.2

hs211216 on Jun 15, 2022

Remove metadata/widgets in notebook file.

pplonski on Apr 29, 2022

Notes: If you are unable to locate the Widget tab on your notebook, which happened to me as well, you will need to install ipywidgets by executing the following command and restart your notebook.

$ pip install ipywidgets

Xu-Justin on Jun 1, 2023

Remove metadata/widgets in notebook file.

Thanks “Widgets” --> “Clear Notebook Widgets State” helped me in jupyter

This helped me too!

–

Note: This requires a traditional notebook layout (e.g. Jupyter Notebook Classic).

See:

Update 2 Feb 2024:

You may need to install nbclassic to the get the widget item in the menu bar, see: https://github.com/jupyter/nbclassic

Install:

pip install nbclassic

Run:

jupyter nbclassic

Otherwise clearing the widget metadata as discussed above may work too, however, I’m not 100% sure.

mrdbourke on Feb 2, 2024