notebook-as-pdf: Failure in gitlab CI

I am really excited for this extension, since it would allow a much leaner docker image in my CI runners.

The runner configuration is pretty simple:

image: python:latest

build:
  stage: build
  script:
    - pip install --no-cache-dir -r requirements.txt
    - jupyter nbconvert --to PDFviaHTML --execute example.ipynb

and of course, notebook-as-pdf is part of requirements.txt.

So I took it for a spin, but it was not a happy camper:

$ jupyter nbconvert --to PDFviaHTML --execute example.ipynb
 [NbConvertApp] Converting notebook example.ipynb to PDFviaHTML
 [NbConvertApp] Executing notebook with kernel: python3
 [NbConvertApp] Executing notebook with kernel: python3
 [W:pyppeteer.chromium_downloader] start chromium download.
 Download may take a few minutes.
 100%|██████████| 108773488/108773488 [00:01<00:00, 105172481.75it/s]
 [W:pyppeteer.chromium_downloader] 
 chromium download done.
 [W:pyppeteer.chromium_downloader] chromium extracted to: /root/.local/share/pyppeteer/local-chromium/588429
 Traceback (most recent call last):
<snip>
 Traceback (most recent call last):
   File "/usr/local/lib/python3.8/site-packages/pyppeteer/launcher.py", line 151, in _close_process
     self._loop.run_until_complete(self.killChrome())
   File "/usr/local/lib/python3.8/asyncio/base_events.py", line 591, in run_until_complete
     self._check_closed()
   File "/usr/local/lib/python3.8/asyncio/base_events.py", line 508, in _check_closed
     raise RuntimeError('Event loop is closed')
 RuntimeError: Event loop is closed
 sys:1: RuntimeWarning: coroutine 'Launcher.killChrome' was never awaited
 RuntimeWarning: Enable tracemalloc to get the object allocation traceback

I’m wondering if my requirements.txt needs something else (why is chromium installed on demand?) but instead of blindly trying things out, I thought I should report it first.

Click to toggle the full log. 
Running with gitlab-runner 13.0.1 (21cb397c)
   on default-runner-7f9b6f57bc-t25l7 JXQu29S9
Preparing the "docker" executor
00:06
 Using Docker executor with image python:latest ...
 WARNING: Container based cache volumes creation is disabled. Will not create volume for "/cache"
 Pulling docker image python:latest ...
 Using docker image sha256:7f5b6ccd03e9da5e02a20ea4377ae61f6ae1b935b05c1bd52fdafe4b958b5e50 for python:latest ...
Preparing environment
00:02
 Running on runner-jxqu29s9-project-94480-concurrent-0 via default-runner-7f9b6f57bc-t25l7...
Getting source from Git repository
00:02
 Fetching changes with git depth set to 50...
 Initialized empty Git repository in /builds/adavid/mylittlelark/.git/
 Created fresh repository.
 From https://gitlab.cern.ch/adavid/mylittlelark
  * [new ref]         refs/pipelines/1751602 -> refs/pipelines/1751602
  * [new branch]      master                 -> origin/master
 Checking out 48add964 as master...
 Skipping Git submodules setup
Restoring cache
00:01
Downloading artifacts
00:02
Running before_script and script
01:23
 $ pip install --no-cache-dir -r requirements.txt
 Collecting jupyter
   Downloading jupyter-1.0.0-py2.py3-none-any.whl (2.7 kB)
 Collecting pympler
   Downloading Pympler-0.8.tar.gz (175 kB)
 Collecting lark-parser
   Downloading lark-parser-0.8.9.tar.gz (294 kB)
 Collecting tqdm
   Downloading tqdm-4.46.1-py2.py3-none-any.whl (63 kB)
 Collecting tables
   Downloading tables-3.6.1-cp38-cp38-manylinux1_x86_64.whl (4.3 MB)
 Collecting notebook-as-pdf
   Downloading notebook_as_pdf-0.2.0-py3-none-any.whl (5.2 kB)
 Collecting ipywidgets
   Downloading ipywidgets-7.5.1-py2.py3-none-any.whl (121 kB)
 Collecting nbconvert
   Downloading nbconvert-5.6.1-py2.py3-none-any.whl (455 kB)
 Collecting jupyter-console
   Downloading jupyter_console-6.1.0-py2.py3-none-any.whl (21 kB)
 Collecting ipykernel
   Downloading ipykernel-5.3.0-py3-none-any.whl (119 kB)
 Collecting notebook
   Downloading notebook-6.0.3-py3-none-any.whl (9.7 MB)
 Collecting qtconsole
   Downloading qtconsole-4.7.5-py2.py3-none-any.whl (118 kB)
 Collecting numexpr>=2.6.2
   Downloading numexpr-2.7.1-cp38-cp38-manylinux1_x86_64.whl (164 kB)
 Collecting numpy>=1.9.3
   Downloading numpy-1.19.0-cp38-cp38-manylinux2010_x86_64.whl (14.6 MB)
 Collecting pyppeteer
   Downloading pyppeteer-0.2.2-py3-none-any.whl (145 kB)
 Collecting PyPDF2
   Downloading PyPDF2-1.26.0.tar.gz (77 kB)
 Collecting widgetsnbextension~=3.5.0
   Downloading widgetsnbextension-3.5.1-py2.py3-none-any.whl (2.2 MB)
 Collecting ipython>=4.0.0; python_version >= "3.3"
   Downloading ipython-7.16.1-py3-none-any.whl (785 kB)
 Collecting traitlets>=4.3.1
   Downloading traitlets-4.3.3-py2.py3-none-any.whl (75 kB)
 Collecting nbformat>=4.2.0
   Downloading nbformat-5.0.7-py3-none-any.whl (170 kB)
 Collecting defusedxml
   Downloading defusedxml-0.6.0-py2.py3-none-any.whl (23 kB)
 Collecting pygments
   Downloading Pygments-2.6.1-py3-none-any.whl (914 kB)
 Collecting jupyter-core
   Downloading jupyter_core-4.6.3-py2.py3-none-any.whl (83 kB)
 Collecting entrypoints>=0.2.2
   Downloading entrypoints-0.3-py2.py3-none-any.whl (11 kB)
 Collecting pandocfilters>=1.4.1
   Downloading pandocfilters-1.4.2.tar.gz (14 kB)
 Collecting mistune<2,>=0.8.1
   Downloading mistune-0.8.4-py2.py3-none-any.whl (16 kB)
 Collecting jinja2>=2.4
   Downloading Jinja2-2.11.2-py2.py3-none-any.whl (125 kB)
 Collecting testpath
   Downloading testpath-0.4.4-py2.py3-none-any.whl (163 kB)
 Collecting bleach
   Downloading bleach-3.1.5-py2.py3-none-any.whl (151 kB)
 Collecting jupyter-client
   Downloading jupyter_client-6.1.3-py3-none-any.whl (106 kB)
 Collecting prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0
   Downloading prompt_toolkit-3.0.5-py3-none-any.whl (351 kB)
 Collecting tornado>=4.2
   Downloading tornado-6.0.4.tar.gz (496 kB)
 Collecting pyzmq>=17
   Downloading pyzmq-19.0.1-cp38-cp38-manylinux1_x86_64.whl (1.1 MB)
 Collecting prometheus-client
   Downloading prometheus_client-0.8.0-py2.py3-none-any.whl (53 kB)
 Collecting Send2Trash
   Downloading Send2Trash-1.5.0-py3-none-any.whl (12 kB)
 Collecting terminado>=0.8.1
   Downloading terminado-0.8.3-py2.py3-none-any.whl (33 kB)
 Collecting ipython-genutils
   Downloading ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
 Collecting qtpy
   Downloading QtPy-1.9.0-py2.py3-none-any.whl (54 kB)
 Collecting appdirs<2.0.0,>=1.4.3
   Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
 Collecting urllib3<2.0.0,>=1.25.8
   Downloading urllib3-1.25.9-py2.py3-none-any.whl (126 kB)
 Collecting websockets<9.0,>=8.1
   Downloading websockets-8.1-cp38-cp38-manylinux2010_x86_64.whl (78 kB)
 Collecting pyee<8.0.0,>=7.0.1
   Downloading pyee-7.0.2-py2.py3-none-any.whl (12 kB)
 Collecting jedi>=0.10
   Downloading jedi-0.17.1-py2.py3-none-any.whl (1.4 MB)
 Collecting pexpect; sys_platform != "win32"
   Downloading pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
 Requirement already satisfied: setuptools>=18.5 in /usr/local/lib/python3.8/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets->jupyter->-r requirements.txt (line 1)) (47.1.1)
 Collecting pickleshare
   Downloading pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
 Collecting backcall
   Downloading backcall-0.2.0-py2.py3-none-any.whl (11 kB)
 Collecting decorator
   Downloading decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
 Collecting six
   Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
 Collecting jsonschema!=2.5.0,>=2.4
   Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
 Collecting MarkupSafe>=0.23
   Downloading MarkupSafe-1.1.1-cp38-cp38-manylinux1_x86_64.whl (32 kB)
 Collecting packaging
   Downloading packaging-20.4-py2.py3-none-any.whl (37 kB)
 Collecting webencodings
   Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
 Collecting python-dateutil>=2.1
   Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
 Collecting wcwidth
   Downloading wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
 Collecting ptyprocess; os_name != "nt"
   Downloading ptyprocess-0.6.0-py2.py3-none-any.whl (39 kB)
 Collecting parso<0.8.0,>=0.7.0
   Downloading parso-0.7.0-py2.py3-none-any.whl (100 kB)
 Collecting pyrsistent>=0.14.0
   Downloading pyrsistent-0.16.0.tar.gz (108 kB)
 Collecting attrs>=17.4.0
   Downloading attrs-19.3.0-py2.py3-none-any.whl (39 kB)
 Collecting pyparsing>=2.0.2
   Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
 Building wheels for collected packages: pympler, lark-parser, PyPDF2, pandocfilters, tornado, pyrsistent
   Building wheel for pympler (setup.py): started
   Building wheel for pympler (setup.py): finished with status 'done'
   Created wheel for pympler: filename=Pympler-0.8-py3-none-any.whl size=164713 sha256=3e8ab23edf19d99c51f5b7a9a1bf74ebbb9f947a6d9ebade2ff8c0a7ad75083f
   Stored in directory: /tmp/pip-ephem-wheel-cache-pivp50fy/wheels/2c/09/38/7c8c5dc224099d3a7194d154a99093d429bb4bda18c241c4b5
   Building wheel for lark-parser (setup.py): started
   Building wheel for lark-parser (setup.py): finished with status 'done'
   Created wheel for lark-parser: filename=lark_parser-0.8.9-py2.py3-none-any.whl size=75357 sha256=990a32661454b7b77d1df1858328ea40daa9bac5ce84bebc0be6cb43a5246b33
   Stored in directory: /tmp/pip-ephem-wheel-cache-pivp50fy/wheels/5c/55/8c/91a8363366ada1c946e50d5c030e63129a2d8f3120e1975997
   Building wheel for PyPDF2 (setup.py): started
   Building wheel for PyPDF2 (setup.py): finished with status 'done'
   Created wheel for PyPDF2: filename=PyPDF2-1.26.0-py3-none-any.whl size=61084 sha256=b1f7402d283104baebace712c6dd94abb940019a8acd38feaf5903eb11954f19
   Stored in directory: /tmp/pip-ephem-wheel-cache-pivp50fy/wheels/b1/1a/8f/a4c34be976825a2f7948d0fa40907598d69834f8ab5889de11
   Building wheel for pandocfilters (setup.py): started
   Building wheel for pandocfilters (setup.py): finished with status 'done'
   Created wheel for pandocfilters: filename=pandocfilters-1.4.2-py3-none-any.whl size=7856 sha256=75791b53889332212a9cbf41d5ba54a30932ae76bec3902bc753298f58305841
   Stored in directory: /tmp/pip-ephem-wheel-cache-pivp50fy/wheels/f6/08/65/e4636b703d0e870cd62692dafd6b47db27287fe80cea433722
   Building wheel for tornado (setup.py): started
   Building wheel for tornado (setup.py): finished with status 'done'
   Created wheel for tornado: filename=tornado-6.0.4-cp38-cp38-linux_x86_64.whl size=427821 sha256=978cecc354d37b4422e9f08fe5949afad94728628d849b74d1c6870ac25c0171
   Stored in directory: /tmp/pip-ephem-wheel-cache-pivp50fy/wheels/88/79/e5/598ba17e85eccf2626eab62e4ee8452895636cd542650d450d
   Building wheel for pyrsistent (setup.py): started
   Building wheel for pyrsistent (setup.py): finished with status 'done'
   Created wheel for pyrsistent: filename=pyrsistent-0.16.0-cp38-cp38-linux_x86_64.whl size=126675 sha256=87e75b33c767b74b59c2ae526d335a2be64aab5cf3b983c519171a6aaceff172
   Stored in directory: /tmp/pip-ephem-wheel-cache-pivp50fy/wheels/17/be/0f/727fb20889ada6aaaaba861f5f0eb21663533915429ad43f28
 Successfully built pympler lark-parser PyPDF2 pandocfilters tornado pyrsistent
 Installing collected packages: six, python-dateutil, tornado, ipython-genutils, decorator, traitlets, jupyter-core, pyzmq, jupyter-client, MarkupSafe, jinja2, defusedxml, pygments, entrypoints, pandocfilters, mistune, testpath, pyrsistent, attrs, jsonschema, nbformat, pyparsing, packaging, webencodings, bleach, nbconvert, parso, jedi, ptyprocess, pexpect, pickleshare, backcall, wcwidth, prompt-toolkit, ipython, ipykernel, prometheus-client, Send2Trash, terminado, notebook, widgetsnbextension, ipywidgets, jupyter-console, qtpy, qtconsole, jupyter, pympler, lark-parser, tqdm, numpy, numexpr, tables, appdirs, urllib3, websockets, pyee, pyppeteer, PyPDF2, notebook-as-pdf
 Successfully installed MarkupSafe-1.1.1 PyPDF2-1.26.0 Send2Trash-1.5.0 appdirs-1.4.4 attrs-19.3.0 backcall-0.2.0 bleach-3.1.5 decorator-4.4.2 defusedxml-0.6.0 entrypoints-0.3 ipykernel-5.3.0 ipython-7.16.1 ipython-genutils-0.2.0 ipywidgets-7.5.1 jedi-0.17.1 jinja2-2.11.2 jsonschema-3.2.0 jupyter-1.0.0 jupyter-client-6.1.3 jupyter-console-6.1.0 jupyter-core-4.6.3 lark-parser-0.8.9 mistune-0.8.4 nbconvert-5.6.1 nbformat-5.0.7 notebook-6.0.3 notebook-as-pdf-0.2.0 numexpr-2.7.1 numpy-1.19.0 packaging-20.4 pandocfilters-1.4.2 parso-0.7.0 pexpect-4.8.0 pickleshare-0.7.5 prometheus-client-0.8.0 prompt-toolkit-3.0.5 ptyprocess-0.6.0 pyee-7.0.2 pygments-2.6.1 pympler-0.8 pyparsing-2.4.7 pyppeteer-0.2.2 pyrsistent-0.16.0 python-dateutil-2.8.1 pyzmq-19.0.1 qtconsole-4.7.5 qtpy-1.9.0 six-1.15.0 tables-3.6.1 terminado-0.8.3 testpath-0.4.4 tornado-6.0.4 tqdm-4.46.1 traitlets-4.3.3 urllib3-1.25.9 wcwidth-0.2.5 webencodings-0.5.1 websockets-8.1 widgetsnbextension-3.5.1
 $ jupyter nbconvert --to html --execute example.ipynb
 [NbConvertApp] Converting notebook example.ipynb to html
 [NbConvertApp] Executing notebook with kernel: python3
 [NbConvertApp] Writing 342781 bytes to example.html
 $ jupyter nbconvert --to PDFviaHTML --execute example.ipynb
 [NbConvertApp] Converting notebook example.ipynb to PDFviaHTML
 [NbConvertApp] Executing notebook with kernel: python3
 [NbConvertApp] Executing notebook with kernel: python3
 [W:pyppeteer.chromium_downloader] start chromium download.
 Download may take a few minutes.
 100%|██████████| 108773488/108773488 [00:01<00:00, 105172481.75it/s]
 [W:pyppeteer.chromium_downloader] 
 chromium download done.
 [W:pyppeteer.chromium_downloader] chromium extracted to: /root/.local/share/pyppeteer/local-chromium/588429
 Traceback (most recent call last):
   File "/usr/local/bin/jupyter-nbconvert", line 8, in <module>
     sys.exit(main())
   File "/usr/local/lib/python3.8/site-packages/jupyter_core/application.py", line 270, in launch_instance
     return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
   File "/usr/local/lib/python3.8/site-packages/traitlets/config/application.py", line 664, in launch_instance
     app.start()
   File "/usr/local/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 340, in start
     self.convert_notebooks()
   File "/usr/local/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 510, in convert_notebooks
     self.convert_single_notebook(notebook_filename)
   File "/usr/local/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 481, in convert_single_notebook
     output, resources = self.export_single_notebook(notebook_filename, resources, input_buffer=input_buffer)
   File "/usr/local/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 410, in export_single_notebook
     output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
   File "/usr/local/lib/python3.8/site-packages/nbconvert/exporters/exporter.py", line 179, in from_filename
     return self.from_file(f, resources=resources, **kw)
   File "/usr/local/lib/python3.8/site-packages/nbconvert/exporters/exporter.py", line 197, in from_file
     return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
   File "/usr/local/lib/python3.8/site-packages/notebook_as_pdf/__init__.py", line 148, in from_notebook_node
     self.pool.submit(
   File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 439, in result
     return self.__get_result()
   File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
     raise self._exception
   File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
     result = self.fn(*self.args, **self.kwargs)
   File "/usr/local/lib/python3.8/asyncio/runners.py", line 43, in run
     return loop.run_until_complete(main)
   File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
     return future.result()
   File "/usr/local/lib/python3.8/site-packages/notebook_as_pdf/__init__.py", line 107, in notebook_to_pdf
     await html_to_pdf(f.name, pdf_path)
   File "/usr/local/lib/python3.8/site-packages/notebook_as_pdf/__init__.py", line 24, in html_to_pdf
     browser = await launch(handleSIGINT=False, handleSIGTERM=False, handleSIGHUP=False)
   File "/usr/local/lib/python3.8/site-packages/pyppeteer/launcher.py", line 305, in launch
     return await Launcher(options, **kwargs).launch()
   File "/usr/local/lib/python3.8/site-packages/pyppeteer/launcher.py", line 166, in launch
     self.browserWSEndpoint = get_ws_endpoint(self.url)
   File "/usr/local/lib/python3.8/site-packages/pyppeteer/launcher.py", line 225, in get_ws_endpoint
     raise BrowserError('Browser closed unexpectedly:\n')
 pyppeteer.errors.BrowserError: Browser closed unexpectedly:
 Error in atexit._run_exitfuncs:
 Traceback (most recent call last):
   File "/usr/local/lib/python3.8/site-packages/pyppeteer/launcher.py", line 151, in _close_process
     self._loop.run_until_complete(self.killChrome())
   File "/usr/local/lib/python3.8/asyncio/base_events.py", line 591, in run_until_complete
     self._check_closed()
   File "/usr/local/lib/python3.8/asyncio/base_events.py", line 508, in _check_closed
     raise RuntimeError('Event loop is closed')
 RuntimeError: Event loop is closed
 sys:1: RuntimeWarning: coroutine 'Launcher.killChrome' was never awaited
 RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Running after_script
00:02
Uploading artifacts for failed job
00:03
 ERROR: Job failed: exit code 1

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 17 (9 by maintainers)

Most upvoted comments

After I installed the apt packages I run the command below and it worked.

jupyter-nbconvert --to PDFviaHTML histogram.ipynb

+2
snmhas on Jul 8, 2020

Grand. Thanks for helping out someone who’s a newb (cough cough).

For navigation help […]

This is exactly what I had in mind when I asked about it.

I agree that pages are arbitrary, but https://site/file.pdf#page=N is a staple for me when sharing information, but I wonder if one could make use of a more granular way.

+1
adavidzh on Jul 1, 2020
Read more comments on GitHub
← openai: System.Text.Json.JsonException The JSON value could not be converted to System.String. Path: $.error.code | LineNumber: 0 | BytePositionInLine: 20.
betfair: stream.stop() breaks when async=True →