readthedocs.org: timeout or excessive memory consumption when migrating to version 2 mkdocs with pdf

Details

Read the Docs project URL: https://readthedocs.org/projects/mapstore/
Build URL (if applicable): https://readthedocs.org/projects/mapstore/builds/21454904/
Read the Docs username (if applicable): https://readthedocs.org/profiles/offtherailz/

I attempted to migrate documentation to version 2 but I have a memory consumption error. I suppose is for the PDF generation of Mkdocs.

Expected Result

The build is successful

Actual Result

After a while I have an error:

Command killed due to timeout or excessive memory consumption

Main files involved

readthedocs.yml

# Read the Docs configuration file for MkDocs projects
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Set the version of Python and other tools you might need
build:
  os: ubuntu-22.04
  tools:
    python: "3.10"

mkdocs:
  configuration: mkdocs.yml

# Optionally declare the Python requirements required to build your docs
python:
  install:
  - requirements: docs/requirements.txt

requirements.txt

mkdocs-material==3.3.0
jinja2==3.1.2
Markdown==3.4.1
git+https://github.com/geosolutions-it/mkdocs-pdf-export-plugin.git
WeasyPrint==52.5

mkdocs.yml

site_name: MapStore
site_url: 'https://mapstore.geosolutionsgroup.com'
use_directory_urls: true
theme:
  name: 'material'
  custom_dir: 'docs/theme'
  language: 'en'
  feature:
    tabs: true
  logo: 'img/mapstore2-logo.png'
  company_url: 'http://www.geo-solutions.it'
  company_contacts: 'GeoSolutions | sales@geosolutionsgroup.com '
  palette:
    primary: 'teal'
    accent: 'teal'
extra:
  repo_icon: 'github'
  social:
    - type: 'github'
      icon: 'fontawesome/brands/github'
      link: 'https://github.com/geosolutions-it'
    - type: 'twitter'
      icon: 'fontawesome/brands/twitter'
      link: 'https://twitter.com/geosolutions_it'
    - type: 'linkedin'
      icon: 'fontawesome/brands/linkedin'
      link: 'https://it.linkedin.com/company/geosolutions-s-a-s-'
extra_javascript:
  - 'https://code.jquery.com/jquery-3.3.1.min.js'
  - 'https://unpkg.com/mermaid@9.1.3/dist/mermaid.min.js'
extra_css:
  - 'theme/css/extra.css'
repo_name: 'geosolutions-it/MapStore2'
repo_url: 'https://github.com/geosolutions-it/MapStore2/'
repo_issues: 'https://github.com/geosolutions-it/MapStore2/issues'
repo_issues_doc: 'https://github.com/geosolutions-it/MapStore2/issues/new/choose'
edit_uri: ''
docs_dir: 'docs'
markdown_extensions:
  - admonition
  - markdown.extensions.attr_list
  - codehilite:
      linenums: true
  - pymdownx.superfences:
      custom_fences:
        - name: mermaid
          class: mermaid
          format: !!python/name:pymdownx.superfences.fence_div_format
nav:
  - Home: 'index.md' # To include in pdf
  - User Guide:
    - Home Page Overview: 'user-guide/home-page.md'
    - Managing Users and Groups:
      - Overview: 'user-guide/managing-users-and-groups.md'
      - Managing Users: 'user-guide/managing-users.md'
      - Managing Groups: 'user-guide/managing-groups.md'
    - Managing Contexts:
      - Overview: 'user-guide/managing-contexts.md'
      - Application Context: 'user-guide/application-context.md'
      - Extension Library: 'user-guide/extension-library.md'
      - Map Catalog: 'user-guide/map-catalog.md'
      - Map Templates: 'user-guide/map-templates.md'
    - Resources Properties: 'user-guide/resources-properties.md'
    - Sharing Resources: 'user-guide/share.md'
    - Exploring Maps:
      - Overview: 'user-guide/exploring-maps.md'
      - Table of Contents: 'user-guide/toc.md'
      - Managing Layer Settings: 'user-guide/layer-settings.md'
      - Filtering Layers: 'user-guide/filtering-layers.md'
      - Attribute Table: 'user-guide/attributes-table.md'
      - Widgets: 'user-guide/widgets.md'
      - Export Layer Data: 'user-guide/export-data.md'
      - MapStore Toolbars: 'user-guide/mapstore-toolbars.md'
      - Printing a Map: 'user-guide/print.md'
      - Importing and Exporting Files: 'user-guide/import.md'
      - Catalog Services: 'user-guide/catalog.md'
      - Performing Measurments: 'user-guide/measure.md'
      - Annotations: 'user-guide/annotations.md'
      - Map Views: 'user-guide/map-views.md'
      - Street View: 'user-guide/street-view.md'
      - Navigation Toolbar: 'user-guide/navigation-toolbar.md'
      - Background Selector: 'user-guide/background.md'
      - Timeline: 'user-guide/timeline.md'
      - Footer: 'user-guide/footer.md'
    - Exploring Dashboards:
      - Overview: 'user-guide/exploring-dashboards.md'
      - Adding widgets: 'user-guide/adding-widgets.md'
      - Connecting widgets: 'user-guide/connecting-widgets.md'
    - Exploring Stories:
      - Overview: 'user-guide/exploring-stories.md'
      - Story Settings: 'user-guide/story-setting.md'
      - Title Section: 'user-guide/title-section.md'
      - Banner Section: 'user-guide/banner-section.md'
      - Paragraph Section: 'user-guide/paragraph-section.md'
      - Immersive Section: 'user-guide/immersive-section.md'
      - GeoCarousel Section: 'user-guide/geocarousel-section.md'
      - Media Section: 'user-guide/media-section.md'
      - Web Page section: 'user-guide/web-section.md'
      - Text Editor Toolbar: 'user-guide/text-editor-toolbar.md'
      - Image Toolbar: 'user-guide/content-image-toolbar.md'
      - Video Toolbar: 'user-guide/content-video-toolbar.md'
      - Map Toolbar: 'user-guide/content-map-toolbar.md'
      - Web Page Toolbar: 'user-guide/content-web-toolbar.md'
      - Media Editor: 'user-guide/media-editor-window.md'
      - Configue the map: 'user-guide/configure-map.md'

  - Developer Guide:
    - Requirements: 'developer-guide/requirements.md'
    - Quick Setup and Run: 'developer-guide/index.md'
    - Main Scripts: 'developer-guide/main-scripts.md'
    - Infrastructure: 'developer-guide/infrastructure-and-general-architecture.md'
    - Developing with MapStore:
      - Introduction: 'developer-guide/developing-with-mapstore-2-intro.md'
      - Folders Structure: 'developer-guide/project-structure.md'
      - Developing and Debugging: 'developer-guide/developing.md'
      - Building and Deploying: 'developer-guide/building-and-deploying.md'
      - Front End Technologies: 'developer-guide/reactjs-and-redux-introduction.md'
      - Plugins architecture: 'developer-guide/plugins-architecture.md'
      - Internationalization: 'developer-guide/internationalization.md'
      - Custom dependencies: 'developer-guide/custom-dependencies.md'
      - Styling and theming: 'developer-guide/customize-theme.md'
      - Working with Extensions: 'developer-guide/extensions.md'
      - Printing Module: 'developer-guide/printing-module.md'
      - How to use CDN: 'developer-guide/how-to-use-a-cdn.md'
      - FAQ: 'developer-guide/dev-faq.md'
    - Guidelines:
      - Code Conventions: 'developer-guide/code-conventions.md'
      - Documentation: 'developer-guide/documentation-guidelines.md'
      - Migration: 'developer-guide/mapstore-migration-guide.md'
      - Release: 'developer-guide/release.md'
    - How to:
      - Dev How To: 'developer-guide/dev-how-to.md'
      - Writing Plugins: 'developer-guide/plugins-howto.md'
      - Writing Epics: 'developer-guide/writing-epics.md'
      - Writing Actions and Reducers: 'developer-guide/writing-actions-reducers.md'
    - Configuration:
      - Configuration Files: 'developer-guide/configuration-files.md'
      - Application Configuration: 'developer-guide/local-config.md'
      - Configuring plugins: 'developer-guide/plugins-documentation.md'
      - Map Configuration: 'developer-guide/maps-configuration.md'
      - Externalize Configuration: 'developer-guide/externalized-configuration.md'
      - Configuration of Application Context Manager: 'developer-guide/context-editor-config.md'
    - Database Setup: 'developer-guide/database-setup.md'
    - Integrations:
      - Users:
        - GeoServer: 'developer-guide/integrations/users/geoserver.md'
        - LDAP: 'developer-guide/integrations/users/ldap.md'
        - OpenID Connect: 'developer-guide/integrations/users/openId.md'
        - Keycloak: 'developer-guide/integrations/users/keycloak.md'
        - Keycloak-SSO: 'developer-guide/integrations/users/keycloak-sso-impl.md'
      - General:
        - Authentication: 'developer-guide/integrations/auth.md'
        - Infrastructure: 'developer-guide/integrations/infrastructure-setups.md'
    - Projects:
      - MapStore Projects: 'developer-guide/mapstore-projects.md'
      - Project Creation Script: 'developer-guide/project-creation-script.md'
    - Communicating with MapStore:
      - API usage: 'developer-guide/API-usage.md'
      - MapViewer query paramaters: 'developer-guide/map-query-parameters.md'
  - Quick Start: 'quick-start.md'
plugins:
  - search
  - pdf-export:
      combined: true
      combined_output_path: pdf/mapstore_documentation.pdf
      enabled_if_env: ENABLE_PDF_EXPORT

About this issue

Original URL
State: closed
Created a year ago
Comments: 15 (8 by maintainers)

Most upvoted comments

At the end we noticed that the main issue was caused by the PDF generation together with lot of big gif files in the documentation, that proabably caused the a lot of memory usage and some stale during the PDF build. I don’t know, but I suspect that this happens with the new build.os value due to some python lib implementation, but honestly I didn’t investigated more on it.

During this investigation I was able to update mkdocs and material theme to the latest versions in my project, and to migrate all my gifs to mp4 files, so the build worked.

Anyway with the new libs, I have a blank first page, and the build took 500s to build, so we decided to temporarely disable PDF export for doc, and wait to re-enable it in the future, using some alternatives (like wtih-pdf or others, that from an experiment takes only ~90seconds and it seems to be more configurable too).

Thank you for the support.

offtherailz on Sep 1, 2023

It looked to me the same error as before. Here the new build is going to take around 350 seconds. Here I restarted it, but it didn’t finshed yet, after 10 minutes: https://readthedocs.org/projects/mapstore/builds/21781683/

So I cancelled it and I’m now trying to force current local versions :

mkdocs==1.5.2
mkdocs-material==3.3.0
jinja2==3.1.2
Markdown==3.4.4
git+https://github.com/geosolutions-it/mkdocs-pdf-export-plugin.git
WeasyPrint==52.5

But also here it is still pending

https://readthedocs.org/projects/mapstore/builds/21781830/

I’m running it locally enabling pdf generation and effectively it takes so long. I’m afraid the problem is only the PDF generation.

offtherailz on Aug 31, 2023

I didn’t find different at Python package level (different dependency versions, or similar). However, I found the build that’s passing is using Python 3.7 and the one that fails is using Python 3.10. I’m not sure that’s the root of the problem, but I’d recommend you to first make a PR that adds a .readthedocs.yaml file without introducing any change on the documentation nor on the Python environment. After that, update the Python version to 3.10 to confirm that’s not the problem and finally make the documentation changes.

Also, note that the build that I posted as passing, shows the following errors in the MkDocs command:

Error converting developer-guide/LayerFilter.md to PDF: 'developer-guide/LayerFilter/' is not in list
ERROR: No anchor #./quick-start/:quick-start for internal URI reference
ERROR: No anchor #./user-guide/home-page/:home-page for internal URI reference
ERROR: No anchor #./developer-guide/:quick-setup-and-run for internal URI reference
ERROR: No anchor #user-guide/layer-settings/./:display for internal URI reference
ERROR: No anchor #user-guide/layer-settings/./:display for internal URI reference
ERROR: No anchor #user-guide/layer-settings/./:visual-editor-style for internal URI reference
ERROR: No anchor #user-guide/layer-settings/./:visual-editor-style for internal URI reference
ERROR: No anchor #user-guide/measure/side-bar:3d-navigation for internal URI reference
ERROR: No anchor #user-guide/map-views/./:3d-views-navigations for internal URI reference
ERROR: No anchor #user-guide/map-views/./:3d-views-navigations for internal URI reference
ERROR: No anchor #developer-guide/plugins-howto/:jspluginsconnectedsamplejsx-2 for internal URI reference
ERROR: No anchor #developer-guide/plugins-howto/:jspluginsconnectedsamplejsx-2 for internal URI reference
ERROR: No anchor #user-guide/side-bar/:identify-tool for internal URI reference

I’m not sure if that’s related, but may be something to investigate as well.

humitos on Jul 31, 2023