readthedocs.org: Fail to run `build.jobs` with single quotes (`'`) in commands

Details

Expected Result

A description of what you wanted to happen

Run build.jobs command successfully

My .readthedocs.yaml:

# .readthedocs.yaml
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Set the version of Python and other tools you might need
build:
  os: ubuntu-22.04
  tools:
    python: "3.8"
  jobs:
    post_install:
      - sed -i -E 's/^     process identity for every yielded instance$/  \0/' "$(python3 -c "print(__import__('psutil').__file__)")"

# Build documentation in the docs/ directory with Sphinx
sphinx:
  builder: html
  configuration: docs/source/conf.py
  fail_on_warning: true

# Optionally declare the Python requirements required to build your docs
python:
  install:
    - requirements: requirements.txt
    - requirements: docs/requirements.txt
  system_packages: true

I want to run command:

sed -i -E 's/^     process identity for every yielded instance$/  \0/' "$(python3 -c "print(__import__('psutil').__file__)")"

after installing the dependencies.

Actual Result

A description of what actually happened

Got errors in raw output:

[rtd-command-info] start-time: 2022-07-02T10:39:42.282806Z, end-time: 2022-07-02T10:39:42.362971Z, duration: 0, exit-code: 1
sed -i -E 's/^ process identity for every yielded instance$/ \0/' "$(python3 -c "print(__import__('psutil').__file__)")"
sed: -e expression #1, char 3: unterminated `s' command

In:

https://github.com/readthedocs/readthedocs.org/blob/62effc771a6171f9252dd8bbaf779cf8c7d0a975/readthedocs/doc_builder/director.py#L339-L341

we split the command with whitespace into a list of command-line options. In my use case, I have:

>>> command = r'''sed -i -E 's/^     process identity for every yielded instance$/  \0/' "$(python3 -c "print(__import__('psutil').__file__)")"'''
>>> print(command)
sed -i -E 's/^     process identity for every yielded instance$/  \0/' "$(python3 -c "print(__import__('psutil').__file__)")"

>>> command.split()  # split with whitespace
[
    'sed',
    '-i',
    '-E',
    "'s/^",
    'process',
    'identity',
    'for',
    'every',
    'yielded',
    'instance$/',
    "\\0/'",
    '"$(python3',
    '-c',
    '"print(__import__(\'psutil\').__file__)")"'
]

Expected split:

[
    'sed',
    '-i',
    '-E',
    "'s/^     process identity for every yielded instance$/  \\0/'",
    '"$(python3 -c "print(__import__(\'psutil\').__file__)")"'
]

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 15 (9 by maintainers)

Most upvoted comments

would be overkill for this, but good to know it’s possible

https://github.com/indico/indico/commit/420ddfb8d42f142f67ca94957ee3a87e0df70e21

    pre_install:
      # RTD python versions are usually behind a bit...
      - sed -i -E "/^python_requires = /d" setup.cfg

This is related to #10103 and it may be fixed by the PR associated as well.

I thought it was a somewhat simple bug fix when I saw it, related to the builders. I definitely don’t want to run things with shell=True, if that’s what is required.

As a workaround, I created a single Bash script fix-psutil-docstring.sh:

#!/usr/bin/env bash

# shellcheck disable=SC2312
exec sed -i -E 's/^     process identity for every yielded instance$/  \0/' "$(python3 -c "print(__import__('psutil').__file__)")"

Then put:

build:
  os: ubuntu-22.04
  tools:
    python: "3.8"
  jobs:
    post_install:
      - bash docs/source/fix-psutil-docstring.sh

in .readthedocs.yaml.