tools: `nf-core` failing from conda on Mac Pro M1

Description of the bug

When trying to make a local module for my non-nf-core workflow, the module creation subcommand throws an error and so just calling nf-core itself. The command works fine in Gitpod, but not on my MacPro M1, so I’m guessing it’s something to do with the environment.

I made a fresh install of the running environment with:

mamba env create -f nextflow_conda-env.yml -n nextflow-env

where nextflow_conda-env.yml is:

channels:
    - conda-forge
    - bioconda
    - defaults
dependencies:
    - nextflow=22.04.0
    - nf-core=2.4.1
    - mamba=0.23.1

Package summary:

  Package                               Version  Build               Channel                     Size
───────────────────────────────────────────────────────────────────────────────────────────────────────
  Install:
───────────────────────────────────────────────────────────────────────────────────────────────────────

  + appdirs                               1.4.4  pyh9f0ad1d_0        conda-forge/noarch        Cached
  + attrs                                21.4.0  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + backports                               1.0  py_2                conda-forge/noarch        Cached
  + backports.functools_lru_cache         1.6.4  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + bleach                                5.0.0  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + boltons                              21.0.0  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + brotlipy                              0.7.0  py310hf8d0d8f_1004  conda-forge/osx-arm64     Cached
  + bzip2                                 1.0.8  h3422bc3_4          conda-forge/osx-arm64     Cached
  + c-ares                               1.18.1  h3422bc3_0          conda-forge/osx-arm64     Cached
  + ca-certificates                 2022.5.18.1  h4653dfc_0          conda-forge/osx-arm64     Cached
  + cattrs                               1.10.0  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + certifi                         2022.5.18.1  py310hbe9552e_0     conda-forge/osx-arm64     Cached
  + cffi                                 1.15.0  py310hf10583b_0     conda-forge/osx-arm64     Cached
  + charset-normalizer                   2.0.12  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + click                                 8.1.3  py310hbe9552e_0     conda-forge/osx-arm64     Cached
  + colorama                              0.4.4  pyh9f0ad1d_0        conda-forge/noarch        Cached
  + commonmark                            0.9.1  py_0                conda-forge/noarch        Cached
  + conda                                4.12.0  py310hbe9552e_0     conda-forge/osx-arm64     Cached
  + conda-package-handling                1.8.1  py310hf8d0d8f_1     conda-forge/osx-arm64     Cached
  + coreutils                               9.1  h1c322ee_0          conda-forge/osx-arm64     Cached
  + cryptography                         37.0.2  py310h94bb23d_0     conda-forge/osx-arm64       1 MB
  + curl                                 7.83.1  h2fcd78c_0          conda-forge/osx-arm64     145 KB
  + dataclasses                             0.8  pyhc8e2a94_3        conda-forge/noarch        Cached
  + docutils                             0.18.1  py310hbe9552e_1     conda-forge/osx-arm64     Cached
  + expat                                 2.4.8  h6b3803e_0          conda-forge/osx-arm64     Cached
  + future                               0.18.2  py310hbe9552e_5     conda-forge/osx-arm64     Cached
  + galaxy-containers                    20.5.0  py_0                bioconda/noarch           Cached
  + galaxy-tool-util                     20.9.1  py_0                bioconda/noarch           Cached
  + galaxy-util                          20.9.1  py_0                bioconda/noarch           Cached
  + gettext                            0.19.8.1  h049c9fb_1008       conda-forge/osx-arm64     Cached
  + git                                  2.35.3  pl5321h8b11ce0_0    conda-forge/osx-arm64      13 MB
  + gitdb                                 4.0.9  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + gitpython                            3.1.27  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + icu                                    70.1  h6b3803e_0          conda-forge/osx-arm64      13 MB
  + idna                                    3.3  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + importlib-metadata                   4.11.3  py310hbe9552e_1     conda-forge/osx-arm64     Cached
  + importlib_resources                   5.7.1  pyhd8ed1ab_1        conda-forge/noarch        Cached
  + iniconfig                             1.1.1  pyh9f0ad1d_0        conda-forge/noarch        Cached
  + itsdangerous                          2.1.2  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + jinja2                                3.1.2  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + jsonschema                            4.5.1  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + krb5                                 1.19.3  hf9b2bbe_0          conda-forge/osx-arm64     Cached
  + libarchive                            3.5.2  h69ec738_2          conda-forge/osx-arm64       2 MB
  + libcurl                              7.83.1  h2fcd78c_0          conda-forge/osx-arm64     300 KB
  + libcxx                               14.0.3  h6a5c8ee_0          conda-forge/osx-arm64     Cached
  + libedit                        3.1.20191231  hc8eb9b7_2          conda-forge/osx-arm64     Cached
  + libev                                  4.33  h642e427_1          conda-forge/osx-arm64     Cached
  + libffi                                3.4.2  h3422bc3_5          conda-forge/osx-arm64     Cached
  + libiconv                               1.16  h642e427_0          conda-forge/osx-arm64     Cached
  + libmamba                             0.23.1  h1c735bf_1          conda-forge/osx-arm64       1 MB
  + libmambapy                           0.23.1  py310h19f903a_1     conda-forge/osx-arm64     206 KB
  + libnghttp2                           1.47.0  he723fca_0          conda-forge/osx-arm64     Cached
  + libsolv                              0.7.22  h1280f1d_0          conda-forge/osx-arm64     386 KB
  + libssh2                              1.10.0  hb80f160_2          conda-forge/osx-arm64     Cached
  + libxml2                              2.9.14  h035c1df_0          conda-forge/osx-arm64     655 KB
  + libzlib                              1.2.11  h90dfc92_1014       conda-forge/osx-arm64     Cached
  + lz4-c                                 1.9.3  hbdafb3b_1          conda-forge/osx-arm64     Cached
  + lzo                                    2.10  h642e427_1000       conda-forge/osx-arm64     Cached
  + mamba                                0.23.1  py310h25b57eb_1     conda-forge/osx-arm64      46 KB
  + markdown                              3.3.7  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + markupsafe                            2.1.1  py310hf8d0d8f_1     conda-forge/osx-arm64     Cached
  + ncurses                                 6.3  h07bb92c_1          conda-forge/osx-arm64     Cached
  + nextflow                            22.04.0  h4a94de4_0          bioconda/noarch           Cached
  + nf-core                               2.4.1  pyh5e36f6f_0        bioconda/noarch           Cached
  + openjdk                            11.0.9.1  he8acd94_3          conda-forge/osx-arm64     Cached
  + openssl                              1.1.1o  ha287fd2_0          conda-forge/osx-arm64       2 MB
  + packaging                              21.3  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + pcre2                                 10.37  hc5de063_0          conda-forge/osx-arm64     Cached
  + perl                                 5.32.1  2_h9b22ae9_perl5    conda-forge/osx-arm64     Cached
  + pip                                  22.0.4  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + pluggy                                1.0.0  py310hbe9552e_3     conda-forge/osx-arm64     Cached
  + prompt-toolkit                       3.0.29  pyha770c72_0        conda-forge/noarch        Cached
  + prompt_toolkit                       3.0.29  hd8ed1ab_0          conda-forge/noarch          5 KB
  + py                                   1.11.0  pyh6c4a22f_0        conda-forge/noarch        Cached
  + pybind11-abi                              4  hd8ed1ab_3          conda-forge/noarch         10 KB
  + pycosat                               0.6.3  py310hf8d0d8f_1010  conda-forge/osx-arm64     Cached
  + pycparser                              2.21  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + pygments                             2.12.0  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + pyopenssl                            22.0.0  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + pyparsing                             3.0.9  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + pyrsistent                           0.18.1  py310hf8d0d8f_1     conda-forge/osx-arm64     Cached
  + pysocks                               1.7.1  py310hbe9552e_5     conda-forge/osx-arm64     Cached
  + pytest                                7.1.2  py310hbe9552e_0     conda-forge/osx-arm64     Cached
  + pytest-workflow                       1.6.0  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + python                               3.10.4  hfc7342c_0_cpython  conda-forge/osx-arm64      12 MB
  + python_abi                             3.10  2_cp310             conda-forge/osx-arm64       4 KB
  + pyyaml                                  6.0  py310hf8d0d8f_4     conda-forge/osx-arm64     Cached
  + questionary                          1.10.0  pyhd8ed1ab_1        conda-forge/noarch        Cached
  + readline                                8.1  hedafd6a_0          conda-forge/osx-arm64     Cached
  + repoze.lru                              0.7  py_0                conda-forge/noarch        Cached
  + reproc                               14.2.3  h3422bc3_0          conda-forge/osx-arm64     Cached
  + reproc-cpp                           14.2.3  hbdafb3b_0          conda-forge/osx-arm64     Cached
  + requests                             2.27.1  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + requests-cache                        0.9.4  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + rich                                 12.4.1  pyhd8ed1ab_1        conda-forge/noarch        Cached
  + rich-click                              1.4  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + routes                                2.5.1  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + ruamel_yaml                         0.15.80  py310he2143c4_1006  conda-forge/osx-arm64     Cached
  + setuptools                           62.3.2  py310hbe9552e_0     conda-forge/osx-arm64     Cached
  + six                                  1.16.0  pyh6c4a22f_0        conda-forge/noarch        Cached
  + smmap                                 3.0.5  pyh44b312d_0        conda-forge/noarch        Cached
  + sqlite                               3.38.5  h40dfcc0_0          conda-forge/osx-arm64     Cached
  + tabulate                              0.8.9  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + tk                                   8.6.12  he1e0b03_0          conda-forge/osx-arm64     Cached
  + tomli                                 2.0.1  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + tqdm                                 4.64.0  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + typing_extensions                     4.2.0  pyha770c72_1        conda-forge/noarch        Cached
  + tzdata                                2022a  h191b570_0          conda-forge/noarch        Cached
  + url-normalize                         1.4.3  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + urllib3                              1.26.9  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + wcwidth                               0.2.5  pyh9f0ad1d_2        conda-forge/noarch        Cached
  + webencodings                          0.5.1  py_1                conda-forge/noarch        Cached
  + wheel                                0.37.1  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + whoosh                                2.7.4  pyhd8ed1ab_2        conda-forge/noarch        Cached
  + xz                                    5.2.5  h642e427_1          conda-forge/osx-arm64     Cached
  + yaml                                  0.2.5  h3422bc3_2          conda-forge/osx-arm64     Cached
  + yaml-cpp                              0.6.3  hc88da5d_4          conda-forge/osx-arm64     122 KB
  + zipp                                  3.8.0  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + zlib                                 1.2.11  h90dfc92_1014       conda-forge/osx-arm64     Cached
  + zstd                                  1.5.2  hd705a24_1          conda-forge/osx-arm64     374 KB

Command used and terminal output

% nf-core modules create mash/screen             
Traceback (most recent call last):
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/bin/nf-core", line 7, in <module>
    from nf_core.__main__ import run_nf_core
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/__main__.py", line 19, in <module>
    import nf_core.lint
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/lint/__init__.py", line 22, in <module>
    import nf_core.modules.lint
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/modules/__init__.py", line 12, in <module>
    from .mulled import MulledImageNameGenerator
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/modules/mulled.py", line 10, in <module>
    from galaxy.tool_util.deps.mulled.util import build_target, v2_image_name
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/galaxy/tool_util/deps/__init__.py", line 16, in <module>
    from galaxy.util.oset import OrderedSet
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/galaxy/util/oset.py", line 7, in <module>
    class OrderedSet(collections.MutableSet):
AttributeError: module 'collections' has no attribute 'MutableSet'

% nf-core   
Traceback (most recent call last):
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/bin/nf-core", line 7, in <module>
    from nf_core.__main__ import run_nf_core
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/__main__.py", line 19, in <module>
    import nf_core.lint
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/lint/__init__.py", line 22, in <module>
    import nf_core.modules.lint
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/modules/__init__.py", line 12, in <module>
    from .mulled import MulledImageNameGenerator
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/modules/mulled.py", line 10, in <module>
    from galaxy.tool_util.deps.mulled.util import build_target, v2_image_name
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/galaxy/tool_util/deps/__init__.py", line 16, in <module>
    from galaxy.util.oset import OrderedSet
  File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/galaxy/util/oset.py", line 7, in <module>
    class OrderedSet(collections.MutableSet):
AttributeError: module 'collections' has no attribute 'MutableSet'

System information

% nextflow info
  Version: 22.04.0 build 5697
  Created: 23-04-2022 18:00 UTC (20:00 CEST)
  System: Mac OS X 12.2.1
  Runtime: Groovy 3.0.10 on OpenJDK 64-Bit Server VM 11.0.9.1+1-LTS
  Encoding: UTF-8 (UTF-8)

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 30 (30 by maintainers)

Most upvoted comments

Pinning python to 3.8 seems to fix the issue:

conda environment file:

channels:
  - conda-forge
  - bioconda
  - defaults
dependencies:
  - python=3.8
  - nextflow=22.04.5
  - nf-core=2.5
  - mamba=0.25.0

fresh install:

mamba env create -f nextflow-env.yml -n test_nf-core

New workflow creation works:

% nf-core create   

                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 2.5 - https://nf-co.re


? Workflow name test
? Description tester
? Author me
? Do you want to customize which parts of the template are used? No
INFO     Creating new nf-core pipeline: 'nf-core/test'                                                                                         create.py:236
INFO     Initialising pipeline git repository                                                                                                  create.py:538
INFO     Done. Remember to add a remote and push to GitHub:                                                                                    create.py:545
          cd /Users/mahpa906/Documents/Sandbox/Bash/test_conda_nfcore/nf-core-test                                                                          
          git remote add origin git@github.com:USERNAME/REPO_NAME.git                                                                                       
          git push --all origin                                                                                                                             
INFO     This will also push your newly created dev branch and the TEMPLATE branch for syncing.                                                create.py:551
INFO     !!!!!! IMPORTANT !!!!!!                                                                                                               create.py:227
                                                                                                                                                            
         If you are interested in adding your pipeline to the nf-core community,                                                                            
         PLEASE COME AND TALK TO US IN THE NF-CORE SLACK BEFORE WRITING ANY CODE!                                                                           
                                                                                                                                                            
         Please read: https://nf-co.re/developers/adding_pipelines#join-the-community

and so does module creation:

% nf-core modules create samtools/tester

                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 2.5 - https://nf-co.re


INFO     Repository type: modules                                                                                                               create.py:93
INFO     Press enter to use default values (shown in brackets) or type your own responses. ctrl+click underlined text to open links.            create.py:97
INFO     Using Bioconda package: 'bioconda::samtools=1.15.1'                                                                                   create.py:165
INFO     Using Docker container: 'quay.io/biocontainers/samtools:1.15.1--h1170115_0'                                                           create.py:191
INFO     Using Singularity container: 'https://depot.galaxyproject.org/singularity/samtools:1.15.1--h1170115_0'                                create.py:192
GitHub Username: (@author): @me
INFO     Provide an appropriate resource label for the process, taken from the nf-core pipeline template.                                      create.py:218
         For example: process_low, process_medium, process_high, process_long                                                                               
? Process resource label: process_low
INFO     Where applicable all sample-specific information e.g. 'id', 'single_end', 'read_group' MUST be provided as an input via a Groovy Map  create.py:232
         called 'meta'. This information may not be required in some instances, for example indexing reference genome files.                                
Will the module require a meta map of sample information? [y/n] (y): 
INFO     Created / edited following files:                                                                                                     create.py:270
           ./modules/samtools/tester/main.nf                                                                                                                
           ./modules/samtools/tester/meta.yml                                                                                                               
           ./tests/modules/samtools/tester/main.nf                                                                                                          
           ./tests/modules/samtools/tester/test.yml                                                                                                         
           ./tests/modules/samtools/tester/nextflow.config                                                                                                  
           ./tests/config/pytest_modules.yml

mahesh-panchal on Aug 31, 2022

% conda create --name tst-glxy python=3.8
<success>
% conda activate tst-glxy
<success>
% conda install --channel base --channel bioconda --channel conda-forge galaxy-tool-util
<success>
% python -c "from galaxy.util.oset import OrderedSet"

So last step seems to have worked.

mahesh-panchal on Aug 31, 2022