singularity-hpc: Unable to set container_base in config

OS Version: Red Hat Enterprise Linux 8.8 (Ootpa)
Kernel: 4.18.0-477.21.1.el8_8.x86_64
System Python3: 3.6.8
Lmod version: Release 4.5.2 (2020-07-30)
singularity-ce version 3.11.5-1.el8
Installing shpc within a venv environment from the public git repo
- shpc version: 0.1.26

On my single node server, I am unable to specify a new location to put pulled & managed container images. shpc config set container_base /usr/share/shpc/image_files/

The error I get is below, which is rather strange as I have successfully done this on an el9 system with Python 3.9 using the same release version (0.1.26). And I set up that machine yesterday, so I don’t think anything has changed within ~24 hours to have borked this functionality.

  File "/opt/shpc/bin/shpc", line 11, in <module>
    load_entry_point('singularity-hpc==0.1.26', 'console_scripts', 'shpc')()
  File "/opt/shpc/lib64/python3.6/site-packages/shpc/client/__init__.py", line 556, in run_shpc
    main(args=args, parser=parser, extra=extra, subparser=helper)
  File "/opt/shpc/lib64/python3.6/site-packages/shpc/client/config.py", line 31, in main
    quiet=args.quiet, settings_file=args.settings_file, validate=validate
  File "/opt/shpc/lib64/python3.6/site-packages/shpc/main/__init__.py", line 36, in get_client
    from shpc.main.modules.lmod import Client
  File "/opt/shpc/lib64/python3.6/site-packages/shpc/main/modules/lmod.py", line 5, in <module>
    from .base import ModuleBase
  File "/opt/shpc/lib64/python3.6/site-packages/shpc/main/modules/base.py", line 14, in <module>
    import shpc.main.container as container
  File "/opt/shpc/lib64/python3.6/site-packages/shpc/main/container/__init__.py", line 1, in <module>
    from .config import ContainerConfig
  File "/opt/shpc/lib64/python3.6/site-packages/shpc/main/container/config.py", line 8, in <module>
    import shpc.main.container.update as update
AttributeError: module 'shpc.main' has no attribute 'container'

About this issue

Original URL
State: open
Created 9 months ago
Comments: 20 (13 by maintainers)

Most upvoted comments

haha but @marcodelapierre … it’s so Groovy!!!

groovy-baby

Can’t help myself, haha. I’ve definitely made that joke one too many times…

vsoch on Oct 17, 2023

Look at some of these new beautiful features in 3.9 (which is already kind of old? I have 3.11 on my system). So nice though!

I know I love programming because I still get excited/happy about the small things. ❤️

vsoch on Oct 17, 2023

We should update here, which is the standard place to put that: https://github.com/singularityhub/singularity-hpc/blob/e1e66adeb56035ce4b3c933b96fe2963df4e9d09/setup.py#L103

I very likely copy pasted that metadata from elsewhere. I’ll update to 3.7 shortly.

vsoch on Oct 14, 2023

it looks like you installed shpc to site packages

So the oddity the above comment is referencing just clicked in my head.

I pip install git+https://url.git directly into a python venv located at /opt/shpc
Set the container_base and module_base to be NOT within that directory structure.
BUT, I did not heed the warning about moving the wrapper_base as well.

Hence

[cloud-user@minicondor ~]$ module load shpc/julia
[cloud-user@minicondor ~]$ which julia
/opt/shpc/lib64/python3.9/site-packages/modules/julia/1.9.3/bin/julia

which will indeed make updating the shpc repo a pain. So will definitely be changing that.

mtwest2718 on Oct 14, 2023