DeepSpeed: Can't install Deepspeed

I also have this issue on ubuntu 22.04

triton 1.0.0 isnt aviable anymore

pip install triton==1.0.0
ERROR: Could not find a version that satisfies the requirement triton==1.0.0 (from versions: 2.0.0.dev20221030, 2.0.0.dev20221031, 2.0.0.dev20221101, 2.0.0.dev20221103, 2.0.0.dev20221105, 2.0.0.dev20221117, 2.0.0.dev20221120, 2.0.0.dev20221202, 2.0.0.dev20230208, 2.0.0.dev20230217, 2.0.0a1, 2.0.0a2, 2.0.0, 2.0.0.post1)
ERROR: No matching distribution found for triton==1.0.0

Thanks, with the steps provided here https://github.com/microsoft/DeepSpeed and afterwards pip install “deepspeed-0.9.1+cc67f22f-py3-none-any.whl” it finally worked for me

@PleezDeez You are seeing an error with the DS_BUILD_OPS environment variable because setting environment variables in Windows Command Prompt or PowerShell is different than on linux: Linux: export DS_BUILD_OPS=0 Command Prompt: set DS_BUILD_OPS=0 PowerShell: $Env:DS_BUILD_OPS=0

Please try setting DS_BUILD_OPS=0 and DS_BUILD_AIO=0 and you will not need to worry about libaio.so

@AndreyRGW I just re-read your original post and noted that you’re on Windows 11. I believe that’s why the versions of triton differ in our pip output.

I don’t currently have access to a system with Windows and GPUs, so I can’t test the install process completely… but in powershell, I can avoid the error you are seeing by disabling pre-compile altogether:

$env:DS_BUILD_OPS = 0
pip install deepspeed

Our support for Windows is something we’re working on, so if this doesn’t work you could also try using Windows Subsystem for Linux (WSL) and install deepspeed in the linux environment that provides.

windows上，我在这里https://pypi.org/project/deepspeed/#files 下载了对应的包，解压之后直接放在虚拟环境里可以成功

Same error: `(venv) C:\Python\venv\Scripts>pip install deepspeed==0.8.3 Collecting deepspeed==0.8.3 Downloading deepspeed-0.8.3.tar.gz (765 kB) ---------------------------------------- 765.4/765.4 kB 4.4 MB/s eta 0:00:00 Preparing metadata (setup.py) … error error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [19 lines of output] test.c LINK : fatal error LNK1181: Eingabedatei “aio.lib” kann nicht ge”ffnet werden. fatal: not a git repository (or any of the parent directories): .git Traceback (most recent call last): File “<string>”, line 2, in <module> File “<pip-setuptools-caller>”, line 34, in <module> File “C:\Users\Sascha\AppData\Local\Temp\pip-install-_o9duc57\deepspeed_fbec153d955b4244ae7a50c9468c627f\setup.py”, line 198, in <module> create_dir_symlink(‘…\…\csrc’, ‘.\deepspeed\ops\csrc’) File “C:\Users\Sascha\AppData\Local\Temp\pip-install-_o9duc57\deepspeed_fbec153d955b4244ae7a50c9468c627f\setup.py”, line 190, in create_dir_symlink os.remove(dest) PermissionError: [WinError 5] Zugriff verweigert: ‘.\deepspeed\ops\csrc’ [WARNING] Torch did not find cuda available, if cross-compiling or running with cpu only you can ignore this message. Adding compute capability for Pascal, Volta, and Turing (compute capabilities 6.0, 6.1, 6.2) DS_BUILD_OPS=0 [WARNING] async_io requires the dev libaio .so object and headers but these were not found. [WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found. [WARNING] sparse_attn cuda is not available from torch [WARNING] sparse_attn requires a torch version >= 1.5 but detected 2.0 [WARNING] please install triton==1.0.0 if you want to use sparse attention Install Ops={‘async_io’: False, ‘cpu_adagrad’: False, ‘cpu_adam’: False, ‘fused_adam’: False, ‘fused_lamb’: False, ‘quantizer’: False, ‘random_ltd’: False, ‘sparse_attn’: False, ‘spatial_inference’: False, ‘transformer’: False, ‘stochastic_transformer’: False, ‘transformer_inference’: False, ‘utils’: False} [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.`

I use Windows 11, which one do you use @AndreyRGW ? And how did you install deepspeed? I used cmd in Admin mode

Same problem in Windows 11

`pip install deepspeed Collecting deepspeed Using cached deepspeed-0.8.3.tar.gz (765 kB) Preparing metadata (setup.py) … error error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [27 lines of output] No CUDA runtime is found, using CUDA_HOME=‘C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1’ test.c Microsoft ® Incremental Linker Version 14.35.32216.1 Copyright © Microsoft Corporation. All rights reserved.

  LINK : warning LNK4044: unrecognized option '/L'; ignored
  LINK : fatal error LNK1181: cannot open input file 'C:PythonLibsite-packages.obj'
  'git' is not recognized as an internal or external command,
  operable program or batch file.
  'git' is not recognized as an internal or external command,
  operable program or batch file.
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "C:\Users\vonpl\AppData\Local\Temp\pip-install-dyziex_l\deepspeed_29dbd8be473f4dcbae7b33482fe3329b\setup.py", line 198, in <module>
      create_dir_symlink('..\\..\\csrc', '.\\deepspeed\\ops\\csrc')
    File "C:\Users\vonpl\AppData\Local\Temp\pip-install-dyziex_l\deepspeed_29dbd8be473f4dcbae7b33482fe3329b\setup.py", line 190, in create_dir_symlink
      os.remove(dest)
  PermissionError: [WinError 5] Access is denied: '.\\deepspeed\\ops\\csrc'
  [WARNING] Torch did not find cuda available, if cross-compiling or running with cpu only you can ignore this message. Adding compute capability for Pascal, Volta, and Turing (compute capabilities 6.0, 6.1, 6.2)
  DS_BUILD_OPS=0
   [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
   [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
   [WARNING]  sparse_attn cuda is not available from torch
   [WARNING]  sparse_attn requires a torch version >= 1.5 but detected 2.0
   [WARNING]  please install triton==1.0.0 if you want to use sparse attention
  Install Ops={'async_io': False, 'cpu_adagrad': False, 'cpu_adam': False, 'fused_adam': False, 'fused_lamb': False, 'quantizer': False, 'random_ltd': False, 'sparse_attn': False, 'spatial_inference': False, 'transformer': False, 'stochastic_transformer': False, 'transformer_inference': False, 'utils': False}
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.`

I also get the same error:

(venv) mzapf@aiworker:~$ DS_BUILD_OPS=1 pip install deepspeed
Collecting deepspeed
  Downloading deepspeed-0.8.2.tar.gz (763 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 763.5/763.5 KB 4.9 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [13 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-vhg78wwk/deepspeed_c3ae6d59752540238f6f4b52f0b123f3/setup.py", line 156, in <module>
          abort(f"Unable to pre-compile {op_name}")
        File "/tmp/pip-install-vhg78wwk/deepspeed_c3ae6d59752540238f6f4b52f0b123f3/setup.py", line 48, in abort
          assert False, msg
      AssertionError: Unable to pre-compile sparse_attn
      DS_BUILD_OPS=1
       [WARNING]  sparse_attn requires a torch version >= 1.5 but detected 2.0
       [WARNING]  using untested triton version (2.0.0), only 1.0.0 is known to be compatible
       [WARNING]  One can disable sparse_attn with DS_BUILD_SPARSE_ATTN=0
       [ERROR]  Unable to pre-compile sparse_attn
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

@AndreyRGW try updating pip ( pip install --upgrade pip) and you should see the latest versions of triton:

venv ❯ pip index versions triton
triton (1.1.1)
Available versions: 1.1.1, 1.1.0, 1.0.0, 0.4.2, 0.4.1, 0.3.0, 0.2.3, 0.2.2, 0.2.1, 0.2.0, 0.1.3, 0.1.2, 0.1.1, 0.1
  INSTALLED: 1.0.0
  LATEST:    1.1.1

I am using the latest version of pip (22.2.2), but triton 1.0.0 still does not appear.