DeepSpeed: [BUG] "Setting ds_accelerator" uses `print` not `logger`

Describe the bug When accelerate tries to run any command while DeepSpeed is available, we get "Setting ds_accelerator ...", which while not explicitly bad, is making a test fail due to a non-clean CLI print when non-deepspeed code is used (we have a check for is_deepspeed_available() which is probably triggering this.

As a nice QOL, it would be good to have get_accelerator (https://github.com/microsoft/DeepSpeed/blob/master/accelerator/real_accelerator.py#L102) and set_accelerator (https://github.com/microsoft/DeepSpeed/blob/master/accelerator/real_accelerator.py#L109) use the already-existing logger in the framework if possible, so that we can disable these annoying prints when theyโ€™re not needed ๐Ÿ˜ƒ

To Reproduce Steps to reproduce the behavior:

  1. pip install accelerate deepspeed -U
  2. Create a file with:
from accelerate.commands.tpu import tpu_command_parser, tpu_command_launcher

parser = tpu_command_parser()
args = parser.parse_args([
    "--config_file", "tests/test_configs/latest.yaml",
    "--install_accelerate",
    "--debug"
])

tpu_command_launcher(args)
  1. Run python {my_file_name.py}
  2. Should print:
Setting ds_accelerator to cuda (auto detect)
Running gcloud compute tpus tpu-vm ssh test-tpu --zone us-central1-a --command cd /usr/share; pip install accelerate -U; echo "hello world"; echo "this is a second command" --worker all

Expected behavior

A configurable option to silence these print statements by having them run through the logging system instead, perhaps as an info or as a debug.

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 21 (9 by maintainers)

Most upvoted comments

@tjruwase I have update to 0.94. And There are still too much message โ€œ[real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect)โ€ printed in std io.

The bug still exists with version 0.10.2

@tjruwase nope, not with DeepSpeed yet (@pacman100 on that one with Accelerate?) itโ€™s on a GPU runner, just our CLI picks up on it due to import (which was also a great way to know that something weird was happening ๐Ÿ˜„ )

No worries. Thanks for reporting the error. I will link a PR asap.

@tjruwase sadly donโ€™t have the bandwidth/familiarity with the code right now to do so, would appreciate it if someone else could pick it up ๐Ÿ™