gpustat: Error on calling nvidia-smi: Command 'ps ...' returned non-zero exit status 1

got above error msg when i run gpustat. but nvidia-smi works on my machine here are some details OS:Ubuntu 14.04.5 LTS Python Version: anaconda3.6

Error on calling nvidia-smi. Use --debug flag for details
Traceback (most recent call last):
  File "/usr/local/bin/gpustat", line 417, in print_gpustat                                                      gpu_stats = GPUStatCollection.new_query()
  File "/usr/local/bin/gpustat", line 245, in new_query
    return GPUStatCollection(gpu_list)
  File "/usr/local/bin/gpustat", line 218, in __init__
    self.update_process_information()
  File "/usr/local/bin/gpustat", line 316, in update_process_information
    processes = self.running_processes()
  File "/usr/local/bin/gpustat", line 275, in running_processes
    ','.join(map(str, pid_map.keys()))
  File "/usr/local/bin/gpustat", line 46, in execute_process
    stdout = check_output(command_shell, shell=True).strip()
  File "/home/xiyun/apps/anaconda3/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/home/xiyun/apps/anaconda3/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'ps -o pid,user:16,comm -p1 -p 14471' returned non-zero exit status 1.

how can i fix this ?

About this issue

Original URL
State: closed
Created 7 years ago
Reactions: 1
Comments: 20 (10 by maintainers)

Most upvoted comments

sorry about that, ~V~. gpustat --debug

Error on querying NVIDIA devices. Use --debug flag for details
Traceback (most recent call last):
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/psutil/_pslinux.py", line 1332, in wrapper
    return fun(self, *args, **kwargs)
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/psutil/_pslinux.py", line 1506, in create_time
    values = self._parse_stat_file()
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/psutil/_common.py", line 313, in wrapper
    return fun(self)
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/psutil/_pslinux.py", line 1367, in _parse_stat_file
    with open_binary("%s/%s/stat" % (self._procfs_path, self.pid)) as f:
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/psutil/_pslinux.py", line 190, in open_binary
    return open(fname, "rb", **kwargs)
FileNotFoundError: [Errno 2] No such file or directory: '/proc/22919/stat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/psutil/__init__.py", line 408, in _init
    self.create_time()
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/psutil/__init__.py", line 734, in create_time
    self._create_time = self._proc.create_time()
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/psutil/_pslinux.py", line 1338, in wrapper
    raise NoSuchProcess(self.pid, self._name)
psutil.NoSuchProcess: psutil.NoSuchProcess process no longer exists (pid=22919)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/gpustat.py", line 371, in print_gpustat
    gpu_stats = GPUStatCollection.new_query()
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/gpustat.py", line 284, in new_query
    gpu_info = get_gpu_info(handle)
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/gpustat.py", line 260, in get_gpu_info
    process = get_process_info(nv_process.pid)
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/gpustat.py", line 221, in get_process_info
    ps_process = psutil.Process(pid=pid)
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/psutil/__init__.py", line 381, in __init__
    self._init(pid)
  File "/home/xiyun/apps/anaconda3/lib/python3.6/site-packages/psutil/__init__.py", line 421, in _init
    raise NoSuchProcess(pid, None, msg)
psutil.NoSuchProcess: psutil.NoSuchProcess no process found with pid 22919

feiwofeifeixiaowo on Sep 6, 2017