tensorboard: Permission denied issue when writing to `/tmp/.tensorboard-info`

My workaround while this issue is resolved involves setting the TMPDIR environment variable. Since tensorboard uses tempfile which will respect user set environment variables (TMPDIR, TMP, etc.), this is possible. Make sure that the directory which TMPDIR points to exists!

You can change and test the new temp directory by running the following (which sets it to /tmp/$USER instead of /tmp).

export TMPDIR=/tmp/$USER; mkdir -p $TMPDIR; python -c "import tempfile; print(tempfile.gettempdir())"

A safe tensorboard invocation is:

export TMPDIR=/tmp/$USER; mkdir -p $TMPDIR; tensorboard --logdir $LOGDIR

@reactivetype: Thanks. It looks like your .tensorboard-info directory was created with an older version of TensorBoard, before the fix went in. You’ll need to manually change its permissions just this once:

sudo chmod 777 /tmp/.tensorboard-info

Hi @tete1030! Thanks for the clear report. This is a good point—I’d considered the multi-user case and determined that it wouldn’t be a problem for reading files in this directory, but didn’t realize that it would not be possible to write new files in the directory.

I think that the following patch should suffice, at least on Unices:

diff --git a/tensorboard/manager.py b/tensorboard/manager.py
index a86c010b..92f7601f 100644
--- a/tensorboard/manager.py
+++ b/tensorboard/manager.py
@@ -235,6 +235,7 @@ def _get_info_dir():
   The directory will be created if it does not exist.
   """
   path = os.path.join(tempfile.gettempdir(), ".tensorboard-info")
+  old_umask = os.umask(0o000)
   try:
     os.makedirs(path)
   except OSError as e:
@@ -242,6 +243,8 @@ def _get_info_dir():
       pass
     else:
       raise
+  finally:
+    os.umask(old_umask)
   return path
 

I’ll have to test this on Windows. If you’re looking for a quick fix, you should be able to patch your TensorBoard install as above. (Or just chmod a+w /tmp/.tensorboard-info, which will work until the next time that /tmp/ is cleared.)

Hey there, just stumbled upon this, I just thought I’d mention that this patch doesn’t fix the issue from the non-sudo user perspective. I added the patch first, but It did not fix the problem, as I do not have write access on that directory anyway. I’m gonna try and change the “.tensorboard-info” name to something unique, and hope this works. I’m thinking it might be nice to be able to customize this location, on a per-user basis ?

Yep, using TMPDIR in this format is specifically encouraged by POSIX. That’s a good workaround; thanks for sharing!

(You probably want to use either /tmp/selgs or /tmp/.selgs in both lines, and you can also just use TMPDIR=/tmp/selgs tensorboard to set an environment variable for one command; no need to use env, which spawns an extra process.)

@tete1030 Any update on this ? @wchargin Please let me know if you want me to keep this issue open until Windows test ?

@alwynmathew: Who owns the /tmp/.tensorboard-info directory? (Find out with stat --format=%U /tmp/.tensorboard-info.) That user—probably whoever created the directory—can chmod it or remove it.

@reactivetype: Thanks. It looks like your .tensorboard-info directory was created with an older version of TensorBoard, before the fix went in. You’ll need to manually change its permissions just this once:

sudo chmod 777 /tmp/.tensorboard-info

hey @wchargin I have installed tensorboard 1.14 but I face the same issues. Seems like I have a .tensorboard-info directory created with an older version of TensorBoard. As I’m not a sudoer, I dont have permission to delete or change the permission of /tmp/.tensorboard-info/. Is there a way to fix it without going to our admin requesting to delete or change the permission of /tmp/.tensorboard-info/?

@wchargin I am using ubuntu with tb-nightly 1.14.0a20190506 and I still have the issue.

File "envs/tf-nightly/lib/python3.6/site-packages/tensorboard/manager.py", line 269, in write_info_file
    with open(_get_info_file_path(), "w") as outfile:
PermissionError: [Errno 13] Permission denied: '/tmp/.tensorboard-info/pid-10141.info'