gogs: Cannot add SSH key when running via systemd

OS: CentOS 7. I have a systemd script setup to start gogs, when I am running via that systemd script I cannot add ssh keys and get the following error:

Sorry, we’re not able to verify your SSH key: ssh-keygen -lf: ssh-keygen: /tmp/keytest422813223: Permission denied

If I run the service by hand it works fine: sudo ./gogs web

Here is my systemd service:

[Unit]
Description=Gogs (Go Git Service)
After=syslog.target
After=network.target
#After=mysqld.service
#After=postgresql.service
#After=memcached.service
#After=redis.service

[Service]
Type=simple
User=root
Group=root
WorkingDirectory=/home/gogs/gogs
ExecStart=/home/gogs/gogs/gogs web
Restart=always
Environment=USER=root HOME=/home/gogs

[Install]
WantedBy=multi-user.target

Originally I had it setup to use the gogs user, but for testing I switched it to root. Still no luck.