hardening: Jenkins service is not able to start up
Hi,
After the OS is harden and reboot. my Jenkins service never able to start up either during boot time or using systemd to start. Please see below for more information. I did try to disable auditd, aide and apparmor but it did not help. any suggestion will much appreciate. Thank you so much
jkwan@ubuntu:~$ ps -ef | grep -i jenkins
jkwan 1977 1964 0 19:08 pts/0 00:00:00 grep --color=auto -i jenkins
jkwan@ubuntu:~$ uptime
19:08:17 up 2 min, 2 users, load average: 0.26, 0.35, 0.16
jkwan@ubuntu:~$ uptime
19:08:21 up 2 min, 2 users, load average: 0.26, 0.35, 0.16
jkwan@ubuntu:~$ ps -ef | grep -i jenkins
jkwan 1982 1964 0 19:08 pts/0 00:00:00 grep --color=auto -i jenkins
jkwan@ubuntu:~$ sudo systemctl start jenkins.service
[sudo] password for jkwan:
Job for jenkins.service failed because the control process exited with error code. See "systemctl status jenkins.service" and "journalctl -xe" for details.
jkwan@ubuntu:~$ sudo systemctl status jenkins.service
● jenkins.service - LSB: Start Jenkins at boot time
Loaded: loaded (/etc/init.d/jenkins; bad; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2018-04-05 19:08:44 +08; 12s ago
Docs: man:systemd-sysv-generator(8)
Process: 1987 ExecStart=/etc/init.d/jenkins start (code=exited, status=7)
Apr 05 19:08:43 ubuntu jenkins[1987]: * Starting Jenkins Automation Server jenkins
Apr 05 19:08:43 ubuntu su[2007]: pam_tally(su:account): unknown option: reset
Apr 05 19:08:43 ubuntu su[2007]: Successful su for jenkins by root
Apr 05 19:08:43 ubuntu su[2007]: + ??? root:jenkins
Apr 05 19:08:43 ubuntu su[2007]: pam_unix(su:session): session opened for user jenkins by (uid=0)
Apr 05 19:08:44 ubuntu jenkins[1987]: ...fail!
Apr 05 19:08:44 ubuntu systemd[1]: jenkins.service: Control process exited, code=exited status=7
Apr 05 19:08:44 ubuntu systemd[1]: Failed to start LSB: Start Jenkins at boot time.
Apr 05 19:08:44 ubuntu systemd[1]: jenkins.service: Unit entered failed state.
Apr 05 19:08:44 ubuntu systemd[1]: jenkins.service: Failed with result 'exit-code'.
jkwan@ubuntu:~$ sudo journalctl -xe
Apr 05 19:08:56 ubuntu audit: PATH item=1 name="/lib64/ld-linux-x86-64.so.2" inode=390623 dev=fc:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
Apr 05 19:08:56 ubuntu audit: PROCTITLE proctitle=7375646F0073797374656D63746C00737461747573006A656E6B696E732E73657276696365
Apr 05 19:08:56 ubuntu sudo[2033]: jkwan : TTY=pts/0 ; PWD=/home/jkwan ; USER=root ; COMMAND=/bin/systemctl status jenkins.service
Apr 05 19:08:56 ubuntu audit[2033]: USER_CMD pid=2033 uid=1000 auid=1000 ses=2 msg='cwd="/home/jkwan" cmd=73797374656D63746C20737461747573206A656E6B696E732E73657276696365 terminal=pts/0 res=success'
Apr 05 19:08:56 ubuntu audit[2033]: CRED_REFR pid=2033 uid=0 auid=1000 ses=2 msg='op=PAM:setcred acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
Apr 05 19:08:56 ubuntu sudo[2033]: pam_unix(sudo:session): session opened for user root by jkwan(uid=0)
Apr 05 19:08:56 ubuntu audit[2034]: SYSCALL arch=c000003e syscall=83 success=no exit=-17 a0=40173c a1=1c9 a2=0 a3=1e5 items=1 ppid=2033 pid=2034 auid=1000 uid=0 gid=1000 euid=0 suid=0 fsuid=0 egid=1000 sgid=1000
Apr 05 19:08:56 ubuntu audit: CWD cwd="/home/jkwan"
Apr 05 19:08:56 ubuntu audit: PATH item=0 name="/tmp/" inode=2 dev=00:25 mode=041777 ouid=0 ogid=0 rdev=00:00 nametype=PARENT
Apr 05 19:08:56 ubuntu audit: PROCTITLE proctitle="/sbin/pam-tmpdir-helper"
Apr 05 19:08:56 ubuntu audit[2034]: SYSCALL arch=c000003e syscall=83 success=no exit=-17 a0=211a0b0 a1=1c0 a2=0 a3=0 items=1 ppid=2033 pid=2034 auid=1000 uid=0 gid=1000 euid=0 suid=0 fsuid=0 egid=1000 sgid=1000
Apr 05 19:08:56 ubuntu audit: CWD cwd="/home/jkwan"
Apr 05 19:08:56 ubuntu audit: PATH item=0 name="/tmp/user/" inode=10 dev=00:25 mode=040711 ouid=0 ogid=0 rdev=00:00 nametype=PARENT
Apr 05 19:08:56 ubuntu audit: PROCTITLE proctitle="/sbin/pam-tmpdir-helper"
Apr 05 19:08:56 ubuntu audit[2033]: USER_START pid=2033 uid=0 auid=1000 ses=2 msg='op=PAM:session_open acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
Apr 05 19:08:56 ubuntu audit[2035]: SYSCALL arch=c000003e syscall=59 success=yes exit=0 a0=5622c89784a8 a1=5622c896f168 a2=5622c8988aa0 a3=5622c898e000 items=2 ppid=2033 pid=2035 auid=1000 uid=0 gid=0 euid=0 sui
Apr 05 19:08:56 ubuntu audit: EXECVE argc=3 a0="systemctl" a1="status" a2="jenkins.service"
Apr 05 19:08:56 ubuntu audit: CWD cwd="/home/jkwan"
Apr 05 19:08:56 ubuntu audit: PATH item=0 name="/bin/systemctl" inode=151444 dev=fc:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
Apr 05 19:08:56 ubuntu audit: PATH item=1 name="/lib64/ld-linux-x86-64.so.2" inode=390623 dev=fc:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
Apr 05 19:08:56 ubuntu audit: PROCTITLE proctitle=73797374656D63746C00737461747573006A656E6B696E732E73657276696365
Apr 05 19:08:57 ubuntu sudo[2033]: pam_unix(sudo:session): session closed for user root
Apr 05 19:08:57 ubuntu audit[2033]: USER_END pid=2033 uid=0 auid=1000 ses=2 msg='op=PAM:session_close acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
Apr 05 19:08:57 ubuntu audit[2033]: CRED_DISP pid=2033 uid=0 auid=1000 ses=2 msg='op=PAM:setcred acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
Apr 05 19:08:58 ubuntu audit[1406]: SYSCALL arch=c000003e syscall=159 success=yes exit=5 a0=7f6fd8ef2bb0 a1=0 a2=862 a3=39fe6f46d items=0 ppid=1 pid=1406 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0
Apr 05 19:08:58 ubuntu audit: PROCTITLE proctitle="/usr/sbin/VBoxService"
Apr 05 19:09:08 ubuntu audit[2037]: SYSCALL arch=c000003e syscall=59 success=yes exit=0 a0=d0de48 a1=e03248 a2=de9008 a3=598 items=2 ppid=1964 pid=2037 auid=1000 uid=1000 gid=1000 euid=0 suid=0 fsuid=0 egid=1000
Apr 05 19:09:08 ubuntu audit: BPRM_FCAPS fver=0 fp=0000000000000000 fi=0000000000000000 fe=0 old_pp=0000000000000000 old_pi=0000000000000000 old_pe=0000000000000000 new_pp=0000003fffffffff new_pi=000000000000000
Apr 05 19:09:08 ubuntu audit: EXECVE argc=3 a0="sudo" a1="journalctl" a2="-xe"
Apr 05 19:09:08 ubuntu audit: CWD cwd="/home/jkwan"
Apr 05 19:09:08 ubuntu audit: PATH item=0 name="/usr/bin/sudo" inode=386 dev=fc:00 mode=0104755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
Apr 05 19:09:08 ubuntu audit: PATH item=1 name="/lib64/ld-linux-x86-64.so.2" inode=390623 dev=fc:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
Apr 05 19:09:08 ubuntu audit: PROCTITLE proctitle=7375646F006A6F75726E616C63746C002D7865
Apr 05 19:09:08 ubuntu sudo[2037]: jkwan : TTY=pts/0 ; PWD=/home/jkwan ; USER=root ; COMMAND=/bin/journalctl -xe
Apr 05 19:09:08 ubuntu audit[2037]: USER_CMD pid=2037 uid=1000 auid=1000 ses=2 msg='cwd="/home/jkwan" cmd=6A6F75726E616C63746C202D7865 terminal=pts/0 res=success'
Apr 05 19:09:08 ubuntu audit[2037]: CRED_REFR pid=2037 uid=0 auid=1000 ses=2 msg='op=PAM:setcred acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
Apr 05 19:09:08 ubuntu sudo[2037]: pam_unix(sudo:session): session opened for user root by jkwan(uid=0)
Apr 05 19:09:08 ubuntu audit[2038]: SYSCALL arch=c000003e syscall=83 success=no exit=-17 a0=40173c a1=1c9 a2=0 a3=1e5 items=1 ppid=2037 pid=2038 auid=1000 uid=0 gid=1000 euid=0 suid=0 fsuid=0 egid=1000 sgid=1000
Apr 05 19:09:08 ubuntu audit: CWD cwd="/home/jkwan"
Apr 05 19:09:08 ubuntu audit: PATH item=0 name="/tmp/" inode=2 dev=00:25 mode=041777 ouid=0 ogid=0 rdev=00:00 nametype=PARENT
Apr 05 19:09:08 ubuntu audit: PROCTITLE proctitle="/sbin/pam-tmpdir-helper"
Apr 05 19:09:08 ubuntu audit[2038]: SYSCALL arch=c000003e syscall=83 success=no exit=-17 a0=afd0b0 a1=1c0 a2=0 a3=0 items=1 ppid=2037 pid=2038 auid=1000 uid=0 gid=1000 euid=0 suid=0 fsuid=0 egid=1000 sgid=1000 f
Apr 05 19:09:08 ubuntu audit: CWD cwd="/home/jkwan"
Apr 05 19:09:08 ubuntu audit: PATH item=0 name="/tmp/user/" inode=10 dev=00:25 mode=040711 ouid=0 ogid=0 rdev=00:00 nametype=PARENT
Apr 05 19:09:08 ubuntu audit: PROCTITLE proctitle="/sbin/pam-tmpdir-helper"
Apr 05 19:09:08 ubuntu audit[2037]: USER_START pid=2037 uid=0 auid=1000 ses=2 msg='op=PAM:session_open acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
Apr 05 19:09:08 ubuntu audit[2039]: SYSCALL arch=c000003e syscall=59 success=yes exit=0 a0=5588bd7db4a8 a1=5588bd7d2168 a2=5588bd7eba70 a3=5588bd7f1000 items=2 ppid=2037 pid=2039 auid=1000 uid=0 gid=0 euid=0 sui
Apr 05 19:09:08 ubuntu audit: EXECVE argc=2 a0="journalctl" a1="-xe"
Apr 05 19:09:08 ubuntu audit: CWD cwd="/home/jkwan"
Apr 05 19:09:08 ubuntu audit: PATH item=0 name="/bin/journalctl" inode=151439 dev=fc:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
Apr 05 19:09:08 ubuntu audit: PATH item=1 name="/lib64/ld-linux-x86-64.so.2" inode=390623 dev=fc:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
Apr 05 19:09:08 ubuntu audit: PROCTITLE proctitle=6A6F75726E616C63746C002D7865
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 25 (11 by maintainers)
Install default jdk solved my problem
The issue seems to be that
systemd
kills the user processes when startingjenkins
withsystemctl
. I wrote a jenkins .service file and added the Jenkins user toKillExcludeUsers
in/etc/systemd/logind.conf
; https://gist.github.com/konstruktoid/1bc96c4f5030f37bd5f5142cc2718b35Hi @joharkwan, any logs when trying
/etc/init.d/jenkins start
or in/var/log/jenkins/
?Nice! Thanks a lot @konstruktoid konstruktoid c