Out of a sudden, we have noticed that all our CentOS clusters, which are running httpd, are no longer running auditd and firewalld. Status of systemctrl –failed is:<br />
<br />
[<a href="mailto:root@cpuk">root@cpuk</a> ~]# systemctl --failed<br />
UNIT LOAD ACTIVE SUB DESCRIPTION<br />
● auditd.service loaded failed failed Security Auditing Service<br />
● firewalld.service loaded failed failed firewalld - dynamic firewall daemon<br />
<br />
LOAD = Reflects whether the unit definition was properly loaded.<br />
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.<br />
SUB = The low-level unit activation state, values depend on unit type.<br />
<br />
2 loaded units listed. Pass --all to see loaded but inactive units, too.<br />
To show all installed unit files use 'systemctl list-unit-files'.<br />
[<a href="mailto:root@cpuk">root@cpuk</a> ~]# <br />
<br />
We looked into everything and issues are consistently pointing to file permissions in /log/messages or /log/audit<br />
<br />
[<a href="mailto:root@cpuk">root@cpuk</a> ~]# systemctl start firewalld<br />
Job for firewalld.service failed because the control process exited with error code. See "systemctl status firewalld.service" and "journalctl -xe" for details.<br />
[<a href="mailto:root@cpuk">root@cpuk</a> ~]# <br />
<br />
[<a href="mailto:root@cpuk">root@cpuk</a> ~]# systemctl status firewalld.service<br />
● firewalld.service - firewalld - dynamic firewall daemon<br />
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)<br />
Active: failed (Result: exit-code) since Sun 2020-01-26 22:29:33 GMT; 21s ago<br />
Docs: man:firewalld(1)<br />
Process: 4936 ExecStart=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS (code=exited, status=1/FAILURE)<br />
Main PID: 4936 (code=exited, status=1/FAILURE)<br />
<br />
Jan 26 22:29:33 cpuk.apache01.int systemd[1]: Starting firewalld - dynamic firewall daemon...<br />
Jan 26 22:29:33 cpuk.apache01.int systemd[1]: firewalld.service: main process exited, code=exited, status=1/FAILURE<br />
Jan 26 22:29:33 cpuk.apache01.int systemd[1]: Failed to start firewalld - dynamic firewall daemon.<br />
Jan 26 22:29:33 cpuk.apache01.int systemd[1]: Unit firewalld.service entered failed state.<br />
Jan 26 22:29:33 cpuk.apache01.int systemd[1]: firewalld.service failed.<br />
[<a href="mailto:root@cpuk">root@cpuk</a> ~]# <br />
<br />
Journalctl -xe shows:<br />
<br />
Jan 26 22:29:33 cpuk.apache01.int rsyslogd[1175]: action 'action 0' resumed (module 'builtin:omfile') [v8.24.0-41.el7_7.2 try <a href="http://www.rsyslog.com/e/2359">http://www.rsyslog.com/e/2359</a> ]<br />
Jan 26 22:29:33 cpuk.apache01.int rsyslogd[1175]: file '/var/log/messages': open error: Permission denied [v8.24.0-41.el7_7.2 try <a href="http://www.rsyslog.com/e/2433">http://www.rsyslog.com/e/2433</a> ]<br />
Jan 26 22:29:33 cpuk.apache01.int rsyslogd[1175]: file '/var/log/messages': open error: Permission denied [v8.24.0-41.el7_7.2 try <a href="http://www.rsyslog.com/e/2433">http://www.rsyslog.com/e/2433</a> ]<br />
Jan 26 22:29:33 cpuk.apache01.int rsyslogd[1175]: action 'action 0' resumed (module 'builtin:omfile') [v8.24.0-41.el7_7.2 try <a href="http://www.rsyslog.com/e/2359">http://www.rsyslog.com/e/2359</a> ]<br />
Jan 26 22:29:33 cpuk.apache01.int rsyslogd[1175]: file '/var/log/messages': open error: Permission denied [v8.24.0-41.el7_7.2 try <a href="http://www.rsyslog.com/e/2433">http://www.rsyslog.com/e/2433</a> ]<br />
Jan 26 22:29:33 cpuk.apache01.int rsyslogd[1175]: file '/var/log/messages': open error: Permission denied [v8.24.0-41.el7_7.2 try <a href="http://www.rsyslog.com/e/2433">http://www.rsyslog.com/e/2433</a> ]<br />
Jan 26 22:29:33 cpuk.apache01.int rsyslogd[1175]: action 'action 0' resumed (module 'builtin:omfile') [v8.24.0-41.el7_7.2 try <a href="http://www.rsyslog.com/e/2359">http://www.rsyslog.com/e/2359</a> ]<br />
Jan 26 22:29:33 cpuk.apache01.int rsyslogd[1175]: file '/var/log/messages': open error: Permission denied [v8.24.0-41.el7_7.2 try <a href="http://www.rsyslog.com/e/2433">http://www.rsyslog.com/e/2433</a> ]<br />
Jan 26 22:29:33 cpuk.apache01.int rsyslogd[1175]: file '/var/log/messages': open error: Permission denied [v8.24.0-41.el7_7.2 try <a href="http://www.rsyslog.com/e/2433">http://www.rsyslog.com/e/2433</a> ]<br />
Jan 26 22:29:33 cpuk.apache01.int rsyslogd[1175]: action 'action 0' suspended, next retry is Sun Jan 26 22:30:03 2020 [v8.24.0-41.el7_7.2 try <a href="http://www.rsyslog.com/">http://www.rsyslog.com/</a><br />
Jan 26 22:29:33 cpuk.apache01.int systemd[1]: firewalld.service: main process exited, code=exited, status=1/FAILURE<br />
Jan 26 22:29:33 cpuk.apache01.int systemd[1]: Failed to start firewalld - dynamic firewall daemon.<br />
-- Subject: Unit firewalld.service has failed<br />
-- Defined-By: systemd<br />
-- Support: <a href="http://lists.freedesktop.org/mailman/listinfo/systemd-devel">http://lists.freedesktop.org/mailman/listinfo/systemd-devel</a><br />
-- <br />
-- Unit firewalld.service has failed.<br />
<br />
So as a first step, we suspected selinux to be an issue and we were successful to run auditd and firewalld, if we disable selinux.<br />
<br />
So we know for sure that it is selinux that is causing these issues.<br />
<br />
Some version info:<br />
<br />
[<a href="mailto:root@cpuk">root@cpuk</a> ~]# rpm -q firewalld<br />
firewalld-0.6.3-2.el7_7.2.noarch<br />
[<a href="mailto:root@cpuk">root@cpuk</a> ~]# <br />
<br />
[<a href="mailto:root@cpuk">root@cpuk</a> ~]# rpm -q kernel<br />
kernel-3.10.0-1062.7.1.el7.x86_64<br />
kernel-3.10.0-1062.9.1.el7.x86_64<br />
[<a href="mailto:root@cpuk">root@cpuk</a> ~]# <br />
<br />
We have looked all through google and also checked the context of /var/log directory. For example, for httpd, we do issue following command to set the context for httpd service.<br />
<br />
chcon -t httpd_sys_rw_content_t /var/www/ -R<br />
chcon -t httpd_sys_rw_content_t /var/log/ -R<br />
<br />
O’wise httpd won’t start either.<br />
<br />
So question is, can you please throw some light on this. Is there a context we need to set. Why all 6 clusters have this issue, is this due to some known version of selinux released recently?<br />
<br />
I would appreciate some thoughts.
↧
