Automatic service restart when some service down in Zabbix

Introduction

Environment

Instruction how install Zabbix

Adjust zabbix-agent configuration

Edit file /etc/zabbix/zabbix_agentd.conf and enable this two options

### Option: EnableRemoteCommands
# Whether remote commands from Zabbix server are allowed.
# 0 - not allowed
# 1 - allowed
#
# Mandatory: no
# Default:
EnableRemoteCommands=1
### Option: LogRemoteCommands
# Enable logging of executed shell commands as warnings.
# 0 - disabled
# 1 - enabled
#
# Mandatory: no
# Default:
LogRemoteCommands=1

Save file and restart service

root@vagrant:/home/vagrant# systemctl restart zabbix-agent
root@vagrant:/home/vagrant# systemctl restart zabbix-agent && systemctl status zabbix-agent
● zabbix-agent.service - Zabbix Agent
Loaded: loaded (/lib/systemd/system/zabbix-agent.service; disabled; vendor preset: enabled)
Active: active (running) since Mon 2020-10-05 17:55:06 UTC; 17ms ago
Process: 15064 ExecStop=/bin/kill -SIGTERM $MAINPID (code=exited, status=0/SUCCESS)
Process: 15065 ExecStart=/usr/sbin/zabbix_agentd -c $CONFFILE (code=exited, status=0/SUCCESS)
Main PID: 15067 (zabbix_agentd)
Tasks: 6 (limit: 1111)
CGroup: /system.slice/zabbix-agent.service
├─15067 /usr/sbin/zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
├─15068 /usr/sbin/zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
├─15069 /usr/sbin/zabbix_agentd: listener #1 [waiting for connection]
├─15070 /usr/sbin/zabbix_agentd: listener #2 [waiting for connection]
├─15071 /usr/sbin/zabbix_agentd: listener #3 [waiting for connection]
└─15072 /usr/sbin/zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
Oct 05 17:55:06 vagrant systemd[1]: Starting Zabbix Agent...
Oct 05 17:55:06 vagrant systemd[1]: zabbix-agent.service: Can't open PID file /run/zabbix/zabbix_agentd.pid (yet?) after start: No such file or directory
Oct 05 17:55:06 vagrant systemd[1]: Started Zabbix Agent.

Verify if zabbix user have permission in your sudoers, ifdon’t edit /etc/sudoers (visudo) and add line:

zabbix ALL=(ALL) NOPASSWD: /etc/init.d/rsyslog restart

Configure Actions in Zabbix WebUI

  • Login into zabbix and go to Configuration -> Actions
  • Create new Action

Explanation:

  • Operation type: We change it to Remote Command
  • Target list: We we should always choose a Host no matter if it’s remote host or localhost
  • Commands: The command to be executed when the trigger is fired

The full configured action should look like this:

Make a test!

  • Create item
  • Create trigger

Now we can stop rsyslog and wait for zabbix trigger

root@vagrant:/home/vagrant# systemctl stop rsyslog

After about 20 seconds a trigger will appear which will restart the stopped service.

The above scenario can be safely applied to other services.

--

--

DevOps Consultant. I’m strongly focused on automation, security, and reliability.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Maciej

DevOps Consultant. I’m strongly focused on automation, security, and reliability.