How to Install Monit (Process and Service Monitoring) on Ubuntu / Debian

Monit - Process and Service monitoring for Linux

Services on any server need to be up and running and do what they are supposed to do.

Automated monitoring is necessary to ensure and maximize the uptime and availability of system services on a server.

It allows system administrators to get notified and fix problems as soon as possible with minimal interruption or down time.

In this post we are going to play with a tool called Monit that can monitor services and processes on a server and send out alert emails when anything fails.

The tool itself is quite light weight and allows users to write "tests" using a very simple script style language.

Install Monit

On Debian and Ubuntu based systems Monit is available in the default repositories. Install using apt-get

$ sudo apt-get install monit

After installation monit would start as a background service and can be verified with the following command

# service monit status
 * monit is running

Configure Monit

The next thing to do is configure monit to monitor specific things. The configuration file for Monit is located at /etc/monit/monitrc on Ubuntu and Debian based systems.

You can either add all configurations to this file - /etc/monit/monitrc, or create additional configuration files in the /etc/monit/conf.d/ directory and monit would automatically read and load all of them

Start editing the configuration file to add the services you want to monitor.

Monit also has a log file located at /var/log/monit.log that would contain valuable information if you need to fix something

After configuration test the new configuration by running the following command

# sudo monit -t

Reload the new configuration without restarting all of monit

# sudo monit reload

Enable the web interface

The web interface allows you to check Monit reports from browser. This is useful when setting up and testing monit. Later on, its better to disable it to keep things secure.

Find the "set httpd" line in the configuration file and uncomment it. If the line doesn't exist, then just create one.

set httpd port 2812 and
#    use address localhost  # only accept connection from localhost
#    allow localhost        # allow localhost to connect to the server and
    allow admin:monit      # require user 'admin' with password 'monit'
#    allow @monit           # allow users of group 'monit' to connect (rw)
#    allow @users readonly  # allow users of group 'users' to connect readonly

Now access http://localhost:2812/ and the Monit web interface should open up.

Setup Email alerts

The whole idea behind monit is to get alerted whenever something unexpected occurs. Monit can automatically send out emails. Find the line "set alert" and specify the email where you want the reports to be delivered. Also specify the mailserver to use.

set alert [email protected]
set mailserver localhost

Make sure that there is an smtp server (MTA) running on localhost and that it is able to deliver mails properly. Use the mail command to verify that the system is correctly configured to send out emails.

Avoid non-alert emails - Monit by default sends out emails for every thing it does or everything that happens. For example when monit starts. However we want only the alert specific emails. Modify the configuration as follows

set alert [email protected] not on {instance}

The above configuration would not email instance reports which are generated upon monit start/stop. Check monit documentation on event filters for alerts to learn more.

This is how a Monit alert mail looks like. It is raised when a disk space test runs and fails on the server.

Resource limit matched Service server_disk 

	Date:        Thu, 11 Jun 2015 15:41:43
	Action:      alert
	Host:        localhost
	Description: space usage 61.4% matches resource limit [space usage>50.0%]

Your faithful employee,
Monit

Monitor services like Apache, Mysql, Postfix

Now you want to monitor your web server, database server, mail server and other daemons to ensure that services are up and running perfectly all the time. This is easy.

Just keep adding instructions in the monit configuration file to specify every service you would like to monitor and Monit would start doing it instantly.

After changing configurations, make sure to run 'sudo monit reload' to bring them in effect.
While testing you can manually turn services on/off and check the monit report from the web interface.

Here are some examples -

1. HTTP service - port 80

The following example will tell Monit to check if the http server is serving on port 80 or not. If not, it would generate an alert

check host http_server with address localhost
    if failed port 80 protocol http then alert

So if there is any problem with the http server(apache or nginx) an alert would be generated
Similarly you can set monitors for other ports like ftp, smtp, imap etc.

2. Monitor website status

Just monitoring port 80 is not sufficient to ensure that website is up and functioning. We need to ensure that fetching a page gives an http response code of 200. Here is the required configuration

# check that website is working proper
check host binarytides with address www.binarytides.com
    if failed
        port 80
        protocol http
        request "/"
        status = 200
    then alert

If either of "port 80" or "http server" or "status 200 for / url" fails then an alert would be generated. This is an effective way to check the that your site is up and available to users.

3. Apache web server - Process Monitoring

Processes are monitored using specific pid files. For example on Debian Apache web server has a pid located at /var/run/apache2/apache2.pid that can be checked to ensure that apache is running or not.

# Monitor Apache and restart if not running, also alert
check process apache with pidfile /var/run/apache2/apache2.pid
    start program = "/etc/init.d/apache2 start"
    stop program  = "/etc/init.d/apache2 stop"

The above tells monit to monitor the process from the specified pid file. Over here, if monit does not find the process running, it would attempt to start it using the command specified in the "start program" parameter. By default, monit will also generate an alert(email) upon restarting.

Monitor only

If you do not want monit to restart apache and just inform about the event, use a different kind of configuration

# Alert if apache not running
check process apache with pidfile /run/apache2/apache2.pid
    if does not exist then alert

4. Check Mysql on port 3306 - Port monitoring

In this example we monitor the mysql database not using the pid but directly by connecting to the service port which is 3306.

To check just the port number with protocol use the following configuration -

# check just port 3306
check host localmysql with address localhost
    if failed port 3306 protocol mysql then alert

The following configuration shall check for mysql process and restart if not running. It would also check for port 3306 service.

check process mysql_server with pidfile /var/run/mysqld/mysqld.pid
   start program = "/etc/init.d/mysql start"
   stop program = "/etc/init.d/mysql stop"
   if failed host 127.0.0.1 port 3306 protocol mysql then alert

5. Monitor Postfix mail server and ports

Now lets setup a configuration to monitor postfix mail server. The configuration is identical to previous ones, just with a different pid file and port number

check host localpostfix with address localhost
    if failed port 25 protocol smtp then alert

To monitor the process and restart upon failure use this -

check process postfix with pidfile /var/spool/postfix/pid/master.pid
   start program = "/etc/init.d/postfix start"
   stop  program = "/etc/init.d/postfix stop"
   if failed port 25 protocol smtp then restart

And do not forget that while postfix is down, monit might not be able to send any emails (unless there are other mail servers specified). But monit would send them once the mail server is up again.

6. Alert repeatedly

By default Monit generates an alert only twice. Once when a test fails and other when it succeeds.

If you wish to get repeated alerts for the same failing test (if its too important for example), then use the following configuration.

set alert [email protected] with reminder on 15 cycles

The above configuration is a global setting, that sends all alerts after every 15 checks/cycles.

To repeat alerts only for specific tests, specify the alert directive within the particular test criteria like this

check filesystem datafs with path /dev/sda1
  alert [email protected] with reminder on 1 cycles
  if space usage > 80% for 5 times within 5 cycles then alert

If you would like to be reminded fewer times but repeatedly then use something like "reminder on 10 cycles".

Check Monit status

This requires the monit http daemon to be running.

$ sudo monit status
The Monit daemon 5.8.1 uptime: 0m 

Process 'apache'
  status                            Running
  monitoring status                 Monitored
  pid                               1335
  parent pid                        1
  uid                               0
  effective uid                     0
  gid                               0
  uptime                            1h 52m 
  children                          7
  memory kilobytes                  29556
  memory kilobytes total            241604
  memory percent                    0.3%
  memory percent total              2.9%
  cpu percent                       0.0%
  cpu percent total                 0.0%
  data collected                    Fri, 05 Dec 2014 11:11:18

System 'enlightened'
  status                            Running
  monitoring status                 Monitored
  load average                      [0.05] [0.26] [0.35]
  cpu                               3.7%us 0.8%sy 0.0%wa
  memory usage                      3613152 kB [44.2%]
  swap usage                        0 kB [0.0%]
  data collected                    Fri, 05 Dec 2014 11:11:18

Or check the web interface at http://localhost:2812/ to check status for everything