I use Salt for configuration management on all my newer servers. I’m particularly fond of its watch requisite, and use it thoroughly to handle restarting all my Salt managed services if need be.
This generally works without much fuss. However, when a service that’s managed by Salt is also being managed by Monit, there’s a chicken and egg problem: when Salt automatically restarts a service using its standard method, Monit thinks the service has died, and problems ensue.
I really wanted my Salt service management goodness and Monit, so after a bit of thinking I came up the the idea of creating a psuedo-service. Salt manages the psuedo-service instead of the regular service, and the psuedo-service simply wraps the Monit commands for starting, stopping, and restarting the service. This allows Salt to trigger service actions via Monit.
This approach solved the issue beautifully without much code or effort. Below I’m including the RHEL/CentOS version of the psuedo-service – other platforms would require some minor adjustments, but the idea is the same.
As seen from the line defining the service name, I manage this script in Salt as a template, so that I can install it for multiple services.
Hope this helps anybody who gets stuck on this same issue!