Simon icon Simon
Flexible server monitoring

Notification when Simon Launches?

Is there a way to send a notification when Simon launches?

I have Simon managed by launchd. I've set it to launch on login for a given user and then have launchd restart it every N minutes. I did it this way instead of the option to always keep it running so that I can stop it and have it out of the way for awhile if I need to administer the machine. Since this install will run on a headless system I'd like to receive a notification when Simon launches (on Reboot, after a Crash etc.).

I will set up a Pulse test to send me a daily "I'm alive" message, but am also interested in any unattended restarts.

Thanks

David Sinclair's picture

Re: Notification when Simon Launches?

There isn't (yet) a way to schedule a test to only run on launch, though I might add that in the future (I have plans for more scheduling options).

However, you could probably do it via a Script-based service. Use it with a test with a short frequency. Have the script use ps or similar to get the launch time of Simon, and load that value from a local file, then compare them to see if the launch time is greater than the saved launch time. If so, save the updated value to the file. Output the start time, and use that with Smart Change Detection to report a change when it changes.

If you write this script, please share it so others can benefit too.

Re: Notification when Simon Launches?

Thanks for the hint. Instead of getting the launch time I'm using ps to get the elapsed running time. I came up with this bash script.

# This will return the elapsed runtime for the Simon.app process
SIMON_UPTIME=`ps -Aeo etime,command | grep -v grep | grep /Simon.app | awk '{ print $1 }'`

# Return 1 (Failure) if Simon has been running for less than 2 minutes.
# A new or relaunch will return a failure.
# Adjust the Regex to match the Simon Checking Options
# My Checking options are:
# After success... 2 minutes
# After failure... 2 minutes
# Timeout      ... 2 minutes
if [[ $SIMON_UPTIME =~ ^0[0-1]\:[0-5][0-9]$ ]]
then
echo "Simon was recently launched ($SIMON_UPTIME)"
exit 1
else
echo "Simon has been running $SIMON_UPTIME"
exit 0
fi

I've done limited testing and this appears to work for my purposes. Any suggestions for improvements are welcome.

David Sinclair's picture

Re: Notification when Simon Launches?

THanks for that. Looks great.

May I post it on the Simon Extras page? I'd want to include your real name with it, if you don't mind.

Re: Notification when Simon Launches?

You'd be welcome to post it. Please let me run it in production for a few days first to verify it works. I'll install it tomorrow when I'm back on site.

The output of the ps etime variable changes over time. I want to ensure that the regex is handling things properly when we are days, weeks or (hopefully) months out from the last launch.

Brad Perkins

David Sinclair's picture

Re: Notification when Simon Launches?

Thank you. Sure, I'll wait for now; please reply again once you're satisfied all is well.

It would be nice if the script could take a Simon custom variable for the number of minutes, so it can be matched to the check frequency by entering an allowed number of minutes in a custom variable field. Modifying the regex for that wouldn't be trivial, though (at least for potential multi-digit numbers of minutes). Maybe the elapsed time could be assigned to a script variable, then the actual vs allowed intervals subtracted? Just a thought.

Re: Notification when Simon Launches?

Or the script could dynamically build the regex based on the check frequency, but as you note that might not be trivial.

If I get some time I'll look into a non-regex solution.

How would the script get the check frequency though? I've looked at the insert-able variables and don't see an obvious candidate.

David Sinclair's picture

Re: Notification when Simon Launches?

You can create Custom Variables in the Script plug-in window; see the User Guide.

The custom variables are presented as fields in the Edit Test window, e.g. like in the FTP Directory Listing one.

Re: Notification when Simon Launches?

I figured that out while you were responding. I think I can improve this by having that variable be in seconds then compare it to the elapsed time in seconds. This wouldn't use a regex (maybe in the elapsed seconds conversion).

What I was asking in my previous question is if there is a predefined Simon variable that returns the defined check interval? It would be nice to add some error checking. As is if I check for uptime under 2 minutes but misconfigure the test to only run once an hour I'll never detect a (re)start.

Thanks!

David Sinclair's picture

Re: Notification when Simon Launches?

No, there isn't a standard variable for the check interval, unfortunately. Perhaps I could add that in a future version.

However, an alternative could be to use the {CheckDate}, {CheckTime}, {TestNextCheckDate} and {TestNextCheckTime} variables. You could convert them into timestamps and subtract them to get the interval between the current check and the next one, which would give you the frequency.

In fact, you could also use the {TestLastCheckDate} and {TestLastCheckTime} variables with the {CheckDate} and {CheckTime} ones. Subtracting those would give you the interval between the last check and this check, which if greater than the calculated frequency (or a custom variable value) would indicate an interruption in Simon's checking due to restart or whatever -- without needing to use ps at all.

The format of those date variables is "%Y-%m-%d", e.g. today is "2010-03-08", and the time ones are "%H:%M:%S", e.g. "13:39:30".

Re: Notification when Simon Launches?

In fact, you could also use the {TestLastCheckDate} and {TestLastCheckTime} variables with the {CheckDate} and {CheckTime} ones. Subtracting those would give you the interval between the last check and this check, which if greater than the calculated frequency (or a custom variable value) would indicate an interruption in Simon's checking due to restart or whatever -- without needing to use ps at all.

The format of those date variables is "%Y-%m-%d", e.g. today is "2010-03-08", and the time ones are "%H:%M:%S", e.g. "13:39:30".

Thanks for noting this, but I'm not sure it would work in my case. Recall that I'm using launchd to keep Simon up. Also recall that what I'm really interested in are unattended restarts. Under launchd Simon will only be down for a few seconds. While testing my script I have observed that my relaunch test does not immediately run on restart. It appears to use the defined check interval and doesn't run the next test until that time has elapsed from the TestLastCheckDate and TestLastCheckTime. So unless the machine was actually shutdown for an extended period I would never get an interval greater than the expected downtime. So, using ps and extracting the elapsed time may still be the safest bet.

David Sinclair's picture

Re: Notification when Simon Launches?

True, in your situation ps would be the best option.

Re: Notification when Simon Launches?

The script has been in service a few weeks. Feel free to post it to the Simon Extras page.

Brad

David Sinclair's picture

Re: Notification when Simon Launches?

Okay, I've added it to the Simon Extras page.

Thanks again for providing this! Hopefully others will also find it useful.