Before I moved the Dejal blog to WordPress, I had a test that was bundled with Simon called “Dejal posts” (you may still have this if you’re a long-time customer). This was a great example of using multiple filters to narrow down the output of a Web Page service. While the URL it monitored is no longer valid, it remains a useful example for you to learn more about creating your own filters.

The general idea of this test was to look at a “Recent Posts” page of the Dejal site, which listed all recent blog, forum, FAQ etc posts and their comments, and output some tidy text describing the most recent one, along with a changed state when a new post or comment is added.

Firstly, here’s the Service page; nothing remarkable here (the cookies are automatically recorded, and unimportant for this test):

The most interesting page is the Filters one:

When you check the test and look in the Activity log, you can see the output from each of those filters (from bottom to top):

Another way to view the output is via the Preview pane, which includes not only the service response and headers, but also the full output of each filter, to help you diagnose each step.

Here’s the output of the service; the full HTML of the web page:

Let’s break down each of the filters, via the Preview filter output.

The first filter, a Block one, takes the service response as its Input, and has Start text of <tbody> and End text of <td class="replies">. This finds the first occurrence of each of those bits of HTML in the service response, which corresponds with the most recent post information:

This filter outputs that:

The second filter is another Block one. It takes the output of the first filter as its input, and narrows it down further to just the title of the post. Notice that it also uses options disclosed on the right-hand-side of the filter configuration: it looks for the second occurrence of the Start text, searching from the beginning of the input:

The output of this filter is the post title:

The third filter is yet another Block (it is one of the most useful filters), but the input is different: this time it uses the output of the first filter, instead of the previous one (as is the default). It also has an option to look for the third occurrence:

It extracts the author information:

Filter number four is different. It uses an Ignore Links filter to extract out just the author name from the previous filter output. The previous filter doesn’t do this as when you were not logged in on the Dejal site, only the name is included (in which case this filter has no effect):

The output is just the non-HTML part of the input:

Next we’re back to a Block filter again, but this time looking at the original service response text to extract the number of replies to the post:

This should always output a number:

We then use a Singular or Plural filter, to take the number found in the previous filter and output “reply” if it is one, or “replies” for any other number:

As seen in the preview:

The last filter puts it all together: an Override Custom filter uses variables to combine the output of several filters in a nice readable way. In this case all the variables are variations of the filter output, but other variables are available too. Something that isn’t immediately obvious is that you can insert numbers to reference specific filters (otherwise it refers to the previous one):

Which results in:

So now that we’ve got some nice output text, what do we do with it? Of course, you can just see it in the Tests list, if you have the last change and failure displayed:

But you’ll probably want to get a notification:

I hope this case study is helpful. Most tests don’t need a series of filters like this, and there are other ways to achieve similar effects (like writing all the logic in a script), but it can be very useful when you want it. You can use similar techniques in your own tests.

If you install Simon on an Apple Silicon (M1, M2, etc) based Mac, you may notice Web Page tests timing out.

That is because the Web Page service uses an embedded helper tool to load the HTML in a separate process, to improve performance and reduce the risk of crashing Simon. But currently this helper is Intel-only. Some other helpers, e.g. to send emails or upload reports to a remote server, are also Intel-only.

This is fine, but it means that you need to install Rosetta, Apple’s emulation layer to enable running Intel apps on Apple Silicon.

The main Simon app is universal, so launching it won’t prompt to install Rosetta, but you can override that by checking the “Open using Rosetta” option in the Finder’s Get Info window:

Just remember to uncheck that option after you’ve opened Simon, so subsequent launches use the universal code.

A customer sent a query, asking how to use Simon to monitor the subscriber counts on YouTube channels.

This is easy for Simon to do, by having Web Page tests for each YouTube channel, each with a filter to extract the subscriber count, and whatever notifier you wish.

The first step is to add a Web Page test for a YouTube channel, and look at the HTML output in the Preview pane, to find how the subscriber count appears.

For example, for my Sinclair Trails channel, the HTML includes:

    "subscriberCountText":{"accessibility":{"accessibilityData":{"label":"30 subscribers"}},"simpleText":"30 subscribers”},

Yes, I currently only have 30 subscribers… rather sad. Please subscribe to help me reach a more respectable number!

A sensible default choice for a filter is the Block one, though in this case a better choice would be the Find Regular Expression filter. But if you want to have several tests with the same notifier configuration, an even better choice is to create a new custom notifier, so you don’t have to configure the notifier for each test.

A regular expression to extract the subscriber count from that HTML could be:

    subscriberCountText.+?simpleText\":\"(.+?)\”\}

This will look for text starting with “subscriberCountText”, some more text, then “simpleText”:"” , then capture the desired text up until “”\}” .

In my case, that will result in:

    30 subscribers

To make a new filter for this, you can go to the Filters page in Simon, and add a new filter named “YouTube subscribers” (or whatever you prefer).

Then choose Find for the Filter Kind, and Regular Expression from the find options menu:

Then enter the expression as the find text, and choose Capture 1 as the output:

Click Done, then go back to your YouTube test, and choose this new filter on the Filters page:

This will output just the subscriber count:

You can see this output in the Activity log:

Or you can add a new Email notifier to email the filter output using the {FilterOutputText} variable:

And of course use that notifier in your test, along with whatever other notifiers you wish:

Then you can add more tests for other YouTube channels you want to monitor, specifying your YouTube subscribers filter for each. For example, CGP Grey has a few more subscribers than me:

I hope this helps!

Here’s a fun example of using Simon for a purpose that may not be entirely obvious.

My wife and I had a big trip to Yellowstone National Park coming up, so I was curious about what the traffic was like to get into the park.

Conveniently, Yellowstone has a number of webcams that watch the park, including the west entrance gate, where we’d be entering.

So I thought I could leverage Simon to help me watch it over time — rather than keeping the web page open and watching it throughout the day, I can have Simon capture the webcam images to a local folder, then review at my convenience.

To find the URL of the webcam image, I used Safari’s web inspector:

Then I added a custom service to Simon to download the webcam images to a folder:

That uses the curl command line tool to download the image and save to a local folder; the `date -v+1H +%Y%m%d%H%M%S` part of the path outputs the date and time in the format YYMMDDHHMMSS, adding one hour from my local time zone, since Yellowstone was an hour ahead of me at the time.

To schedule the captures, I added a test in Simon that checks once per minute, using that service. No filters or notifiers needed.

That results in a folder in the Finder with files like this:

I can arrow through them to quickly view them. And I can also do other things, like use the excellent Retrobatch app to convert them to a GIF:

Click to view an example of the resulting GIF, showing the traffic flow from 06:00 to 18:00 on one day (the GIF is 61 MB, despite being scaled, so might take a moment to load); each frame of the GIF represents one minute, six frames per second:

That seems typical; it gets really busy around 06:30, and eases off around 12:30 every day. So a tip for Yellowstone visitors; enter in the afternoon! Of course, that really only works if you’re staying in the park, since it takes ages to get anywhere in the park. It’s a big place.

If you’re curious, I did several posts about this Yellowstone trip on my Sinclair Trails blog. Check it out!