As part of a work spike we recently looked at cleaning up some technical debt within our application performance monitoring. Through previous iterations our existing monitoring is all over the place, with metrics captured at multiple points including Spring handler interceptors, exception resolvers and annotation-aspects.
While our team differed on the exact approach, we all agreed the current situation was unacceptable and we needed to unify our approach. At my company we have many internal APIs, including one for monitoring. However, in our experience many of them suffer from Not Invented Here syndrome: symptoms including idiosyncratic coding standards.
But the biggest problem is that internal APIs are often stale, with unfixed bugs, no obvious means to push upstream changes, and often using outdated dependencies (such as Spring 2.5.x). We simply lack the resources to keep them up-to-date.
With this in mind, we decided it would be better to use an open source monitoring library. The main benefit being that public, open source libraries tend to attract more developers and are, so the theory went, better supported. So I took the responsibility to review the current Java, open-source monitoring libraries. To keep the spike time-limited, we specified a check-list of possible features we’d like to see:
- Counts or hits
- Timings (including average timings)
- Standard deviations
- Distinguish timings and counts between success and failure
- Spring support
- JMX support
- SPI for customising monitoring if some of the above aren’t fulfilled
So here are our findings:
Jamon is probably the most established monitoring framework, so it was one of the first ones we looked at. While it certainly covers all the metrics we wanted (and then some) there was no JMX support and limited Spring support. With poor documentation and no obvious means to customise its functionality, we decided not to go with it.
Another contender is perf4j, which takes an interesting approach to monitoring by extending logging frameworks. It has support for many familiar logging APIs, including log4j, slf4j, commons-logging and java.util.logging. This makes a lot sense and with helpful documentation and out-of-the box Spring/JMX support this seemed like a good candidate.
However, while it covered most of our timing metrics (as you would expect) it didn’t seem to support counts/hits at all. What is more, you have to extend the framework via various logging framework SPIs which seemed like a lot of work.
The new kid on the block, Java Simon, seemed like the likeliest candidate: it comes with Spring and JMX support, handles all the required metrics and has a clean, simple API that is easy to extend. There also appear to be some performance gains over Jamon, although these are probably marginal at best.
Even though we settled no Java Simon, none of the above really met our exact requirements. But Java Simons long-term support (version 3.0 was released this month) and easy to extend SPI made it the likeliest candidate for us.