I got this idea a little while ago….
There where several unit’s, who wanted to know how many errors/instances occurred of a certain type in the log files on a middleware platform. This was because they wanted to detect and correlate errors/instances with problems they had on their application, communicating thru the middleware platform.
I therefore created a website, where you could specify the search whit a regular expression. This is possible thru an Admin site.
Here you can create, modify and delete different searches.
You specify the search with a regular expression and the title for the graph.
Each search created, creates a graph which is displayed on a site like the picture below.
The search sends a request to a log server, where I’ve created an interface which then parses the log files and sends back the granulated information from the log file.
I then create a graph for each search, displaying the granularity within 1 minute.