By the power of R, statistical computing at your fingertips

5 December, 2007 – 1:07 pm

I’ve explored in previous posts the use of tools such as onboard Analytics (LoadRunner), off-the-shelf tools (Excel) and custom web based implementations (JGraph, ChartDirector) used to analyze the nitty gritty of performance metrics.

All of these tool’s use are governed by some common factors being:

the Expediency factor - the timeliness of data being analyzed as measured between capturing and analysis of raw data.

the Pimp factor - the ‘prettiness’ in the presentation of data. Particularly important for benchmarking or external (public) facing documents. Never underestimate the importance in presentation of your results.

the Share-ability factor - how portable the analysis needs to be. Particularly important when working with different technology groups such as middleware, storage or network subject matter experts.

the Proprietary factor - sometimes you just can’t escape this. Your heart may lie with support open source, but often your pay check dictates that you must use proprietary formats, templates and the like as already setup by the client. Particularly pertinent with the use of tools like Load Runner, QALoad and the like.

Enter stage right, the R-Project.

R is a free software environment for statistical computing and graphics. It compiles and runs on a wide variety of UNIX platforms, Windows and MacOS.

Using R escapes the proprietary factor although some employers may object to you using GPL in their environment. It is a tad weak on the share-ability factor but is very strong on expediency and the subsequent timeliness of your analysis. Oh and the pimp factor, well, that just depends on your imagination. For this demo I’m sticking with corporate grey on a white background.

So how do you use it?
Essentially the reason I’m finding R so powerful, is its ability to quickly analyze and present meaningful statistical information from a raw data source. In my current implementation, I’m using a sequence of simple MySQL import scripts to import raw data (e.g. iostat, vmstat etc) in a database table. For my demo, the table name is based on the server I am sampling.

Then using a simple R script, that is executed from the R console, I loop through that table (don’t even need to bother with what the column names are, as it automatically enumerates those) and present the data as a series of line charts with a lowess curve fitted.

The script itself is as simple as this.
library(RMySQL)
MySQL
m <- dbDriver("MySQL")
con <- dbConnect(m, group="DatabaseName")

perfstat <- function(metric, table, dtgFrom, dtgTo) {
    print(metric)
    query<-paste("SELECT ", metric, " result FROM ", table, "
    WHERE dtg BETWEEN ", dtgFrom, " AND ", dtgTo, sep="")
    rs<-dbSendQuery(con, query)
    df<-fetch(rs,n=-1)
    quartz()
    plot(df$result, yaxs ="i", type="l", col="grey", xlab="Elapsed Time",
    ylab=metric, main=paste("Line chart of ",metric), lwd=1)
    lines(lowess(df$result,f=.05,iter=100), col="blue", lwd="0.25")
}
 
table<-"perfstats_ServerName"
dtgFrom<-"'2007-11-29 19:00:00'"
dtgTo  <-"'2007-11-30 19:00:00'"
fields <- dbListFields(con, table)
for (i in fields) {
if (i == "dtg") print("skipping") else
perfstat(paste("`",i,"`",sep=""),table, dtgFrom, dtgTo) }
 
dbDisconnect(con)

In that example, I’m sampling performance stats from a remote server spanning a 24 hour period, from which data was sampled at a 5 second interval, and presenting that info as follows.

With a slight twist of the code, you can plot result distributions using my favourite representation (boxplot) to help show the distribution of your data. Code like this:
library(RMySQL)
MySQL
m <- dbDriver("MySQL")
con <- dbConnect(m, group="mysql")

perfstat <- function(metric, table, dtgFrom, dtgTo) {
    print(metric)
    query<-paste("SELECT ", metric, " result, device FROM ", table, "
    WHERE dtg BETWEEN ", dtgFrom, " AND ", dtgTo, " ORDER BY device", sep="")
    rs<-dbSendQuery(con, query)
    df<-fetch(rs,n=-1)
    quartz()
  # set plot parameters
  op <- par(las=2, lwd=0.5, omd=c(0,1,0,1), mar=c(5,15,4,2), cex=0.8)
 
    boxplot(df$result ~ df$device, data = df, col="lightgray", horizontal=TRUE, lwd=0.5, range=0, notch=TRUE,
    xlab=metric, main=paste("Boxplot of ",metric))
}
 
table<-"_iostat"
dtgFrom<-"'2007-11-13 18:00:00'"
dtgTo  <-"'2007-11-14 18:00:00'"
fields <- dbListFields(con, table)
for (i in fields) {
if (i == "dtg") print("skipping") else
perfstat(paste("`",i,"`",sep=""),table, dtgFrom, dtgTo) }
 
dbDisconnect(con)
Will present very useful boxplots (grouped by device ID) something like this:

Which makes it very easy to spot performance bottlenecks from a 20,000′ level.

I’m a realtive noob with this program, so will be exploring better ways to script and automate with it in future. But for the time being, its ability to quickly analyze data in a meaningful manner makes it one of the sharper tools in your toolbox.

I’m currently using it from my Macbook Pro, but hope to integrate it on some production Windows servers in the near future (using its ability to output straight to png). This a great product, it’s free and relatively easy to use. You could use this tool to automate ongoing performance benchmarking aspects of your application development life cycle, or as a simple health check for your key systems.