Formatting Data for Import into LoadRunner Analysis

HP LoadRunner Analysis is a free tool right? Well, maybe not “free, as in free beer”, but you can certainly install the trial version of LoadRunner 9.x, and Analysis does not appear to be time limited.

All right, I realise that the Controller is time limited, so how do you go about importing data into LoadRunner Analysis after your time trial has expired, and you can no longer use the Controller to ping away at your server resource metrics during runtime? Simply use the Import Data functionality built into Analysis…

In this example, I’m collecting performance metrics from remote Solaris hosts, which coincidentally I can’t get access to due to security reasons, hence the reason for going through this pain of importing data manually post test. First things first, I’ve asked for access to the raw metrics provided from commands like vmstat, iostat and sar to get memory, disk and cpu utilization respectively.

Then quickly parse that data using some custom Ruby scripts to help format those results into the csv format that LoadRunner Analysis expects. I’ve included some smarts in it to figure out the date time of each metric so you can better correlate your results with an actual scenario.

For example, vmstat info that comes in its raw format like this:
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr m0 m1 m2 m1 in sy cs us sy id
0 0 0 40563488 10311984 4 21 1 0 0 0 0 0 0 0 0 413 280 280 0 0 99
0 0 0 31902256 1908536 1 20 0 0 0 0 0 0 0 0 0 637 793 831 1 0 99


Can be turned into LoadRunner Analysis friendly format like this:
date,time,r,b,w,swap,free,re,mf,pi,po,fr,de,sr,m0,m1,m2,m1,in,sy,cs,us,sy,id
13/4/2008,17:36:22,0,0,0,31792512,2202496,0,2,0,0,0,0,0,0,0,0,0,577,699,706,1,0,99
13/4/2008,17:36:17,0,0,0,31779792,2201592,0,2,0,0,0,0,0,0,0,0,0,568,699,695,0,0,99

Simply by running it through a Ruby script like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

file = ARGV[0] || 'D:\vmstat.results.csv'
freq = ARGV[1] || 5
stat = File.stat(file)
now  = stat.mtime
 
puts "date,time,r,b,w,swap,free,re,mf,pi,po,fr,de,sr,m0,m1,m2,m1,in,sy,cs,us,sy,id"
File.readlines(file).reverse_each { |line|
	line.chomp
	if line.match(/^\s+\d+/m) then
        now  = now - freq
        date = "#{now.day}/#{now.month}/#{now.year}"
        time = "#{now.hour}:#{now.min}:#{now.sec}"
		data = line.gsub(/^\s+/m,"").gsub(/[\s\t]+/m,",").gsub(/,$/m,"")
		puts "#{date},#{time},#{data}"
	end
}

In this case, the date and time is figured out by looking at the last modified date, and subtracting the frequency interval (of 5 seconds) for each row. Pretty simple huh?

Here are the other Ruby scripts you’ll need for iostat and sar respectively:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

file = ARGV[0] || 'D:\iostat.results.csv'
freq = ARGV[1] || 5
 
stat = File.stat(file)
now  = stat.mtime
puts "date,time,device,r/s,w/s,Mr/s,Mw/s,wait,actv,wsvc_t,asvc_t,%w,%b"
File.readlines(file).each { |line|
	line.chomp
    if line.match(/^(\d+)$/m): $epoch = $1 end
    if line.match(/^\d+\.\d+/m) then
        dtg = Time.at($epoch.to_i).strftime("%d/%m/%Y,%H:%M:%S")
        if line.match(/,([\w\d]+)$/m): dvc = $1 end
        line = line.gsub(/,[\w\d]+$/m,"")
		puts "#{dtg},#{dvc},#{line}"
	end
}

In this case, using the -Tu unix timestamp switch in iostat provided a formatted time in epoch for each results, so I was able to figure out date and time for each block. If you don’t have that switch available, as for HP-UX, then you can improvise with the method shown in the vmstat example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

file = ARGV[0] || 'D:\sar.results.csv'
 
puts "date,time,%usr,%sys,%wio,%idle"
File.readlines(file).each { |line|
	line.chomp
    if line.match(/(\d+\/\d+\/\d+$)/m): $date = $1 end
    if line.match(/^(\d+:\d+:\d+)/m) then
        time = $1
        unless line.match(/usr/m) then
            data = line.gsub(/\d+:\d+:\d+/m,"").gsub(/^\s+/m,"").gsub(/[\s\t]+/m,",").gsub(/,$/m,"")
		    puts "#{$date},#{time},#{data}"
        end
	end
}

If you want a hand actually importing the data (aside from following the help manual) just sing out. The instructions worked fine for me, just look under ‘Import Data’. The only gotcha is to ensure that you set the time zone correctly for your format template files.

Cheers,
Tim Koopmans