In my last post I made a tongue-in-cheek observation that load testing really doesn’t work by today’s perceived standards for web automation and testing.
Part of my grievance for that was based on frustration in dealing with different load testing platforms. Not frustration born out of incompetence, more the frustration born out writing tests for increasingly complex browsers with an antiquated toolset.
That’s not to say that protocol based application load testing (or the languages they sit on) aren’t powerful in their own right. LoadRunner, JMeter et al. has their place in the load testing world. But I would like to relegate them to the background as useful, sometimes expensive tools to spin up the wheels so to speak. Nothing like a bit of JMeter infused fun to get the wheels smoking on your system under test. But for the real part, measuring user experience on the web platform?
Continue reading Going Parallel … distributed testing across a grid network using Watir
I was going to write why LoadRunner does not work for the web as it is today and into the foreseeable future, but decided against that because I don’t think the following points are necessarily limited to a specific toolset.
Load testing does not work for web as it is today primarily because…
Continue reading Why Load Testing Does Not Work for the Web `Date::today >> 36`
I really like the concept of Getting Things Done. I’m also a fan of Inbox Zero. Well, at least try to be. Here’s how I implement GTD using my gmail account…
Continue reading GMail Getting Things Done – a poor man’s version
Here’s an idea… Ever been in a LoadRunner contract where you’ve got a ton of scripts to write for business transactions that have many many design steps?
Even worse, have you had to write those scripts against a development environment with an unstable code base and shocking performance? Sound familiar? Ever felt the frustration of re-recording manual test cases back into LoadRunner in said environments. Why wouldn’t you just automate the script recording process?
Continue reading Watir Powered LoadRunner Scripts
In mathematics, Fibonacci numbers are the following sequence of numbers:
0, 1, 1, 2, 3, 5, 8, 13, …
By definition, the first two Fibonacci numbers are 0 and 1, and each remaining number is the sum of the previous two. This naturally occurring sequence is useful for determining the amount of time to sleep whenever you are polling a resource.
I’m often required to poll a database resource in some sort of daemon. For example, check a table for some asynchronous tasks to complete. If you have a lot of threads polling this resource at quick intervals, the impact on the resource can be excessive/high. Rather than poll at a set interval, I use a Fibonacci sequence to determine what the next sleep interval will be. Following are code examples of this polling technique hosted on GitHub in my most commonly used programming languages…
Continue reading Polling a Resource with Fibonacci Determined Sleep Intervals
Version control is not something you often associate with performance testing scripts. The reasons generally vary. A significant one is the lack of any version control support in commercial toolsets like LoadRunner. Another reason is the perceived short-term shelf life of test scripts. You tend to write them once and throw them away right?
If you work with more agile teams/developers you will probably start to require a lot of re-use out of your scripts, especially when the release cycle increases in terms of frequency. It starts to make sense to use version control and some form of scripting framework.
In this post I’m going to use JMeter as the tool of choice here and I’m going to re-introduce concepts which I normally force upon others whilst contracting. That is, use of Application Simulation Models (ASMs) to hep define the load and use of the Application Performance Index (Apdex) to help present results. I’m also going to use GitHub (or git) as my tool for maintaining version control and implementing a wiki as a dashboard replacement for other test management tools.
Continue reading A Framework for JMeter using GitHub and Ruby
A colleague recently asked me if %D, when used in your LogFormat for apache2 access logs includes the ‘download’ time of transmitting the response back to the client.
Following are observations from my apache2 access logs when using the %D format code. The first column is %D in the following log entries.
# 5 second pause on server side code. In this test case I deliberately induce a 5 second delay on server side code using the php sleep function. This delay can be seen in the timestamp (microseconds)
5003909 138.130.86.92 - - [11/Sep/2009:16:43:50 +1000] "GET /test.php HTTP/1.1" 200 139 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3"
# Static image open bandwidth. In this test case I let the browser download a static image as fast as my mobile broadband permits. The time takes approx 4.8 secs to download approx 900KB.
4860133 138.130.86.92 - - [11/Sep/2009:17:00:51 +1000] "GET /gorilla.jpg HTTP/1.1" 200 912128 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3"
# Static image restricted bandwidth. In this case I restrict the bandwidth to any destination port of 80 to 1KB/sec using ipfw[1]. This time it takes approx 12.8 secs to download the same file.
12891663 138.130.86.92 - - [11/Sep/2009:17:02:56 +1000] "GET /gorilla.jpg HTTP/1.1" 200 912128 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3"
[1] to restrict bandwidth on my mac I did this:
sudo ipfw pipe 1 config bw 1KByte/s
sudo ipfw add 1 pipe 1 dst-port 80
sudo ipfw delete 1
To sum up, when using %D in LogFormat for apache2, this timestamp is increased by any:
1. server side processing delays
2. delays attributed to restricted bandwidth etc
QED
Here’s a simple Ruby script that will parse a remote GC log using Perl for specific date time entries. The scenario in which you might use this script is where you have rather large remote GC logs sitting on another platform like AIX that doesn’t have Ruby installed. It’s useful in the sense you can execute a load test, then just parse the GC log for the entries from the native_stderr.log that represent your test run. YMMV.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
| # notes: make sure you escape slashes and quotes in the regex
# libs: you will need to install SSH libraries for Ruby from DOS
# set HTTP_PROXY=http://username:password@proxy.local:80
# gem install net-ssh
# gem install net-sftp
require 'net/ssh'
require 'net/sftp'
@HOST = "remote.host"
@USER = "username"
@PASS = "********"
@PATH =
"/opt/websphere/portalserver/6.0/portalserver/native_stderr.log"
@SDATE = "Aug 11 10:0\\d+"
@EDATE = "Aug 11 11:0\\d+"
Net::SSH.start(@HOST, @USER, :password => @PASS) do |ssh|
# parse log
ssh.exec "perl -e '
open(I, \"< #{@PATH}\");
open(O, \"> /tmp/native_stderr.log\");
while(<i>) {
$found = 1 if /#{@SDATE}/;
$found = 0 if /#{@EDATE}/;
print O $_ if $found > 0;
}
close(I);
close(O)'"
end
Net::SFTP.start(@HOST, @USER, :password => @PASS) do |sftp|
# download the truncated log
sftp.download!("/tmp/native_stderr.log", "C:/native_stderr.log")
end |
A colleague recently asked me if it was possible to test SAP portals using JMeter. The short answer is yes.
Some people get confused with the additional protocols that LoadRunner sports for SAP, particularly the SAP GUI. But if your SAP portal uses plain old HTTP (or HTTPS) then JMeter can do the job.
There are a few gotchas which I will continue to update on this post, namely around correlation. In JMeter, correlation is particularly *easier* because you don’t have to worry about where to position a correlation rule. The equivalent function to use is a Post Processor -> Regular Expression Extractor.
Continue reading Testing SAP Web Portals with JMeter
I’ve been thinking about some promotional material for the company I work for. What inspired me was seeing a vi coffee mug… Unfortunately only available in the US. I’ve been endeavouring to wean myself off textmate and the like so thought this was a brilliant idea. But with limited space, what essential commands should go on a vim cheat sheet? What other cheat sheets would you like to see in this format?
Continue reading Vim Cheat Sheets on a Mug
