An exercise in correlation - playing MySpace music via iTunes

[UPDATE] Mr Gecko, has since implemented this logic and sorted out downloading woes (for u US folk) in a nifty little OSX app here …

In between contracts, I have been practicing the art of correlation by scraping data from public web sites. The legality of this is perhaps a little unclear so I’ve aimed not to infringe on copyrights by not storing the content, but just changing the way in which content is displayed from a live site. One such example was to open up MySpace songs via iTunes, rather than using the flash player MySpace provide. This required a fair analysis of web server to client traffic, in order to piece together the necessary conversation required to achieve all this via Perl …

Read more »

Posted in Perl, Programming, Automation | 8 Comments »

Using Selenium RC to automate your web based testing

Selenium is an awesome free alternative to apps such as Quick Test Pro and the like, providing you with a browser based automation suite for web applications.

Selenium uses JavaScript and Iframes to embed a test automation engine in your browser. This technique should work with any JavaScript-enabled browser. Selenium Remote Control provides a Selenium Server, which can automatically start/stop/control any supported browser. It works by using Selenium Core, a pure-HTML+JS library that performs automated tasks in JavaScript.

In this demo, I’ve used Selenium-RC and Perl to automate the checking of a promotional website called FreeStuffDay where every month you can have the chance to secure an advertised item for free. Instead of me sitting behind a browser checking the availability of free items manually, I’ve written a quick demo script to show you how it’s done from Perl and Selenium-RC.
Read more »

Posted in Testing, Automation | No Comments »

Another chart API for your performance metrics

Google have got some charting code worth looking at, which basically allows you to create charts by modifying parameters in the query string.

Something as simple as this:
http://chart.apis.google.com/chart?chtt=Tim's%20Beer%20Intake&cht=p3&chd=s:9h&chs=250x100&chco=00ff00&chl=Unemployed|Employed produces this:

Being on holidays at the moment, I’m lacking the um, ‘project’ on which to apply this technology, but am curious to see how functional it is. They give some examples on how to serialize datasets, and I think up to 4095 data points can be used. Not bad for those dashboard type applications …

Posted in Analysis, Productivity, Programming | 3 Comments »

Using bees to find bottlenecks

On a basic level, the honeybee’s dilemma is a tale of two flower patches. If one patch is yielding better nectar than the other, how can the hive use its workforce most efficiently to retrieve the best supply at the moment? The solution, which earned Austrian zoologist Karl von Frisch a Nobel Prize, is a communication system called the waggle dance.

Saw this on Slashdot … The waggle dance

Posted in General | No Comments »

Risk management for dummies

This video is a play on Pascal’s Wager which really gets you thinking on possible outcomes for a given scenario being climate change … Is it a better bet to accept doomsday predictions for climate changes, or rest on our haunches and let the world pass by? If you haven’t thought about it yet, get onto it. Good find Ted, pass the word and do your bit …

Posted in General | 1 Comment »

Leopard Perl 5.8.8 installation throws errors when compiling (makefile)

If you’re stuck with this error when trying to compile your own apps:

No rule to make target `/System/Library/Perl/5.8.8/darwin-thread-multi-2level/CORE/config.h', needed by `Makefile'.

You will probably find that your Leopard installation is lacking some necessary files.
If you copy the files from your old(er) installation of perl (5.8.6) you should be able to get around this …

sudo cp /System/Library/Perl/5.8.6/darwin-thread-multi-2level/CORE/* /System/Library/Perl/5.8.8/darwin-thread-multi-2level/CORE/

Posted in OSX, Perl, Programming | 18 Comments »

By the power of R, statistical computing at your fingertips

I’ve explored in previous posts the use of tools such as onboard Analytics (LoadRunner), off-the-shelf tools (Excel) and custom web based implementations (JGraph, ChartDirector) used to analyze the nitty gritty of performance metrics.

All of these tool’s use are governed by some common factors being:

Read more »

Posted in Statistics, Analysis | 5 Comments »

Win32::PerfMon is unable to enum objects with multiple instances

I’ve been looking for some easy ways to enumerate all of the available performance counters on a Win32 platform programatically.

Rather than trawl through the PerfMon counter GUI, or regress back into the millions of raw counters available via WMI, I was looking for a solution somewhere in between.

The ActiveState version of Perl has a Win32::PerfMon module which lets you enumerate cooked performance counter metrics on a Win32 platform. So I started to write a simple script that would list all Objects->Instances->Counters recursively. Sounds simple right?

Unfortunately I found that the module appears faulty when trying to enum Objects that have multiple instances, such as PhysicalDisk and LogicalDisk. Naturally, these are the counters I’m most interested in! After googling to the end of the known universe, I couldn’t find a solution to this problem.

Windows comes with a command line utility called Typeperf which is great for what I ultimately need to script (an unattended batch job that collects performance metrics).

But in order to develop a list of ALL available counters I wrote a mashup in Perl using the Win32::Perfmon module and calls to Typeperf to discover performance counters related to any particular object. That way, I can top and tail the output to my chosen performance counters and not have to worry about the exact typing of these finicky little buggers.
Read more »

Posted in Perl, Win2K3, Analysis, Tools, Tuning, Windows | No Comments »

Leopard, blue screens, 3 Mobile and a hot MBP!

Good grief, I paid the price of tweaking my OS 10.4.x install after installing Leopard and dealing with the myriad of problems previous tweaking posed!

The first problem was after a very long upgrade process (1+ hours) for Leopard, I was presented with a nice blue screen, a mouse pointer and that was all! Several reboots later, and a google on blue screens I found that I needed to:

1. Start up in single-user mode by holding Command-S after restarting the computer.

2. Type these commands, each on a single line followed by Return:
/sbin/fsck -fy
/sbin/mount -uw /
rm -rf /System/Library/SystemConfiguration/ApplicationEnhancer.bundle
reboot

So that got me up and running with the new Leopard OS. Turns out, running the application enhancer bundle for ‘ClearDock’ wasn’t such a hot idea for the Leopard upgrade process.

Speaking of hot, my macbookpro was running extremely hot! Greater than 60 degrees celsius and seemingly climbing! Even worse I couldn’t hear the fans spinning. So I checked out my FanControl tweak and confirmed that no fans were running. A check of their website confirmed that a Leopard friendly update existed, so I unistalled the old version, reinstalled the new, rebooted, and got control of my fans again …

Lastly I rock up to work and try to attach my Novatel 3 Mobile HSDPA broadband express card and alas, the previous configuration had been wiped. I tried to install the old drivers with no success. Eventually I found some instructions here that sorted it all out. Thankfully Mac OSX 10.5 has support for the Novatel modem integrated with the OS now. No more 3rd party drivers required, yipee!

So the upgrade process really wasn’t that smooth, and it reminded me of windoze pain from the past. Apple lost a bit of shine for me, but thankfully I’m through it all, and cruising with what really is a lush operating system.

Posted in OSX, Hardware | 1 Comment »

Chasing symptoms, not cause

Any job in which you need to ‘fix’ something requires you to correctly analyze the root cause. Treat just the symptoms and you’ll find yourself on the never-get-fixed roundabout …

Performance tuning fits this profile neatly. In an array of available metrics, how do you avoid chasing the symptoms and never identifying root cause?
Read more »

Posted in Analysis, Testing | 2 Comments »

Syncing Outlook mail with Apple Mail 10.4.x

It’s a real pain for Apple users to synchronize Microsoft Outlook with Apple Mail, but Parallels (or VMWare) brings us one step closer. You can virtualize your work environment (typically a PC with Outlook/Exchange access) and host your own local IMAP server, to sync mail between your virtual PC and say your MacBookPro. The concept is, for each client I might work for, I run a virtual PC that has all their security/access requirements catered for. Since the majority of clients I work for use some version of XP with Outlook Exchange, I decided to host a single IMAP server locally that these virtual PCs can then synchronize with. That way I can continue to use my mac offline, and get the other benefits of time management apps like GTD Inbox.

Following are some less than trivial installation instructions pieced together from an article here and more help on the subject here. These installations are suitable for Mac OSX (10.4.x)
Read more »

Posted in General, OSX, Windows | 3 Comments »

“move a little, do a lot”

Have just stumbled across a very cool productivity app for OSX. I’m already a big fan of Quicksilver, and FlyGesture builds on that style of functionality albeit from your trackpad.

FlyGesture is activated like Exposé or Dashboard, bringing up a transparent window of “guides” to move your mouse through. Moving your mouse through the guides lets FlyGesture know what action or actions you want to be performed, such as closing a window or opening an application. Not getting it? Here’s a movie that demonstrates creating and using a new gesture to open Safari.

And like all great Mac apps, it’s free!

Posted in General, OSX, Tools, Productivity | No Comments »

Quick tip: split a large file into smaller files

Needed to break a large file containing many messages, each message separated on a new line into many new small files containing only one message per file … Started thinking in perl, but almost looked past the Unix split command:

split -l 1 -a 3 bigfile.txt smallfile_

-l ‘n’ will put ‘n’ lines per file
-a ‘n’ defines the length of the suffix, in this case 3 letters

QED

Posted in Analysis, Unix | No Comments »

Use of dashboards for performance tuning

Jason Gorman describes in his article 6 requirements for the practical use of dashboards in performance related testing.

As a performance tester, I find that I often need to provide snapshot summaries of systems performance ala dashboards. The key requirement imposed is normally timeliness and relevance of data being presented.

So sticking with Jason’s requirements, how do you implement good dashboard design?
Read more »

Posted in Statistics, Analysis, Tuning | No Comments »

Sharing data safely between Java threads

Am really starting to enjoy the power and portability of using Java in my load testing efforts, especially when the environments in which you work are often limited in terms of tool sets you’re allowed to use. For example the current Solaris environment I work in doesn’t have a C compiler native to the OS, so I am automatically limited in the amount of Perl code I can write, as I am lacking more powerful modules (that need to be compiled). Enter Java; it already comes installed and for the types of mid range services I’m working on (MQ, JMS etc) has loads of great APIs you can incorporate into your own custom test harnesses.

All good test harnesses are going to need to be multi-threaded, but I’ve had no idea on how to safely share data between threads! This problem has been annoying me for sometime, partly due to my own ignorance and lack of understanding of Java (indeed OO principles perhaps). But in any case, I was stuck in old (bad) habits of sharing data between threads. Normally I just assign a global variable but in OO design, that’s considered poor design, so here is my attempt at getting spawned threads of a ‘worker’ class sharing data via a common ‘datastore’ class.
Read more »

Posted in Java, Testing | 2 Comments »

Humour that only cat lovers can appreciate

I don’t normally blog on the personal side of things, but this video I certainly identify with. Absolute classic!
Read more »

Posted in General, Humour | 1 Comment »

65 thousand rows, the end of the Excel universe

It has long since annoyed me and no doubt numerous others, the inability of Excel to get beyond 65,536 rows of data or more simply 2^16 …

A deft upgrade to Office 2007 sees an improvement in this limitation and am now able to have at least 1M rows of data. This is good news for us performance testers stuck between learning a dedicated statistics package like R or dialled in to on-board analytics with tools like Load Runner.

It’s nice to be able to analyse a greater sample size using standard tools in the workplace, although MSO2007 is probably not standard just yet …
Read more »

Posted in General, Analysis, Tools | No Comments »

Gmail goes big(ger)! (4GB)

Techdebug has spotted the size increase in Gmail standard user accounts. Guess that means storage is getting pretty cheap for the heavy weights. Coincidentally DPHOTO uses the GMail service for its production users, which pretty much rules out the need for me(us) to ever have to re-install MailEnable again … woot!

Posted in General | No Comments »

Turning off Mac OSX text anti aliasing selectively

I’ve been experimenting with a couple of different IDEs lately for Java development, and the two at the top of my list are NetBeans and Eclipse. A frustrating thing is that Mac OSX tends to ignore anti aliasing rules for text in these applications. I think it has something to do with the way the applications themselves are rendering the text (slightly higher scale).

You can obviously turn off text smoothing for font sizes in your global system preferences, but I like having anti aliased text for other applications like mail and internet.

Roger Braunstein identified a neat hack here which essentially rescaled a proportional bit map font (ProggySquare) by a factor of around 300%. Thus when you use the font in an application like NetBeans, you simply set the ProggySquareHuge font (size=4) and your standard font smoothing rules still apply, eg. smooth everything below size 10.

This is a great solution for NetBeans and works a treat, however I couldn’t get it to work properly for Eclipse. It looks like the bold font types really turn out to be 4pt in size, so basically unreadable.

Reading between the lines in some of the user comments, it is possible to set font smoothing rules on an application by application basis. Simply type the following from the terminal for eclipse
defaults write org.eclipse.eclipse AppleAntiAliasingThreshold 20

This means all fonts under the size of 20 will not be anti aliased for this particular application. You can do this for any application as listed in /Library/Preferences/*

This also works for the ActiveState Komodo IDE
defaults write com.activestate.Komodo AppleAntiAliasingThreshold 20

Posted in OSX, Programming | 3 Comments »

Installing the CVS bundle for TextMate for Mac OSX

More for my own reference than anything else, here is how you install the CVS bundle for TextMate using a subversion client.

1. If you don’t already have a subversion, try the precompiled version here.

2. Set the LC_CTYPE variable to use UTF-8
export LC_CTYPE=en_US.UTF-8

3. Make a bundles directory if you have not already installed others
mkdir -p /Library/Application\ Support/TextMate/Bundles

4. Change to your bundles directory
cd /Library/Application\ Support/TextMate/Bundles/

5. Install the CVS bundle using the svn client
/usr/local/bin/svn co http://macromates.com/svn/Bundles/trunk/Bundles/CVS.tmbundle

6. Relaunch TextMate and you should now see a CVS bundle!

Posted in OSX, Programming | 2 Comments »