HttpWatch, a Free Alternative Using Ruby

When performance testing I often need to spy on TCP (http) traffic between the client and server on a frequent basis. This is pretty much a staple of any performance test script development effort.

In the past I’ve relied heavily on trace options within LoadRunner and the like, but often find this a tad cumbersome. LiveHTTPHeaders is a great plugin for Firefox but it only captures header information. WireShark is great if you need to ‘go deep’, but often equates to opening a walnut with a sledgehammer. FireBug and IE Developer Toolbar are great to see what’s happening on a page-by-page basis. TCPdump is great if I’m on a Mac or Linux box, but it can be tricky parsing binary data. In any case most client sites are windows based for me.

So that’s a great list of tools to help you correlate and sniff out dynamic data, but the closest I’ve found for IE to doing what I want, that is, capturing all HTTP data going back and forth with the ability to filter for specific information from a console/command line is HttpWatch. A $295USD price tag / crippled basic (free) edition really forced me down this path…to roll my own HttpWatch using Ruby and the WEBBrick proxy.

Read on for a description of the code.

The script itself when run, will listen on a port (e.g. 9090) which you configure your browser to send all requests through. (Note: FoxyProxy plugin is great for fast switching between different configurations) An optional parameter argument allows you to search using regex for the information you are trying to find. For example, I frequently need to know where all occurrences of a hidden field might turn up in the response body, so that I know exactly where to correlate my data in the corresponding previous requests. This is important for both LoadRunner and JMeter scripts (or any web-based performance test harness for that matter).

When executed, the script will then parse all requests and responses (with options to display header information as well if required) and just output the results to your console/command line. the following screenshot demonstrates:
http_proxy_watch

Using an ANSI colour scheme (win32console gem on windows or highline gem on mac) any lines containing the search filter will be highlighted in green with the actual line number in the response body where it occurs. You can also see which GET/POST request triggered this. The code itself is pretty simple as the following demonstrates:

# A HttpWatch proxy clone (without the license!)
require 'rubygems'
require 'webrick/httpproxy'

begin
 require 'Win32/Console/ANSI' if PLATFORM =~ /win32/
 require 'highline/import'    if PLATFORM =~ /darwin/
rescue LoadError
 raise 'You must gem install win32console or highline to use color on Windows/OSX'
end

@proxy_port    = ARGV[0] || 9090
@search_body   = ARGV[1]

# Optional flags
@print_headers  = false
@print_body     = true
@pretty_colours = true

server = WEBrick::HTTPProxyServer.new(
    :Port => @proxy_port,
    :AccessLog => [], # suppress standard messages

    :ProxyContentHandler => Proc.new do |req,res|
        puts "-"*75
        puts ">>> #{req.request_line.chomp}n"
        req.header.keys.each do |k|
            puts "#{k.capitalize}: #{req.header[k]}" if @print_headers
        end

        puts "<<<" if @print_headers
        puts res.status_line if @print_headers
        res.header.keys.each do |k|
            puts "#{k.capitalize}: #{res.header[k]}" if @print_headers
        end
        unless res.body.nil? or !@print_body then
            body = res.body.split("n")
            line_no = 1
            body.each do |line|
              if line.to_s =~ /#{@search_body}/ then
                puts "n<<< #{line_no} #{line.gsub(/#{@search_body}/,
                  "e[32m#{@search_body}e[0m")}" if   @pretty_colours
                puts "n<<< #{line_no} #{line}" unless @pretty_colours
              end
              line_no += 1
            end
        end
    end
)
trap("INT") { server.shutdown }
server.start

If you can’t get the win32 console binaries from your dos prompt (i.e. behind a firewall or proxy) just install the attached zip file to librubysite_ruby1.8
download http_watch_proxy

Note: this proxy won’t support HTTPS communication, so you will need to disable the proxy on whatever page uses HTTPS (typically logon screens etc) then switch back to it for remaining transactions…

Enjoy :)

About these ads

7 thoughts on “HttpWatch, a Free Alternative Using Ruby

  1. Have you tried http://www.fiddler2.com/fiddler2/ ?

    Despite being from Microsoft, it is actually a useful tool

  2. Hi Charlie, in short yes, but thanks for the reminder … Despite the name, I mean ‘fiddler’? eewwww! Fiddler is a great tool performance testers should have at hand. There is also a HttpWatch equivalent for FireFox called HttpFox…

    However, I couldn’t get those tools to easily filter out variables and show me where they occur. Which I guess is what I was initially trying to achieve. I just wanted a filtered watchpoint if you like, to show me when and where variables are occurring in the response body …

    Cheers,
    Tim

  3. Another tool that I have used in the past is the most excellent chaosreader.pl perl script. You can take a .pcap capture from tcpdump or Wireshark and run it through chaosreader and extract all sorts of handy HTTP information. On top of that, it will also extract images and also generate a proxy style log of HTTP request along with a detailed list of HTTP requests and responses.

    Highly recommended!

  4. Cool tip, thanks! =)

  5. can use it under the linux

  6. The download link does not seem to work :-(

  7. [...] own proxy is to use an existing one. For all my work on this series of posts I have been using a proxy written in Ruby and based on the Webrick web server. It is fairly simple and doesn’t do everything quite [...]

Comments are closed.