JMeter Tips and Tricks – Application Simulation Models

I’ve been working with JMeter of late and after speaking with some people at conferences and the like, I thought it would be good to share what I’ve learned along the way in building, structuring and executing JMeter test plans.

One of the first things I like to do when performance testing is establish in reasonable detail what the Application Simulation Model (ASM) will entail. You can create an ASM visually using something like UCML. This is a good starting point for performance testers looking to structure their test planning efforts.

UCMLâ„¢ is a set of symbols that can be used to create visual system usage models and depict associated parameters. When applied to Performance Testing these symbols can serve to represent the workload distributions, operational profiles, pivot tables, matrixes, and Markov chains that performance testers often employ to determine what activities are to be included in a test and with what frequency they’ll occur.

More often than not though, I create an ASM using a spreadsheet format as I find this easiest to communicate with people from the business teams. If people are interested I will share some of the spreadsheet templates. The following screenshot gives you an idea of what I’m talking about:
example asm

The pertinent points for an ASM, is that it should:

  • describe the user community distibution in terms the amount of concurrent users and their daily/hourly profiles in terms of concurrent load.
  • e.g. “we have 3 user groups with a total of 150 users, 80% of them are helpdesk support, 10% are team leads and 10% are management”

  • describe transaction quantities and percentage distibutions.
  • e.g. “we’re aiming for around 240 trans/hour, of which 50% are searches, 25% are updates and 25% are reports”

    Read on to discover how to turn ASMs into working JMeter test plans.

    One of things I’m aiming for is to use relative random distributions of transactions within a loop controller which simulates the transaction distributions in real life. The following screenshot shows a Loop Controller, with three children Throughput Controllers (renamed to 20%, 50% and 30%). Each Throughput Controller then has a Transaction Controller, which is the parent container for all subsequent requests that make up that transaction (there are typically tens or hundreds of requests here depending on the complexity of the web application). Note I have also renamed the Transaction Controllers to something more representative of the transactions being simulated.
    loop controller jmeter

    The Loop Controller will effectively control the number of iterations and I often set this to run forever when threads are shut down manually during test runs. I also use a Constant Throughput Timer, which alleviates the need to set iteration counts and pacing between iterations which LoadRunner users would be familiar with. When using a Constant Throughput Timer, you can simply set the targeted transaction rate in terms of samples/minute (in this case it’s measured on collective Transaction Controller’s throughput). JMeter will then self-throttle the load to try and match that transaction rate. If you want more granular control, you can move CTT to be child elements of selected Transaction Controllers, thus applying different transaction rates for different transactions. The benefit in using Throughput Controllers with Percent Execution settings specified, is that transactions will be randomly distributed according to those percentages down to each subsequent Transaction Controller. The following screenshot demonstrates those settings used for the 20% Throughput Controller:
    throughput controller

    This means that if I specified 100 iterations in my Loop Controller, then 20 transactions will be distributed to the 20% Throughput Controller. This is done in a relatively random fashion, it does not process transactions in sequence which is good for random simulations.

    Once you’ve established your transaction groups with percentage distribution, you can then create your user groups using the Thread Group element. In my example I’m using two user groups (Group A and B), each with differing user quantities. The purpose of these is just to pigeon hole my user groups and quantities as detailed in the corresponding ASM. Each user group is run in parallel, and is subject to the startup (ramp-up) and loop count settings for the threads (users) created. The following screenshot shows some typical settings:
    jmeter thread group

    A more complete test plan looks like the following screenshot:
    jmeter simple test plan

    You can see two user groups in there, along with Loop Controllers and Throughput Controllers. You can also see all the sample requests under each of the Transaction Controllers. In my example I’m using HTTP request samplers, but this could just as easily be a web service, ftp or some other form of JMeter sampler. I often start with this template in mind, then record and edit transactions in the Workbench before importing back into my Transaction Controllers.

    Basically JMeter will run through the test plan from the top to the bottom. For this reason I often keep my configuration elements such as HTTP cookie managers, user parameters and reporting listeners towards the top of the test plan so they get initialized. Then, as it hits the Thread Groups it will launch those according to the settings specified. You can also see in the screenshot some results from the View Results Tree listener for Thread Group A. Essentially this demonstrates the randomized distribution of Transaction Controllers according to their parent Throughput Controllers.

    In coming posts I’ll walk through some other features of JMeter that help you control thread execution and delve deeper into the reporting functionality that comes with JMeter. You can download a copy of the jmeter test plan referred to in this post here.

    About these ads

    34 thoughts on “JMeter Tips and Tricks – Application Simulation Models

    1. Hi

      Interesting blog entry. Do you have any experience to share in relation to simulating varying bandwidths also? I very recently started looking at JMeter for a project I am working on but it does not (from what I found) provide varying bandwidth simulation. I came across something called Sloppy (Slow Proxy) which has potential in this area. I was thinking of using it in conjunction with JMeter.

      Thanks.

    2. Thanks!

      Your approach is fine, I’ve done something similar in the past where I would throttle individual load generators using a software solution. There are many available. For the commercial side, look at

      http://www.shunra.com/

      If you’re trying to keep it free, look at something like

      http://bandwidthcontroller.com/trafficShaperXp.html

      You can also see some others via this wiki

      http://en.wikipedia.org/wiki/Bandwidth_throttling

      If you don’t want to go the software-on-a-pc route, you could try something with your networking setup. I once got an experienced Cisco engineer to replicate different WAN speeds using software/hardware settings on switches etc.

      Regards,
      Tim Koopmans

    3. Thanks for the prompt response.

    4. Hi,

      Very interesant blog, congratulations!! I’m new using Jmeter and I don’t know if you can recomended (or send me) some docemuentation that you think could be very helpfull to start with Jmeter. I research the web and I learned about recording, create Test Plans manually and more, but I don’t understand very well how I can analyze the Jmeter results and wich are the limits of jmeter. If you can help me, I will apreciate it.

      Thanks in advance!!

      Jose

      Jose

    5. Hi,

      Very interesant blog, congratulations!! I’m new using Jmeter and I don’t know if you can recomended (or send me) some docemuentation that you think could be very helpfull to start with Jmeter. I research the web and I learned about recording, create Test Plans manually and more, but I don’t understand very well how I can analyze the Jmeter results and wich are the limits of jmeter. If you can help me, I will apreciate it.

      Thanks in advance!!

      Jose

    6. Hi Jose,

      Thanks for the comments. You might find this book an interesting read:

      http://www.bookwormr.com/book/show/isbn/1847192955

      I assume you’ve already been to the main site:

      http://jakarta.apache.org/jmeter/

      With regards to limits of JMeter, that’s a very broad question. If I’ve ever had any doubts, such as as “How many concurrent users can I run off a single load generator” then my answer would be to set it up and try for yourself! Post back your results here =) Or maybe ask me a more specific question which I can investigate for you …

      Another good resource for limitations and problems running jmeter are the usergroups hosted here:

      http://markmail.org/message/yeuceiaemv3y3ykz

      Analyzing results is another matter all together and I’ve yet to find a single *great* resource for this. Although depending on the system you’re working with, there are normally excellent performance tuning guides available. For example, this book is an excellent buy/read for understanding Solaris performance metrics (and tools available):

      Sorry I can’t be more specific, I guess I’ve just picked up these skills along the way through lots of reading etc. I don’t think there is any special path to follow =)

      Regards,
      Tim Koopmans

    7. PS. Don’t forget other people’s blogs and conferences! This recent post at performance engineer.com has a wealth of information =)

      http://www.performanceengineer.com/blog/velocity-conference-roundups/

      Regards,
      Tim Koopmans

    8. Tim,

      Thank you very much for your response, it’s very helpful for me, I will continue research about Jmeter, It’s a very interesant tool for the Performance tests.

      Thanks!!!

      Jose

    9. Tim,

      Do you know any technique or calculations to estimate the time it takes to develop the test plan and the execution of Performance Test with Jmeter?

      I have requirements, but I don’t know how to calculate the time; do you have any thoughts or advice?

      Thanks in advence!!!

      Jose

    10. Hi Jose,

      That’s a difficult question, because it depends on the circumstances and obviously complexity of the test effort / environment.

      However as a rough rule of thumb, I would plan on 2 – 5 days familiarizing yourself with the environment, then about 1 day per end to end scenario to document the manual test case and data requirements, then probably 2 – 3 days to write the script. This would obviously shorten with the more transactions/scenarios you simulate as you become more familiar with the application under test.

      In terms of test execution, it depends on the client’s requirements. A simple test effort might only be 5 days in duration for multiple iterations, but this is largely dependent on what type of load testing (load, stress, soak, tune etc) you want to conduct (and how fast your defect turnaround is).

      It really is hard to estimate without more information. As a total ‘pluck’ I would say a 20 day effort would normally be the minimum for a simple web based app, with multiple test runs and aiming for 5 – 10 simulated transactions/scenarios tops…

      If you want to discuss this via email please do at tim.koops at gmail.com

      Regards,
      Tim

    11. Ok, I will send you an email

      Thanks!!!

    12. Hi, nice tips and tricks :).

      The throughput controller seems to work only with loops.

      My problem is, I need to run 60 threads in 60 seconds (with 1 loop), throughput controller does not seem to control the threads throughput.

      I have tried many other ways without luck yet :(. What can I do to control threads throughput? (not loop throughput).

      Thanks for help.

    13. Hi Kong, not sure I quite understand…

      Do you want to run 60 threads simultaneously, with each thread completing only 1 transaction?

      When you say 60 seconds, is this a single transaction duration? Or do you mean 60 x 1 second transactions, with 60 threads executing each one…

      Can you please explain in more detail?

      Regards,
      Tim Koopmans

    14. Hi Tim!
      Could you please define more exactly how you created you ASM? I understand that it is just an example but anyway…
      To be more precise, I’d like to understand how you calculated concentrated to 12 hours day distribution, factoring+20% hourly spike and other data.
      I am a beginner at load testing, and your example seems very useful to me. And if you recommend any reference sources about it, I will be very much appreciated.

      Thanks In Advance
      Alexey

    15. Sorry, it seems I wrote something wrong in my previous post. Please, excuse me, my English is not perfect.
      I hope, inspite of this, you will answer me.
      Best regards,
      Alexey

    16. Hi Alexy, got your comments, no problems. Just a tad busy with work at the moment. Based on your comments and some other work stuff I think I’ll do another post dedicated to application simulation models (with templates) in near future …

      Regards,
      Tim

    17. Thanks, Tim!
      I’ll be waiting for your new post.
      Best regards,
      Alexey

    18. Hi,

      Your blog is a really usefull one, only people who have really worked on Jmeter can write such a useful blog.Howevere I have one question
      1. When I try to record using proxy servere the recording never happens.why? do we need to change anything in the jmeter properties.
      2. I want to a Transaction mix & the load mix ex: I have 2 types of users say user & manager performing requests I want to distribute them 60% & 40% and within that I want to distribute the requests .how do I do that? It is very urgent please let me know.
      3. I read some where that with different thread groups if we record all the request will get recorded in the first thread group it self, how do we solve this.

    19. Another excellent post Tim. When you have time can you make some templates available that you find handy for creating ASM’s and usage profiles?
      It would be neat to be able to tie an excel sheet template to fill out to autogenerate a UCML, Barber style doc. ;)

      Regards,
      Mordechai

    20. How to analyse the result in jmeter…can anybody guide me….for analysing the result of jmeter.

    21. I am currently using JMeter for performance testing. I have recorded some scripts to create some new records in the application. But when i run it for say 7 users, the records is being created 7 times in the database. How can i avoid the record being created.

    22. Thanks for sharing this!

    23. Hi,
      The article is good. Can you please share the ASM templates?

    24. If you have more complex user behavior, you can use Markov chains in Jmeter, too. http://www.trustsoft.uni-oldenburg.de/en/33145.html

      The plugin is working good. Getting around the configuration model is a bit harder. Once figured out it works like it is supposed to.

    25. That’s some interesting stuff, thanks Christian.

    26. Hi — Thanks for the write-up. The links to the screenshots and test plan are broken, though.

    27. How to analyse the result in jmeter…can anybody guide me….for analysing the result of jmeter and maximum how many users we can run.

    28. Hi Tim

      I found you blogs helping in understand Jmeter.

      I am beginner level developer, I am test a site for load testing. Eventhing is fine I am quite comfortable with Jmeter, Recording test cases, using parametrized variables and running test plans.
      but I am confused about multiple users,

      The application I am testing is having mock tests which have 100 question with multiple choices, User can answer questions/ move to next Q/previous Q / select a choice are ajax calls. Total time for test is 8100 seconds
      I am taking an average test case where a user goes to next Q 150 times /previous Q 50 times /select choice 100/ change choice 50 times which makes around 350 hits.
      As i have 8100 sec to finish the test that means 8100/350 = 23 secs per hit now I have add a Gaussian Timer with Deviation of 5s and constant time 18s.

      if I am running this test with hundred user it is taking 6-7 hours to finish.

      Can Your Please help to find out which timer and how much think time i need to use to get proper results

      Thanks

    29. Suresh Rajachar

      Hi Tim,

      The article is very informative. But, the links for screenshots and test plan are broken.
      Can you please provide us the alternate link ?

    30. Hi Tim,
      You got any idea, like how to keep on looping the thread for a constant time duration.
      i have set my thread loop count as forever and i want to stop the loop or set the loop count to ZERO after an hour. Is this possible in Jmeter , how we can acheive this ?

    31. Hey Tim,
      I wanted to know how to simulate multiple users that each have different username and passwords thanks

    32. how jmeter works from the proxy network
      is it ok to use badboy for the recording

    33. Hi Tim,

      Information here is very good?helping too.

      I have samll doubt?while reading some document I come to know that we can configare the mail from Jmeter?
      Maik like when results are complated then it automatically sent the report and in same way failed condiction it will send the report.

      but I am unable to find where I need to configar this setting can you help me in this regard.

    34. Hi christian
      Markov is very good addin for test plan.Its very nice and simple to create a test plan fro jmeter. but can you elaborate behavioral mix or Behavior Models.

      Thanks In Advance
      Piyush Sharma

    Comments are closed.