Well, as a performance test lead I’ve been getting more involved in the hiring process.
To cut through the chaff, these are some of the questions I like to ask potential candidates. What’s your favourite questions? How might you answer these?
I also like to follow up these questions with a half hour practical scripting in VuGen (or your tool of choice). You can learn alot about someone by the way they write their scripts.
The point? Determine if the person has recently used the tool (you’d be surprised) and suss out general knowledge of performance testing…
1. What are essential components of a performance test plan? Give an
example of a non functional requirement.
Test objectives, scope, scripts, scenarios, management, schedule, data,
environment, access, defect management, traceability requirements, roles &
responsibilities.
An example NFR might be response time and/or volumetric requirements. e.g.
this application/transaction is targeting a 5 second response time, with 20
concurrent users and peak 20 trans/min.
2. Describe steps you’d take to setup a script / load harness and confirm
correct operation / configuration.
Document manual test case, dry run, record 1st time, record 2nd time to
help diff dynamic data, parametize & correlate, replay. Validate using
server logs, database or other areas which demonstrate correct application
functionality.
3. What OS level resource metrics would you use to detect bottlenecks?
CPU, memory, disk, network
4. Which OS commands would you typically use to collect those metrics
(Solaris, AIX etc)?
sar, prstat, vmstat, iostat, netstat etc
5. What are 2 advantages of using Performance Center?
Provide a central repository of scripts and their results
Simplify licensing and distribution of software
6. Describe differences between correlation and parameterization in a LR
script.
Correlation
To simplify or optimize your code
For example, if you perform a series of dependent queries one after
another, your code may become very long. To reduce the size of the code,
you can nest the queries, but then you lose precision and the code becomes
complex and difficult to understand. Correlating the statements enables you
to link queries without nesting.
To generate dynamic data
Many applications and Web sites identify a session by the current date and
time. If you try to replay a script, it will fail because the current time
is different than the recorded time. Correlating the data enables you to
save the dynamic data and use it throughout the scenario run.
To accommodate unique data records
Certain applications (for example databases) require the use of unique
values. A value that was unique during recording is no longer unique for
script execution. For example, suppose you record the process of opening a
new bank account. Each new account is assigned a unique number which is
unknown to the user. This account number is inserted into a table with a
unique key constraint during recording. If you try to run the script as
recorded, it tries to create an account with the recorded number, rather
than a new unique number. An error will result because the account number
already exists.
Parameterization
When you record a business process, VuGen generates a script that contains
the actual values used during recording. Suppose you want to perform the
script’s actions (query, submit, and so forth) using different values from
those recorded. To do this, you replace the recorded values with
parameters. This is known as parameterizing the script.
7. What LR function would you use to save a parameter in a web-based
script?
web_reg_save_param()
8. Name 3 types of performance test types or methodologies you might employ
in a typical test effort?
Shakeout, Baseline, Load, Volume, Stress, Soak, Tuning
9. Which blogs/user groups/forums would you use to help solve VUGen
scripting difficulties?
Google
HP support site
SQA forums
Yahoo or Google user groups
90kts 
10. Broadly describe typical run-time settings you’d employ in your VUGen
script.
Run logic set to number of iterations
Pacing to control start of iterations
Log settings appropriate for test type
Think time as recorded or random percentage
Browser emulation with/without cache
11. What other types of resource/application level metrics might you
monitor? Which tools would you require?
Web Servers – availability, bytes tx/rx, workers, requests, memory, cpu
Application Servers – JDBC pools, thread pools, servlet sessions, EJBs,
JVM (memory util, garbage collection)
Database Servers – server, tablespace and database metrics, locks, buffer
ppols, reads, writes, slow queries
Messaging Servers – queue stats, message flows, node statistics
Network Devices – packets/bytes rx/tx, errors, discards
12. How does SiteScope facilitate agentless monitoring of unix hosts?
Uses ssh sessions to log in and retrieve OS and app level metrics from
command line.
13. Define the differences between response time, throughput and latency.
Response time is normally end to end time taken for a round trip response. Can include many sub-components of the system under test.Normally expressed in milliseconds or seconds.
Throughput is amount of work a process can do in a period of time. Normally
expressed as transactions/second or similar.
Latency is a period of delay. Normally expressed in milliseconds or
seconds.
Thanks Tim .. it would certainly help the PT to attend interviews confidently
I did a performance test interview last week and they had a written test related to Load Runner. Had to write out the answers with good ‘ol pen and paper. It’s hard to hit F1 with a dead tree system to get help.
I found the written test to be an interesting way to gauge knowledge of Load Runner since there was no help system to help write the code. Either I knew it or I didn’t.
I have found that just asking the question “Explain to me the difference between correlation and parameterization” gets 99% of the “resume artists”.