Many years ago I had a team member who would always answer “yes” when I asked if he had finished the LoadRunner script he was working on. Invariably he had not finished the script.
So I sat down and wrote a checklist for him, and told him that he should only answer “yes” if he had been through the checklist and completed every item.
The checklist didn’t stop him always insisting that he had finished his work when he hadn’t, so I fired him and wrote the script myself. :)
Perhaps you will find the checklist useful, even if he didn’t…
Note that this checklist has some items that only apply to web-based scripts.
- All values that should be correlated have been correlated
- Text checks (using web_reg_find) have been added before each web_url and web_submit_data function
- ContentCheck Rules have been added for all known error messages
- All server requests (web_url, web_submit_data) are being measured with a transaction (lr_start_transaction, lr_end_transaction, lr_set_transaction)
- Script runs without causing errors, and any warnings are due to legitimate reasons (e.g. download filters)
- File-based parameters have the correct “select next row” and “update value on” settings
- Correct runtime settings have been set. This means:
- Action blocks have been weighted with correct percentages in Run Logic
- Pacing intervals have been set to the correct value that will achieve target throughput with the given number of vusers
- Full logging has been disabled, and size of “send messages only when an error occurs” lgo cache has been increased
- Think time set to replay as a random percentage, and think time values in script are not ridiculously large. No think time inside transactions.
- Any needed download filters have been set
- All other runtime settings have been set as per internal standards
Hi Stuart,
Thanks a lot for the useful checklist.
I would add 2 tips:
– test data/test users for the script is/are set up
– The script related information is documented in a specification
All too familiar Stuart. There are many ‘experts’ among us in this industry, who seem to fake their way into interviews (probably been reading allot of our bogs and ‘interview questions’ sites) who are quickly discovered in their first few days of being employed.
It’s a shame what is happening, but I still believe if you are passionate about your work you will still do very well.
I like this list also, I too constructed a similar list, however, I’d like to make a slight contribution to your list.
web_reg_save_param to have “notfound=warning”, and to allow the web_reg_find to do the error capturing, or you will get duplicate errors in controller, and many errors when a page fails, also, as part of runtime settings, that the ‘continue on error’ is not checked, to stop a domino affect on errors.
Cheers.
Sameh, I strongly agree with you about not using “continue on error”, as performance testers sometimes end up with an unmanageable number of errors at the end of their test (and huge logs if they have “snapshot on error” and a high level of logging with “send messages only when an error occurs” enabled).
Generally I don’t worry about getting multiple errors when a single page fails, as most of the time I get duplicates anyway as a web_reg_find() will fail as well as a Content Check Rule (if it is a known error message), but testers should definitely not be using web_reg_save_param() failing to do their error checking for them.
I would be keen to link to your checklist if you post it on your blog. :)
Cheers,
Stuart.
Okay, I’ve had a few people ask about the “firing”.
In this case it was a contractor who had claimed to be a LoadRunner expert on their CV, and had been brought in to help with load testing during the final 6 weeks of a short project.
It quickly became obvious that the contractor did not have even a basic understanding of C, and they had never created a VuGen script for a web application before.
He was quickly sent back to the recruitment company who had provided him.
The funny thing was that I ran into him a week later. He had been hired by a different project at the same large Australian retailer.
I am from Bangalore, currently working in M&S London.I have been using below techniques to reduce error rates.
1) I maintain param file size calculation sheet for all scripts
2) Add 50% buffer to the size calculated from unique. each iteration and abort type of param simulation to compensate for early exit due to error @ browse page the process in agile perf engg .
3) And importantly adding web_reg_find is not mandatory when u have web_reg_save_param without notfound=warning. All this internal string functions consume lot of injector resources.
4) For huge loadtest dont use random thinktime as this increases overhead on controller 2 calculate.
5) Error detection should be logged as custom messages to detect.
Somehow it s required to identify the work that lower skilled people can do :) Hope he utilised ur checklist 2 attend next company interview after being fired.
Regards,
Manjunatha