Understanding Aggregate Variance within LoadRunner Analysis

From time to time, you may notice variances within the Loadrunner Analysis tool. This is most apparent when reviewing the Transaction Response Time (Percentile) graph.
 
If you look at the following Transaction Summary Report:

And look at the following Transaction Response Time (Percentile) graph:

 
As you can see there are some conflicting values:
 

 
Firstly, I’ll state that all these values are correct. The LoadRunner Analysis engine calculates all values correctly, and all variances can be easily explained by examining the data.
 

Median vs Mean vs Average vs Percentile

The first concept to understand and permanently implant into your brain, is the difference between these four aggregate types. An “Aggregate” is a single value that represents an underlying set of multiple values, commonly known aggregates are Average, Minimum and Maximum.
Most people understand what an “Average” is…that is the sum of a value set divided by the count of the value set. “Average” is also known as “Mean”. “Median” however is a different concept, a median value is calculated by sorting a set of data values from smallest to largest value then dividing this sorted set by two. The “Median” is the middle value at the place you divided the set. Median is identical to the 50th Percentile. Percentiles are an extension on the median concept, but instead of dividing the sorted value set by two…we instead divide the value set by a percentage (starting with the smallest value). So the 15th Percentile, refers to the value 15% along the sorted set…the 75th Percentile refers to the value 75% along the sorted set. Also the 0th Percentile is undefinable value.

Based on this information, we can safely separate the values into the following two groups.

 
Clearly the median value for the Graph Mouseover and the Summary Report are identical, so for the rest of the article I’ll will no longer refer to the Graph Mouseover.
 

Graph Values vs Raw Values

When the Percentile graph is generated (or any other graph for that matter), the analysis tool generates a small sample set for the graph. The sample set size is configurable in some graphs by setting the graph’s granularity, however the Percentile graph is fixed at a sample set of 100 data values…with each corresponding to a separate percentile value.
Therefore the values for Average and Median for the Graph Legend are calculated from the Graph data, not the Raw data.
Keeping this in mind, the Graph Average value is simply the average of the 1th, 2nd, 3rd…100th Percentile. Whilst this is graph average is close, it is not as accurate as the Summary Report average (which is calculated from the complete raw data set).
 
Okay, so up to this point we’ve explained all the value variations, except for the difference with the Median value. This problem is interesting, remember I mentioned that the median value is simply the sorted value set divided into two…and that the 0th Percentile is undefined. Well, the analysis tool equally divides the graph data result set into a set of values representing the 1th Percentile through to the 50th Percentile, and the 51st Percentile through to the 100th Percentile. As there is no clearly defined midpoint in this data set the analysis engine uses the next value…in this case the 51st Percentile.
So in summary. the Graph Median value actually the value of the 51st Percentile…not the 50th Percentile. This may be considered a minor bug.
 

In Summary

In summary, the graph value data is adequate for producing a graph…however for aggregate values I recommend only using the Summary Report data as the Summary report is the only report in the analysis engine that is guaranteed to display accurate data based on the complete data set.