Tag: LoadRunner

Case Study: Flash Group optimises performance of the Global Corporate Challenge Website

Case Study, Entertainment, Micro Focus, Professional ServicesJDS
In 2009, Flash experienced some performance issues with the Global Corporate Challenge (GCC) website—the world’s first virtual health program that encourages corporations to help their employees get active and healthy—that resulted in speed degradation, functionality errors and site downtime

With the number of GCC participants predicted to double in 2010 to 120,000, Flash needed to drive a higher level of application performance and mitigate the risks it had previously. As a result, the company turned to HP Preferred Partner, JDS Australia, for a solution. The company adopted a Business Technology Optimization (BTO) approach to application performance with HP LoadRunner software for predicting the behaviour and performance of the GCC website under load.

Objective

Flash Group wanted to mitigate the risk of performance issues for the launch of the 2010 Global Corporate Challenge (GCC) website.

Approach

Flash engaged HP Preferred Partner, JDS Australia and adopted a Business Technology Optimization (BTO) strategy with HP LoadRunner software to obtain an accurate picture of end-to-end system performance.

IT improvements

  • Ensured the quality and performance of the GCC website for the 2010 programme.
  • Established a standardised procedure for load testing the website.
  • Identified and eliminated performance bottlenecks to tune for better performance.
  • Matured its website development methodology.
  • Raised its profile and credibility as an organisation that produces high-performing, user-friendly websites.
  • Delivered 99.99 per cent uptime on its systems with web servers only reaching 20 percent system capacity, and page response times of less than two seconds which resulted in a high-quality user experience and enhanced the programme’s brand value.

About Flash Group

Flash Group (Flash) is one of Australia’s fastest growing full-service advertising agencies, offering integrated services including above and below the line advertising with in-house digital, strategy and design.

The company’s 30 staff are dedicated to servicing a group of high profile clients that spans retail, healthcare, travel, fashion, hardware, consumer electronics and entertainment. This includes leading brands such as Pioneer, Stanley, Global Corporate Challenge, Contiki, Origin Energy, Clive Peeters, and more.

Every year, the company assists Global Corporate Challenge (GCC), a world-first virtual health programme that encourages corporations to help their employees get active and healthy. The programme sees people from around the globe form teams and don pedometers for a period of 16 weeks and record their daily step count on the GCC website, which was designed and built by Flash.

Industry

Marketing and Advertisement

Primary hardware

  • Multiple Virtual Web and Database Servers hosted externally running Windows Server 2008

Primary software

  • HP LoadRunner software

Predicting system behaviour and application performance

“The stability and performance of the GCC website is critical to the long-term success of the programme,” explains Carla Clark, digital producer, Flash Group.

“While we undertook some basic testing in 2009, we did not have adequate visibility to obtain an accurate end-to-end picture of the website’s performance, particularly at peak loads. This was apparent when we experienced issues during the 2009 program and it was the impetus for us to seek a performance validation solution.

“Despite the broad experience of our team, we wanted to leverage specialised expertise in performance validation, so we invited JDS Australia to recommend an appropriate software solution. We settled on HP LoadRunner software, due to its functionality, reliability and versatility.”

Partnership provides expertise and speeds time to benefit

An HP Platinum Partner and winner of the coveted HP Software and Solutions Partner of the Year Award for the past four years, JDS is widely regarded as a leader in the BTO space. The company provides extensive and in-depth knowledge of HP’s suite of testing and monitoring solutions, offering support to clients in a variety of industries.

The account manager at JDS Australia believes this is quite an unusual project, as Flash is one of the first creative agencies he has come across that realised the importance of performance validation for a website it had developed. “Ensuring that mission-critical systems such as the GCC website are available and performing as intended is something that all organisations grapple with. However, we don’t often see creative agencies trying to predict system behaviour and application performance at this level – that’s usually the domain of IT teams or developers.

“For organisations (such as Flash) that don’t have in-house performance testing expertise, getting a partner on-board takes the hassle out of deployment. In this instance, JDS provided a roadmap to help Flash mitigate the risk of deploying the GCC website and prevent the costly performance problems it had previously incurred. We helped the team stress test the website to handle the large increase in participants and determine the peak loads and transactional volumes, which in turn enabled us to recommend how best to setup the IT infrastructure. The testing also identified bottlenecks, which the website developers rectified this year.”

Carla Clark believes that having an HP partner involved made all the difference to this project. She says, “Having JDS on board meant that we could focus on our core competencies, while allowing them to do what they do best—provide the services needed to ensure the GCC website would be available and performing as and when required. JDS has assisted Flash in getting the most out of HP LoadRunner in a short space of time.”

Mitigating risk and gaining confidence

The company’s vision in adopting HP LoadRunner was to ensure the GCC website would be scalable in line with the rising number of users. “We wanted to adopt a long-term approach to this project and create a robust website to keep pace with the programme’s planned growth,” explains Tim Bigarelli, senior developer at Flash. “This also entailed the migration to a new IT infrastructure to further enhance our ability to support the website’s evolution.”

Flash began preparations for the launch of this year’s website by having JDS test the previous application on the old infrastructure to establish performance benchmarks. The next round of tests were applied to the new code base using both the old and new infrastructure. “The results uncovered were extremely beneficial as it enabled us to redevelop the website for maximum performance and functionality. But more importantly, it provided us with complete visibility into the performance of the application from end-to-end, which enabled us to verify that the new application would sustain loads of 1,000 concurrent users over the first peak hour on the launch day with an average login time of 7-8 minutes per user and average response times for all pages under two seconds to avoid abandonment,” adds Bigarelli.

Better decisions, operational efficiencies and improved client satisfaction

As a result of deploying HP LoadRunner to validate the performance of the GCC website, Flash has realised considerable benefits. The organisation has facilitated better decision-making, particularly on the development side, experienced operational efficiencies and improved client satisfaction.

Clark says, “HP LoadRunner software takes the guesswork out of the GCC website’s development. It provides confidence that the application will work as intended and it gives us the data we need to support our decisions. In short, it helps us avoid application performance problems at the deployment stage.

“By giving us a true picture of end-to-end performance, diagnosing application and systems bottlenecks and enabling us to tune for better performance, we mitigated the risk of failure for the GCC website. And with access to facts, figures and baseline measurements, we were able to tune the application for success.”

Putting the website to the test

Following considerable testing, Flash launched the GCC website on May 13, 2010. As expected, traffic was extremely high, with an average of 130,000 visitors on the first two days, and a peak of 8,403 visitors in the first hour.

“The GCC website performed according to our expectations and we are delighted with the business outcomes of HP LoadRunner software,” says Clark.

“Thanks to the preparative measures we put in place, our systems thrived and delivered 99.99 percent uptime, with our web servers only reaching 20 percent system capacity and page response times of under two seconds. This enabled us to provide a high-quality user experience, which is enhancing the programme’s brand value.

“Overall, HP LoadRunner software helped us solve key issues this year and identify areas for performance improvements for next year. We have benefited from knowing that performance testing prevents potential failures - such as the ones we experienced last year. As a result, we have considerably reduced the opportunity cost of defects, while driving productivity and quality in our operational environment to deliver a robust GCC website this year, that performs as intended.”

Business Benefits

  • Mitigated the risks of poor performance with a consistent approach to load testing.
  • Adopted a consistent approach to load testing to make confident, informed decisions about the performance and scalability of the GCC website.
  • Gained a true picture of end-to-end performance, which enabled better decision-making and functionality changes.
  • Increased client satisfaction through a fast, high-performing website.
  • Resolved issues with the production architecture and configuration before users were impacted.
  • Gained understanding and confidence in the performance characteristics of the website prior to going live.

Looking ahead

HP will continue to play a key role as the performance validation backbone of the GCC website. By leveraging the functionality and flexibility of HP LoadRunner software, Flash will continue to derive value from predicting system behaviour and application performance. The company is also exploring options to extend its HP investment by utilising the HP LoadRunner scripts with HP Business Availability Center software to monitor the performance and availability of the GCC website from an end user perspective.

In the future, Clark is keen to have someone in the team take the lead on testing. She says: “This project has demonstrated to us just how important testing really is, so we are focused on ensuring it becomes part of our routine development. We are also keen to share the functionality of HP LoadRunner to other clients with similar-sized projects.

“On the whole, HP LoadRunner software has helped Flash mature its website development methodology. We deployed a higher quality GCC website, improved client satisfaction and raised our profile and credibility as an organisation that produces high-performing, user-friendly and scalable websites,” concludes Clark.

Case Study: Bendigo Bank delivers a higher quality customer experience with HP

Case Study, Financial Services, Micro FocusJDS
Bendigo Bank provides banking and wealth management services to individual and small to medium businesses. It is represented in all states and territories with almost 900 outlets, including more than 190 company-owned branches, 250 locally-owned Community Bank® branches, 90 agencies and 800 ATMs.

With a tradition of adding value for customers through quality personal service, the bank recently began to look to technology as the enabler of service delivery and business performance. Realising its existing systems were account-centric and not customer-focused, the bank embarked on an ambitious program to align technology more closely with its business strategy. The result? It purchased Siebel Customer Relationship Management (CRM) and Universal Customer Master (UCM) applications to streamline customer-facing operations.

Known as ‘Enable Customer Phase 1’, the objective of this 18-month project was to introduce CRM and UCM capability across the organisation. As this would significantly impact 5,000 users and would result in considerable change management, the bank knew it had to deliver high-quality applications that functioned and performed at the levels demanded by the business.

“Enable Customer Phase 1 is the single largest implementation undertaken across the Bank in the past 15 years,” explains Robert Murphy, the project’s Technical Implementation Manager. “We had one chance to get it right and we knew quality assurance had to play a big part in the equation. We decided to make use of HP Quality Center software, which has been in the organisation for the past seven years. By leveraging an existing quality management solution, we could reduce our total cost of ownership and ensure a smoother transition to our new CRM platform.”

Objective

To drive the business value of its new customer facing solutions, Bendigo Bank sought to standardise system quality and performance 

Approach

Bendigo Bank adopted a quality and performance assurance approach using HP Quality Center software and HP LoadRunner software

IT improvements

  • Standard platform manages every aspect of system quality and performance
  • Centralisation enhances productivity
  • Isolated and fixed defects quickly
  • Established benchmarks for future enhancements
  • Fine-tuned testing efforts around data migration

About Bendigo Bank

The Bendigo Bank is the retail arm of the Bendigo and Adelaide Bank Group, an Australian company formed in November 2007 as a result of the merger between Bendigo Bank and Adelaide Bank. A publicly listed company, the group is owned by more than 82,000 shareholders.

Industry

Banking and Finance

Primary applications

  • Siebel CRM
  • Siebel URM

Primary software

  • HP LoadRunner software
  • HP Quality Center v9

Partnership provides valuable and timely expertise

To complete the quality approach and ensure all aspects of the new system were tested, the bank appointed JDS Australia to provide services in load testing and performance management.

An HP Platinum Partner and winner of the coveted HP Software Partner of the Year Award for the past four years, JDS is widely regarded as a leader in the Quality and Performance testing space. The company provides extensive and in-depth knowledge of the HP suite of testing and monitoring solutions offering support to clients in a variety of industries.

Steve Smith, JDS Australia’s Account Manager, believes that validating performance of newly deployed mission-critical systems is the key to achieving high user adoption and enhancing the consumer experience. Ensuring that applications are available and performing as intended is something that all organisations grapple with. JDS assisted Bendigo Bank by deploying HP LoadRunner to stress-test its Siebel CRM/URM system to ensure it could handle the peak loads and transactional volumes it would be subjected to, once live.

A quality ownership imperative

Prior to the adoption of HP Quality Center, the bank performed quality assurance on its core systems using a mixture of spreadsheets and documents. Following two mergers, the bank expanded rapidly and decided it needed to standardise its approach to quality assurance as a way of gaining some unity across the business and driving competitive advantage in a tough financial market. Today, the bank is firmly focused on retaining and growing its customer relationships, increasing loyalty and delivering personalised and consistent service experiences.

“We began the quality assurance part of the Enable Customer Phase 1 project by putting the ownership of quality in the hands of the business. We sought to make the business accountable for its operational outcomes. In short, we wanted quality management to be part of everyone’s mandate and HP Quality Center enabled us to do just that,” says Murphy.

“The quality management structure of this project was somewhat unusual. We used an iterative approach to development and put the business analysts, testers and developers into the one team. This allows us to fast-track time to success by facilitating communication and collaboration. But more importantly, it bridged the gap between business and technology expert, aligning testing more closely to business outcomes.”

Standardised processes improve decision-making

By providing a seamless, repeatable process for gathering requirements, planning and scheduling tests, analysing results and managing defects, HP has brought structure to managing quality for this project.

Murphy explains, “HP Quality Center creates an end-to-end quality management infrastructure to enforce standardised processes and best practices, such as our policy of ‘no work without a ticket’. It has given us the ability to streamline the management of defects, so that we can make effective ‘go/no-go’ decisions.

“By standardising on one quality platform we can do a lot of work in a short space of time, knowing that it is all contributing to our overall quality objectives. We can monitor the advancement of our work against these objectives to determine whether we are on track, on budget and on time. Having such insight into our progress delivers good governance and greatly improves decision-making.”

Testing what’s needed reduces risk

With quality firmly embedded in the centre of the organisation’s development mandate, ensuring that testing is prioritised according to business need was vital to achieving timely results for the bank.

HP Quality Center provides risk-based quality management to objectively assess and prioritise the highest-risk, highest-priority requirement, so testing efforts can be fine-tuned based on quantifiable business risk.

“HP Quality Center supports our approach of not wanting to test everything,” adds Murphy. “It enabled us to marry testing priorities with risk. We focused our testing efforts around data migration from our legacy systems into Siebel, as this was an integral part of future functionality.

“Prioritising our testing was also cost-effective in terms of centralisation and reusability. It meant that our people could store tests in one central location, review test planning information and reuse entire test plans or amend test cases across project components. Plus, having access to quality metrics put the business at ease because we could show that elements had been effectively tested and would work as intended.”

Validating performance

Gaining an understanding of how the Enable Customer project would meet the performance and scalability of the business was another objective the bank sought to achieve. Specifically, it wanted to obtain an accurate picture of end-to-end system performance before going live.

HP LoadRunner software was used to emulate the bank’s working environment with thousands of concurrent users. It stressed the application from end-to-end, applying consistent, measurable and repeatable workloads and identified issues that would affect its users in production.

“As we drove loads against the system, HP LoadRunner captured end-user response times for key transactions. It showed us that had we gone live, our users would have experienced slow performance when printing following a query. We rectified the issue in five days, but without HP LoadRunner it could easily have taken us a month or more to fix it.

“In the end, HP LoadRunner verified that our new Siebel CRM/URM system would meet specified performance requirements including sub-second response times,” confirms Murphy.

Quality, confidence, and success

After extensive testing and a successful pilot in two branches, the bank recently went live on Enable Customer Phase 1 without any showstoppers.

“We are delighted with the success of the project’s deployment and have achieved good outcomes through quality and performance testing,” adds Murphy. “Throughout the course of the project, we were able to isolate and fix defects quickly, automate quality processes and establish benchmarks for future enhancements. Quite simply, we delivered a high-quality, high-performing, robust system to support our people.”

“The value that HP Quality Center has brought to Bendigo Bank can be summarised in terms of standardisation, visibility and insight. We gained an end-to-end quality management infrastructure that gave us visibility into every element of the system and the insight we needed to make good decisions.”

Business Benefits

  • Gained 360-degree visibility into application quality
  • Went live on the single largest IT implementation in 15 years (Siebel CRM/URM) which functioned and performed at levels demanded by 5,000 users
  • Rectified performance issue in five days instead of a month
  • Aligned testing to business outcomes by facilitating communication and collaboration among business analysts, testers and developers
  • Reduced application deployment risk
  • Streamlined management process to assist with go/no-go decisions
  • Monitored the progress of work against objectives to track timeliness, budget and readiness

Looking ahead

HP Software will continue to play a key role as the backbone of Bendigo Bank’s quality and performance validation engine.

“We have successfully deployed one of the largest customer-facing projects in the history of the bank. Our focus now is on continuing to manage quality and performance of this system on a quarterly basis, ensuring that updates, changes, and upgrades are validated prior to release.

“Overall, HP has helped Bendigo Bank set the benchmark for ensuring our mission critical applications are high in quality and give the best performance to support our users in delivering excellent products and services,” concludes Murphy.

Case Study: RMIT implementation of LoadRunner

Case Study, Higher Education, Micro FocusJDS
Twice a year, RMIT publishes course results for its students on the student portal of the university’s website. These two days in the calendar generate the highest volume of website traffic. With such huge spikes experienced in a 24-hour period, RMIT’s student portal was being saturated and not handling the volume effectively. This resulted in poor performance and students reported difficulties in accessing their results in a timely fashion.

Tim Ash, RMIT’s senior production assurance and test manager, explains, “The single most important aspect of our students’ lives, apart from the education they receive, is knowing how well they are doing in their studies. Having a stable student portal is critical to satisfy our students’ needs and deliver a high level of service. It also impacts on our credibility and future success.

“For this reason, we identified performance testing as an essential part of future application delivery. With limited resources available in-house and a tight deadline, we invited JDS Australia (JDS), an HP Platinum Partner, to help us to predict future system behaviour and application performance."

Objective

Predict system behaviour and application performance to improve student satisfaction

Approach

RMIT University verified that its new critical student results application (called MyResults) met specified performance requirements

IT improvements

  • Achieved 100 percent uptime on MyResults
  • Delivered student results 60 per cent faster than predicted
  • Gained a true picture of end-to-end performance of MyResults
  • Emulated peak loads of 20,000+ student logins per hour – more than six times the average loads
  • Uncovered and rectified functional issues prior to going live
  • No helpdesk complaints logged for poor system performance
  • Optimisation opportunities were identified that enabled the student portal to be scaled more appropriately

About RMIT University

As a global university of technology and design, RMIT University is one of Australia’s leading educational institutions, enjoying an international reputation for excellence in work-relevant education.

RMIT employs almost 4,000 people and has 74,000 students studying at its three Melbourne campuses and in its Vietnam campuses. The university also has strong links with partner institutions which deliver RMIT award programmes in Singapore, Hong Kong, mainland China, and Malaysia, as well as research and industry partnerships on every continent.

RMIT has a broad technology footprint, counting its website and student portal as mission-critical applications essential to the day-to-day running of the university.

Industry

Higher Education

Primary applications

  • SAP Financials
  • PeopleSoft Student Administration

Primary software

  • HP LoadRunner software
  • HP Quality Center
  • HP QuickTest Professional software

Selecting an industry standard

A long-term user of HP software including HP Quality Center, HP QuickTest Professional and HP Business Availability Center, RMIT chose HP LoadRunner software, as its performance validation platform.

“HP LoadRunner is the industry standard for performance testing,” explains Tim. “It was a natural choice for RMIT, due to its functionality and integration capabilities as well as our investment in other HP software.”

Predicting system behaviour

In seeking to prevent future student portal performance problems, RMIT had identified a number of potential solutions.

Tim explains, “Due to the complex nature of our student portal, it was unclear which solution design would provide the best performing architecture. HP LoadRunner was used to obtain an accurate picture of end-to-end system performance by emulating the types of loads we receive on result days. We tested all the options. Our objective was to handle the loads and satisfy 100 percent of our students’ results queries in under five seconds.”

Emulating loads

By leveraging the experience of JDS, HP LoadRunner was used to emulate peak loads of 20,000+ student logins per hour, which is more than six times the average loads experienced on non-results days. “The data from our tests revealed that we needed our student portal platform to have the ability to scale considerably, to handle traffic up to six times the usual volume on result days,” explains Tim. “As we drove loads against the various design options, we also captured end-user response times.

“Based on this, we selected the design solution that had the best performance. There were several options to choose from with some going through a portal which put unnecessary load on the system. The brand new application chosen is called MyResults, which, during testing, met our performance objective and delivered response times in less than two seconds.”

The JDS account manager for the case says, “As well as testing the performance of the various design solutions, optimisation opportunities were identified that enabled the student portal to be scaled more appropriately. Using HP LoadRunner, JDS consultants identified bottlenecks in the application and platform. By working with RMIT, they were able to direct efforts to remediate the problems prior to going live.”

Successful go-live drives student satisfaction

While MyResults is a simple application that delivers results, its structure is quite complex as it receives inputs from various databases. Ensuring it would perform as intended on results day was key.
Tim says, “By using HP LoadRunner, we significantly decreased the risk of deploying an application that would not meet our performance requirements. On results day, MyResults proved an outstanding success. It handled the loads and spikes extremely well, consistently delivering results in timeframes sub-two seconds. This would not have happened if we had not validated performance beforehand.

“Our students certainly noticed the difference. We received a number of tweets praising the system’s performance. Many of our students couldn’t believe how quickly they obtained their results. Another great indication of our success was the low impact on our helpdesk. They didn’t receive complaints or issues regarding the system and that’s a big plus.”

Boosting reputation

As a result of deploying HP LoadRunner to validate the performance of MyResults, RMIT has realised considerable benefits. The institution has facilitated better decision-making as information is more readily available, and experienced operational efficiencies.

Tim says, “We are delighted with the outcomes of HP LoadRunner. First and foremost, we rectified poor performance issues to provide a results system that exceeded our own goals and those of our students. Thanks to the preparative measures we put in place, our system thrived and delivered 100 percent uptime. This enabled us to provide a high-quality student experience, which culminated in increased user satisfaction.

“Operationally, HP LoadRunner helped us to identify the most suitable option to improve our performance. It gave us confidence (prior to release) in MyResults’ ability and allowed us to make informed business decisions, which reduced our deployment risk. In addition, we saved money, because we were more efficient and did not experience any downtime. Plus, we were able to fix issues in development which is always a cheaper option, and we saved considerable time during testing by having the ability to re-use and repeat tests."

Business Benefits

  • The university's reputation was boosted by providing students with reliable and fast access to results—students tweeted their satisfaction.
  • Money was saved through having zero downtime and the ability to fix issues early.
  • RMIT could make informed decisions and reduce the risk of deploying poor quality applications.
  • University and student confidence in IT systems was improved.

Next steps

HP LoadRunner will continue to play a key role as the performance validation backbone of MyResults. Tim believes RMIT is certainly up-to-date with testing technology with the HP Solutions it utilises, as well as adoption of a stringent testing methodology.

“Students are early adopters of technology. So it’s logical that our next big push is mobility and wireless applications. More and more, our students are accessing RMIT’s website using mobile devices and we need to make sure our applications are optimised accordingly. We want students to be able to log in from home and attend a lecture from their laptop or access their results from their mobile phones. HP LoadRunner will be used to ensure we can meet the wireless requirements of the university’s future.

“Looking at the big picture, HP LoadRunner is an essential component of RMIT’s technology footprint. It allows us to perform due diligence on our applications, make go live decisions with confidence and provide statistical information that is trusted and relied upon by the institution.

“Ultimately, HP LoadRunner gives me peace of mind that RMIT’s systems will work as intended and deliver the quality of service our students and staff expect.”