DevOps

Is DevPerfOps a thing?

Is DevPerfOps a thing?

New technology terms are constantly being coined. One of our lead consultants answers the question: Is DevPerfOps a thing?


Hopefully it’s no surprise that traditional performance testing has struggled to adapt to the new paradigm of DevOps, or even Agile. The problems can be boiled down to a few key factors:

  1. Performance testing takes time—time to script, and time to execute. A typical performance test engagement is 4-6 weeks, during which time the rest of the project team has made significant progress. 
  2. Introducing performance testing to a CI/CD pipeline is really challenging. Tests typically run for an hour or more and can hold up the pipeline, and they are extremely sensitive to application changes as the scripts operate at the protocol level (typically HTTP requests). 
  3. Performance testing often requires a full-sized, production-like infrastructure environment. These aren’t cheap and are normally unavailable during early development, making “early performance testing” more of an aspirational idea, rather than something that is always practical.

All the software vendors will tell you that DevOps with performance testing is easy, but none of them have solved the above problems. For the most part, they have simply added a CI/CD hook without even attempting to challenge the concept of performance testing in DevOps at all.

A new world

So what would redefining the concept of performance testing look like? Allow me to introduce DevPerfOps. We would define the four main activities of a DevPerfOps engineer as:

  1. Tactical Load Tests
  2. Microbenchmarking
  3. APM and Code Reviews
  4. Container Workloads

Let’s start at the top.

Tactical Load Testing

This is can be broadly defined as running performance tests within the limitations of a CD/CI pipeline. Is it perfect? No. Is it better than not doing anything or struggling with a full end-to-end test? Absolutely, yes!

Tactical Load Testing has the following objectives:

  • Leverage CI toolkits like Jenkins for execution and results collection.
  • Use lightweight toolsets for execution (JMeter, Taurus, Gatling, LoadRunner, etc. are all perfectly good choices).
  • Configure performance tests to run for short durations. 15-20 minutes is the ideal balance. This can be achieved by minimising the run logic, reducing think times, and focusing on only testing individual business processes.
  • Expect to run your performance tests on a scaled-down environment, or a single server node. Think about what this will do to your workload profile.

Microbenchmarking

These are mini unit or integration tests designed to ensure that a component operates within a performance benchmark and that deviations are detected.

Microbenchmarking can target things like core application code, SQL queries, and Web Service integrations. There is an enormous amount of potential scope here; for example, you can write a microbenchmark test for a login, or a report execution, or a data transformation--anything goes if it makes sense.

Most importantly, microbenchmarking is a great opportunity to test early in the project lifecycle and provide early feedback.

APM and Code Reviews

In past years, having access to a good APM tool or profiler, or even the source code, has been a luxury. Not anymore--these tools are everywhere, and while a fully featured tool like AppDynamics or New Relic is beneficial for developers and operations, a lot can be achieved with low-cost tools like YourKit Profiler or VisualVM.

APM or profilers allow slow execution paths to be identified, and memory usage and CPU utilisation to be measured. Resource intensive workloads can be easily identified and performance baked into the application.

Container Workloads

Containers will one day rule the world. If you don’t know much about containers, you need to learn about them fast.

In terms of performance engineering, each container handles a small unit of workload and container orchestration engines like Kubernetes will auto-scale that container horizontally across the entire data centre. It’s not uncommon for applications to run on hundreds or thousands of containers.

What’s important here is that the smallest unit of workload is tiny, and scaling is something that a well-designed application will get for free. This allows your performance tests to scale accordingly, and it allows the whole notion of “what is application infrastructure” to be challenged. In the container world, an application is a service definition… and the rest is handled for you.

So, is DevPerfOps a thing? We think so, and we think it will only continue to grow and expand from here. It’s time that performance testing meets the needs of 2018 IT teams, and JDS can help. If you have any questions or want to discuss more, please get in touch with our performance engineering team by emailing contactus@jds.net.au. If you’re looking for a quick, simple way of getting actionable insights into the performance health of your website or application, check out our current One Second Faster promotion.

Our team on the case

Every day, do something that people want.

Nick Wilton

Consultant

Length of Time at JDS

8.5 years

Skills

Primary: Software security, Performance optimisation

Secondary: DevOps, Software development, Technical sales

Workplace Solutions

I help clients to solve problems like:
  • Is my application secure?
  • How do I manage threats?
  • Will my application perform when I need it to?

Workplace Passion

It’s all about managing risk whilst driving business confidence in technology and software solutions. That’s what I’m passionate about.

Read more Tech Tips

Posted by JDS Admin in Tech Tips
PagerDuty

PagerDuty

Digital Operations Management: Turn signals into real-time insight and action to mobilise teams, resolve disruptions, and deliver high-performing digital services.

START A FREE 14-DAY TRIAL


Integrate machine data with human intelligence

PagerDuty arms organisations with powerful real-time context and automation to accelerate resolution times and delight customers. The PagerDuty platform harnesses digital signals and applies machine learning to get the right data in the hands of the right people at the right time. Teams are empowered to quickly mitigate threats as well as action opportunities.

The digital operations management platform

The PagerDuty Digital Operations Management Platform gives organisations the actionable insights they need to turn data into opportunity across every operational use case—including DevOps, ITOps, Security, Support, and beyond. Resolve and prevent issues, improve customer and employee retention, and gain back time for innovation.

Why PagerDuty?

Connect your data

Connect any tool in minutes with self-service integrations to visualise and action data

Automate events

Make relevant context accessible when it’s needed most with automated escalations and flexible workflows

Empower Agile teams

A team-oriented experience helps teams mobilise fast and understand service impact rapidly

Resolve and prevent issues

Drive response times down to seconds and reduce resolution times to boost productivity

Analyse and learn

Learn from postmortems and gain actionable insights across system and team health

JDS capabilities with PagerDuty

Integration

Setup of the platform and integration with your existing environment is integral to successful deployment of PagerDuty. With JDS experience, your organisation can hit the ground running and build with best practices. More than 200 out-of-the-box integrations in PagerDuty and the support of JDS will help you build custom integrations that ensure your use of PagerDuty is maximised.

Workflow automation

JDS can help your organisation interface and deal with the large volumes of data and eventing as a result of integrating your tools into PagerDuty. The centralisation of data and eventing allows you to define new workflows for event management and incident response, as well as automation where valuable. JDS consultants are skilled in identifying these opportunities and quickly implementing valuable solutions in PagerDuty.

Best practice

PagerDuty incident response notifies the right people in your team immediately so they can quickly assess, triage, and resolve issues when they occur. Implementing this tool with the expert assistance of JDS consultants—who leverage their cross-industry experience to knowledge of ITSM and ITOM—will ensure you meet SLAs and best practice standards for IT management.

DevOps support

JDS applies Continuous Integration and Continuous Delivery principles to software delivery and use this experience to uplift the DevOps capabilities of your organisation. Effective delivery-lifecycle shortening is the primary goal and is achieved with PagerDuty’s positive impact to communication and efficacy of defect/incident management. DevOps teams are also given more opportunity to grow and innovate when application and infrastructure issues are resolved quickly.

Enterprise enhancement

JDS is partnered with AppDynamics (Application Performance Management), Splunk and ServiceNow (IT Service Management). These partnerships will ensure that your journey with PagerDuty is supported by broader, cross-product knowledge and business acumen. JDS consultants come as skilled practitioners in these products and regularly support IT Operations, NOCs, and DevOps teams to improve on organisational agility, performance, and uptime.

Why choose JDS?

JDS is the leading PagerDuty integrator in Australia and a solutions partner of AppDynamics, Atlassian, Micro Focus, ServiceNow, and Splunk. We provide local, skilled, and responsive services to support IT projects and operations. Bringing together expert services, the latest technology, and best practices, JDS achieves improved IT outcomes for businesses. We do this by giving independent advice, providing training and ongoing support, and implementing IT testing, monitoring, and management solutions.

Our PagerDuty stories