The great testing pyramid of devops

Utilising the test pyramid effectively in software development requires thoughtful consideration before reaping its benefits. Let’s delve into the principles of the Cheops DevOps’ Great Testing Pyramid and ensure its proper application for maximum value.
What is the test pyramid?

The test pyramid, much like actual pyramids, may seem simple on the surface but proves to be intricate and challenging to navigate internally, without the inclusion of deadly traps. While not a perilous structure, its complexity requires careful consideration for effective use in software development.

In its essence, the test pyramid serves as a heuristic for guiding efforts in test automation projects. Originating in agile circles around 2009, it proposes that the longer tests take to run, the fewer of those tests should be implemented. A more detailed explanation can be found on the same site, with Mike Cohn introducing it in his work “Succeeding with Agile,” though it has gained recognition through Google’s testing blog as well.

While some hail it as a silver bullet for agile development, there is a contrasting viewpoint from those less impressed, a group I happen to align with. In this post, my aim is to elucidate why I hold this perspective.

What is the problem with it?

The initial challenge arises from the fact that unit tests, despite their name, do not hold substantial value from a Quality Assurance (QA) perspective. They tend to uncover fewer bugs and, notably, miss many common bug types, such as integration bugs. Additionally, unit tests demand significant maintenance during the early phases of projects and may not be suitable for certain types of code. It’s important to note that perceptions of unit tests may differ among individuals, and my perspective might not align exactly with yours.

The second, more apparent issue stems from the lack of a standardised definition for terms like unit tests and integration tests. Varied interpretations among different practitioners contribute to the confusion, making consensus on these definitions challenging. This lack of agreement is a fundamental issue and a primary cause of ongoing debates regarding best practices in the field.

The third challenge involves diverse requirements for different teams based on factors like size, project type, and codebase. A best practice endorsed by a larger organisation like Google may not necessarily translate well into a smaller shop, as the optimisation efforts of a larger entity may not address problems encountered by smaller organisations. This variability in needs adds another layer of complexity to the ongoing discussions surrounding best, better, and good practices in the field.

Can we fix the problems?

It’s quite common for people to engage in online disputes over differing definitions, and I found myself falling into that pattern with the test pyramid. Initially, I never looked beyond my own interpretations of unit and integration tests, dismissing the entire concept as a somewhat silly notion—a new silver bullet for agile consultants and evangelists to wield against their teammates. However, I later discovered that, with appropriate definitions of units and integrations, the test pyramid can indeed be meaningful.

Despite my personal reservations about the definitions of unit tests and integration tests, when it comes to understanding the benefits of the pyramid, these specific interpretations prove to be spot-on. It’s a reminder that sometimes adopting different perspectives can shed new light on established practices, even if those definitions might initially seem inappropriate from a personal standpoint.

Unit

Code-only tests. Possibly testing multiple genuine classes interacting. Only testing non-trivial code (no getters, setter, or otherwise trivial functions).

Integration

System-level integrations: External dependencies and running services needed by the app.

E2E

UI-driven testing. Click, type, and get-text your way to glory.

The internal model of what those should be is very different.

Unit

One class (or one file, or one function). Test the contents in absolute isolation. To verify and document correct interface behavior.

Integration

ALL THE OTHER CODE-BASED TESTS. If you don’t mock all the things you now have an integration test.

E2E

All the tests that require actual running software to be able to run.

Finally

To effectively use the test pyramid, it’s essential to grasp the specific testing levels your project requires. Consider the unique needs of unit tests, integration tests, and end-to-end (e2e) tests, recognising that not every project is suitable for unit tests, and traditional UI-based e2e tests may not be universally applicable. Projects differ in terms of quality expectations, execution speed, and available resources.

  1. Begin by clearly defining the terms specific to your project. What constitutes a unit test, an integration test, or an end-to-end (e2e) test in your context?
  2. Evaluate how these tests should be grouped and determine when and how they will be executed. Explore ways to organise and execute them effectively.
  3. Discard the pyramid metaphor and shape the testing strategy to align with the unique requirements of your project. Find the structure that best suits your needs.
  4. Identify heuristics that fit your project and figure out how to apply them. Tailor these principles to your specific context for optimal results.

What do they call this process? It’s simply test strategy and test planning, infused with a bit of test design. Surprisingly, these three activities prove too intricate to be adequately represented by a simplified three-layer pyramid.

For more information

Leave your details and our team will contact you:

Copyright RQTech.  All rights reserved.  Click here to view our privacy and data policy.

Get a quick overview so you can spend your time improving your processes.

You can easily define and set up processes according to your own needs using a convenient layout tool. In addition, since all types of workflow processes can be managed, you get all case management in one single system. 

Analyses, statistics and reports give you an unbeatable overview with real-time status and division of responsibility. Nothing has to fall through the cracks anymore and you always have an accurate and up-to-date base for decision-making.

CANEA Process is a tool that allows you to model and share business processes in an easy-to-use graphical web interface.

Seeing is understanding

Visualisation gives all employees an understanding of the organisation’s processes, activities, responsibilities and information flows.

Living processes

Identifying working methods while making it easy to update is the basis for continuous improvement of processes.

Create a complete picture

Linking together documents, information and tools with clickable process maps create an intuitive and comprehensive management system.

CANEA Workflow is an IT solution that automates, quality-assures and speeds up administrative workflow processes.

You can easily define and set up processes according to your own needs using a convenient layout tool. In addition, since all types of workflow processes can be managed, you get all case management in one single system. 

Analyses, statistics and reports give you an unbeatable overview with real-time status and division of responsibility. Nothing has to fall through the cracks anymore and you always have an accurate and up-to-date base for decision-making.

By creating executable processes in CANEA Workflow, you get both better control of the situation and smoother processing.

Streamlining the work

Ensure compliance and that handovers are done correctly and with the right information.

Correct decision support

You get an unbeatable overview of the processes in real time, with both clear reports and clear diagrams.

Improve processes continuously

Our process support can be constantly adapted to changing needs and requirements. In this way, we give you the best possible conditions for your daily work.

A document management system without complicated folder structures.

CANEA Document combines simple and intuitive search features with powerful features for managing documents from a life cycle perspective.

Maximum availability
Search and find information quickly based on what you need, not where it is stored.

High security
Ensure accuracy, changes and access to all information – with high traceability.

Fulfils requirements
Manage information according to standards, legislation and other requirements for document management.

CANEA Document supports everything from production, publication and modification to archiving and deletion – with full traceability and version management.

CANEA Document provides secure management of all types of documents – in one place. Tagging the information with metadata creates a virtual, multidimensional folder structure. This means that a document appears in multiple locations at the same time, with authorisation-controlled access. The right information in the right place for the right users at the right time!

You can usefully add your company’s process-oriented management system to CANEA Document. All production and distribution of documents is quality-assured and streamlined. In addition, your employees always have access to the most up-to-date version of the documents – directly via intranet, tablets and mobile phones.

Improve the performance of your project activities

CANEA Project is a comprehensive project tool with integrated support for all types of projects and parties– such as management, resource owners, clients, project managers and project participants. CANEA Project gives you an excellent overview of your entire project portfolio, including profitability and status, making it easier to prioritize and make decisions.

CANEA Project shares all important project information with both internal and external members.

Let your project manager focus on management rather than administration and distribution of the information.

Gather all project information in one place and make it available to both internal and external members.

Helps you with prioritization of projects, resource management and analysis of portfolios and programs.

Make reality of the strategy

CANEA Strategy makes it possible for organisations of all sizes to create a unique common thread from the strategic work to the daily operations. We do not just provide performance management tools but rather a completely new generation of IT support for strategy activation. The system guarantees and provides support throughout the chain from strategy to execution. You get no results without action. CANEA Strategy makes it possible in practise! 

CANEA Strategy ensures and provides support throughout the entire journey from strategy to execution.

Create a shared understanding of the strategy, the goal to achieve and how.

Gives management an unbeatable overview of what’s happening, how it’s progressing, and why.

Creates a clear common thread from the strategy to project, initiatives and actions.