Test Optimization Models: Cross-App vs. App-Level

Overview: Two Models for Test Optimization

Test Optimization supports two primary models for generating test recommendations, ensuring efficiency across both single-application and complex, multi-component testing environments. The model used depends on whether your test stage is coupled to a specific application build or targets a whole environment.

1. App-Level Test Stages

This is the traditional model for test stages that execute against a single application build.

• Definition: An App-Level Test Stage is explicitly linked to a single application. App-Level stages are primarily executed within CI pipelines and are often the first testing step. The onboarding process is simpler as it only requires linking to a single application repository and a unique build identifier.

• Applicability: This model is for tests where coverage and test recommendations need to be tightly coupled to a single application's source code, such as:

  • Unit Tests: Testing a single application component.

  • Component Tests: Testing a service in isolation.

  • Any testing where the scope is limited to one application.

• Recommendation Logic: Test Optimization calculates recommendations by comparing the current application build against the immediately preceding build of that same application. This provides a precise view of code changes only within that single application.

2. Cross-App Test Stages

The Cross-App Test Stage provides a streamlined approach to testing in dynamic lab environments. It is onboarded directly onto a lab/environment and executes tests against all applications currently deployed in that environment.

• Definition: A Cross-App Test Stage is configured directly to a lab or testing environment and is not tied to any single application or build. They are designed for Integration or System Test Environments where multiple services interact.

• Applicability: This model is ideal for integration, system, or end-to-end (E2E) test stages that span multiple services, where the state of the entire environment dictates the required test subset.

• Recommendation Logic: Test Optimization supports this concept by generating test recommendations at the Lab/Environment level, rather than the individual application or build level. This allows for accurate test impact analysis across complex, multi-component deployments.

• Tracking Included Applications: It is easy to track precisely which applications (AUTs - Applications Under Test) were considered for the recommendation calculation. This information is visible in the UI on the Savings Breakdown page, under the AUT tab. This ensures transparency in the optimization process.