Test Stage Cycle

Test Stage Cycles: A New Approach to Test Execution

SeaLights recognizes the complexities and inefficiencies often associated with traditional testing practices. To address these challenges, we have introduced Test Stage Cycles.

Test Stage Cycles represent the execution of a collection of tests from the same test stage against a specific lab, independent of the Application Under Test (AUT). This concept introduces a level of abstraction that decouples test execution from the build itself, providing greater flexibility and value for customers, with more efficient test re-runs and targeted recommendations.

Key Benefits

• Faster Value Realization: By separating test execution from the build process, customers can more quickly realize the benefits of SeaLights, including improved test efficiency and recommendations.

• Reduced Onboarding Effort: In the future, Test Stage Cycles will further simplify the onboarding process, requiring less effort and coordination from multiple teams.

How Test Stage Cycles Work

1. Test Stage Definition: Define test stages within your SeaLights project.

2. Data Collection: SeaLights collects data on test execution, coverage (of the applications running on the same lab), and other relevant metrics.

3. Analysis and Recommendations: The system analyzes the data to provide recommendations for future test cycles, focusing on failed tests and recommended tests that were not executed (assuming the builds of the apps running on the lab didn't change).

Test Stage Cycle Calculation

Test Stage Cycles are automatically calculated based on the following criteria:

• Explicit Definition (via API): If customers use the SeaLights API to explicitly define a Test Stage Cycle, the start and end points are determined by these API calls.

• Implicit Calculation: In the absence of explicit API calls, Test Stage Cycles are calculated automatically based on:

  • Deployments: A new Test Stage Cycle is created when a deployment occurs, capturing all test executions initiated after the deployment.

  • Timeout: If no test executions are open within a specified timeout period, the current open Test Stage Cycle will be closed and a new one will be created as soon as a new execution will be started.

This approach ensures that Test Stage Cycles accurately reflect the actual test execution patterns within your environment, allowing for granular analysis and optimization.

Value for Customers

• Optimized Re-Runs: Reduce the time and effort required for re-runs by focusing on relevant tests. When re-running the same test stage, Test Stage Cycles enable SeaLights to provide more targeted recommendations, focusing only on failed tests and recommended tests that were not previously executed.

• Granular Analysis: Analyze coverage data and test execution data at the Test Stage Cycle level, gaining deeper insights into test execution performance.

Test Stage Cycles provide a powerful mechanism for managing and analyzing test execution within SeaLights. By understanding the concept of Test Stage Cycles and leveraging their benefits, customers can optimize their testing processes and achieve greater efficiency and effectiveness.