Test Stage Run
Test Stage Runs (TSRs): A New Approach to Test Execution
SeaLights recognizes the complexities and inefficiencies often associated with traditional testing practices. To address these challenges, we have introduced Test Stage Runs (TSRs).
TSRs (Test-Stage-Runs) represent the execution of a collection of tests from the same test stage against a specific lab, independent of the Application Under Test (AUT). This concept introduces a level of abstraction that decouples test execution from the build itself, providing greater flexibility and value for customers, with more efficient test re-runs and targeted recommendations.
Key Benefits
Faster Value Realization: By separating test execution from the build process, customers can more quickly realize the benefits of SeaLights, including improved test efficiency and recommendations.
Reduced Onboarding Effort: In the future, TSRs will further simplify the onboarding process, requiring less effort and coordination from multiple teams.
How TSRs Work
Test Stage Definition: Define test stages within your SeaLights project.
Data Collection: SeaLights collects data on test execution, coverage (of the applications running on the same lab), and other relevant metrics.
Analysis and Recommendations: The system analyzes the data to provide recommendations for future test runs, focusing on failed tests and recommended tests that were not executed (assuming the builds of the apps running on the lab didn't change).
Test Stage Run (TSR) Calculation
TSRs are automatically calculated based on the following criteria:
Explicit Definition (via API): If customers use the SeaLights API to explicitly define a TSR, the start and end points are determined by these API calls.
Implicit Calculation: In the absence of explicit API calls, TSRs are calculated automatically based on:
Deployments: A new TSR is created when a deployment occurs, capturing all test executions initiated after the deployment.
Timeout: If no test executions are open within a specified timeout period, the current open TSR will be closed and a new one will be created as soon as a new execution will be started.
This approach ensures that TSRs accurately reflect the actual test execution patterns within your environment, allowing for granular analysis and optimization.
Value for Customers
Optimized Re-Runs: Reduce the time and effort required for re-runs by focusing on relevant tests. When re-running the same test stage, TSRs enable SeaLights to provide more targeted recommendations, focusing only on failed tests and recommended tests that were not previously executed.
Granular Analysis: Analyze coverage data and test execution data at the TSR level, gaining deeper insights into test execution performance.
TSRs provide a powerful mechanism for managing and analyzing test execution within SeaLights. By understanding the concept of TSRs and leveraging their benefits, customers can optimize their testing processes and achieve greater efficiency and effectiveness.
Last updated