Introducing Test Stage Runs
Last updated
Last updated
SeaLights is excited to announce a new concept: Test Stage Runs (TSRs). This new feature revolutionizes how we manage and analyze test execution, empowering you to achieve greater efficiency and insight.
TSRs represent the execution of a set of tests from the same test stage, independent of the specific build. By decoupling test execution from the build process, TSRs enable more granular analysis and optimized test re-runs.
The Coverage Dashboard now offers a granular view of TSR data, for each build and test stage. You can now view detailed information about each test stage run, including the number of tests, pass/fail rates, and execution times.
The Savings Breakdown page now offers a TSR view, providing a detailed breakdown at the TSR level.
Note: The choice of view (TSR or Build) is independent of the TIA recommendation mode. You can switch between these views regardless of whether TIA recommendations are based on builds or TSRs.
Build View: Each build card now displays the number of associated TSRs, providing a quick overview of the test activity within that build.
TSR View: Each TSR is represented by an individual card, allowing you to drill down into specific test executions. View details about each test within a TSR, including its status, code changes, and recommendations.
The associated build is clearly indicated on each TSR card, providing context for the test execution.
A new tab is available when on a Build View. For each given build you can now see the list of TSRs with their status, number of tests and test overview.
The Build View now includes a new tab that provides detailed information about the Test Stage Runs associated with a specific build. This allows you to see the status of each TSR, including whether it's completed or still running, and gain insights into the number of tests executed within each TSR.
The charts within the Savings Breakdown have been updated to provide more granular insights at the TSR level. When viewing data in TSR mode, charts will display information specific to individual test stage runs.
You can now choose whether to base TIA recommendations on builds or TSRs.
Build-Based Recommendations: If multiple test runs are associated with the same build, they will share the same set of recommended tests. This approach may lead to redundant test executions, especially for frequently re-run builds (e.g. re-running passed tests, even though no code changes were made).
TSR-Based Recommendations: Recommendations are generated for each TSR, focusing on tests that failed or were not executed in previous runs within the same build. This approach optimizes test execution by avoiding unnecessary test re-runs.
By leveraging TSRs and the enhanced features in the Savings Breakdown, you can optimize your testing processes, reduce costs, and improve software quality.
