Test Stage Strategy Overview

The Test Stage Strategy Overview page provides a dedicated dashboard for individual Cross-App Test Stages within SeaLights. It serves as a central hub for monitoring the effectiveness of Test Optimization, offering detailed performance metrics, historical savings data, and administrative controls to adjust optimization strategies.

This page allows teams to move beyond high-level summaries and analyze the specific impact of Test Optimization on a per-stage basis, ensuring that testing cycles are both efficient and reliable.

Strategy Status & Configuration

At the top of the page, users can view and manage the current optimization state:

• Optimization Level: Displays the current mode: Conservative, Moderate, or Aggressive.

• Configure Strategy Button: Enabled only for Admin/DevOps Users. Opens the Configuration panel to modify the strategy settings.

• Full Run Button: Allows users to manually trigger a full test suite execution for the next cycle.

  • If a full run is already scheduled, the button is disabled until the pending request is done.

Overview Metrics

Primary KPIs provide an immediate snapshot of the stage's efficiency:

• Average Time Savings Rate: Average of the percentage of time saved with Test Optimization, in the last three full months of data.

• Average Time Saved: Average of the total duration (Hours/Minutes) removed from the testing cycle, based on Test Optimization, in the last three full months of data.

• Tracking Period: Total months of active Test Optimization tracking.

• Annual Trend Chart: A 12-month visual representation of savings rate trends.

Monthly Savings Breakdown Table

This table provides a month-by-month historical trends (up to 12 months) of all test cycles executed with Test Optimization On, to validate ROI.

Table Columns

Column	Description
Month	Pending tag for the current month or Final for completed months.
Test Cycles	Total cycles executed that utilized Test Optimization.
Original Time	Estimated time to run all tests without optimization.
Actual Time	The actual time spent executing optimized test sets.
Saved Time	See calculation here.
Time Savings Rate	See calculation here.

Customization & Interactions

• Column Customizer: Users can toggle additional columns such as Original/Actual/Saved Test Counts and % Test Savings. These preferences are saved per user.

• Breakdown Charts: Clicking the chart icon on a row opens a side panel with visual distributions of test results and cycle data.

• Breakdown Analysis: Clicking the icon opens the Savings Breakdown page in a new tab for deep-dive investigation and troubleshooting (Available to Admin/DevOps users only).

---

FAQ

Can I see historical data if Test Optimization is currently turned Off?

Yes. If the stage has historical data collected while Test Optimization was previously On, the metrics and history table will remain available for review even if the current status is Off.

Why is the "Configure Strategy" button disabled for me?

Only users with Admin or DevOps roles have the permissions required to modify optimization strategies.

What is the difference between "Pending" and "Final" months in the table?

A month is marked as Pending as long as the calendar month is still active. Once the month concludes, the data is locked and marked as Final.

How is "Saved Time" calculated?

See full explanation in Savings Calculation.

Does this report replace the Savings Breakdown page?

No. The Savings Breakdown page remain available for deep dive analysis.