Cross-App Test Stage Strategy Configuration

The Strategy Configuration panel is the centralized interface for managing Test Optimization for cross-app test stages. It provides intuitive, level-based configuration with granular controls over execution scope (e.g., Pull Requests vs. Baseline runs) and automatic full runs.

Access and Permissions

• Availability: Currently available only for Cross-App Test Stages.

• Navigation: Accessed via the "Configure Strategy" button on the Single Test Stage Overview page.

• Roles: Only users with Admin or DevOps roles can modify these settings. For all other roles, the configuration button is disabled.

Enabling Test Optimization

At the top of the panel, a master toggle controls the activation state of Test Optimization for the stage.

• Persistence: If optimization is turned OFF, the configuration is saved. If re-enabled later, the last saved settings are restored.

• Historical Data: Turning optimization OFF does not affect the status of historic test cycles that ran while it was active.

Test Optimization Levels

SeaLights offers three predefined levels to define the "aggressiveness" of test selection:

Level	Behavior	Best Fit For...
Conservative	Basic Test Optimization. Runs every test identified as impacted. No additional pruning. Ideal for CD/frequent deployments.	Teams with CD/hourly deployments requiring guaranteed full validation daily.
Moderate	Default. Uses the basic Test Optimization algorithm plus a 80% "Maximum Capacity" filter. Prioritizes high-confidence testing for standard CI cycles.	Standard weekly/bi-weekly releases where automation is the primary gate.
Aggressive	Uses the basic Test Optimization algorithm plus a 60% "Maximum Capacity" filter. Selects the most critical subset for the fastest possible feedback.	Teams with high-volume code pushes or manual QA/Staging environments.

Note: The specific percentages for Moderate/Aggressive are configurable manually per customer.

Learn more about how recommendations are being generated for each optimization level on Generating Test Recommendations.

Periodic Full Run

To mitigate the risk of skipping tests over time, users can schedule automated full runs.

• Scope:

  • No Automatic Full Run: Only runs on-demand (Read more about on demand full run onTest Stage Strategy Overview).

  • Skipped Impacted Tests: Runs only the tests that were skipped by the optimization level in previous cycles.

  • Entire Test Suite: Runs all tests, regardless of impact.

• Frequency: Options include Daily, Weekly, Monthly, or a Custom interval (e.g., every 3 cycles, every 2 weeks).

Automated Risk Management: Periodic full runs ensure that deferred risks (skipped tests) are eventually addressed automatically.

Dynamic Reset: Changing the Optimization Level (e.g., from Moderate to Aggressive) will automatically update the "Full Run" frequency to match system defaults for that level.

Optimization Scope

This section defines which types of executions get optimized:

• All Executions (Default): Optimizes every run.

• PR Executions Only: Baseline runs are always "Full Runs"; only PRs are optimized.

• Non-PR Executions Only: PRs are always "Full Runs"; baseline runs are optimized.

Important: "Full Run" settings defined in the Strategy panel are automatically ignored for PR-level executions to ensure maximum speed during code review.

Learn how to configure Test Optimization for automatic testing, manual testing and shift-left testing, with our detailed playbooks.

---

FAQ

If I turn Test Optimization OFF, will I lose my custom settings?

No. SeaLights persists your configuration. If you toggle it back ON, your previous levels, filters, and full-run schedules will be restored.

Does the "Maximum Capacity" filter apply to PRs?

Yes, if the "Optimization Scope" includes PRs, the selected Strategy Level will be applied to PR executions. However, the Periodic Full Run schedule is ignored for PRs.

How does "Custom" frequency work for Full Runs?

You can set a specific integer value and a unit (Days, Weeks, Months, or Cycles). For example, setting it to "5 Cycles" ensures that every 5th execution is a full run, regardless of time elapsed.