LogoLogo
Product
  • Knowledge Base
  • What's New
  • Guides
  • User Story Coverage
    • Getting Started
    • User Story Challenges & Solution
      • Typical Implementation
      • The Challenges
      • The Solution
    • User Story Coverage Report Overview
      • Release Go / No Go Report
        • How to Generate / Edit the Report
      • User Story Quality Overview
        • How to Generate the User Story View
      • User Story Coverage Analysis
        • How to Generate the Analysis View
      • Uncovered Methods View
        • How to Generate the View
      • Customization
      • Integration
    • Use Cases by Persona
      • Managers
        • Informed Go/No Go Decisions Making
        • Effective Resources Prioritization
        • Overall Progress Monitoring
      • Developers
        • Code Quality Ownership
        • Seamless Collaboration with QA
        • Code Review Facilitator
      • QA Engineers
        • Test Execution Progress Monitoring
        • Testing Effort Prioritization
        • Testing Strategy Planing
    • Technical Overview
      • User Story Coverage Mechanism
      • Technical Architecture
      • Deployment Guide
        • US1_getResults.sh
        • US2_createReport.sh
        • US_UpdateConfluence.sh
  • Test Optimization
    • Getting Started
    • Test Execution Challenges & Solution
      • The Challenges
      • Test Optimization Solution
      • Test Optimization Main Advantages
    • Test Optimization Overview
      • Automated Test Optimization
      • Manual Test Optimization
      • Test Optimization for Pull Request
      • Test Selection Policies
        • Full Run Policy
        • No Code Changes Policy
        • Common Code Policy
        • Fastest Path to 100% Coverage Policy
      • Integrations
    • Use Cases by Persona
      • Managers
        • Fast Delivery
        • Resource Optimization
        • Thorough Testing in Tight Schedule
      • Developers
        • Exploring Only Relevant Test Failures
        • Faster Feedback Loop
        • Shift Left Testing
      • QA Engineers & Manual Testers
        • Faster & Focused Manual Testing
        • Optimizing Test Suite
        • Having Stable Product for Testing
    • Technical Overview
      • Test Optimization Mechanism
        • Associating Code With Tests
          • Statistical modeling
          • One-to-One Mapping
          • Calibration
        • Detecting Modified Code
        • Generating Test Recommendations
      • Technical Architecture
      • Deployment Guide
  • Quality Improvement
    • Getting Started
    • Challenges & Approach Comparison
      • The Challenges
      • Quality Improvement Approaches
      • Choosing the Right Approach
    • Quality Improvement Solution Overview
      • Test Gaps Analysis Report
        • How to Generate / Edit the Report
      • Coverage Trend Report
        • How to Generate / Edit the Report
      • Proof of Testing Report
        • How to Generate / Edit the Report
      • Release Quality Improvement Guide
        • STEP 1: Deploy SeaLights
        • STEP 2: Take a Quality Snapshot
        • STEP 3: Prioritize Code Areas
          • Add Code Labels
          • Ignore Irrelevant Code
          • Perform a Deep CSV Analysis
        • STEP 4: Set Baseline & Threshold
        • STEP 5: Analyze Test Gaps
        • STEP 6: Write Tests
        • Step 7: Make a Go / No Go Decision Based on Quality Gate
        • STEP 8: Measure Defect Escape Rate
      • Over Time Quality Improvement Guide
        • STEP 1: Deploy SeaLights
        • STEP 2: Take a Quality Snapshot
        • STEP 3: Prioritize code areas
          • Add Code Labels
          • Ignore Irrelevant Code
          • Perform a Deep CSV Analysis
        • STEP 4: Set Baseline & Goal
        • STEP 5: Set timeline
        • STEP 6: Write tests
        • STEP 7: Monitor progress
        • STEP 8: Measure Defect Escape Rate
    • Use Cases by Persona
      • Managers
        • Effective Prioritization & Budget Allocation
        • Tracking Progress & Measuring Impact
        • Data-Driven Release Decisions
        • Transparency & Communication
      • Developers
        • Mastering Code Coverage
        • Seamless Collaboration with QA
        • Code Quality Ownership
      • QA Engineers
        • Prioritizing Test Efforts
        • Contributing to Release Informed Decisions
        • Seamless Collaboration with Developers
        • Evaluating Testing Strategy
    • Technical Overview
      • Solution Mechanism
      • Technical Architecture
      • Deployment Guide
  • Value Proposition
    • Overview
    • Quality Use Cases
      • Go/No Go Decisions
      • Quality Improvement & Test Gaps
      • Governance & Quality Gates
      • Compliance & Proof of Testing
    • Test Optimization Use Cases
      • Reduce Costs & Infrastructure
      • Shorten Release Cycles
      • Reduce Troubleshooting
Powered by GitBook
On this page
  • Determining Full Run Frequency
  • Setting the Policy

Was this helpful?

  1. Test Optimization
  2. Test Optimization Overview
  3. Test Selection Policies

Full Run Policy

PreviousTest Selection PoliciesNextNo Code Changes Policy

Was this helpful?

Test Impact Analysis (TIA) provides the option to recommend a full test run based on policy. This strategic approach offers two key benefits:

  • Enhanced Mapping Accuracy: Statistical modeling forms the foundation of TIA, and the more test executions TIA captures, the more precise its connection map becomes. Therefore, scheduling regular full test runs ensures that TIA maintains accurate mappings, addressing changes in tests and modifications to test configurations, identifying deleted or renamed tests, and improving the statistical modeling of relatively new tests.

  • Comprehensive Coverage for Critical Changes: When dealing with critical or sensitive code modifications, a full test run is essential to ensure thorough coverage and comprehensive testing. This can be achieved by either setting up a recurring full run or utilizing the option to designate the next build as the full run.

Determining Full Run Frequency

While periodic full test runs are crucial for TIA's accuracy, it's essential to balance this necessity with the goal of maximizing savings and adhering to the TIA's purpose of reducing test execution time. Here's a recommended frequency for full runs based on test stage execution patterns:

  • For test stages running a few times a day: Set the full run policy to once a day, typically at the start of the day (00:00).

  • For test stages running a few times a week: Set the full run policy to once a week, typically on Monday at the start of the week (00:00).

  • For extensive manual test stages: Set the full run policy to once a month, typically at the 1st of the month (00:00).

  • For any other use cases: Set the full run policy to every X builds for a specific test stage.

Important! After utilizing TIA for a while, evaluate the savings achieved for a particular app and test stage. If savings are not as expected, consider decreasing the frequency of full runs to optimize testing efficiency.

Setting the Policy

  • Define a default policy: Apply to all apps and test stages with TIA enabled via the SeaLights Settings / TIA page.

  • Customize for specific needs: Tailor the policy for individual apps, branches, and test stages through the Settings page or the Test Optimization - Build Breakdown page (using the TIA configuration button).

By leveraging both pre-defined and customizable policies, you can ensure SeaLights Test Optimization delivers recommendations that perfectly align with your unique testing strategy.