LogoLogo
Product
  • Knowledge Base
  • What's New
  • Guides
  • Welcome!
  • Intro to SeaLights
    • What is SeaLights
      • Glossary
      • Working in Conjunction with Your Quality / Coverage Tools
        • SeaLights vs SonarQube
        • SeaLights vs JaCoCo
    • Technical Overview
      • Test Stage Cycle
    • FAQ
  • User Story Coverage
    • Getting Started
    • User Story Challenges & Solution
      • Typical Implementation
      • The Challenges
      • The Solution
    • User Story Coverage Report Overview
      • Release Go / No Go Report
        • How to Generate / Edit the Report
      • User Story Quality Overview
        • How to Generate the User Story View
      • User Story Coverage Analysis
        • How to Generate the Analysis View
      • Uncovered Methods View
        • How to Generate the View
      • Customization
      • Integration
    • Use Cases by Persona
      • Managers
        • Informed Go/No Go Decisions Making
        • Effective Resources Prioritization
        • Overall Progress Monitoring
      • Developers
        • Code Quality Ownership
        • Seamless Collaboration with QA
        • Code Review Facilitator
      • QA Engineers
        • Test Execution Progress Monitoring
        • Testing Effort Prioritization
        • Testing Strategy Planing
    • Technical Overview
      • User Story Coverage Mechanism
      • Technical Architecture
      • Deployment Guide
        • US1_getResults.sh
        • US2_createReport.sh
        • US_UpdateConfluence.sh
  • Test Optimization
    • Getting Started
    • Test Execution Challenges & Solution
      • The Challenges
      • Test Optimization Solution
      • Test Optimization Main Advantages
    • Test Optimization Overview
      • Automated Test Optimization
      • Manual Test Optimization
      • Test Optimization for Pull Request
      • Test Selection Policies
        • Full Run Policy
        • No Code Changes Policy
        • Common Code Policy
        • Fastest Path to 100% Coverage Policy
      • Integrations
    • Use Cases by Persona
      • Managers
        • Fast Delivery
        • Resource Optimization
        • Thorough Testing in Tight Schedule
      • Developers
        • Exploring Only Relevant Test Failures
        • Faster Feedback Loop
        • Shift Left Testing
      • QA Engineers & Manual Testers
        • Faster & Focused Manual Testing
        • Optimizing Test Suite
        • Having Stable Product for Testing
    • Technical Overview
      • Test Optimization Mechanism
        • Associating Code With Tests
          • Statistical modeling
          • One-to-One Mapping
          • Calibration
        • Detecting Modified Code
        • Generating Test Recommendations
      • Technical Architecture
      • Deployment Guide
  • Quality Improvement
    • Getting Started
    • Challenges & Approach Comparison
      • The Challenges
      • Quality Improvement Approaches
      • Choosing the Right Approach
    • Quality Improvement Solution Overview
      • Test Gaps Analysis Report
        • How to Generate / Edit the Report
      • Coverage Trend Report
        • How to Generate / Edit the Report
      • Proof of Testing Report
        • How to Generate / Edit the Report
      • Release Quality Improvement Guide
        • STEP 1: Deploy SeaLights
        • STEP 2: Take a Quality Snapshot
        • STEP 3: Prioritize Code Areas
          • Add Code Labels
          • Ignore Irrelevant Code
          • Perform a Deep CSV Analysis
        • STEP 4: Set Baseline & Threshold
        • STEP 5: Analyze Test Gaps
        • STEP 6: Write Tests
        • Step 7: Make a Go / No Go Decision Based on Quality Gate
        • STEP 8: Measure Defect Escape Rate
      • Over Time Quality Improvement Guide
        • STEP 1: Deploy SeaLights
        • STEP 2: Take a Quality Snapshot
        • STEP 3: Prioritize code areas
          • Add Code Labels
          • Ignore Irrelevant Code
          • Perform a Deep CSV Analysis
        • STEP 4: Set Baseline & Goal
        • STEP 5: Set timeline
        • STEP 6: Write tests
        • STEP 7: Monitor progress
        • STEP 8: Measure Defect Escape Rate
    • Use Cases by Persona
      • Managers
        • Effective Prioritization & Budget Allocation
        • Tracking Progress & Measuring Impact
        • Data-Driven Release Decisions
        • Transparency & Communication
      • Developers
        • Mastering Code Coverage
        • Seamless Collaboration with QA
        • Code Quality Ownership
      • QA Engineers
        • Prioritizing Test Efforts
        • Contributing to Release Informed Decisions
        • Seamless Collaboration with Developers
        • Evaluating Testing Strategy
    • Technical Overview
      • Solution Mechanism
      • Technical Architecture
      • Deployment Guide
  • Value Proposition
    • Overview
    • Quality Use Cases
      • Go/No Go Decisions
      • Quality Improvement & Test Gaps
      • Governance & Quality Gates
      • Compliance & Proof of Testing
    • Test Optimization Use Cases
      • Reduce Costs & Infrastructure
      • Shorten Release Cycles
      • Reduce Troubleshooting
  • What's New
    • What's New Highlights
    • 2024
      • July 2024
        • New Dashboard Design
        • New Coverage Report
        • Test Optimization - Build Breakdown Improvements
      • August 2024
        • New Navigation Bar
        • Enhanced Code Changes Calculation in TGA Report
      • September 2024
        • Executive Summary Report for Multi-Branch Environments
      • October 2024
        • Enhanced TIA Settings
        • Test Optimization Timeline
        • Test Optimization - Application Under Test
        • Saved Views on Dashboard
        • Group Coverage Trend Report for All Branches
      • November 2024
      • December 2024
        • Simplified TGA Report Creation
    • 2025
      • January 2025
      • February 2025
        • Introducing Test Stage Cycles
      • March 2025
        • Introducing Our New Tricentis-SeaLights Logo!
        • Enhanced Security & Control with SeaLights Tokens
        • Component Level Coverage For All Test Stages
        • Quality Gates - Flexible Failure Criteria
        • Test Gaps: Coverage Focus
  • Features
    • Coverage Dashboard
    • Coverage Report
    • Test Gaps Report
      • Code Changes Calculation
        • Hidden Changes Detection
    • Test Optimization - Savings Breakdown
      • TIA Configuration
    • Settings
      • Token Access & Management
      • Quality Gates
Powered by GitBook
On this page
  • Test Recommendation Principles
  • Test Selection Policies
  • The Impact On Code Coverage

Was this helpful?

  1. Test Optimization
  2. Technical Overview
  3. Test Optimization Mechanism

Generating Test Recommendations

After identifying all code changes in a new build and leveraging the Code-To-Test mapping, TIA determines which tests are affected by these changes. It subsequently compiles a list of recommended tests, which is accessible for viewing or consuming via an API and can be integrated back into the test framework for executing only the necessary tests. Alternatively, the list of tests can be incorporated into the customer's CI pipeline.

By utilizing TIA, teams can run only the relevant tests associated with code changes, saving time and enhancing efficiency.

Test Recommendation Principles

TIA follows specific principles to construct a precise list of essential tests that ensures high quality and instills confidence in teams and organizations. These principles guide the creation of the test recommendation list:

  • Tests impacted by code changes. Meaning tests related to modified methods.

  • Tests that failed in the last test run on the specific test stage for comprehensive coverage.

  • Pinned tests, which must consistently run.

  • New tests not yet mapped to any code section.

  • Dependent tests, which should run alongside related recommended tests.

  • Previously recommended tests intended for the previous test recommendation but not executed, often due to failures or blockers.

Tests that do not fall into any of the principles above are excluded from the recommendation list and automatically skipped. This refined test list maintains quality while reducing the test suite.

Test Selection Policies

SeaLights empowers you to customize the recommended test list to perfectly align with your specific testing needs. This flexibility allows you to:

  • Broaden coverage: Ensure comprehensive testing by incorporating additional tests.

  • Optimize test volume: Prioritize critical tests to streamline execution time.

SeaLights empowers you to find the perfect balance between thoroughness and efficiency in your testing process, ultimately saving time and resources.

The Impact On Code Coverage

TIA offers the advantage of skipping irrelevant tests for each build and release. Consequently, this selective approach may result in lower overall code coverage. This can be a concern if quality gates within an organization rely on code coverage metrics and require a specific threshold for validation.

Similarly, other systems that depend on full test execution, such as SonarCube, may be affected by TIA's selective test runs. It's essential to recognize this potential impact and communicate it proactively with all relevant stakeholders.

In cases where all tests must run for specific reasons, perhaps due to a critical and sensitive code change necessitating a comprehensive system test, you have the option to temporarily disable TIA, or to choose a full run policy for the next build to ensure all tests are executed without skipping any. This allows you to meet the requirements of sensitive code changes while maintaining comprehensive coverage.

Remember that balancing the advantages of TIA's efficiency with the need for comprehensive testing is crucial. Effective communication with stakeholders is key to managing expectations and ensuring quality throughout the development process.

PreviousDetecting Modified CodeNextTechnical Architecture

Last updated 10 months ago

Was this helpful?

Test Selection Policies