LogoLogo
Product
  • Knowledge Base
  • What's New
  • Guides
  • User Story Coverage
    • Getting Started
    • User Story Challenges & Solution
      • Typical Implementation
      • The Challenges
      • The Solution
    • User Story Coverage Report Overview
      • Release Go / No Go Report
        • How to Generate / Edit the Report
      • User Story Quality Overview
        • How to Generate the User Story View
      • User Story Coverage Analysis
        • How to Generate the Analysis View
      • Uncovered Methods View
        • How to Generate the View
      • Customization
      • Integration
    • Use Cases by Persona
      • Managers
        • Informed Go/No Go Decisions Making
        • Effective Resources Prioritization
        • Overall Progress Monitoring
      • Developers
        • Code Quality Ownership
        • Seamless Collaboration with QA
        • Code Review Facilitator
      • QA Engineers
        • Test Execution Progress Monitoring
        • Testing Effort Prioritization
        • Testing Strategy Planing
    • Technical Overview
      • User Story Coverage Mechanism
      • Technical Architecture
      • Deployment Guide
        • US1_getResults.sh
        • US2_createReport.sh
        • US_UpdateConfluence.sh
  • Test Optimization
    • Getting Started
    • Test Execution Challenges & Solution
      • The Challenges
      • Test Optimization Solution
      • Test Optimization Main Advantages
    • Test Optimization Overview
      • Automated Test Optimization
      • Manual Test Optimization
      • Test Optimization for Pull Request
      • Test Selection Policies
        • Full Run Policy
        • No Code Changes Policy
        • Common Code Policy
        • Fastest Path to 100% Coverage Policy
      • Integrations
    • Use Cases by Persona
      • Managers
        • Fast Delivery
        • Resource Optimization
        • Thorough Testing in Tight Schedule
      • Developers
        • Exploring Only Relevant Test Failures
        • Faster Feedback Loop
        • Shift Left Testing
      • QA Engineers & Manual Testers
        • Faster & Focused Manual Testing
        • Optimizing Test Suite
        • Having Stable Product for Testing
    • Technical Overview
      • Test Optimization Mechanism
        • Associating Code With Tests
          • Statistical modeling
          • One-to-One Mapping
          • Calibration
        • Detecting Modified Code
        • Generating Test Recommendations
      • Technical Architecture
      • Deployment Guide
  • Quality Improvement
    • Getting Started
    • Challenges & Approach Comparison
      • The Challenges
      • Quality Improvement Approaches
      • Choosing the Right Approach
    • Quality Improvement Solution Overview
      • Test Gaps Analysis Report
        • How to Generate / Edit the Report
      • Coverage Trend Report
        • How to Generate / Edit the Report
      • Proof of Testing Report
        • How to Generate / Edit the Report
      • Release Quality Improvement Guide
        • STEP 1: Deploy SeaLights
        • STEP 2: Take a Quality Snapshot
        • STEP 3: Prioritize Code Areas
          • Add Code Labels
          • Ignore Irrelevant Code
          • Perform a Deep CSV Analysis
        • STEP 4: Set Baseline & Threshold
        • STEP 5: Analyze Test Gaps
        • STEP 6: Write Tests
        • Step 7: Make a Go / No Go Decision Based on Quality Gate
        • STEP 8: Measure Defect Escape Rate
      • Over Time Quality Improvement Guide
        • STEP 1: Deploy SeaLights
        • STEP 2: Take a Quality Snapshot
        • STEP 3: Prioritize code areas
          • Add Code Labels
          • Ignore Irrelevant Code
          • Perform a Deep CSV Analysis
        • STEP 4: Set Baseline & Goal
        • STEP 5: Set timeline
        • STEP 6: Write tests
        • STEP 7: Monitor progress
        • STEP 8: Measure Defect Escape Rate
    • Use Cases by Persona
      • Managers
        • Effective Prioritization & Budget Allocation
        • Tracking Progress & Measuring Impact
        • Data-Driven Release Decisions
        • Transparency & Communication
      • Developers
        • Mastering Code Coverage
        • Seamless Collaboration with QA
        • Code Quality Ownership
      • QA Engineers
        • Prioritizing Test Efforts
        • Contributing to Release Informed Decisions
        • Seamless Collaboration with Developers
        • Evaluating Testing Strategy
    • Technical Overview
      • Solution Mechanism
      • Technical Architecture
      • Deployment Guide
  • Value Proposition
    • Overview
    • Quality Use Cases
      • Go/No Go Decisions
      • Quality Improvement & Test Gaps
      • Governance & Quality Gates
      • Compliance & Proof of Testing
    • Test Optimization Use Cases
      • Reduce Costs & Infrastructure
      • Shorten Release Cycles
      • Reduce Troubleshooting
Powered by GitBook
On this page

Was this helpful?

  1. Quality Improvement
  2. Use Cases by Persona
  3. Managers

Tracking Progress & Measuring Impact

Turning testing efforts into tangible results requires insightful measurement. This use case equips you to monitor your testing journey, evaluate the effectiveness of your strategies, and identify areas needing a course correction. By consistently tracking key metrics and analyzing their impact, you'll ensure continuous progress towards quality software.


Step-by-Step Tutorial

1. Identify Key Metrics
  • Code Coverage: Using the Proof of Testing report to track the percentage of code covered by tests (overall code and modified code), aiming for gradual improvement over time.

  • Test Execution Rate: Monitor the number of tests successfully executed to gauge overall testing activity.

  • Defect Detection Rate: Measure how many bugs tests unearth, indicating their effectiveness in identifying issues.

  • Defect Escape Rate: Measure how many escaping defects were found in production.

2. Monitor Progress Through Data Visualization
  • Utilize charts and graphs on SeaLights' Coverage Trend Report to visualize metric trends, spotting areas of progress, stagnation, or regression.

  • Benchmark your metrics against industry standards or previous releases to assess relative performance.

3. Analyze Impact for Informed Decisions
  • Compare code coverage trends with defect detection rates: Assess if increased coverage leads to more bug identification.

  • Compare code coverage trends with defect escape rates: Assess iff increased coverage leads to less escaping defects.

  • Analyze trends across different features or modules: Identify specific areas requiring additional testing efforts.

4. Identify Areas for Improvement
  • Low code coverage areas: Prioritize additional testing for these sections to enhance overall quality.

  • High defect detection areas: Invest in more focused testing or code reviews to further reduce vulnerabilities.

5. Refine & Iterate
  • Based on your analysis, adjust testing strategies, allocate resources, and introduce new testing tools as needed.

  • Continuously revisit your key metrics and refine your tracking approach for lasting quality improvement.

By embracing this data-driven approach, you can transform the "black box" of testing into a transparent and measurable journey, ensuring your efforts deliver maximum quality impact.

PreviousEffective Prioritization & Budget AllocationNextData-Driven Release Decisions

Was this helpful?