Coverage Dashboard

New Dashboard Design Available! 🎉

We've redesigned the dashboard to be more intuitive and user-friendly, while still offering all the features you know and love.

New Dashboard Design

Coverage Dashboard

SeaLights Coverage Dashboard provides a centralized hub for visualizing and analyzing your code coverage data. It empowers you to gain valuable insights into the effectiveness of your testing efforts and identify areas for improvement. This guide offers a detailed explanation of the key features and functionalities within the Coverage Dashboard.

Key Coverage Metrics

• Overall Coverage: This metric represents the percentage of methods within your codebase that have been exercised by at least one test stage during the latest build. It's calculated as:

Overall Coverage (%) = Number of Tested Methods / Overall Number of Methods

• Code Changes Coverage: This metric focuses on recently modified methods within your codebase. It reflects the percentage of these modified methods that have been covered by at least one test stage in the latest build. The calculation is as follows:

Modified Code Coverage (%) = Number of Modified & Tested Methods / Overall Number of Modified Methods

• Untested Code Changes: This metric represents the number of modified methods that have not been executed by any of your test stages during the latest build. These untested code changes represent areas that potentially lack sufficient test coverage and considered as quality risks. It is recommended to have 0 untested code changes (or at least 0 critical untested code changes).

Understanding Coverage Calculations

The coverage numbers displayed on the dashboard are aggregated across all test stages executed against a specific app/branch in the latest build. Importantly, this coverage is calculated at the method level, providing a granular view of how effectively your tests exercise your codebase.

SeaLights compares the current build's methods with those in the reference build to determine which methods have been added or modified. This allows you to focus on the coverage specifically for these recently changed sections of your codebase.

Understanding Reference Builds

A Reference build serves as the baseline for calculating Code Changes Coverage and Untested Code Changes metrics.

By default, the reference build is the previous build and is defined per branch. The current reference build can be identified in the "Reference Build" column for each application.

You can customize the reference build and set a specific build (e.g., the latest production release) as the baseline. This can be done through the SeaLights public API or manually within the UI. Once set, all subsequent builds within the branch will utilize this chosen build for comparison purposes when calculating code changes coverage and identifying untested code changes.

Reference Build Button

Reference Build Popup

Different builds can have different reference builds assigned. However, SeaLights does not allow for retroactive modification of the Reference Build for a processed build.

Navigating Build History

The Coverage Dashboard allows you to explore your build history, providing insights into how code coverage has evolved over time for each app/branch within your project.

Build History Button

Build History Popup

Quality Gate Status

Quality Gates define thresholds that determine if a build meets your predefined quality standards. Three parameters influence the Quality Gate status:

• Code Changes Coverage

• Overall Code Coverage

• Failed Tests

By clicking on the status within the dashboard, you can access a pop-up window that offers a summary of the specific build status and its associated Quality Gate definitions.

Quality Gate Status per Build

Quality Gate Summary - Passed

Quality Gate Summary - Failed

There are four potential statuses for each build reported per application. The table below summarizes these statuses and their corresponding reasons:

Status	Reasons
Passed	All Quality Gate conditions were met.
Failed	The code did not meet one or more quality thresholds.
Missing Data	This status is assigned when all criteria for Quality Gate evaluation cannot be assessed due to missing data: No Reported Tests: A criterion relies on the quantity of failed tests, but no tests were executed or reported. Missing Test Stage Data: A criterion depends on coverage data, but no test stages were defined or executed. No Detected Code Changes: A criterion is based on code changes, but no modifications were found between the current and reference builds.
Scan Issue	The build scan encounters errors during the reporting process.

Applying Filters and Code Labels

While filters and code labels can help you focus on specific app/branches within your project, it's important to note that they may also influence the coverage metrics calculations for those filtered elements.

Unveiling Test Stage Details

By clicking on any line within the Coverage Dashboard table, you'll be presented with a deeper dive into the coverage data. This expanded view empowers you to understand the nuances of your testing strategy and identify potential areas for improvement. Here's what you can expect:

• Test Stage List: Gain a comprehensive view of all the test stages executed against the selected app/branch in the latest build. This list provides a clear understanding of the various testing perspectives employed to assess your codebase.

• Coverage per Test Stage: Analyze the coverage details for each individual test stage. This breakdown allows you to pinpoint which test stages are effectively exercising specific sections of your code and identify any potential gaps in coverage.

• Impact Analysis: While not explicitly mentioned in previous sections, SeaLights offer functionalities to understand factors that could have affected the coverage data. This could include the ability to see if the build was executed with Test Impact Analysis (TIA) enabled, potentially influencing which tests were run. It is possible to directly access detailed tables showcasing the specific tests executed within each test stage. This can provide even more granular insights into your testing process.

Test Stage List