Coverage Dashboard
Last updated
Last updated
New Dashboard Design Available! 🎉
We've redesigned the dashboard to be more intuitive and user-friendly, while still offering all the features you know and love.
SeaLights Coverage Dashboard provides a centralized hub for visualizing and analyzing your code coverage data. It empowers you to gain valuable insights into the effectiveness of your testing efforts and identify areas for improvement. This guide offers a detailed explanation of the key features and functionalities within the Coverage Dashboard.
Overall Coverage: This metric represents the percentage of methods within your codebase that have been exercised by at least one test stage during the latest build. It's calculated as:
Code Changes Coverage: This metric focuses on recently modified methods within your codebase. It reflects the percentage of these modified methods that have been covered by at least one test stage in the latest build. The calculation is as follows:
Untested Code Changes: This metric represents the number of modified methods that have not been executed by any of your test stages during the latest build. These untested code changes represent areas that potentially lack sufficient test coverage and considered as quality risks. It is recommended to have 0 untested code changes (or at least 0 critical untested code changes).
The coverage numbers displayed on the dashboard are aggregated across all test stages executed against a specific app/branch in the latest build. Importantly, this coverage is calculated at the method level, providing a granular view of how effectively your tests exercise your codebase.
SeaLights compares the current build's methods with those in the reference build to determine which methods have been added or modified. This allows you to focus on the coverage specifically for these recently changed sections of your codebase.
A Reference build serves as the baseline for calculating Code Changes Coverage and Untested Code Changes metrics.
By default, the reference build is the previous build and is defined per branch. The current reference build can be identified in the "Reference Build" column for each application.
You can customize the reference build and set a specific build (e.g., the latest production release) as the baseline. This can be done through the SeaLights public API or manually within the UI. Once set, all subsequent builds within the branch will utilize this chosen build for comparison purposes when calculating code changes coverage and identifying untested code changes.
Different builds can have different reference builds assigned. However, SeaLights does not allow for retroactive modification of the Reference Build for a processed build.
The Coverage Dashboard allows you to explore your build history, providing insights into how code coverage has evolved over time for each app/branch within your project.
Quality Gates define thresholds that determine if a build meets your predefined quality standards. Three parameters influence the Quality Gate status:
Code Changes Coverage
Overall Code Coverage
Failed Tests
By clicking on the status within the dashboard, you can access a pop-up window that offers a summary of the specific build status and its associated Quality Gate definitions.
There are four potential statuses for each build reported per application. The table below summarizes these statuses and their corresponding reasons:
| Status | Reasons |
|---|---|
Passed | All Quality Gate conditions were met. |
Failed | The code did not meet one or more quality thresholds. |
Missing Data | This status is assigned when all criteria for Quality Gate evaluation cannot be assessed due to missing data:
|
Scan Issue | The build scan encounters errors during the reporting process. |
While filters and code labels can help you focus on specific app/branches within your project, it's important to note that they may also influence the coverage metrics calculations for those filtered elements.
By clicking on any line within the Coverage Dashboard table, you'll be presented with a deeper dive into the coverage data. This expanded view empowers you to understand the nuances of your testing strategy and identify potential areas for improvement. Here's what you can expect:
Test Stage List: Gain a comprehensive view of all the test stages executed against the selected app/branch in the latest build. This list provides a clear understanding of the various testing perspectives employed to assess your codebase.
Coverage per Test Stage: Analyze the coverage details for each individual test stage. This breakdown allows you to pinpoint which test stages are effectively exercising specific sections of your code and identify any potential gaps in coverage.
Impact Analysis: While not explicitly mentioned in previous sections, SeaLights offer functionalities to understand factors that could have affected the coverage data. This could include the ability to see if the build was executed with Test Impact Analysis (TIA) enabled, potentially influencing which tests were run. It is possible to directly access detailed tables showcasing the specific tests executed within each test stage. This can provide even more granular insights into your testing process.
