Multi-App Coverage Report
The Multi-App Coverage Report is a powerful new tool designed to provide deep, component-level visibility into quality metrics across complex, integrated systems spanning multiple applications and services. Unlike traditional reports that require rigid, single-structure data, this report eliminates constraints by allowing users to precisely select which applications and branches to include in their analysis. This high degree of customization ensures granular, accurate, and cumulative coverage metrics, aggregating test results across all relevant environments (Dev, QA, Staging) to deliver an adaptable and scalable snapshot of quality that aligns perfectly with modern, real-world development and testing processes.
Accessing Multi-App Coverage Reports
The new Multi-App Coverage Reports, along with your existing Test Gaps (TGA) reports, are now consolidated under a new main navigation menu item: Custom Coverage Reports.
You can set the "Multi-App Coverage" page as your default landing page on My Profile page.
Multi-App Coverage Reports List
The Multi-App Coverage Reports list provides a quick overview of all your saved reports.
Report List Table Columns
By: Displays the creator of the report. Hover to see a tooltip with the creator's full name and email (if available).
Last Edited: Reports are sorted by this column by default, with the latest reports appearing first. When generating or when failed to generate a report, a visual indicator appears next to the date.
Report Name: The name you assigned to the report.
Date Range: Displays the range selected during creation, or the the earliest/latest build within that range. There is no range when the report failed or is being generated.
Code Changes Coverage: % of modified methods tested by at least one test stage. Visualized as a bar with the percentage displayed next to it. On hover, a tooltip shows x / y methods tested.
Overall Code Coverage: % of methods tested by at least one test stage. Visualized as a bar with the percentage displayed next to it. On hover, a tooltip shows x / y methods tested.
Untested Code Changes: Number of modified methods not covered by any test stage. Hidden by default; can be shown via column customization.
Untested Overall Code: Number of methods not covered by any test stage. Hidden by default; can be shown via column customization.
Customizing Table Columns
Use the Customize button in the table toolbar to show or hide columns. The available columns are: Code Changes Coverage, Overall Code Coverage, Untested Code Changes, and Untested Overall Code. By default, only the two coverage columns are shown. Your column preferences are saved per user in the browser.
Report & Table Actions
Hover over an individual report in the list to reveal context-specific action buttons on the right side: Edit, Duplicate, Delete, and Download as a CSV file. At the top of the table, you can manage the entire report list using the action bar to Refresh reports currently in "Generating" status, Search by report name, and Filter to display "My Reports Only".
Creating a New Report
Note on Future Capabilities: In this initial version, Branching Out (code changes based on another branch) and Code Labels (filtering by arbitrary code tags) are not supported. These capabilities are planned for future releases.
The Multi-App Coverage report offers significantly more customization than the standard TGA report, supporting numerous applications, flexible code changes calculations, and multiple branches for coverage analysis. A guided, four-step wizard simplifies the configuration process:
Step 1: Applications
This step allows you to select the specific applications you want to include in your report. You can do this in one of two ways:
Option 1: Select Specific Apps
Manually select applications from a multi-select dropdown. The list shows all applications in your organization that have at least two builds. A Select All checkbox lets you select all currently visible apps at once (up to 200 items at a time). You can refine the list using the search bar, selections made across different searches are cumulative and will not be lost when you change the search term.
Option 2: Use a Saved Filter
Choose a pre-defined saved filter from your Coverage Dashboard. The filter dynamically determines which applications and branches are included in the report. Only apps you have permission to view and that have at least two builds will be included. When the report is regenerated, the filter is re-evaluated to reflect any changes to the app/branch list.
Creating a New Filter from the Wizard (Admin and DevOps roles only)
If the filter you need doesn't exist yet, click Create New Filter directly below the saved filter dropdown. This opens an inline Advanced Filter builder using Regex-based rules and conditions, without leaving the wizard. Once saved, the new filter is automatically selected and also becomes available in the Coverage Dashboard.
Step 2: Code Scope
This step defines the scope of code included in your report. The build range you select here will apply to all applications and main branches chosen for this report. SeaLights will then analyze all builds within this specified range for all selected apps and branches. The Base Build for your report will be the earliest build identified within this range, and the Target Build will be the latest build.
For overall coverage, the code scope is determined by the Target Build.
Default Values
Base Build defaults to: Earliest build on date (today).
Target Build defaults to: Latest build on branch.
You can adjust these using the following options:
Earliest/Latest build on Date: Provide a specific date.
Earliest/Latest build on Branch: No additional value needed; the range is determined by the earliest/latest builds available on the selected branches.
Branch Selection
For each application, you must choose a main branch for code change calculation:
Auto-selection: If an application has only one branch, it is automatically selected.
Select Branch: If an application has more than one branch, click the link next to the app to open a dropdown and select your desired branch. The selected branch name is displayed inline for a quick overview of your configuration.
Quick Selection: For reports spanning many applications, use the Quick Selection tool to bulk-assign a main branch to all applications that are still missing one. Enter a branch name and choose either Contains (default) or Equals matching logic. The system scans each unassigned app's available branches and applies the match. The result may be a full success (all unassigned apps assigned), a partial success (some apps assigned, others not found), or a failure (no matches found). Apps that already have a main branch selected are not affected. You cannot proceed to the next step until all apps have a main branch assigned.
When using a Saved Filter, the available branches in this step are restricted to those currently included in the filter's output.
Step 3: Code Coverage
This step allows you to define how code coverage data is collected for your report.
"Collect coverage from additional branches" Checkbox:
By default, coverage is only collected from the branches specified in Step 2.
Check 'Collect coverage from additional branches' to include coverage data from additional branches. Two options appear:
Select Specific Branches: Manually choose specific branches per application.
Select All Relevant Branches: Automatically include every branch in the report scope that had at least one active build within the build range defined in Step 2, plus a one-week buffer on both ends. When a Saved Filter is used, the scope is limited to branches within the filter's output.
Branches selected in Step 2 appear as checked and disabled, they are always included.
Unchecking the parent checkbox ignores any selections below it; re-checking it restores your previous selections.
Branch names are displayed inline next to each app. When more than one additional branch is selected, the count of extra branches is shown alongside the main branch name.
When using a Saved Filter, the available branches for additional coverage are restricted to those currently included in the filter's output.
Why collect coverage from multiple branches? In modern CI/CD pipelines, different test stages run on different branches. For example, feature tests may run on a dev branch while regression or E2E tests run on main or release. Collecting coverage from all relevant branches ensures a complete, cumulative quality view — every executed test stage contributes to the final coverage score, regardless of where it ran.
Step 4: Report Name
The final step allows you to give your report a meaningful name.
Report Name Input: Enter a name for your report. The name does not need to be unique.
Viewing a Report
The Multi-App Coverage report page serves as essential evidence for testing and release readiness. It helps you track your release quality progress, allowing you to clearly demonstrate testing rigor to stakeholders and identify areas not tested before deployment.
Code Changes tab is shown by default. The report provides two tables, Code Changes and Overall Code, each with a three-level hierarchy (Application > File > Method).
Report Tables:
Code Changes (default tab) and Overall Code tables each display a three-level hierarchy:
Application: The top level, showing aggregated data for each app.
Files: Expanding an application shows the relevant files.
Methods: Expanding a file shows the individual methods.
Search: Search for specific files and methods within each table.
Coverage Columns: Displayed as a bar visualization with the percentage shown next to the bar. Hover to see a tooltip showing x / y methods tested.
Contributors & SCM Integration: "Contributor" and "SCM" buttons provide insights into who made changes and links to your source control.
CSV Export: The download button generates a CSV file based on the currently active table:
On the Overall Code table: exports all methods (file name: Untested Code).
On the Code Changes table: exports only modified methods (file name: Untested Code Changes).
Metrics Cards: The top section features metrics cards showing Code Changes Coverage and Untested Code Changes first, followed by Overall Code and Untested Overall Code. Each card includes a bar visualization and combined tested/total counts.
Report Metadata: A metadata line shows the number of apps, date range, and test stages. Click Apps to open a popup with the full list of applications and branches used for both code change calculation and coverage data, including build range details per app and branch.
Test Stages Filter: The "Test Stages" link shows what is currently presented (e.g., "All Test Stages"). Click to filter the report by a specific test stage.
Report Rename: Hover over the report name and click Edit (or click the name directly) to rename the report. The updated name is reflected in the report list.
Managing Your Reports
SeaLights offers maximum flexibility for managing your Multi-App Coverage reports, allowing you to change any parameter on an existing report or quickly duplicate one.
Editing & Regenerating a Report
Click Edit on a report in the list to open the "Update Report" wizard.
Navigate to any step and change any parameters.
The Regenerate Report button is available on all steps for quick updates.
Regenerating creates a new version of the report, replacing the previous one in the list.
Duplicating a Report
Click Duplicate on a report in the list to open the "Create Report" wizard with all parameters pre-filled (except the report name).
Click Create Report on Step 4 to create a new independent report without affecting the original.
Last updated
Was this helpful?