Multi-App Coverage Report
The Multi-App Coverage Report is a powerful new tool designed to provide deep, component-level visibility into quality metrics across complex, integrated systems spanning multiple applications and services. Unlike traditional reports that require rigid, single-structure data, this report eliminates constraints by allowing users to precisely select which applications and branches to include in their analysis. This high degree of customization ensures granular, accurate, and cumulative coverage metrics, aggregating test results across all relevant environments (Dev, QA, Staging) to deliver an adaptable and scalable snapshot of quality that aligns perfectly with modern, real-world development and testing processes.
Accessing Multi-App Coverage Reports
The new Multi-App Coverage Reports, along with your existing Test Gaps (TGA) reports, are now consolidated under a new main navigation menu item: Custom Coverage Reports.
Multi-App Coverage Reports List
The Multi-App Coverage Reports list provides a quick overview of all your saved reports.
Report List Table Columns
By: Displays the creator of the report. Hover to see a tooltip with the creator's full name and email (if available).
Last Edited: Reports are sorted by this column by default, with the latest reports appearing first. When generating or when failed to generate a report, a visual indicator appears next to the date.
Report Name: The name you assigned to the report.
Date Range: Displays the range selected during creation, or the the earliest/latest build within that range. There is no range when the report failed or is being generated.
Overall Code Coverage: % of methods tested by at least on test stage. Visualized as a bar for quick scanning. On hover, a tooltip appears with the percentage.
Untested Overall Code: Number of methods not covered by any test stage.
Code Changes Coverage: % of modified methods tested by at least on test stage. Visualized as a bar for quick scanning. On hover, a tooltip appears with the percentage.
Untested Code Changes: Number of modified methods not covered by any test stage.
Report & Table Actions
Hover over an individual report in the list to reveal context-specific action buttons on the right side: Edit, Duplicate, Delete, and Download as a CSV file. At the top of the table, you can manage the entire report list using the action bar to Refresh reports currently in "Generating" status, Search by report name, and Filter to display "My Reports Only."
Creating a New Report
The Multi-App Coverage report offers significantly more customization than the standard TGA report, supporting numerous applications, flexible code changes calculations, and multiple branches for coverage analysis. A guided, four-step wizard simplifies the configuration process:
Step 1: Applications
This step allows you to select the specific applications you want to include in your report.
The multi-select dropdown lists all applications in your organization that have at least two builds. You can select any number of applications from this list.
Step 2: Code Changes
This step is crucial for defining the scope of code changes included in your report. The build range you select here will apply to all applications and main branches chosen for this report. SeaLights will then analyze all builds within this specified range for all selected apps and branches. The Base Build for your report will be the earliest build identified within this range, and the Target Build will be the latest build.
Base Build and Target Build (Defining the Range): You can define your desired code change range in the following ways:
Earliest/Latest build on Date: You must provide a specific date.
Earliest/Latest build on Branch: No additional value is needed for this option, as the range will be determined by the earliest/latest builds available on the selected branches.
Branch Selection: For each application selected in Step 1, you must choose a main branch for code change calculation.
Auto-selection: If an application has only one branch, that branch will be automatically selected.
"Select Branch" Option: If an application has more than one branch, you'll see "Select Branch" next to the app name. Click this to open a dropdown and select your desired branch for that specific application.
Step 3: Code Coverage
This step allows you to define how code coverage data is collected for your report.
"Collect coverage from additional branches" Checkbox:
By default, coverage will only be collected from the branches specified in Step 2.
Select the 'Collect coverage from additional branches' checkbox to include coverage data from branches beyond those specified in Step 2.
You can select any branch of an application (no limitation).
Branches selected in Step 2 for code changes calculation will appear as checked and disabled. For these branches, only builds within your defined build range are considered for coverage.
Unchecking the main "Collect coverage from additional branches" box will ignore any selections made in the list below it. Re-checking it will restore your previous selections.
The Need for Multi-Branch Coverage Collection: In modern CI/CD pipelines, various test stages are executed on different branches. For example, feature-specific tests might run on a dev branch, while robust regression or E2E tests might only be executed on the consolidated main or release branch. To create a complete, cumulative view of your quality, this feature allows you to aggregate coverage results from all relevant branches, ensuring that every executed test across your pipeline contributes to the final coverage score, regardless of where that specific test stage was run.
Step 4: Report Name
The final step allows you to give your report a meaningful name.
Report Name Input: Enter a name for your report. The name does not need to be unique.
Viewing a Report
The Multi-App Coverage report page serves as essential evidence for testing and release readiness. It helps you track your release quality progress, allowing you to clearly demonstrate testing rigor to stakeholders and identify areas not tested before deployment.
The report provides a detailed breakdown of your quality metrics, featuring two distinct tables: Overall Code and Code Changes. It also introduces a new 3-level hierarchy (Application > File > Method) to enhance clarity and consumption.
Report Tables:
There are two primary tables: Overall Code and Code Changes. The Code Changes table is displayed by default.
3-Level Hierarchy: Each table now displays a three-level hierarchy:
Application: The top level, showing aggregated data for each app.
Files: Expanding an application shows the relevant files.
Methods: Expanding a file shows the individual methods.
Search: You can search for specific files and methods within each table.
Coverage Columns: Displays coverage as a bar visualization, with detailed percentage tooltips on hover.
Contributors & SCM Integration: "Contributor" and "SCM" buttons function similarly to the TGA report, providing insights into who made changes and links to your source control.
CSV Export: The CSV export file for both tables now includes an "app" column.
Metrics Cards: The top section of the report page features redesigned metrics cards that visually represent your overall code and code changes coverage, including bars for quick visualization. Untested and total numbers for each type are combined in one intuitive card.
Report Metadata: A metadata line provides key information about the report:
Number of Apps, Date Range, and Test Stages: These are displayed concisely.
"Apps" Link: Pressing this link opens a popup displaying:
The full list of applications and branches used for the code coverage calculation. For each app, the relevant branch(es) appear, along with a "Build Range" link. Hover over this link to see a tooltip with the base and target builds.
The full list of applications and branches used for code coverage data. For each app, the relevant branch(es) appear, with a "Build Range / All Builds" link. Hover over this link to see a tooltip with the build range.
Test Stages Filter: The "Test Stages" link includes an indication of what is currently presented in the report (e.g., "All Test Stages"). Pressing this link opens a dropdown allowing you to select either "All Test Stages" (default) or a specific test stage to filter the report data.
Report Rename: You can easily rename the report by hovering over the report name and clicking the "Edit" button (or clicking the name directly). Once a new name is saved, it will be available on the report list table as well.
Managing Your Reports
SeaLights offers maximum flexibility for managing your Multi-App Coverage reports, allowing you to change any parameter on an existing report or quickly duplicate one.
Editing & Regenerating a Report
Pressing the "Edit" button for a specific report on the list page opens the "Update Report" wizard.
You can navigate to any step and change any parameters.
The "Regenerate Report" button is available on all steps, allowing for quick updates.
Regenerating a report creates a new version of it, replacing the older version on the report list. The page will always show the latest version of each report.
Duplicating a Report
Pressing the "Duplicate" button for a specific report on the list page opens the "Create Report" wizard popup.
All parameters from the original report are pre-filled, except for the report name.
Pressing the "Create Report" button on Step 4 creates a brand new report without affecting the original report that was duplicated.
Last updated
Was this helpful?