Coverage Analysis FAQ
Access & Permissions
Who can create and manage views?
Users with Admin or DevOps roles can create, edit, and delete views. All other users can access, filter, and export any view that has been created for them, no special permissions needed to use a view.
Can I see coverage data for all apps in a view, regardless of my permissions?
No. Coverage Analysis respects your existing app-level permissions. If a view includes apps you're not authorized to see, those apps are excluded from the data and metrics displayed to you. A banner at the top of the view will notify you that some data has been restricted. The metrics you see accurately reflect only the apps you have access to.
Can I edit or delete a view someone else created?
Users with DevOps role can edit and delete any view. Other users can only edit and delete views they created themselves.
Metrics
What is the difference between Overall Coverage and Change Coverage?
Overall Coverage tells you what percentage of all methods in your scope were executed by tests during the selected period - it's a broad picture of your codebase's test health. Change Coverage narrows this to only the methods that were modified during the selected window. It answers a more targeted question: "Did we test the code we actually changed?". These two metrics serve different purposes.
Why would I use Change Coverage instead of Overall Coverage?
High overall coverage does not guarantee that recently modified code was tested. A codebase can have 80% overall coverage while new and changed features have little to no coverage, because the existing tests cover stable, unchanged code. Change Coverage surfaces this distinction, making it the right metric for sprint-level quality validation and change risk assessment.
Can I include both Overall Coverage and Change Coverage in the same view?
Yes. When both are selected during view creation, the view shows them as switchable tabs. You can compare both perspectives without needing separate views.
What does the Failed Tests count represent?
Failed Tests shows the number of unique tests that are currently failing (or their last run) within your scope and date range, not the total number of test executions that failed. A test that fails repeatedly in multiple runs is counted once. This gives you a clearer sense of how many distinct issues exist, rather than how many times they occurred.
Why does my overall coverage look lower than expected when Test Optimization is active?
Test Optimization intentionally runs only a subset of tests per cycle to save time and resources. Because not every test runs in every cycle, not every method gets executed in every reporting period, which can result in a lower overall coverage number. This is expected behavior, not a gap in your test suite. An indicator appears next to any test stage where Test Optimization was active, with a tooltip explaining the impact.
Scope & Configuration
What is the easiest way to define my application scope?
The simplest approach is to select a Lab. Since labs are deployed with the services your team works on, selecting the relevant lab automatically includes all associated services, without needing to manually list every app, branch, or configuration. If you need more precise control, you can define a custom scope using App, Branch, Lab, or Code Label filters and save it for reuse.
Can I reuse the same code scope across multiple views?
Yes. You can create a Saved Scope and apply it to as many views as you like. This is useful for teams or product areas that have a consistent scope definition they want to use across sprint, release, and monitoring views.
Can I delete a saved code scope?
Only if it isn't linked to any active views. A saved scope used by one or more views is protected from deletion. To delete it, you first need to remove or reconfigure the views that reference it. Deletion is done from the Coverage Dashboard.
What is the difference between the date range modes (Last, Since, Fixed)?
Last is a rolling window - "Last 7 days" always means the 7 days ending right now, updating automatically as time passes. Best for ongoing monitoring.
Since grows forward from a fixed start date to the present - useful for tracking an in-progress sprint or release from its kickoff until now.
Fixed is a locked period with a defined start and end - useful for reviewing a completed sprint, pre-defining a release window or a planned testing period before it concludes.
Can the change baseline for Change Coverage be different from the date range?
Yes. When Change Coverage is enabled, you can set the change baseline independently - either tied to the start of the date range, or set to a specific reference date such as a previous release or sprint start. This lets you, for example, measure test results over the past two weeks while measuring changes relative to the last release date.
Views & Data
Are views shared with my whole team?
Yes. Views are shared across all users in your organization. Anyone can open any view from the view selector. Filters applied within a view are personal, each user's filter state is saved independently and doesn't affect what others see.
Can two views have the same name?
Yes. There is no uniqueness requirement on view names, so teams can use naming conventions that work for them (e.g., "Sprint 44 – Payments Team" alongside "Sprint 44 – Auth Team").
How current is the data in a view?
Views refresh automatically every 15 minutes, as long as the view has been accessed in the past month. The "Data as of" timestamp in the view header shows when the last successful refresh occurred.
What happens if I leave the page while a view is being generated?
Generation continues in the background. You can navigate freely within SeaLights while the view processes. When it's ready, it will be available in the view selector. If you return to Coverage Analysis before it finishes, the view is shown with a "Generating" status so you know it's still in progress.
What if a view fails to generate?
An error message will appear in the main area with an option to regenerate. Clicking it retries the generation using the same configuration, no need to reconfigure the view from scratch.
Exports
What does the CSV export include?
A full method-level table with: App, File Path, File Name, Method name, Line Number, and a column per test stage showing whether each method was covered. If Change Coverage is included in the view, additional columns show whether each method was changed and who the contributors are.
Last updated
Was this helpful?