> For the complete documentation index, see [llms.txt](https://docs.sealights.io/knowledgebase/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.sealights.io/knowledgebase/setup-and-configuration/integrations/user-story-coverage/faq.md).

# FAQ

Common answers about setup, tagging, reporting, and integrations.

{% hint style="info" %}
Use [Troubleshooting](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/troubleshooting.md) when a run fails, returns no data, or produces the wrong result.
{% endhint %}

### General Questions

<details>

<summary>What is this tool for?</summary>

User Story Coverage connects issue tracking data and SeaLights coverage data.

It tags code changes with ticket IDs and generates coverage reports at the user story level.

This helps teams see which features have enough test coverage.

</details>

<details>

<summary>Do I need to install anything on my servers?</summary>

No.

This is a Python command-line tool that runs wherever Python 3.7 or later is available.

It connects to your existing systems through their APIs.

</details>

<details>

<summary>Which platforms are supported?</summary>

**Issue trackers**

* Jira Cloud
* Jira on-premises with plugin support
* Azure DevOps

**Source control**

* GitHub
* GitLab
* Bitbucket
* Azure DevOps Git

**Coverage platform**

* SeaLights

</details>

<details>

<summary>How much does it cost?</summary>

The tool is included as part of the SeaLights integration.

You still need active access to SeaLights, your issue tracker, and your source control platform.

</details>

### Setup and Configuration

<details>

<summary>How do I get started?</summary>

Follow the shortest working path below.

{% stepper %}
{% step %}

### Install and run once

Complete [Installation and first run](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/installation.md).
{% endstep %}

{% step %}

### Fill in your settings

Use [Configuration](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/configuration.md) for the full settings model.
{% endstep %}

{% step %}

### Pick a workflow

Continue with the matching path in [Deployment patterns](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/examples.md).
{% endstep %}
{% endstepper %}

</details>

<details>

<summary>Where do I get API tokens?</summary>

Get tokens from each source system:

* **SeaLights** — create an API token in **Settings → API Tokens**
* **Jira Cloud** — create an Atlassian API token
* **GitHub** — create a personal access token with repository read access
* **Azure DevOps** — create a PAT with the required work item permissions

Use [Credential Management](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/security-considerations.md) for storage, rotation, and minimum-permission guidance.

</details>

<details>

<summary>Can I use environment variables instead of a settings file?</summary>

Yes.

All settings can be overridden with environment variables.

Use [Environment variables](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/cli-reference.md#environment-variables) for the full variable list and examples.

</details>

<details>

<summary>Can I use multiple settings files?</summary>

Yes.

You can pass multiple `--settings` values and merge configurations from left to right.

Use [Configuration](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/configuration.md) for merge rules and examples.

</details>

### Tagging Questions

<details>

<summary>How does the tool extract ticket IDs?</summary>

The tool uses a regex pattern to extract ticket IDs from commit messages, branch names, and pull request titles.

Configure the pattern in your SCM settings with `key_pattern_regex`.

</details>

<details>

<summary>Do I need to tag code before generating reports?</summary>

It depends on your tagging mode.

* If SeaLights auto-tagging is enabled, you can skip the source-tagging scripts.
* If auto-tagging is not enabled, run the tagging phase before the reporting phase.

Use [Source Tagging](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/cli-reference.md#source-tagging) for the tagging commands and [Installation and first run](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/installation.md) for the shortest working path.

</details>

<details>

<summary>What is SeaLights auto-tagging and how do I enable it?</summary>

SeaLights auto-tagging extracts ticket IDs from git commit data during build reporting.

It removes the need to run the `US_SRC_*` tagging scripts.

To enable it, contact your SeaLights Customer Success representative and provide your ticket ID regex pattern plus the app and branch scope.

</details>

<details>

<summary>How often should I run tagging?</summary>

It depends on your workflow:

* **Daily or continuous** — run after merges or on a nightly schedule
* **Sprint-based** — run at sprint start, mid-sprint, and sprint end
* **On-demand** — run before releases or for a specific report request

If SeaLights auto-tagging is enabled, you do not need a separate tagging schedule.

</details>

<details>

<summary>Can I tag retroactively?</summary>

Yes.

Use the `--since-date` option with the GitHub PR tagging flow when you need historical tagging.

Use [Source Tagging](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/cli-reference.md#source-tagging) for the exact command and supported tagging paths.

</details>

<details>

<summary>What if I don't use pull requests?</summary>

Use the git history tagging flow instead of the PR-based flow.

This works well when your team merges directly to branches or does not rely on pull requests.

Use [Source Tagging](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/cli-reference.md#source-tagging) for the history-based command and `--days-back` usage.

</details>

### Reporting Questions

<details>

<summary>What formats are reports available in?</summary>

Reports are available as:

* JSON data files
* HTML pages
* Confluence pages
* Jira or Azure DevOps field updates

</details>

<details>

<summary>Can I customize the HTML reports?</summary>

Not through standard settings.

HTML output is generated from templates embedded in the scripts.

</details>

<details>

<summary>How long does report generation take?</summary>

Runtime depends on ticket count, app and branch scope, SeaLights API response time, and `coverage_days_back`.

A typical run takes 5 to 15 minutes for 20 to 50 tickets.

</details>

<details>

<summary>Can I speed up report generation?</summary>

Yes.

Start with these changes:

* reduce `coverage_days_back`
* narrow your JQL or WIQL query
* reuse an existing report with `--report-id`
* process tickets in smaller batches

</details>

<details>

<summary>What if a ticket has no coverage data?</summary>

The report shows `0%` coverage.

This usually means the code was not tagged with that ticket ID, no tests ran against the tagged code, or the lookback period missed the relevant test runs.

</details>

<details>

<summary>Can I customize report formatting?</summary>

Standard settings do not expose full report templating.

HTML and Confluence output use embedded templates in the scripts.

Check [Configuration](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/configuration.md) before changing templates directly.

</details>

<details>

<summary>Can I export data to other tools?</summary>

Yes.

`ReportInfo_*.json` files contain structured data that you can parse and send to other systems for dashboards, notifications, and gates.

Use [Report Generation](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/cli-reference.md#report-generation) for the report outputs and [Deployment patterns](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/examples.md) for automation patterns.

</details>

### Ticket Management Questions

<details>

<summary>What JQL should I use?</summary>

It depends on your reporting goal.

**Current sprint**

```
sprint in openSprints() AND project = MYPROJ
```

**Last 30 days**

```
project = MYPROJ AND created >= -30d
```

**Specific epic**

```
"Epic Link" = MYPROJ-100
```

**By labels**

```
project = MYPROJ AND labels = sprint-42
```

Use [Data sources](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/configuration/data-sources.md) for the full Jira query settings model.

</details>

<details>

<summary>How do I find custom field IDs?</summary>

Inspect a Jira issue through the Jira REST API and locate the matching `customfield_XXXXX` entry in the response.

Use [Publication targets](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/configuration/publication-targets.md) for legacy Jira publishing behavior and field-mapping context.

</details>

<details>

<summary>Do I need admin access to Jira?</summary>

No, but you do need:

* read access to projects in your JQL
* write access if you update custom fields through Jira publishing
* plugin admin rights if you use the Jira plugin

Some Jira environments require elevated credentials when updates are sent with `notifyUsers=false`. If your token cannot do that, set `jira.notify_users` or `JIRA_NOTIFY_USERS` to `true`.

</details>

<details>

<summary>What WIQL should I use?</summary>

Choose a query that matches your work item scope.

**User stories in last 30 days**

```
SELECT [System.Id] FROM WorkItems
WHERE [System.WorkItemType] = 'User Story'
AND [System.ChangedDate] >= @Today - 30
```

**Specific iteration**

```
SELECT [System.Id] FROM WorkItems
WHERE [System.IterationPath] = 'MyProject\\Sprint 42'
```

**By area path**

```
SELECT [System.Id] FROM WorkItems
WHERE [System.AreaPath] UNDER 'MyProject\\Team A'
```

Use [Data sources](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/configuration/data-sources.md) for the full Azure DevOps query settings model.

</details>

<details>

<summary>How do I test my WIQL?</summary>

Validate the query in Azure DevOps before running the reporting scripts.

{% stepper %}
{% step %}

### Open queries

Go to **Boards → Queries**.
{% endstep %}

{% step %}

### Switch to editor mode

Create a new query and switch to **Editor** mode.
{% endstep %}

{% step %}

### Paste and run

Paste your WIQL, run it, and verify the results.
{% endstep %}
{% endstepper %}

</details>

<details>

<summary>What permissions do I need in Azure DevOps?</summary>

Create a PAT with these permissions:

* **Work Items** — Read
* **Extension Data** — Read and Write, if you use the Azure DevOps plugin

</details>

### CI/CD Questions

<details>

<summary>Can I run this in CI/CD?</summary>

Yes. Deployment patterns has runnable pipeline configurations for GitHub Actions, Jenkins, GitLab CI, and Azure Pipelines.

</details>

<details>

<summary>What does a typical pipeline look like?</summary>

The reporting flow usually follows this order:

{% stepper %}
{% step %}

### Tag source changes

Run the tagging phase first if you are not using SeaLights auto-tagging.
{% endstep %}

{% step %}

### Generate report data

Create the report data files for the selected ticket scope.
{% endstep %}

{% step %}

### Build HTML output

Generate HTML reports for human review.
{% endstep %}

{% step %}

### Publish or archive results

Publish artifacts or push results to downstream systems.
{% endstep %}
{% endstepper %}

</details>

<details>

<summary>Can I run reports on a schedule?</summary>

Yes.

Run the reporting flow from your CI/CD scheduler or a cron-based trigger.

</details>

<details>

<summary>Can I fail builds based on coverage thresholds?</summary>

Yes.

Parse the generated JSON output and apply your own threshold gate in the pipeline.

Use [Report Generation](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/cli-reference.md#report-generation) for the output files and [Deployment patterns](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/examples.md) for workflow patterns.

</details>

<details>

<summary>What outputs should I publish as artifacts?</summary>

Publish the outputs that match your pipeline goal:

* HTML reports for review
* `ReportInfo_*.json` files for automation and gating
* `confluence_links.json` if you publish Confluence pages

</details>

<details>

<summary>Can I send notifications from the pipeline?</summary>

Yes.

The JSON outputs can feed Slack, Microsoft Teams, email, or custom webhook notifications.

</details>

<details>

<summary>Can I export report data to other tools?</summary>

Yes.

Use the generated JSON files for downstream automation, dashboards, and notifications.

</details>

<details>

<summary>Do I need a SeaLights agent in the reporting job?</summary>

No.

The reporting tool reads coverage through the SeaLights REST API.

You still need SeaLights agents running during tests and builds reported to SeaLights with the correct app and branch names.

</details>

### Repository Mapping Questions

<details>

<summary>Does it work with monorepos?</summary>

Yes.

Configure multiple entries in `repo_list`, either for different paths or for the same repository with different filters.

</details>

<details>

<summary>Can I track coverage across multiple repositories?</summary>

Yes.

List all repositories in `repo_list` and map each one to the matching SeaLights application.

</details>

<details>

<summary>What if my app name in SeaLights is different from my repo name?</summary>

That is fine.

Map the repository name to the SeaLights application name in `repo_list`.

</details>

<details>

<summary>What are integration builds?</summary>

Integration builds combine coverage from multiple component apps.

Enable them in `coverage_app_list` and make sure each component also exists in `repo_list`.

</details>

<details>

<summary>Can I use regex for app and branch names?</summary>

Yes.

Use `app_regex` and `branch_regex` in `coverage_app_list` to match multiple applications or branches.

</details>

### Test Stage Questions

<details>

<summary>What are test stages?</summary>

Test stages represent phases of testing such as unit, integration, and end-to-end.

They must match the stage names configured in SeaLights.

</details>

<details>

<summary>How do I know what my test stages are called?</summary>

Check your SeaLights dashboard or Test Gap Analysis reports.

Use the same names in your settings.

</details>

<details>

<summary>What are grouped test stages?</summary>

Grouped test stages aggregate coverage from multiple individual stages.

For example, one grouped stage can combine unit, integration, and end-to-end coverage into a single rollup.

Use `grouped_test_stages` for this configuration.

</details>

<details>

<summary>Can I add custom test stages?</summary>

Yes, as long as they already exist in SeaLights.

Add them to the `test_stages` array in your settings.

</details>

### Performance Questions

<details>

<summary>Why is tagging so slow?</summary>

Common causes include a large time range, many pull requests or commits, large repositories, and network latency to your SCM platform.

Start by reducing `--days-back`, limiting repositories per run, and running closer to your SCM platform when possible.

</details>

<details>

<summary>Why is report generation slow?</summary>

Common causes include many tickets in one run, long `coverage_days_back` windows, and first-run processing in SeaLights.

Start by reducing `coverage_days_back`, splitting large ticket sets into batches, and reusing an existing report with `--report-id` when possible.

</details>

<details>

<summary>Can I process repositories in parallel?</summary>

Yes.

You can run multiple tagging jobs at the same time when repositories are independent.

Use [Deployment patterns](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/examples.md) for end-to-end workflow patterns.

</details>

#### Advanced Questions

<details>

<summary>Can I run this in Docker?</summary>

```dockerfile
FROM python:3.9-slim
WORKDIR /app
COPY . .
RUN pip install requests
CMD ["python3", "US1_RPT_create_report_info_files.py", "--settings", "settings"]
```

</details>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.sealights.io/knowledgebase/setup-and-configuration/integrations/user-story-coverage/faq.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.