# FAQ

Common questions about the User Story Coverage Tagging tool.

## General Questions

### What is this tool for?

This tool connects issue tracking systems (Jira, Azure DevOps) with SeaLights test coverage platform. It automatically tags code changes with ticket IDs and generates coverage reports at the user story level, helping teams track which features have adequate test coverage.

### Do I need to install anything on my servers?

No. This is a Python command-line tool that runs wherever Python 3.7+ is available. It connects to your existing systems (SeaLights, Jira/ADO, GitHub) via their APIs.

### Which platforms are supported?

**Issue Trackers:**

* Jira Cloud
* Jira On-Premises (with plugin)
* Azure DevOps

**Source Control:**

* GitHub
* GitLab
* Bitbucket
* Azure DevOps Git

**Coverage Platform:**

* SeaLights

### How much does it cost?

The tool itself is free (part of SeaLights integration). You need existing subscriptions to:

* SeaLights (for test coverage data)
* Your issue tracker (Jira or Azure DevOps)
* Your source control platform (GitHub, GitLab, etc.)

***

## Setup and Configuration

### How do I get started?

1. Install Python 3.7+ and the `requests` library
2. Copy `settings.sample` to `settings`
3. Fill in your credentials and configuration
4. Run the Quick Start example

See the [Quick Start Guide](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/quick-start.md) for a complete walkthrough.

### Where do I get API tokens?

**SeaLights:**

* Log into SeaLights web UI
* Navigate to Settings → API Tokens
* Generate a new token

**Jira Cloud:**

* Go to <https://id.atlassian.com/manage/api-tokens>
* Create API token
* Use with your email: `echo -n "email:token" | base64`

**GitHub:**

* Go to <https://github.com/settings/tokens>
* Generate new token with `repo` scope

**Azure DevOps:**

* User Settings → Personal Access Tokens
* Create token with Work Items (Read) and Extension Data (Read & Write) permissions

### Can I use environment variables instead of a settings file?

Yes! All settings can be overridden with environment variables. This is recommended for CI/CD pipelines.

```bash
export SEALIGHTS_API_TOKEN="your-token"
export JIRA_BASE_URL="https://yourcompany.atlassian.net"
python3 US1_RPT_create_report_info_files.py --settings settings
```

See [Configuration](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/configuration.md) for the complete list.

### Can I use multiple settings files?

Yes. Specify multiple `--settings` flags to merge configurations:

```bash
python3 US1_RPT_create_report_info_files.py \
  --settings base_config \
  --settings team_config \
  --settings credentials
```

Settings are merged left to right, with later files overriding earlier ones.

***

## Tagging Questions

### How does the tool extract ticket IDs?

The tool uses regex patterns to extract ticket IDs from:

* Commit messages
* Branch names
* Pull request titles

Configure the pattern in settings:

```json
{
  "scm": {
    "key_pattern_regex": "PROJ-[0-9]+"
  }
}
```

### Do I need to tag code before generating reports?

**It depends on your SeaLights configuration.**

If your CI/CD pipeline reports builds to SeaLights with git commit information (commit messages, branch names), SeaLights may already be auto-tagging your code with ticket IDs using regex patterns configured by your Customer Success representative. **If auto-tagging is enabled, you can skip the tagging scripts entirely** and go directly to report generation.

If auto-tagging is not configured, yes—you need to run the tagging scripts first:

1. **Tag Phase** - Run US\_SRC\_\* scripts to associate code changes with ticket IDs in SeaLights
2. **Report Phase** - Run US\_RPT\_\* scripts to query SeaLights and generate reports

**To check if you need tagging scripts:**

* Ask your SeaLights Customer Success representative if auto-tagging is configured for your apps/branches
* Try generating a report—if tickets show 0% coverage despite having tests, tagging may be needed

### What is SeaLights auto-tagging and how do I enable it?

SeaLights auto-tagging is a feature where SeaLights automatically extracts ticket IDs from git commit information (commit messages or branch names) when builds are reported to SeaLights. This eliminates the need to run the US\_SRC\_\* tagging scripts.

**How it works:**

* Your CI/CD pipeline reports builds to SeaLights including git commit information
* SeaLights uses configured regex patterns to extract ticket IDs
* Code is automatically tagged with those ticket IDs
* You only need to run the report generation scripts (US\_RPT\_\*)

**How to enable it:**

* Contact your SeaLights Customer Success representative
* Provide: ticket naming regex pattern (e.g., `PROJ-[0-9]+`) and scope (apps/branches)
* SeaLights CS will configure auto-tagging for you
* This is not a self-service feature

**Benefits:**

* No need to run tagging scripts
* Tagging happens automatically with each build
* Simpler workflow—just reporting phase
* No separate tagging infrastructure needed

### How often should I run tagging?

It depends on your workflow:

**Daily/Continuous:**

* Run after PRs are merged
* Run nightly for all recent changes

**Sprint-based:**

* Run at sprint start to capture previous sprint
* Run mid-sprint for progress tracking
* Run at sprint end for final reports

**On-demand:**

* Before releases
* When specific reports are needed

### Can I tag retroactively?

Yes! Use `--since-date` to tag historical changes:

```bash
python3 US_SRC_tag_repos_by_github_prs.py \
  --settings settings \
  --since-date 2026-01-01
```

### What if I don't use pull requests?

Use the git history tagging script:

```bash
python3 US_SRC_tag_repos_by_history.py \
  --settings settings \
  --days-back 30
```

This analyzes commit history directly instead of PRs.

***

## Reporting Questions

### What formats are reports available in?

* **JSON** - Raw data files (ReportInfo\_\*.json)
* **HTML** - Interactive web pages
* **Confluence** - Wiki pages
* **Jira/ADO Fields** - Coverage percentages in issue tracker

### Can I customize the HTML reports?

Not directly through configuration. The HTML reports are generated from templates embedded in the scripts. You can modify the scripts themselves to customize the output.

### How long does report generation take?

Varies based on:

* Number of tickets (1-100 tickets: minutes, 100+ tickets: 10-30 minutes)
* Number of apps/branches
* SeaLights API response time
* `coverage_days_back` setting

Typical: 5-15 minutes for 20-50 tickets.

### Can I speed up report generation?

Yes:

* Reduce `coverage_days_back`
* Process fewer tickets (narrow JQL/WIQL)
* Use `--report-id` to reuse existing coverage report
* Process tickets in batches

### What if a ticket has no coverage data?

The report will show 0% coverage. This usually means:

* Code hasn't been tagged with this ticket ID
* No tests were run against the tagged code
* The lookback period doesn't include test runs

***

## Integration Questions

### Can I integrate this with CI/CD?

Yes! See [Examples - CI/CD Integration](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/examples.md#ci-cd-integration) for:

* GitHub Actions
* Jenkins
* GitLab CI/CD
* Azure Pipelines

### Can I fail builds based on coverage thresholds?

Yes. Parse the generated JSON files and check coverage values:

```bash
coverage=$(jq -r '.overallResults.overall_coverage' reports/ReportInfo_PROJ-123.json)
if (( $(echo "$coverage < 70" | bc -l) )); then
  exit 1
fi
```

See [Example 19](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/examples.md#example-19-conditional-reporting-based-on-coverage).

### Can I send notifications?

Yes. The tool outputs structured JSON that you can parse and send to:

* Slack
* Microsoft Teams
* Email
* Custom webhooks

See [Example 20](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/examples.md#example-20-notification-on-low-coverage).

### Does it work with monorepos?

Yes. Configure multiple entries in `repo_list`, each pointing to different paths or the same repo with different filters.

### Can I track coverage across multiple repositories?

Yes. List all repositories in `repo_list`:

```json
{
  "repo_list": [
    {"repoName": "frontend", "SealightsAppName": "frontend-app"},
    {"repoName": "backend", "SealightsAppName": "backend-api"},
    {"repoName": "mobile", "SealightsAppName": "mobile-app"}
  ]
}
```

***

## Test Stage Questions

### What are test stages?

Test stages represent different phases of testing (unit, integration, E2E, etc.). They must match the stage names in your SeaLights configuration.

### How do I know what my test stages are called?

Check your SeaLights dashboard or Test Gap Analysis (TGA) reports. The stage names used there should match the `name` field in your settings.

### What are grouped test stages?

Grouped test stages aggregate coverage from multiple individual stages. For example, "Overall" might combine Unit + Integration + E2E.

```json
{
  "grouped_test_stages": [
    {
      "name": "Overall",
      "stages": ["Unit_Tests", "Integration_Tests", "E2E_Tests"]
    }
  ]
}
```

### Can I add custom test stages?

Yes, as long as they exist in SeaLights. Add them to the `test_stages` array in settings.

***

## Jira-Specific Questions

### What JQL should I use?

Depends on your goal:

**Current sprint:**

```
sprint in openSprints() AND project = MYPROJ
```

**Last 30 days:**

```
project = MYPROJ AND created >= -30d
```

**Specific epic:**

```
"Epic Link" = MYPROJ-100
```

**By labels:**

```
project = MYPROJ AND labels = sprint-42
```

### How do I find custom field IDs?

1. Open a ticket in Jira
2. View the ticket JSON:
   * Jira Cloud: `https://yourcompany.atlassian.net/rest/api/3/issue/PROJ-123`
   * Use browser dev tools or curl
3. Search for your field in the JSON
4. Note the `customfield_XXXXX` ID

### Do I need admin access to Jira?

No, but you need:

* Read access to projects in your JQL
* Write access if updating custom fields (US4\_RPT\_update\_jira.py)
* Plugin admin rights if using the Jira plugin

For `US4_RPT_update_jira.py`, the script defaults to sending Jira updates with `notifyUsers=false`. Some Jira environments require elevated/admin-capable credentials for that query parameter. If your token cannot do that, set `jira.notify_users` (or `JIRA_NOTIFY_USERS`) to `true` so the update is sent with notifications enabled instead.

***

## Azure DevOps-Specific Questions

### What WIQL should I use?

**User stories in last 30 days:**

```
SELECT [System.Id] FROM WorkItems
WHERE [System.WorkItemType] = 'User Story'
AND [System.ChangedDate] >= @Today - 30
```

**Specific iteration:**

```
SELECT [System.Id] FROM WorkItems
WHERE [System.IterationPath] = 'MyProject\\Sprint 42'
```

**By area path:**

```
SELECT [System.Id] FROM WorkItems
WHERE [System.AreaPath] UNDER 'MyProject\\Team A'
```

### How do I test my WIQL?

In Azure DevOps:

1. Go to Boards → Queries
2. Create new query
3. Switch to "Editor" mode
4. Paste your WIQL
5. Run and verify results

### What permissions do I need in ADO?

* **Work Items:** Read
* **Extension Data:** Read & Write (if using ADO plugin)

Create a PAT with these permissions.

***

## SeaLights Questions

### Do I need a SeaLights agent running?

No. The tool uses the SeaLights REST API. However, you need:

* SeaLights agents running during your tests (to collect coverage)
* Builds reported to SeaLights with your app/branch names

### What if my app name in SeaLights is different from my repo name?

That's fine and common. Map them in `repo_list`:

```json
{
  "repo_list": [
    {
      "repoName": "my-git-repo",
      "SealightsAppName": "My Application Name in SeaLights"
    }
  ]
}
```

### What are integration builds?

Integration builds are SeaLights builds that combine multiple component apps. Enable with:

```json
{
  "coverage_app_list": [
    {
      "app_regex": "my-integration-build",
      "branch_regex": "main",
      "integration_build": true
    }
  ]
}
```

The tool will validate that all components exist in `repo_list`.

### Can I use regex for app and branch names?

Yes! The `app_regex` and `branch_regex` fields are regex patterns:

```json
{
  "coverage_app_list": [
    {"app_regex": "frontend-.*", "branch_regex": "main|master"}
  ]
}
```

***

## Performance Questions

### Why is tagging so slow?

Common causes:

* **Large time range** - Reduce `--days-back`
* **Many PRs/commits** - Process repos in parallel
* **Large repositories** - Git clone/pull takes time
* **Network latency** - Run closer to your git server

### Why is report generation slow?

Common causes:

* **Many tickets** - Process in batches
* **Long lookback** - Reduce `coverage_days_back`
* **SeaLights processing** - First report takes longer, subsequent reports are cached

### Can I process repositories in parallel?

Yes, use GNU Parallel or run multiple scripts simultaneously:

```bash
python3 US_SRC_tag_repos_by_github_prs.py --settings settings_repo1 &
python3 US_SRC_tag_repos_by_github_prs.py --settings settings_repo2 &
wait
```

***

## Troubleshooting Questions

### Where can I find error logs?

Enable debug logging and redirect to a file:

```bash
python3 US1_RPT_create_report_info_files.py \
  --settings settings \
  --log-level DEBUG \
  2>&1 | tee debug.log
```

### The tool says "No tickets found" but I can see them in Jira/ADO

Verify:

1. Your JQL/WIQL returns results in the web UI
2. API credentials have read access to those tickets
3. Settings file has correct base URL and authorization

Test API directly:

```bash
curl -H "Authorization: Basic ..." \
  "https://yourcompany.atlassian.net/rest/api/3/search?jql=project=MYPROJ"
```

### Reports show 0% coverage for all tickets

This means code hasn't been tagged. Run tagging scripts first:

```bash
python3 US_SRC_tag_repos_by_github_prs.py --settings settings --days-back 30
```

Then generate reports:

```bash
python3 US1_RPT_create_report_info_files.py --settings settings
```

### How do I debug regex pattern matching?

Test in Python:

```bash
python3 -c "
import re
text = '[PROJ-123] Add feature'
pattern = r'PROJ-[0-9]+'
match = re.search(pattern, text)
print(f'Match: {match.group()}' if match else 'No match')
"
```

***

## Best Practices

### How should I organize my settings files?

**Recommended structure:**

```
base_settings          # Non-sensitive defaults
team_settings          # Team-specific config
credentials            # Tokens (gitignored)
```

Use multiple files:

```bash
python3 script.py --settings base_settings --settings credentials
```

### Should I commit settings to git?

* **Commit:** Structure, regex patterns, test stages, non-sensitive URLs
* **Don't commit:** API tokens, passwords, production credentials

Add to `.gitignore`:

```
settings
credentials
*_credentials.json
```

### How often should I generate reports?

**Automated:**

* Nightly for active sprints
* After each deployment

**On-demand:**

* Before sprint planning/review
* Before releases
* When coverage questions arise

### What's the recommended lookback period?

* **Tagging:** 30-90 days (captures most active development)
* **Coverage:** 14-30 days (balances data freshness vs performance)

Adjust based on your development velocity.

***

## Advanced Questions

### Can I customize report formatting?

HTML/Confluence reports use embedded templates in the scripts. To customize, modify the Python scripts directly (search for HTML template strings).

### Can I export data to other tools?

Yes! The ReportInfo JSON files contain structured data you can parse and send anywhere:

```bash
# Example: Send to custom endpoint
for report in reports/ReportInfo_*.json; do
  curl -X POST https://your-api.com/coverage \
    -H "Content-Type: application/json" \
    -d @"$report"
done
```

### How do I handle private repos with 2FA?

Use SSH keys or personal access tokens (PATs), not passwords. See [Installation - Git Authentication](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/installation.md#git-authentication-setup-only-for-git-based-tagging).

### Can I run this in Docker?

Yes:

```dockerfile
FROM python:3.9-slim
WORKDIR /app
COPY . .
RUN pip install requests
CMD ["python3", "US1_RPT_create_report_info_files.py", "--settings", "settings"]
```

***

## More Help

* [Troubleshooting Guide](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/troubleshooting.md) - Detailed error solutions
* [Configuration Guide](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/configuration.md) - All settings explained
* [Examples](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/examples.md) - Working code samples
* [CLI Reference](/knowledgebase/setup-and-configuration/integrations/user-story-coverage/cli-reference.md) - Complete command documentation


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.sealights.io/knowledgebase/setup-and-configuration/integrations/user-story-coverage/faq.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.