Quick Start

This guide will walk you through running your first coverage report in minutes.

Prerequisites

Before starting, ensure you have:

• Python 3.7+ installed

• requests library installed (pip install requests)

• Access credentials for:

  • SeaLights API (token and domain)

  • Jira or Azure DevOps (authorization token)

  • GitHub (optional, only if manually tagging from PRs)

Note About Tagging:

If your CI/CD pipeline reports builds to SeaLights with git commit information, SeaLights may already be auto-tagging your code with ticket IDs. This is configured by your SeaLights Customer Success representative. If auto-tagging is enabled, you can skip all tagging steps and jump directly to report generation (Step 3 below). The tagging scripts (US_SRC_*) are only needed if you're not using SeaLights auto-tagging.

Step 1: Create a Minimal Settings File

Create a file named settings with the following minimal configuration:

{
    "output_folder": "./reports",
    "sealights": {
        "api_token": "your-sealights-api-token",
        "domain": "yourcompany.sealights.co",
        "coverage_days_back": 14
    },
    "jira": {
        "authorization": "Basic your-base64-encoded-credentials",
        "jql": "project = MYPROJECT AND created > startOfMonth(-1)",
        "child_levels": 1,
        "base_url": "https://yourcompany.atlassian.net",
        "api_version": 3
    },
    "scm": {
        "key_pattern_regex": "MYKEY-[0-9]*",
        "type": "github"
    },
    "test_stages": [
        {
            "name": "Unit_Tests",
            "reportJsonKey": "unit_tests",
            "reportTitle": "Unit Tests"
        }
    ],
    "coverage_app_list": [
        {
            "app_regex": "my-app",
            "branch_regex": "main"
        }
    ],
    "repo_list": []
}

Replace the following values:

• your-sealights-api-token - Your SeaLights API token

• yourcompany.sealights.co - Your SeaLights domain

• your-base64-encoded-credentials - Your Jira credentials (see Security Considerations)

• yourcompany.atlassian.net - Your Jira instance URL

• MYPROJECT - Your Jira project key

• MYKEY-[0-9]* - Regex pattern matching your ticket IDs

• my-app - Your application name in SeaLights

Step 2: Create Output Directory

mkdir -p reports

Step 3: Generate Coverage Report Data

Run the script to fetch tickets from Jira and generate coverage data:

python3 US1_RPT_create_report_info_files.py --settings settings --ticket-source jira

Expected Output

2026-02-02 10:15:32 INFO: Reading settings file 'settings
2026-02-02 10:15:32 INFO: working in folder /path/to/UserStoryCoverageTagging/reports
2026-02-02 10:15:33 INFO: Found 5 tickets matching JQL query
2026-02-02 10:15:33 INFO: Processing ticket MYKEY-123
2026-02-02 10:15:34 INFO: Generating report, id=abc123def456
2026-02-02 10:15:45 INFO: Coverage report ready
2026-02-02 10:15:46 INFO: Creating report info file 'reports/ReportInfo_MYKEY-123.json'
...
2026-02-02 10:16:05 INFO: Generated 5 report files

Step 4: Generate HTML Reports

Create human-readable HTML reports from the JSON data:

python3 US2_RPT_create_html_reports.py --settings settings

Expected Output

2026-02-02 10:17:01 INFO: Reading settings file 'settings
2026-02-02 10:17:01 INFO: working in folder /path/to/UserStoryCoverageTagging/reports
2026-02-02 10:17:02 INFO: Found 5 ReportInfo JSON files
2026-02-02 10:17:02 INFO: Generating HTML for MYKEY-123
2026-02-02 10:17:03 INFO: Created reports/MYKEY-123.html
...
2026-02-02 10:17:10 INFO: Created reports/Summary.html
2026-02-02 10:17:10 INFO: Generated 6 HTML files

Step 5: View the Reports

Open the generated reports in your browser:

# Open summary report
open reports/Summary.html  # macOS
xdg-open reports/Summary.html  # Linux
start reports/Summary.html  # Windows

# Or open individual ticket reports
open reports/MYKEY-123.html

The HTML reports will show:

• Ticket information (ID, title, status, assignee)

• Coverage percentages by test stage

• List of tested and untested methods

• Links to source code files

Alternative: Quick Start with Azure DevOps

If you use Azure DevOps instead of Jira, modify the settings file:

{
    "output_folder": "./reports",
    "sealights": {
        "api_token": "your-sealights-api-token",
        "domain": "yourcompany.sealights.co",
        "coverage_days_back": 14
    },
    "ado": {
        "wiql": "SELECT [System.Id] FROM WorkItems WHERE [System.WorkItemType] = 'User Story' AND [System.ChangedDate] >= @Today - 30",
        "authorization": "Basic your-pat-token",
        "organization": "your-org",
        "project": "your-project",
        "child_levels": 1
    },
    "scm": {
        "key_pattern_regex": "[0-9]+",
        "type": "ado"
    },
    "test_stages": [
        {
            "name": "Unit_Tests",
            "reportJsonKey": "unit_tests",
            "reportTitle": "Unit Tests"
        }
    ],
    "coverage_app_list": [
        {
            "app_regex": "my-app",
            "branch_regex": "main"
        }
    ],
    "repo_list": []
}

Then run:

python3 US1_RPT_create_report_info_files.py --settings settings --ticket-source ado
python3 US2_RPT_create_html_reports.py --settings settings

Common First-Run Issues

No tickets found

Symptom: No tickets found, exiting

Solution: Check your JQL/WIQL query returns results in your issue tracker's web interface. Adjust the date range or query criteria.

Failed to create coverage report

Symptom: Failed to create the coverage report

Solution: Verify:

1. The app name in coverage_app_list matches your SeaLights application name exactly

2. The branch name matches

3. The app has builds reported in the last coverage_days_back days

No coverage data found

Symptom: Report files created but show 0% coverage

Solution: This usually means:

1. Code changes haven't been tagged yet (see Examples for tagging workflows)

2. The ticket IDs don't match any tagged code in SeaLights

3. The coverage lookback period (coverage_days_back) doesn't include your test runs

Next Steps

Now that you have basic reporting working:

1. Set up source tagging - Tag your code changes with ticket IDs (Examples)

2. Configure test stages - Define your test pipeline stages (Configuration)

3. Automate in CI/CD - Integrate into your build pipeline (Examples)

4. Push to issue trackers - Update Jira/ADO with coverage data (CLI Reference)

Full Example: End-to-End Workflow

Here's a complete workflow from tagging to reporting:

# 1. Tag code changes from GitHub PRs (last 30 days)
python3 US_SRC_tag_repos_by_github_prs.py --settings settings --days-back 30

# 2. Generate report data from Jira tickets
python3 US1_RPT_create_report_info_files.py --settings settings --ticket-source jira

# 3. Create HTML reports
python3 US2_RPT_create_html_reports.py --settings settings

# 4. (Optional) Create Confluence pages
python3 US3_RPT_create_confluence_reports.py --settings settings

# 5. (Optional) Update Jira with coverage data
python3 US4_RPT_update_jira.py --settings settings

For detailed configuration options and advanced usage, see the Configuration Guide and CLI Reference.