Introduction

What is User Story Coverage Tagging?

User Story Coverage Tagging is a Python-based integration system that connects issue tracking platforms (Jira and Azure DevOps) with the SeaLights test coverage platform. It enables development teams to track and report test coverage at the user story level, providing visibility into which code changes have adequate test coverage.

What Does This Tool Do?

The tool automates the process of:

1. Extracting ticket IDs from source control - Identifies which user stories are associated with specific code changes by parsing commit messages, branch names, or pull request titles

2. Tagging code changes in SeaLights - Links modified code to ticket IDs in the SeaLights platform for coverage analysis

3. Generating comprehensive coverage reports - Creates coverage reports in multiple formats (HTML, Confluence, JSON) for each user story

4. Pushing coverage data to issue trackers - Updates Jira or Azure DevOps with coverage metrics and summaries

Audience

This tool is designed for:

• DevOps Engineers - Automating test coverage reporting in CI/CD pipelines

• QA Engineers - Monitoring test coverage for specific user stories and features

• Engineering Managers - Tracking team-level test coverage metrics across sprints

• Release Managers - Ensuring adequate test coverage before production releases

• Development Teams - Using Jira or Azure DevOps for issue tracking and SeaLights for test intelligence

High-Level Use Cases

Use Case 1: Sprint Coverage Reporting

Generate coverage reports for all user stories completed in a sprint:

1. Configure the tool with your Jira/ADO query (e.g., stories completed in last 30 days)

2. Tag code changes by processing GitHub pull requests

3. Generate HTML reports showing coverage per story

4. Create a Confluence page summarizing sprint coverage

5. Update Jira tickets with coverage percentages

Use Case 2: Pull Request Coverage Analysis

Track coverage for code changes in specific pull requests:

1. Tag modified files from GitHub PRs linked to tickets

2. Generate coverage reports for those specific tickets

3. Push coverage metrics back to the PR or issue tracker

4. Ensure minimum coverage thresholds are met before merge

Use Case 3: Feature Coverage Tracking

Monitor test coverage for a specific feature across multiple tickets:

1. Use JQL/WIQL to query all tickets related to a feature (epic, parent story, labels)

2. Tag code changes from commit history for all related repositories

3. Generate aggregated coverage reports showing coverage by test stage

4. Track untested methods and files that need additional testing

Use Case 4: CI/CD Integration

Integrate into continuous integration pipelines:

1. Run tagging scripts automatically when PRs are merged

2. Schedule nightly report generation for active tickets

3. Fail builds if coverage drops below configured thresholds

4. Automatically update issue trackers with latest coverage data

How It Works

The system operates in two phases, with Phase 1 being optional if you use SeaLights auto-tagging:

Phase 1: Source Tagging (Optional)

Option A: SeaLights Auto-Tagging (Recommended)

If your CI/CD pipeline reports builds to SeaLights with git commit information (commit messages, branch names), SeaLights can automatically extract and tag ticket IDs using regex patterns. This is configured by your SeaLights Customer Success representative—you provide the ticket naming pattern and scope (apps/branches), and SeaLights handles tagging automatically. When auto-tagging is enabled, you can skip the US_SRC_ tagging scripts entirely.*

Option B: Manual Tagging Scripts

If auto-tagging isn't available or suitable for your workflow, use the provided scripts to extract ticket IDs from source control metadata (commits, branches, PRs) and call the SeaLights API to tag modified code. This creates the linkage between user stories and code changes.

Phase 2: Reporting (Always Required)

Scripts query SeaLights for coverage data associated with tagged tickets, then generate reports and distribute them to various destinations (HTML files, Confluence pages, Jira/ADO fields). This phase is required regardless of which tagging approach you use.

Key Benefits

• Traceability - Direct linkage between user stories and test coverage

• Automation - Eliminates manual coverage reporting

• Flexibility - Supports multiple SCM platforms (GitHub, GitLab, Bitbucket, Azure DevOps)

• Multi-format reporting - HTML, Confluence, JSON, and direct issue tracker updates

• Configurable - Extensive settings for test stages, coverage thresholds, and report formatting

• CI/CD Ready - Designed for integration into automated pipelines

Next Steps

• Installation Guide - Set up Python and install dependencies

• Quick Start - Run your first coverage report in minutes

• Configuration Guide - Configure settings for your environment