Deployment patterns

Use practical deployment patterns for tagging, reporting, publishing, and CI/CD.

Choose the pattern that matches your rollout goal.

Use the expandables for the shortest path. Use the linked reference pages for field details.

Use Installation and first run for the first successful run.

Prefer the Jira plugin or ADO plugin paths when possible. Use Jira custom fields only for legacy Jira setups.

End-to-end reporting workflows

SeaLights auto-tagging

Use this when SeaLights already tags code changes during build reporting.

Confirm the data path

Confirm your ticket source in Data sources.

Confirm your target in Publication targets.

Generate report data

Run the report-generation flow from Report Generation.

Use --ticket-source jira or --ticket-source ado to match your source system.

Publish the result

Publish issue KPIs or detailed pages from Publish Analytics.

Expected outcome

No separate tagging step runs.
Coverage comes from existing SeaLights tagging data.
The flow stays short for recurring reports.

Manual tagging

Use this when auto-tagging is unavailable or too limited for your build flow.

Tag source changes

Choose the tagging command in Source Tagging.

Use --days-back for pull-request tagging.

Use --since-date for history-based tagging.

Generate report data

Run the report-generation flow from Report Generation.

Publish the result

Publish issue KPIs or detailed pages from Publish Analytics.

Expected outcome

Source changes are tagged before coverage is calculated.
Ticket-linked coverage works without auto-tagging.
Downstream publishing stays unchanged.

Use this path only when auto-tagging does not fit your workflow.

Issue coverage KPIs

Sprint coverage

Use this when you want sprint tickets updated where teams already review work. Choose the platform that matches the KPI destination.

✅ Prerequisites

Your JQL returns the sprint tickets you want.
Jira publishing is configured.
Report data generation already works.

🚀 Steps

Configure the ticket query

Set the JQL filter in Data sources.

Generate report data

Run the Jira report flow from Report Generation with --ticket-source jira.

Publish sprint coverage

Use the Jira publishing path in Publish Analytics.

📤 Expected outcome

Jira issues show updated sprint coverage values. Teams review coverage without leaving Jira.

✅ Prerequisites

Your WIQL returns the sprint work items you want.
Azure DevOps publishing is configured.
Report data generation already works.

🚀 Steps

Configure the work item query

Set the WIQL in Data sources.

Generate report data

Run the Azure DevOps report flow from Report Generation with --ticket-source ado.

Publish sprint coverage

Use the Azure DevOps publishing path in Publish Analytics.

📤 Expected outcome

Azure DevOps work items show updated sprint coverage. Coverage follows the WIQL result set.

✅ Prerequisites

Jira custom field IDs are defined.
Jira write access is available.
Report data generation already works.

🚀 Steps

Configure the query and field IDs

Set the JQL filter in Data sources. Set the field mapping in Publication targets.

Generate report data

Run the Jira report flow from Report Generation with --ticket-source jira.

Publish sprint coverage

Use the legacy Jira publishing path in Publish Analytics.

📤 Expected outcome

Jira custom fields are updated with sprint coverage values. Existing legacy dashboards keep working.

Detailed coverage reports

Feature coverage

Use this when teams need page-based coverage for a feature, epic, or cross-ticket review. Choose the destination that matches where teams review full coverage pages.

✅ Prerequisites

Confluence access is configured.
The target space and parent page are known.
Report data generation already works.

🚀 Steps

Configure the target

Set the Confluence target in Publication targets.

Generate report data

Run the report flow from Report Generation.

Publish detailed pages

Use the Confluence publishing path in Publish Analytics.

📤 Expected outcome

A summary page is created or updated in Confluence. Ticket-level pages are published under the target parent page. Teams get stable links for feature-level review.

Azure DevOps Wiki is a planned target. It is not the current production path. Azure DevOps Wiki support is planned by EOY2026.

🚀 Steps

Keep the current production target

Continue to publish detailed pages to Confluence for now.

Track the future destination

Record Azure DevOps Wiki as the planned target in your rollout plan.

📤 Expected outcome

Detailed publishing still goes to Confluence today. Azure DevOps Wiki remains the planned future destination.

CI/CD automation

Use this when coverage must run on a schedule or as part of delivery gates. Choose the platform tab that matches your pipeline.

# .github/workflows/coverage-report.yml
name: User Story Coverage Report

on:
  schedule:
    - cron: '0 2 * * *'  # Daily at 2 AM
  workflow_dispatch:

jobs:
  generate-coverage-report:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.9'

      - name: Install dependencies
        run: pip install requests

      - name: Tag PRs
        env:
          SEALIGHTS_API_TOKEN: ${{ secrets.SEALIGHTS_TOKEN }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          JIRA_AUTHORIZATION: ${{ secrets.JIRA_AUTH }}
        run: |
          python3 US_SRC_tag_repos_by_github_prs.py \
            --settings settings \
            --days-back 30

      - name: Generate reports
        env:
          SEALIGHTS_API_TOKEN: ${{ secrets.SEALIGHTS_TOKEN }}
          JIRA_AUTHORIZATION: ${{ secrets.JIRA_AUTH }}
        run: |
          python3 US1_RPT_create_report_info_files.py \
            --settings settings \
            --ticket-source jira
          python3 US2_RPT_create_html_reports.py \
            --settings settings

      - name: Upload reports
        uses: actions/upload-artifact@v3
        with:
          name: coverage-reports
          path: reports/*.html

      - name: Update Jira
        env:
          JIRA_AUTHORIZATION: ${{ secrets.JIRA_AUTH }}
        run: |
          python3 US4_RPT_update_jira.py --settings settings

// Jenkinsfile
pipeline {
    agent any

    environment {
        SEALIGHTS_API_TOKEN = credentials('sealights-token')
        JIRA_AUTHORIZATION = credentials('jira-auth')
        GITHUB_TOKEN = credentials('github-pat')
    }

    stages {
        stage('Setup') {
            steps {
                sh 'pip install requests'
            }
        }

        stage('Tag Code Changes') {
            steps {
                sh '''
                    python3 US_SRC_tag_repos_by_github_prs.py \
                        --settings settings \
                        --days-back 30 \
                        --log-level INFO
                '''
            }
        }

        stage('Generate Reports') {
            steps {
                sh '''
                    python3 US1_RPT_create_report_info_files.py \
                        --settings settings \
                        --ticket-source jira
                    python3 US2_RPT_create_html_reports.py \
                        --settings settings
                '''
            }
        }

        stage('Publish') {
            parallel {
                stage('Confluence') {
                    steps {
                        sh 'python3 US3_RPT_create_confluence_reports.py --settings settings'
                    }
                }
                stage('Jira') {
                    steps {
                        sh 'python3 US4_RPT_update_jira.py --settings settings'
                    }
                }
            }
        }
    }

    post {
        always {
            archiveArtifacts artifacts: 'reports/**/*.html', fingerprint: true
        }
    }
}

# .gitlab-ci.yml
variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"

cache:
  paths:
    - .cache/pip

stages:
  - tag
  - report
  - publish

tag_code_changes:
  stage: tag
  image: python:3.9
  before_script:
    - pip install requests
  script:
    - |
      python3 US_SRC_tag_repos_by_github_prs.py \
        --settings settings \
        --days-back 30
  only:
    - schedules

generate_reports:
  stage: report
  image: python:3.9
  before_script:
    - pip install requests
  script:
    - |
      python3 US1_RPT_create_report_info_files.py \
        --settings settings \
        --ticket-source jira
      python3 US2_RPT_create_html_reports.py \
        --settings settings
  artifacts:
    paths:
      - reports/
    expire_in: 7 days
  only:
    - schedules

publish_confluence:
  stage: publish
  image: python:3.9
  before_script:
    - pip install requests
  script:
    - python3 US3_RPT_create_confluence_reports.py --settings settings
  only:
    - schedules

publish_jira:
  stage: publish
  image: python:3.9
  before_script:
    - pip install requests
  script:
    - python3 US4_RPT_update_jira.py --settings settings
  only:
    - schedules

# azure-pipelines.yml
trigger: none

schedules:
  - cron: "0 2 * * *"
    displayName: Daily coverage report
    branches:
      include:
        - main
    always: true

pool:
  vmImage: 'ubuntu-latest'

variables:
  - group: sealights-coverage-secrets

stages:
  - stage: Tag
    jobs:
      - job: TagCodeChanges
        steps:
          - task: UsePythonVersion@0
            inputs:
              versionSpec: '3.9'
          - script: pip install requests
            displayName: 'Install dependencies'
          - script: |
              python3 US_SRC_tag_repos_by_github_prs.py \
                --settings settings \
                --days-back 30
            displayName: 'Tag PRs'
            env:
              SEALIGHTS_API_TOKEN: $(SEALIGHTS_API_TOKEN)
              GITHUB_TOKEN: $(GITHUB_TOKEN)
              JIRA_AUTHORIZATION: $(JIRA_AUTHORIZATION)

  - stage: Report
    dependsOn: Tag
    jobs:
      - job: GenerateReports
        steps:
          - task: UsePythonVersion@0
            inputs:
              versionSpec: '3.9'
          - script: pip install requests
            displayName: 'Install dependencies'
          - script: |
              python3 US1_RPT_create_report_info_files.py \
                --settings settings \
                --ticket-source jira
              python3 US2_RPT_create_html_reports.py \
                --settings settings
            displayName: 'Generate reports'
            env:
              SEALIGHTS_API_TOKEN: $(SEALIGHTS_API_TOKEN)
              JIRA_AUTHORIZATION: $(JIRA_AUTHORIZATION)
          - publish: reports
            artifact: coverage-reports

  - stage: Publish
    dependsOn: Report
    jobs:
      - job: PublishJira
        steps:
          - task: UsePythonVersion@0
            inputs:
              versionSpec: '3.9'
          - script: pip install requests
            displayName: 'Install dependencies'
          - script: python3 US4_RPT_update_jira.py --settings settings
            displayName: 'Update Jira'
            env:
              JIRA_AUTHORIZATION: $(JIRA_AUTHORIZATION)

Store SEALIGHTS_API_TOKEN, GITHUB_TOKEN, and JIRA_AUTHORIZATION in an Azure DevOps variable group named sealights-coverage-secrets, marked as secret.

CI/CD reference material

Sample ReportInfo JSON structure

{
  "key": "PROJ-123",
  "title": "Implement user authentication",
  "overallResults": {
    "overall_coverage": 78.5,
    "overall_count": 120,
    "unit_tests_coverage": 85.0,
    "unit_tests_count": 80,
    "integration_tests_coverage": 65.0,
    "integration_tests_count": 40
  },
  "methods": [
    {
      "signature": "com.example.auth.AuthService.login",
      "coverage": 100,
      "testStages": ["Unit_Tests", "Integration_Tests"]
    }
  ]
}

Run multiple repositories in parallel

#!/bin/bash
# parallel_tagging.sh
cat > repo_list.txt <<EOF
frontend-app
backend-api
mobile-app
admin-portal
EOF

cat repo_list.txt | parallel -j 4 '
  echo "Processing {}"
  python3 US_SRC_tag_repos_by_github_prs.py \
    --settings settings_{} \
    --days-back 30
'

Fail the build on low coverage

#!/bin/bash
# coverage_gate.sh
THRESHOLD=70
FAILED=false

python3 US1_RPT_create_report_info_files.py --settings settings --ticket-source jira
python3 US2_RPT_create_html_reports.py --settings settings

for report in reports/ReportInfo_*.json; do
  ticket=$(basename "$report" | sed 's/ReportInfo_//;s/.json//')
  coverage=$(jq -r '.overallResults.overall_coverage // 0' "$report")
  if (( $(echo "$coverage < $THRESHOLD" | bc -l) )); then
    echo "FAIL: $ticket at ${coverage}% (below ${THRESHOLD}%)"
    FAILED=true
  fi
done

if [ "$FAILED" = true ]; then
  echo "Coverage gate failed."
  exit 1
fi

Send a low-coverage notification

#!/bin/bash
# notify_low_coverage.sh
THRESHOLD=60
SLACK_WEBHOOK="$SLACK_WEBHOOK_URL"

python3 US1_RPT_create_report_info_files.py --settings settings --ticket-source jira

for report in reports/ReportInfo_*.json; do
  ticket=$(basename "$report" | sed 's/ReportInfo_//;s/.json//')
  coverage=$(jq -r '.overallResults.overall_coverage // 0' "$report")
  title=$(jq -r '.title' "$report")
  if (( $(echo "$coverage < $THRESHOLD" | bc -l) )); then
    message="Low coverage alert: $ticket - $title has ${coverage}% coverage (threshold: ${THRESHOLD}%)"
    curl -X POST "$SLACK_WEBHOOK" -H 'Content-Type: application/json' -d "{\"text\": \"$message\"}"
  fi
done

Analytics scope patterns

One app and one repo

Use this when one repository maps to one SeaLights app and one main branch.

{
  "coverage_app_list": [
    {
      "app_regex": "my-app",
      "branch_regex": "main"
    }
  ],
  "repo_list": [
    {
      "repoName": "frontend-app",
      "SealightsAppName": "my-app",
      "SealightsBranchName": "main"
    }
  ]
}

Expected outcome

Coverage is pulled from one app and mapped back to one repo cleanly.

Several repos in one reporting flow

Use this when one report must include several repositories.

{
  "repo_list": [
    {
      "repoName": "component-a",
      "SealightsAppName": "component-a",
      "SealightsBranchName": "main"
    },
    {
      "repoName": "component-b",
      "SealightsAppName": "component-b",
      "SealightsBranchName": "main"
    }
  ]
}

Expected outcome

The same run can cover more than one repository.

Raw stages and grouped stages together

Use this when you want both detailed stage coverage and a rolled-up KPI.

{
  "test_stages": [
    {
      "name": "Unit_Tests",
      "reportJsonKey": "unit_tests",
      "reportTitle": "Unit Tests"
    },
    {
      "name": "Integration_Tests",
      "reportJsonKey": "integration_tests",
      "reportTitle": "Integration Tests"
    }
  ],
  "grouped_test_stages": [
    {
      "name": "Overall",
      "reportJsonKey": "overall",
      "reportTitle": "Overall Coverage",
      "stages": ["Unit_Tests", "Integration_Tests"]
    }
  ]
}

Expected outcome

Reports show both stage detail and grouped coverage.

Integration build scope

Use this when one reported build combines several components.

{
  "coverage_app_list": [
    {
      "app_regex": "my-integration-build",
      "branch_regex": "main",
      "integration_build": true
    }
  ],
  "repo_list": [
    {
      "repoName": "component-a",
      "SealightsAppName": "component-a",
      "SealightsBranchName": "main"
    },
    {
      "repoName": "component-b",
      "SealightsAppName": "component-b",
      "SealightsBranchName": "main"
    }
  ]
}

Expected outcome

User Story Coverage validates that all required components are in scope.

Multi-system examples

Jira source to Jira plugin output

Use this when Jira is both the ticket source and the KPI destination.

Select the tickets

Use the Jira query path in Data sources.

Generate the report data

Run the Jira report flow from Report Generation.

Publish the KPI

Use the Jira plugin path in Publish Analytics.

Expected outcome

Jira issues are selected by JQL.
Coverage is calculated from SeaLights data.
Jira issues are updated with coverage KPIs.

Azure DevOps source to Confluence output

Use this when work items live in Azure DevOps and teams review full pages in Confluence.

Select the work items

Use the WIQL path in Data sources.

Generate the report data

Run the Azure DevOps report flow from Report Generation.

Publish detailed pages

Use the Confluence path in Publish Analytics.

Expected outcome

Work items are selected by WIQL.
Coverage is matched to those work items.
Detailed pages are published to Confluence.

SeaLights and Git tagging to issue coverage KPIs

Use this when ticket mapping comes from tagging before report generation.

Tag the linked changes

Choose the tagging command in Source Tagging.

Generate the report data

Run the report flow from Report Generation.

Publish the KPI

Use the target-specific path in Publish Analytics.

Expected outcome

Recent code changes are tagged with ticket IDs.
Coverage is linked to those tickets.
Teams see issue-level coverage in the target system.

Tagging workflows

Pull request coverage

Use this when pull requests are the main source of ticket-linked changes.

Choose the tagging source

Use GitHub pull requests when merged PRs are your source of truth.

Use Jira-linked pull requests when tagging must stay limited to Jira query results.

Run the tagging path

Use US_SRC_tag_repos_by_github_prs.py or US_SRC_tag_repos_by_jira_github_prs.py from Source Tagging.

Use --days-back to set the PR lookback window.

Continue to reporting

Generate report data in Report Generation.

Expected outcome

Recent merged pull requests are scanned and matching ticket IDs are tagged in SeaLights.

Git history tagging

Use this when pull requests are not your source of truth.

Set the time range

Choose a stable cutoff date for the tagging window.

Run the history path

Use US_SRC_tag_repos_by_history.py from Source Tagging.

Use --since-date to set the history cutoff.

Continue to reporting

Generate report data in Report Generation.

Expected outcome

Commits are scanned directly and matching ticket IDs are tagged from commit history.

Jira-linked pull requests

Use this when tagging must stay limited to tickets returned by the Jira query.

Configure the Jira query

Set the JQL filter in Data sources.

Run the constrained tagging path

Use US_SRC_tag_repos_by_jira_github_prs.py from Source Tagging.

Continue to reporting

Generate report data in Report Generation.

Expected outcome

Only pull requests linked to the selected Jira tickets are processed.

PreviousPublication targets

NextSecurity Considerations

Last updated 2 months ago

Was this helpful?