Handling Unexpected Closure of Testing Executions

If a test framework fails or a pipeline stops early, some testing executions can remain open. These open executions can block coverage calculation and leave execution state inaccurate.

Most teams manage cleanup by environment. This is the most common pattern for functional testing, where one environment can host multiple related executions. This page focuses on that use case.

Use the Get Executions Status List API to:

1. Query executions with status=created.
2. Filter by labId.
3. Optionally add testStage.
4. Close each matching test session.

You can also manage cleanup by application version. This is common for unit tests and other in-process tests, where executions are tied to one specific app version. In that case, use bsid to target one specific build session for that version.

Details on the Get Executions Status List API are available in the API reference.

Below are sample implementations that query remaining open executions and close them one by one as part of post-run cleanup.

Jenkins/Bash

    post {
        aborted{
            echo '[Sealights] Cleanup executions left open.'
            withCredentials([string(credentialsId: 'sl.agent.token', variable: 'SL_AGENT_TOKEN')]) {
                sh '''
                    set +x
                    SL_DOMAIN="yourcustomdomain.sealights.co"
                    SL_AGENT_TOKEN=$(cat ./sl-agent-token.txt)

                    SL_LABID="MyLabID"
                    SL_TESTSTAGE="Functional Tests"

                    SL_TEST_EXECUTION_IDs=(`curl -sX GET "https://$SL_DOMAIN/sl-api/v1/executions?labId=$SL_LABID&testStage=$SL_TESTSTAGE&status=created" \
                                         -H "Authorization: Bearer $SL_AGENT_TOKEN" \
                                         -H "Content-Type: application/json" \
                                         | jq -r '.data.list | map(.executionId) | join(" ")'`)

                    # Optional: filter based on Test Stage and/or LabID

                    echo "Found ${#SL_TEST_EXECUTION_IDs[@]} executions"

                    for id in ${SL_TEST_EXECUTION_IDs[@]}
                    do 
                       echo -n "Closing Test Session ID $id: "
                       curl -isX DELETE "https://$SL_DOMAIN/sl-api/v1/test-sessions/$id" \
                         -H "Authorization: Bearer $SL_AGENT_TOKEN" \
                         -H "Content-Type: application/json" | grep HTTP
                    done
                '''
            }
        }
    }

GitHub Actions/Python

GitHub Actions YAML executing the Sealights cleanup step when the Functional Tests job fails or is cancelled.

  functional-tests:
    needs: build
    runs-on: ubuntu-latest
    env:
      SL_DOMAIN: yourcustomdomain.sealights.co
      SL_LABID: MyLabID
      SL_TESTSTAGE: "Functional Tests"
    steps:
      ## ...
      - name: Run Functional Tests
        id: functionaltests
        continue-on-error: true
        run: ## Functional test command
      - name: Sealights - Cleanup Open Executions
        if: (cancelled() || steps.functionaltests.outcome == 'failure') && env.SEALIGHTS_ENABLED == 'true'
        run: |
          pip install requests
          python -u scripts/sealights-utils.py --cleanup \
            "${{ env.SL_DOMAIN }}" \
            "${{ secrets.SL_AGENT_TOKEN }}" \
            "${{ env.SL_LABID }}" \
            "${{ env.SL_TESTSTAGE }}"

Python script

import requests


def cleanup_open_executions(SL_DOMAIN, SL_AGENT_TOKEN, SL_LABID, SL_TESTSTAGE=None):
    SL_BASE_URL = f"https://{SL_DOMAIN}/sl-api"
    SL_HEADERS = {
        "Authorization": f"Bearer {SL_AGENT_TOKEN}",
        "Content-Type": "application/json",
    }
    SL_PARAMS = {"labId": SL_LABID, "status": "created"}
    if SL_TESTSTAGE:
        SL_PARAMS = {
            "labId": SL_LABID,
            "testStage": SL_TESTSTAGE,
            "status": "created",
        }

    print(
        f"[Sealights] Cleanup executions left open for LabID '{SL_LABID}'"
        + (
            f" and Test Stage '{SL_TESTSTAGE}'"
            if SL_TESTSTAGE
            else ""
        )
    )

    try:
        response = requests.get(
            f"{SL_BASE_URL}/v1/executions",
            headers=SL_HEADERS,
            params=SL_PARAMS,
            timeout=30,
        )
        response.raise_for_status()
    except Exception as e:
        print(f"Failed to query executions: {e}")
        return 0

    data = response.json()
    SL_TEST_EXECUTION_IDs = [
        item["executionId"]
        for item in data.get("data", {}).get("list", [])
    ]
    print(f"Found {len(SL_TEST_EXECUTION_IDs)} open execution(s)")

    closed = 0
    failed = 0
    for execution_id in SL_TEST_EXECUTION_IDs:
        delete_url = f"{SL_BASE_URL}/v1/test-sessions/{execution_id}"
        try:
            resp = requests.delete(delete_url, headers=SL_HEADERS, timeout=30)
            if resp.status_code in (200, 202, 204):
                print(f"Closing Test Session ID {execution_id}: HTTP {resp.status_code}")
                closed += 1
            else:
                print(
                    f"Closing Test Session ID {execution_id}: HTTP {resp.status_code} - {resp.text}"
                )
                failed += 1
        except Exception as e:
            print(f"Closing Test Session ID {execution_id}: {e}")
            failed += 1

    print(f"Closed {closed} execution(s)")
    if failed:
        print(f"Failed to close {failed} execution(s)")

    return closed

ADO/Powershell

steps:
- task: Bash@3
  name: RunFunctionalTests
  displayName: "Run Functional Tests"
  inputs:
    targetType: inline
    script: |
      ./run-functional-tests.sh
# ========================================================================
# 2. Sealights cleanup only if RunFunctionalTests failed or was cancelled
# ========================================================================
- task: PowerShell@2
  displayName: "Sealights cleanup for Functional Tests"
  condition: or(failed('RunFunctionalTests'), canceled('RunFunctionalTests'))
  env:
    SL_AGENT_TOKEN: $(sl.agent.token)
    SL_DOMAIN: $(DOMAIN)
    SL_LABID: $(LABID)
    SL_TESTSTAGE: Functional Tests
  inputs:
    targetType: inline
    script: |
      Write-Host "[Sealights] Cleanup executions left open."

      # Build API URL
      $encodedTestStage = [System.Uri]::EscapeDataString($env:SL_TESTSTAGE)
      $url = "https://$env:SL_DOMAIN/sl-api/v1/executions?labId=$env:SL_LABID&testStage=$encodedTestStage&status=created"

      # Prepare headers
      $headers = @{
        "Authorization" = "Bearer $env:SL_AGENT_TOKEN"
        "Content-Type"  = "application/json"
      }

      # Get execution IDs
      try {
        $response = Invoke-RestMethod -Method GET -Uri $url -Headers $headers -ErrorAction Stop
        $SL_TEST_EXECUTION_IDs = $response.data.list | ForEach-Object { $_.executionId }
      }
      catch {
        Write-Host "Failed to fetch execution list: $($_.Exception.Message)"
        exit 0   # do not fail pipeline
      }

      if (!$SL_TEST_EXECUTION_IDs) {
        Write-Host "No executions found."
        exit 0
      }

      Write-Host "Found $($SL_TEST_EXECUTION_IDs.Count) executions"

      # Close each open Sealights test session
      foreach ($id in $SL_TEST_EXECUTION_IDs) {
        Write-Host ("Closing Test Session ID {0}: " -f $id) -NoNewline

        $deleteUrl = "https://$env:SL_DOMAIN/sl-api/v1/test-sessions/$id"

        try {
          $deleteResponse = Invoke-WebRequest -Method DELETE -Uri $deleteUrl -Headers $headers -ErrorAction Stop
          if ($deleteResponse.StatusCode -in 200, 202, 204) {
            Write-Host ("HTTP {0}" -f $deleteResponse.StatusCode)
          }
          else {
            Write-Host ("HTTP {0}" -f $deleteResponse.StatusCode)
          }
        }
        catch {
          if ($_.Exception.Response -and $_.Exception.Response.StatusCode) {
            Write-Host ("HTTP {0}" -f [int]$_.Exception.Response.StatusCode)
          }
          else {
            Write-Host "Failed: $($_.Exception.Message)"
          }
        }
      }

Best Practices

• URL-encode testStage values that contain spaces or special characters before sending the request.
• Verify the HTTP status for each DELETE request. Treat non-2xx responses as cleanup failures and log the execution ID for follow-up.
• If the execution list spans more than one API response, iterate through every page before building SL_TEST_EXECUTION_IDs. Do not assume a busy shared lab fits in one response.