Testing Frameworks Integration

This documentation page assumes the two preliminary steps of scan and runtime instrumentation for the application under test are already completed.

When your integration is successful, you’ll see

• In Coverage Dashboard, the Test Stage name is associated with your application entry, detailing coverage and test results (passed, failed, skipped)

• In the Test Optimization pages, for Supported Frameworks, you can see the potential and/or actual savings of your tests (including details on the recommendation reasons) for this specific application/branch/test stage.

Integration for Supported Testing Frameworks

Built-in Integrations with other Tricentis Solutions

Framework name	Documentation link	Test Optimization Support
Tricentis Tosca	Tosca - SeaLights integration (Starting from Tosca 2025.1)
Tricentis Testim	Testim Docs - Sealights Integration

Integrations with the most common Testing Frameworks

This document provides a tabulated summary of the integration support offered by SeaLights for various testing frameworks, categorized by the underlying technology and linked to their respective documentation pages or sections. For more detailed guidance on these framework integration processes, follow the documentation links provided in the table or refer to related sections within the SeaLights support documentation.

Framework name	Underlying Technology	Documentation link	Test Optimization
Cucumber JS	JavaScript	Cucumber JS framework
Cypress	JavaScript	sealights-cypress-plugin - npm
Playwright JS	JavaScript	sealights-playwright-plugin - npm
TestCafe	JavaScript	testcafe-reporter-sealights - npm
Jest	JavaScript	Jest framework
Mocha	JavaScript	Mocha test framework
Karma	JavaScript	Karma test framework
AVA	JavaScript	AVA framework
Protactor	JavaScript	sealights-protractor-plugin - npm
Robot	Python	Robot Framework Sealights Custom Listener
Pytest	Python	Running tests with pytest
Behave	Python	Running tests with behave
Cucumber Java	Java	If executed via Maven: Running Tests (Only) If executed via Gradle: Running Tests (Only) (JSON file)
JUnit, TestNG	Java	If executed via Maven: Running Tests (Only) If executed via Gradle: Running Tests (Only) (JSON file) If executed via Java CLI: Working with a supported framework
SoapUI	Java	Running test listener with SoapUI
JMeter	Java	Running test listener with JMeter
xUnit	.NET	Running tests with MSTest, NUnit, or XUnit
nUnit	.NET	Running tests with MSTest, NUnit, or XUnit
MSTest, VSTest	.NET	Running tests with MSTest, NUnit, or XUnit
Go Test	Go	Running Tests
Ginkgo	Go	Using Ginkgo
Godog, Testify, GoConvey	Go	Framework relying on go test

Integrations maintained by Third-Parties

Framework name	Underlying Technology	Documentation link	Test Optimization
Katalon Studio	Java	Katalon.com - Sealights Integration plugin (Dynamic Test Suites) Katalon.com - Sealights Test Listener (Static Test Suites)
Ginger by Amdocs	.NET	Ginger by Amdocs Help - Sealights Integration

Generic Integration for Unsupported Framework

Sealights Public API

The SeaLights Public API provides integration capabilities with unsupported testing frameworks by allowing users to manage and optimize their test executions. Below are the detailed steps and API calls required for integrating Test Impact Analysis (TIA) using the SeaLights Public API:

1. Creating a Test Session: To begin a test session, you need to create a session using the dedicated API call.

2. Getting Test Recommendations: To optimize the testing process, you can retrieve a list of recommended tests to run and use them to dynamically reduce the list of tests executed

3. Run Your Tests based on Recommendations: Assuming your application under test is properly instrumented with Sealights

4. Reporting Test Events: Once tests are complete, report the test results with the timestamps for each test of its start and end time.

5. Closing a Test Session: Finally, close the test session (delete request) complete the testing cycle, and trigger the processing of all the related metadata (coverage and test results).

All the relevant details of the API calls are described in our dedicated section of the portal: Tricentis SeaLights Public API

Agents' legacy commands

When using the legacy approach of a SeaLights Agent to run your tests, you can report and capture coverage and test results data effectively for your applications under test. The following steps are to follow:

1. Start Session: This command initializes a Test Stage Run for your build, which will be used to link all related test and coverage data. It's critical for associating results from different test stages.

2. Run Your Tests: Assuming your application under test is properly instrumented with Sealights

3. Upload Results: After executing your tests, you need to upload the test results (success, failed, skipped).

4. End Session: Conclude the session to finalize the data collection. This ensures that the session is properly closed and all related data (coverage and test results) is processed.

This approach doesn’t support Sealights Test Optimization and will not allow you to skip irrelevant tests in a seamless way. If this is necessary, please refer to the Public API section.

Technology	Documentation link
Java	Working without a supported framework If you're using Maven or Gradle to trigger a testing framework that isn't directly supported by us but uses a supported test runner (like JUnit or TestNG) under the hood, please refer to the relevant integration guides above or contact Support for assistance.
Javascript	Running Tests
.Net	Legacy approach If you're using a testing framework that isn't directly supported by us but relies on a supported test runner (like xUnit, NUnit, or MSTest) under the hood, please refer to the relevant integration guides above or contact Support for assistance.
Python	No Supported Framework
Go	Running Your Tests with the Legacy Approach