Integrating Test Executions from various Testing Frameworks

You can integrate any Testing Framework with Sealights using our Public API to capture Tests Events like Start and End of Test Stage, Tests results, etc. We’re listing below some examples of integrations for your reference.

The sample code listed in this article is not supported by Sealights as an official plugin, agent, or product and is shared as a courtesy.

Sealights Integration samples repository

Sealights had published a repository of several sample integrations on Github:

GitHub - Sealights/sealights-integration-examplesGitHub

Known Third Party Integrations

Robot Framework Sealights Custom Listener

This section demonstrates applying SeaLights with Test Optimization to tests executed with the Robot framework (Python): It uses a reference implementation of the Robot Listener interface to facilitate calls to SeaLights API at appropriate phases of the test suite's lifecycle. The SeaLights integration is implemented in the file SLListener.py available as an open-source project for you to be able to customize it to your specific configuration.

By default, the command line to add the Sealights' listener to the Robot execution is as follows: --listener “SLListener.py:<CustomerDomain>:<Token>:<buildSessionId>:<Test Stage Name>:<LabId>”.

The command arguments are specified after the listener name (or path) using a colon (:) as a separator:

• Customer Domain URL

• Token

• buildSessionId

• Test Stage name

• (Optional) LabId

For example:

robot --listener "SLListener.py:your-domain.sealights.co:`cat sltoken.txt`:`cat buildSessionId.txt`:Robot Tests" some_tests.robot

or

robot --listener "SLListener.py:your-domain.sealights.co:${SL_TOKEN}:${SL_BUILD_SESSION_ID}:Robot Tests:${SL_LAB_ID}" some_tests.robot

For more information about Robot Framework Interfaces, please refer to the official documentation: Robot Framework User Guide

Katalon Studio Sealights plugin

This plugin is publicly available on Katalon Store.

Ginger by Amdocs

Since Release 4.5, Ginger by Amdocs has the ability to integrate with Sealights using a built-in plugin and allows to:

• Publish Automation execution data to Sealights, to deliver Faster, High-Quality, and controlled releases

• Get Test Recommendations from Sealights for Ginger Runset execution to optimize the efforts and reduce testing cycle time

More details are available in the dedicated section on the Ginger documentation portal: Ginger-Sealights Integration

Sample Code

Converting Results Report from XML

Several frameworks provide the ability to report the results of test executions into an XML format that can be reused to feed Sealights.

For example, the Windows Powershell commands below allow to convert a TestNG XML format into the JSON format expected by Sealights API.

If your Testing Frameworks uses TestNG, please prefer Sealights' Out-Of-the-Box integration with TestNG. If your framework simply outputs its result in this format, the sample below is relevant for you.

$SealightsJson = ConvertTo-Json -InputObject @( [xml]$(Get-Content -Path .\testng-results.xml) | Select-Xml -XPath "//test-method" |foreach {
    [PSCustomObject]@{
        name = $_.node.signature
		start = ([DateTimeOffset](Get-Date $($_.node.'started-at'))).ToUnixTimeMilliseconds()
		end = ([DateTimeOffset](Get-Date $($_.node.'finished-at'))).ToUnixTimeMilliseconds()
		status = $(if ($_.node.status -eq 'PASS') { 'passed' } else { $(if ($_.node.status -eq 'FAIL') { 'failed' } else { 'skipped' }) })
    }
} ) )

If you’re converting an NUnit XML report, you’ll be mapping test-case object and its following properties: fullname, start-time, end-time, and result (with values Passed, Failed, and Skipped)

For better accuracy of Sealights Test Optimization, please make sure to set your XML report timestamps in milliseconds. For example, you can change the parameter timestampFormat in your XML Reporter settings from yyyy-MM-dd'T'HH:mm:ss'Z' to yyyy-MM-dd'T'HH:mm:ss.SSS'Z', for that purpose.