Supported Framework

Gathering coverage and test information using the SeaLights Python agent is done in a few steps:

See 'Generating an Agent token' for instructions on how to generate a token

Test Duration Requirements for Coverage Calculation Please note that tests must run for at least 1 second for SeaLights to calculate coverage. If your test duration is under 1 second, adding a brief pause will allow SeaLights to complete the coverage sampling process.

Running your tests - Functional Tests

Before running your functional tests you need to set up the backend server to receive the test footprints. See Running Backend Server Using SeaLights Agent.

Once set up you now run your tests with one of the following flags

Running tests with unittest

sl-python unittest --teststage "Unit Tests" {--cov-report /path/to/report.xml} <your args...>

Running tests with pytest

Sealights agent conflicts with the pytest coverage option (--cov), and this option must be removed when using the sl-python command below.

sl-python pytest <Options ...> <pytest args...>

Options:
--buildsessionidfile                   <Text>                     Path to a file to save the build session id in (default: <user.dir>/buildSessionId.txt)
--buildsessionid                       <Text>                     Provide build session id manually, case-sensitive
--tokenfile                            <Text>                     A path to a file where the program can find the token. Case-sensitive
--token                                <Text>                     Token (mandatory. Can also be provided by 'tokenfile' argument). Case-sensitive
--labid                                <Text>                     Lab Id, case-sensitive
--teststage                            <Text>                     {E.g: integration tests, regression, [Unit Tests]}
--cov-report                           <Path>                     {E.g: /path/to/report.xml}
--per-test                             <StrToBool>                {E.g: false, [true]}
--interval                             <Integer in milli second>  {E.g: 20000, [10000], 5000}
-tsd, --test-selection-disable         <Flag> 
-tsri, --test-selection-retry-interval <Integer in second>        {E.g: 1, [5], 10}
-tsrt, --test-selection-retry-timeout  <Integer in second>        {E.g: 30, [60], 120} 

If the pytest-cov module exists in your project with an older version than defined in the sl-python dependencies, coverage collection will fail. This can be avoided by removing pytest-cov from your dependencies, as sl-python will install this automatically.

Running tests with unittest2

sl-python unit2 --teststage "Unit Tests" {--cov-report /path/to/report.xml} <your args...>

Running tests with nose

Sealights agent conflicts with nose coverage options (-with-coverage --cover-package=server for example), and they must be removed when capturing coverage via the sl-python command below

sl-python nose --teststage "Unit Tests" {--cov-report /path/to/report.xml} <your args...>

Running tests with behave

sl-python behave --tokenfile sltoken.txt --teststage "Functional Tests" --labid my_app_env <your args...>