Running tests
Running tests with MSTest, NUnit, or XUnit
You have two approaches to capture tests from a compatible framework: wrapping the command or using the Profiler-Initiated-Collector configuration. With the later approach, you don’t have to modify your startup command.
Minimum System Requirements
MSTest Framework 1.4.0 (a.k.a. MSTest v2) and higher
NUnit 3.10.0 and higher
XUnit 2.3.0 and higher
VSTest.Console.exe 17.2 and higher
Wrapper to the test command
When using MSTest, NUnit, or XUnit, you must wrap the execution with the agent and the test listener will monitor the tests. When test execution is completed, test results will be uploaded automatically.
Here is a sample command with the LabID-only setup:
dotnet ./sl-dotnet-agent/SL.DotNet.dll run --tags mytag --instrumentationMode tests --labId <lab Id> --testStage "Functional Tests" --target "c:\tools\testrunner.exe" --workingDir $(pwd) --targetArgs "MyTest.dll --flag1 true --flag2 false"
PIC Integration
Set environment variables required for profiling and PIC. In the below sample configuration, we consider that agent binaries are unpacked to the $SL_AGENT_ROOT
, and the $SL_AGENT_ROOT
variable contains an absolute path (relative path could cause issues). Env variables could be set by sourcing script, by setting env variables for docker, using env file for docker/ docker-compose / k8s, etc…
Environment variables should be set in the same session where test runner will be executed.
Avoid setting PIC or Profiling environment variables system-wide or user-wide. It could lead to unpredictable consequences because the profiler and collector will be started for each .NET process in the system. Also, it will conflict with other dotnet agents running in the same scope.
# enable .net profiler
Env:COR_ENABLE_PROFILING=1
Env:COR_PROFILER="{01CA2C22-DC03-4FF5-8350-59E32A3536BA}"
Env:COR_PROFILER_PATH_32="$SL_AGENT_ROOT\SL.DotNet.ProfilerLib.Windows_x86.dll"
Env:COR_PROFILER_PATH_64="$SL_AGENT_ROOT\SL.DotNet.ProfilerLib.Windows_x64.dll"
Env:CORECLR_ENABLE_PROFILING=1
Env:CORECLR_PROFILER="{01CA2C22-DC03-4FF5-8350-59E32A3536BA}"
Env:CORECLR_PROFILER_PATH_32="$SL_AGENT_ROOT\SL.DotNet.ProfilerLib.Windows_x86.dll"
Env:CORECLR_PROFILER_PATH_64="$SL_AGENT_ROOT\SL.DotNet.ProfilerLib.Windows_x64.dll"
# configure Sealights PIC params.
Env:SL_PROFILER_INITIALIZECOLLECTOR=1
Env:SL_PROFILER_INSTRUMENTATIONMODE="tests"
#Env:SL_COVERAGECOLLECTOR_STARTUPINACTIVITYTIMEOUTSEC=3
#Env:SL_COVERAGECOLLECTOR_IDLETIMEOUTSEC=3
# configure env specific params (token, bsid, labid...)
Env:SL_SESSION_TOKENFILE="$SL_AGENT_ROOT\sltoken.txt"
Env:SL_SESSION_BUILDSESSIONIDFILE="$SL_AGENT_ROOT\buildSessionId"
Env:SL_GENERAL_TESTSTAGE="Functional Tests"
#Env:SL_LABID="integ_poc_app"
You can now execute your tests in the same session, using regular commands with any needed set of params:
dotnet test ./SomeAPITests.dll --params /settings -flags -cookies -filters -kittens
Legacy approach
Starting a test stage
Before running your tests, you must send the SeaLights server a start execution event with the test stage's name.
dotnet ./sl-dotnet-agent/SL.DotNet.dll startExecution --buildSessionIdFile buildSessionId.txt --testStage "Functional Tests"
Running your tests
Now you can run your tests while capturing coverage: SeaLights .NET Core agent - Capturing coverage from the app process
If you’re using a non-supported testing framework, execute your tests with your regular command/script and once the tests are finished, you can upload one or more report files. Supported formats are MSTest (TRX files), xUnit.Net, and NUnit (XML files)
dotnet ./sl-dotnet-agent/SL.DotNet.dll uploadTestReport --buildSessionIdFile buildSessionId.txt --testStage "Functional Tests" --report myReport.trx
Ending the test stage
Once you run your tests and upload the results, you must send the SeaLights server an end execution event with the test stage's name.
dotnet ./sl-dotnet-agent/SL.DotNet.dll endExecution --buildSessionIdFile buildSessionId.txt --testStage "Functional Tests"
Sample scripts
NUnit Tests integration via PIC
# enable .net profiler
$Env:COR_ENABLE_PROFILING=1
$Env:COR_PROFILER="{01CA2C22-DC03-4FF5-8350-59E32A3536BA}"
$Env:COR_PROFILER_PATH_32="C:\Sealights\agent\SL.DotNet.ProfilerLib.Windows_x86.dll"
$Env:COR_PROFILER_PATH_64="C:\Sealights\agent\SL.DotNet.ProfilerLib.Windows_x64.dll"
$Env:CORECLR_ENABLE_PROFILING=1
$Env:CORECLR_PROFILER="{01CA2C22-DC03-4FF5-8350-59E32A3536BA}"
$Env:CORECLR_PROFILER_PATH_32="C:\Sealights\agent\SL.DotNet.ProfilerLib.Windows_x86.dll"
$Env:CORECLR_PROFILER_PATH_64="C:\Sealights\agent\SL.DotNet.ProfilerLib.Windows_x64.dll"
# configure Sealights PIC params.
$Env:SL_PROFILER_INITIALIZECOLLECTOR=1
$Env:SL_PROFILER_INSTRUMENTATIONMODE="tests"
$Env:SL_COVERAGECOLLECTOR_STARTUPINACTIVITYTIMEOUTSEC=3
$Env:SL_COVERAGECOLLECTOR_IDLETIMEOUTSEC=3
# configure env specific params (token, bsid, labid...)
$Env:SL_SESSION_TOKENFILE="C:\Sealights\agent\sltoken.txt"
$Env:SL_LABID="TestLab"
$Env:SL_GENERAL_TESTSTAGE="Functional Tests"
# Logging
$Env:SL_LOGGING_ENABLED="true"
$Env:SL_LOGLEVEL=6
$Env:SL_LOGGING_TOFILE="true"
$Env:SL_LOGDIR="C:\Sealights\logs\Functional"
$Env:SL_LOGGING_FILENAME="C:\Sealights\logs\Functional\pic.log"
#Execute your regular testing command
NUnit Tests integration with command wrapper
# Download the SL DotNetCore Agent
$global:ProgressPreference = "SilentlyContinue"
iwr -OutFile sealights-dotnet-core-agent.zip -UseBasicParsing -Uri https://agents.sealights.co/dotnetcore/latest/sealights-dotnet-agent-windows-self-contained.zip
Expand-Archive .\sealights-dotnet-core-agent.zip -DestinationPath sl-dotnet-agent -Force
Write-Output "[Sealights] .NetCore Agent version is: $(Get-Content .\sl-dotnet-agent\version.txt)"
# Place the token in the agent folder
Copy-Item "C:\Sealights\sltoken.txt" -Destination ".\sl-dotnet-agent\"
#Out-File -InputObject $env:SEALIGHTS_AGENT_TOKEN -NoNewline -Force -FilePath ".\sl-dotnet-agent\sltoken.txt"
$env:SL_LABID = "my_labID"
$env:SL_TESTSTAGE_NAME = "Functional Tests"
# Setup Sealights log level
$Env:SL_LogDir = ".\sl-logs\"
$Env:SL_LogLevel = "6"
$Env:SeaLights_EnableInjectedLibLogFile = "1"
# Pass the NUnit Command to SL Proceess Wrapper using Predefined Agent's Env variables to avoid challenges with quotes, file paths and other variables interpetation
# Prefer using only double quotes " and escape quotes ` when passing a complex command line to the SL Agent (Powershell)
$Env:SL_PROFILER_TARGET="C:\QATools\.nuget\packages\nunit.consolerunner\3.10.0\tools\nunit3-console.exe"
$Env:SL_PROFILER_TARGETARGS="`"C:\AutomationTests\ui-e2e-tests\AutomationTests\Tests\bin\Debug\UI.E2E.Tests.dll`" --inprocess --where cat=E2E_P1 --params=APP_OS=Linux;CollectLogs=`"True`" "
# Execute the Tests using SL Agent as a wrapper to the NUnit process
.\sl-dotnet-agent\SL.DotNet.exe run --tags nunit --instrumentationMode tests --labId $env:SL_LABID --testStage $env:SL_TESTSTAGE_NAME --workingDir $(pwd) --enablePerformanceMonitoring true
Last updated
Was this helpful?