Performance Impact

Adding the Sealights agent to your Application Under Test will have varying levels of impact on application performance. This page offers benchmarking data so that our customers can assess the impact of each integration.

---

AWS Lambda - Sealights Layer

AWS Lambda pricing is based on number of invocations, time of execution and memory used. The following tables detail some approximate performance related statistics to help assess the cost and performance impact of this. Note that AWS Lambda allocates CPU power in proportion to memory so results will vary widely.

The benchmarking data below is sectioned by each of the languages for which Sealights provides a Lambda Solution.

SL Java Lambda Layer

These benchmarks were collected against two differently sized Lambda projects:

• Short-lived execution with fewer methods executed.

• Long-lived execution involving CPU-intensive use of the same methods.

Each Lambda was tested under these scenarios:

1. No Sealights Layer (baseline): Execution of the Lambda function alone

2. Lambda with Sealights layer: Execution of the Lambda function with the Sealights layer applied

3. No Active Test Execution (Standby Mode): Execution of the Lambda function with the Sealights layer applied , but without an active/open test execution.

Table 1. Average execution times

Handler execution time	Metric	No Sealights Layer (baseline)	Lambda with Sealights layer	No Active Test Execution (Standby Mode)
Short-Lived	AVG time [ms]	45.8	81.6	73.2
Long-Lived	AVG time [ms]	4,750.3	5,380.6	4,852.85

Table 2. Init time and memory consumption

Handler execution time	Metric	No Sealights Layer (baseline)	Lambda with Sealights layer	No Active Test Execution (Standby Mode)
Short-Lived	AVG memory usage [MB]	107.2	239.3	198.1
Long-Lived	AVG memory usage [MB]	124.7	246.6	213.5

Recommendations

• When creating the Lambda function, ensure that the allocated memory is appropriate for your use case. As a starting point, consider doubling the memory for Lambdas using the Sealights Java Layer.

• The Lambda Layer does not impact ephemeral storage, so there is no need to increase it when using the Sealights Layer.

• Increase the Lambda timeout to suit your use case, as the initial execution and initialization may take longer.

SL NodeJS Lambda Layer

These benchmarks were collected against two Lambda functions:

• Short-lived execution (ping, with baseline execution time 167ms)

• Long-lived execution (getComments, with baseline execution time 3411.46ms)

Each Lambda was tested under these scenarios:

1. No Sealights Layer (baseline): Execution of the Lambda function alone

2. Lambda with Sealights layer: Execution of the Lambda function with the Sealights layer applied (with an active test execution)

3. No Active Test Execution (Standby Mode): Execution of the Lambda function with the Sealights layer applied , but without an active/open test execution.

Each Lambda was tested with resource allocation of:

• 128MB RAM

• 1024MB RAM

Metrics Collected:

• Absolute Time Impact – the number of ms added to the Lambda function under the different testing scenarios

• Memory Utilization – the memory used by the Lambda function under the different testing scenarios

Table1: Average Execution Times (128MB RAM)

Handler execution time	Metric	No Sealights Layer (baseline)	Lambda with Sealights layer	No Active Test Execution (Standby Mode)
Short-Lived	AVG time [ms]	332	1,473	324
Long-Lived	AVG time [ms]	4,082	6,189	4,829

Table2: Average Memory Consumption (128MB RAM)

Handler execution time	Metric	No Sealights Layer (baseline)	Lambda with Sealights layer	No Active Test Execution (Standby Mode)
Short-Lived	AVG memory usage [MB]	91	113	108
Long-Lived	AVG memory usage [MB]	107	114	113

Table: Average Execution Times (1024MB RAM)

Handler execution time	Metric	No Sealights Layer (baseline)	Lambda with Sealights layer	No Active Test Execution (Standby Mode)
Short-Lived	AVG time [ms]	52	224	86.6
Long-Lived	AVG time [ms]	573	1064	796

Table: Average Memory Consumption (1024MB RAM)

Handler execution time	Metric	No Sealights Layer (baseline)	Lambda with Sealights layer	No Active Test Execution (Standby Mode)
Short-Lived	AVG memory usage [MB]	92	136	117
Long-Lived	AVG memory usage [MB]	100	170	143

Recommendations

• Optimize RAM allocation to at least 512-1024MB for layered executions to minimize performance penalties.

• Avoid using a remote collector outside the VPC, especially when network latency is a concern.

• Use additional layers selectively to balance runtime performance and required functionality.

SL Python Lambda Layer

These benchmarks were collected against two Lambda functions:

• Short-lived execution

• Long-lived execution

Each Lambda was tested under these scenarios:

1. No Sealights Layer (baseline): Execution of the Lambda function alone

2. Lambda with Sealights layer: Execution of the Lambda function with the Sealights layer applied (with an active test execution)

3. No Active Test Execution (Standby Mode): Execution of the Lambda function with the Sealights layer applied , but without an active/open test execution.

Each Lambda was tested with resource allocation of:

• 128MB RAM

• 1024MB RAM

Metrics Collected:

• Absolute Time Impact – the number of ms added to the Lambda function under the different testing scenarios

• Memory Utilization – the memory used by the Lambda function under the different testing scenarios

Table1: Average Execution Times (128MB RAM)

Handler execution time	Metric	No Sealights Layer (baseline)	Lambda with Sealights layer	No Active Test Execution (Standby Mode)
Short-Lived	AVG time [ms]	75	750	518
Long-Lived	AVG time [ms]	5212	6004	5365

Table2: Average Memory Consumption (128MB RAM)

Handler execution time	Metric	No Sealights Layer (baseline)	Lambda with Sealights layer	No Active Test Execution (Standby Mode)
Short-Lived	AVG memory usage [MB]	64	67	66
Long-Lived	AVG memory usage [MB]	65	67	67

Table: Average Execution Times (1024MB RAM)

Handler execution time	Metric	No Sealights Layer (baseline)	Lambda with Sealights layer	No Active Test Execution (Standby Mode)
Short-Lived	AVG time [ms]	57	409	191
Long-Lived	AVG time [ms]	5100100	5394	5184

Table: Average Memory Consumption (1024MB RAM)

Handler execution time	Metric	No Sealights Layer (baseline)	Lambda with Sealights layer	No Active Test Execution (Standby Mode)
Short-Lived	AVG memory usage [MB]	64	67	66
Long-Lived	AVG memory usage [MB]	66	67	66

Recommendations

• Optimize RAM allocation to at least 512-1024MB for layered executions to minimize performance penalties.

• Avoid using a remote collector outside the VPC, especially when network latency is a concern.

• Use additional layers selectively to balance runtime performance and required functionality.