Test Project ID

The Test Project ID is an organizational parameter used to distinguish between different test cycles within the same test stage in SeaLights. It allows you to run multiple independent test cycles using the same test stage name (e.g., "regression" or "smoke") while keeping the test execution history and optimization logic completely separate.

Why It Is Useful

Who Should Use This?

• DevOps Engineers: Onboarding test pipelines for multiple products/teams using shared infrastructure.

• QA Managers: Managing independent automated frameworks that run against the same environment.

• Shared Lab Environments: Organizations where multiple teams run independent test suites but use common stage names (e.g., "Integration").

The Problem This Solves

Without a Test Project ID, if Team A and Team B both start a "regression" cycle in the same lab, SeaLights merges them into a single cycle. This leads to:

• Merged Results: You cannot see which team's specific tests failed in the dashboard.

• Conflicting Optimization: Team A's test history starts influencing Team B's recommendations, reducing the accuracy of Test Optimization for both groups.

The Solution

By assigning a unique Test Project ID (e.g., Team-A and Team-B), SeaLights generates two distinct test cycles in the UI: regression (Team-A) and regression (Team-B).

Detailed Functionality

• Execution Isolation: The Project ID acts as a unique label for the test run. This ensures that the test execution history used for Test Optimization is isolated to that specific ID, providing accurate recommendations for each suite.

• UI Visibility: In the SeaLights Test Optimization pages, the ID appears in parentheses immediately after the test stage name, allowing users to filter and view pass/fail results for their specific suite.

• Coverage Aggregation (Mandatory Rule): It is important to note that code coverage is always aggregated at the Test Stage level. There is currently no way to view or filter coverage for a specific Test Project ID. If multiple Project IDs run under the "Regression" stage, their coverage is combined into one unified "Regression" coverage report.

• Optimization Accuracy: Because the execution history is isolated per ID, SeaLights provides recommendations based only on the tests and code changes relevant to that specific project's testing history.

Best Practices

• Shared Environments: Use a unique Test Project ID whenever multiple teams share the same test lab environment or the same test stage names (e.g., "regression", "smoke").

• Test Organization: Leverage IDs to group related tests by test suite, functional area, or component. This maintains visibility into different logical sets while keeping consolidated coverage metrics at the stage level.

• Naming Conventions: Use descriptive names that clearly identify the product, team, or project (e.g., PaymentService instead of generic names like test1).

• Character Restrictions: Avoid using special characters that might cause issues in URLs or file systems when naming your Test Project ID.

• Consistency: Maintain the same Test Project ID across all runs of a specific suite. This ensures the optimization engine has a stable, consistent history to learn from.

• Persistence: Avoid changing the ID during an active test cycle, as this will result in the creation of a new, separate cycle in the dashboard.

---

Common Use Cases

Use Case 1: Multiple Products in Shared Lab

Scenario: Two products deploy to the same central testing lab

Configuration:

• Product A Pipeline: --test-stage=regression --test-project-id="Product A"

• Product B Pipeline: --test-stage=regression --test-project-id="Product B"

Result: Two separate regression cycles in Test Optimization

Use Case 2: Multiple Teams Testing Same Application

Scenario: Frontend team and Backend team both run "integration" tests

Configuration:

• Frontend Team: --test-stage=integration --test-project-id="Frontend"

• Backend Team: --test-stage=integration --test-project-id="Backend"

Result: Separate integration cycles: integration (Frontend) and integration (Backend)

Use Case 3: Different Product Versions

Scenario: Testing v1.0 and v2.0 simultaneously

Configuration:

• Version 1: --test-stage=regression --test-project-id="v1.0"

• Version 2: --test-stage=regression --test-project-id="v2.0"

Result: Clear separation between version testing cycles

Use Case 4: Splitting Test Suites Within Same Test Stage

Scenario: You want to run API tests and UI tests as separate cycles, but track their combined coverage under "regression"

Configuration:

• API Tests: --test-stage=regression --test-project-id="API-Suite"

• UI Tests: --test-stage=regression --test-project-id="UI-Suite"

Result: Two distinct test cycles in Test Optimization (regression (API-Suite) and regression (UI-Suite)), while coverage for both rolls up to the "regression" test stage

---

FAQ

Can I see a coverage report for just one Test Project ID?

No. In the current version of SeaLights, coverage is always aggregated to the Test Stage. You can see independent results and savings, but coverage is viewed collectively for the stage.

I don't see my Test Project ID in the cycle name.

Verify the parameter was set correctly in your test runner logs and ensure the environment variable was exported before the SeaLights agent initialized.

Does changing the Test Project ID affect my historical optimization?

Yes. Since optimization history is tied to the ID, changing it will start a "fresh" history for the new ID, and the engine will need a few runs to re-optimize effectively.