# Quality Gates

SeaLights Quality Gate feature allows you to define thresholds for build quality and ensure that only builds meeting your standards are promoted. Quality Gates act as checkpoints, evaluating builds against specific criteria and providing a clear indication of whether a build is ready for release or requires further attention.

## Quality Gate Parameters

Quality Gates evaluate builds based on three key parameters:

* **Code Changes Coverage (Mandatory):** The percentage of code changes covered by tests. This metric helps ensure that recent modifications are adequately tested. You can specify minimum coverage for all test stages or for specific ones.
* **Overall Code Coverage (Optional):** The percentage of the entire codebase covered by tests. This metric provides a holistic view of test coverage across the application/component. You can specify minimum coverage for all test stages or for specific ones.
* **Failed Tests (Optional):** This parameter offers two options for defining acceptable failure thresholds:
  * **No Failures (Strict):** The Quality Gate fails if *any* tests fail. This is a stringent requirement suitable for projects where zero failures are the target. You can apply this to all test stages or specific ones.
  * **Percentage of Failures (Flexible):** Allows you to define a threshold for acceptable failure rates. The Quality Gate will pass if the percentage of failed tests is below the defined value. This option provides more flexibility and can be tailored to projects with varying risk tolerance. You define the acceptable percentage. You can apply this to all test stages or specific ones.

## Configuring Quality Gates

Quality Gates can be configured at different levels:

* **SeaLights Default:** SeaLights provides default settings for Quality Gates. These are automatically applied as the default for new customer accounts.
* **Customer Default:** Customers can edit the SeaLights default settings to create their own customer default. This customer default can then be applied to all applications.
* **Application-Specific:** It's possible to define different Quality Gates for individual applications. These settings override the customer default. Applications can revert to the customer default or the SeaLights default at any time.

### **Management**

* **Default Settings (SeaLights & Customer)** 
  * Managed from **Settings / Quality Gates** page
* **Application Quality Gate Settings** 
  * Managed from the **Coverage Dashboard**
  * Managed from the **Coverage Report**
  * Managed from the **Settings / Quality Gates** page
  * Managed using [Sealights Public API](https://api-doc.sealights.io/#9f90aef5-d9ec-4d33-9aa0-2632fb8a5d2f).  

## Failed Test Counting Methodology

SeaLights offers two methods for counting failed tests:

* **All Results Method (Current Behavior):** Counts all test runs, even if subsequent runs of the same test pass. This can inflate the perceived number of failures.
* **Latest Result Method (New):** Counts only the latest test run result for each unique test. This provides a more accurate representation of actual failures.

### **Example**

| Test | Run 1  | Run 2  |
| ---- | ------ | ------ |
| A    | Passed | Passed |
| B    | Failed | Failed |
| C    | Failed | Passed |
| D    | Failed | Failed |

* **All Results Method:** 5 failures out of 8 test runs.
* **Latest Result Method:** 2 failures out of 4 tests.

**Note:** Skipped tests are not counted in either method.

#### **Configuration**

This setting is currently managed manually with the help of your SeaLights customer success representative. Changing this setting will likely impact the calculated percentages of failed tests and potentially the Quality Gate status of existing builds.\
Changing this setting affects all applications in the account. 

## Understanding Quality Gate Status

Each build is evaluated against the defined Quality Gate criteria, resulting in one of the following statuses:

* **Passed:** All Quality Gate conditions were met. \
  The build successfully satisfies the predefined quality thresholds.
* **Failed:** The code did not meet one or more quality thresholds. \
  The build failed to meet the minimum requirements defined in the Quality Gate.
* **Missing Data:** The Quality Gate could not be fully evaluated due to one or more of the following:
  * No Reported Tests
  * Missing Test Stage Data
  * No Detected Code Changes
* **Scan Issue:** The build scan encountered errors during the reporting process. \
  This indicates a problem with the scan itself, rather than the code quality. Investigate the scan logs to identify the root cause of the issue.

### Quality Gate Status Display

<figure><img src="https://1255034336-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxSagdStwf6CJ9cI6lA68%2Fuploads%2FiqlccsRADKhygJ8JEEBD%2FScreenshot%202025-02-24%20at%2023.17.23.png?alt=media&#x26;token=5fe909cb-10f1-4035-bbf4-8bca770db83a" alt="" width="563"><figcaption><p>Quality Gate Status</p></figcaption></figure>

The Quality Gate status is displayed on the Coverage Dashboard and Coverage Report. Clicking on the status opens a pop-up window summarizing the build status and Quality Gate definitions, including:

* Overall Quality Gate status.
* Defined thresholds for each parameter.
* Actual values measured for each parameter.
* Clear indication of which criteria were met and which were not.

## Identifying Code Changes

By default, each build is compared to its previous build to identify code modifications. However, you have the flexibility to define a *reference build*, which can be any prior build. In this case, When a reference build is defined, code modifications between the current build and the reference build are considered. For example, you can select the production release build as your reference build to ensure all code changes since the last release are accounted for.

## **Coverage Calculation Specificity**

While code changes are identified based on the comparison to the reference build, the coverage calculation is specific to the tests executed within the *current* build being measured. This means that SeaLights only considers the test coverage achieved by tests running directly against the build under evaluation, and does not include coverage from tests executed in previous builds, even those since the reference build.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.sealights.io/knowledgebase/settings/quality-gates.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.