Generating Test Recommendations
After identifying all code changes in a new build and leveraging the Code-To-Test mapping, TIA determines which tests are affected by these changes. It subsequently compiles a list of recommended tests, which is accessible for viewing or consuming via an API and can be integrated back into the test framework for executing only the necessary tests. Alternatively, the list of tests can be incorporated into the customer's CI pipeline.
By utilizing TIA, teams can run only the relevant tests associated with code changes, saving time and enhancing efficiency.
Test Recommendation Principles
TIA follows specific principles to construct a precise list of essential tests that ensures high quality and instills confidence in teams and organizations. These principles guide the creation of the test recommendation list:
Tests impacted by code changes. Meaning tests related to modified methods.
Tests that failed in the last test run on the specific test stage for comprehensive coverage.
Pinned tests, which must consistently run.
New tests not yet mapped to any code section.
Dependent tests, which should run alongside related recommended tests.
Previously recommended tests intended for the previous test recommendation but not executed, often due to failures or blockers.
Tests that do not fall into any of the principles above are excluded from the recommendation list and automatically skipped. This refined test list maintains quality while reducing the test suite.
Test Selection Policies
SeaLights empowers you to customize the recommended test list to perfectly align with your specific testing needs. This flexibility allows you to:
Broaden coverage: Ensure comprehensive testing by incorporating additional tests.
Optimize test volume: Prioritize critical tests to streamline execution time.
SeaLights empowers you to find the perfect balance between thoroughness and efficiency in your testing process, ultimately saving time and resources.
The Impact On Code Coverage
TIA offers the advantage of skipping irrelevant tests for each build and release. Consequently, this selective approach may result in lower overall code coverage. This can be a concern if quality gates within an organization rely on code coverage metrics and require a specific threshold for validation.
Similarly, other systems that depend on full test execution, such as SonarCube, may be affected by TIA's selective test runs. It's essential to recognize this potential impact and communicate it proactively with all relevant stakeholders.
In cases where all tests must run for specific reasons, perhaps due to a critical and sensitive code change necessitating a comprehensive system test, you have the option to temporarily disable TIA, or to choose a full run policy for the next build to ensure all tests are executed without skipping any. This allows you to meet the requirements of sensitive code changes while maintaining comprehensive coverage.
Remember that balancing the advantages of TIA's efficiency with the need for comprehensive testing is crucial. Effective communication with stakeholders is key to managing expectations and ensuring quality throughout the development process.
Last updated