Test Optimization for Manual Tests
Manual tests are fully supported across all SeaLights products. While our general coverage and Test Gap Analysis (TGA) reports provide broad analytics, the Test Impact Analysis (TIA) engine requires precise identification and learning of each individual manual test. This granular understanding is crucial for TIA to accurately build connections between tests and the code parts they are related to, provide effective recommendations, and optimize your test cycles.
Three Ways to Integrate Manual Tests with TIA
SeaLights Chrome Extension: Install and launch the extension for easy test reporting alongside manual test execution.
Manual Tests Wizard within SeaLights Interface: Access the wizard within the Test Optimization page for a UI-based reporting experience similar to the extension.
SeaLights API: Integrate the API into your existing testing platform to report manual tests automatically.
For all options, once the test is executed, enter the result and submit it through the respective interface.
Viewing TIA Test Recommendations
There are three options to consume the TIA recommendations:
On SeaLights UI
Access the Test Optimization page and select the appropriate app, test stage, and build.
You can view recommended tests for both past and new builds.
Download recommended tests as a CSV file for easy execution reference.
Use SeaLights API to get the list of tests to skip.
Use the wizard to view the recommended tests to run for the selected app, test stage, and build.
Step-by-Step Guide for Manual Testing
This section outlines the general steps for performing manual tests with SeaLights, applicable to both the Chrome Extension and the SeaLights Interface.
Key Considerations for Manual Testers
When conducting manual tests with SeaLights, it's crucial to understand a fundamental principle to ensure all data is correctly captured:
One Test at a Time: Manual testers should focus on executing one distinct test at a time. This means completing the steps for one test case before initiating another.
Start and Submit Test Results Promptly: It is imperative to start the test before execution and submit the test results at the end of each manual test run. Submitting the test results after each test ensures that SeaLights recognizes the precise end time of the run, which is vital for Test Impact Analysis (TIA) to accurately build the connections between the tests and the code parts they are related to. TIA requires both the start and end times of a test to function properly. Failing to start or submit the test results will result in incomplete or lost data.
1. Install and Configure (Chrome Extension Only):
Install the "SeaLights Chrome extension" from the Chrome store on all computers used by manual testers.
How to install the extension?
Go to SeaLights Chrome extension page.
Press the "Add to Chrome" button.
Verify the installation by pressing the "Add extension" button.|
SeaLights extension will be added to chrome. Make sure the pop-ups are not blocked.
Pin the extension to the browser bar for easy access and open the extension.
Copy the extension token from the SeaLights UI (found under Settings > Integrations > Browser extension token) and paste it into the extension settings.
Learn how to generate an extension token.
2. Access the Manual Test Interface:
For Chrome Extension: Click the pinned SeaLights extension icon in your browser and press "New Manual Test".
For SeaLights Interface: Click the the "Run Manual Test" button at the top right menu of SeaLights UI (after logging in with your credentials).
3. Define Your App
The default setting is to report the tests according to App / Branch / Build. Make sure the information there is correct for the test being performed.
Lab ID is identified by default.
If you encounter this message, please contact your SeaLights admin.
4. Define Your Test:
Enter the test stage.
Enter your test name:
You can enter a single test name for the current run.
If the test was previously run and reported to SeaLights, an auto-complete will help you find it quickly.
Alternatively, you can upload a list of tests from a CSV file if you have multiple tests to execute in sequence.
The file should be of a .csv format: first column should include a header and then a list of test names, each test name in a separate row (see example file below).
After choosing a file and uploading it, you should see the list of all tests that the file contains.
You can now: Uncheck a test that you do not wish to execute this time.
Enter a new test one by clicking on the plus sign on the top of the page.
Change the tests order by grabbing one test and dragging it.
Once the list of tests in the extension represents the tests as you plan to perform them, click on ‘Close’.
The TIA engine can also recommend tests for you to run, helping to optimize your manual test cycle.
5. Start the Test:
After defining your test(s), initiate the test run through the extension or SeaLights UI, by clicking the "Start" button.
The interface will minimize and indicate that the test has started, showing the test name.
6. Perform Manual Testing:
Execute the steps of your manual test case on your application's UI.
Interact with the application as required by your test scenario. SeaLights will capture the code coverage during this period.
7. Stop and Submit Results:
Once you have completed your manual test, click the "stop" button in the extension or wizard.
You will then be prompted to enter the result of the test (e.g., Pass, Fail).
Finally, click "Submit and Done" to finalize the current test run, or "Submit and start next test" if you are working through a list of tests. This submission is crucial for SeaLights to accurately record the test's end time and associate coverage.
Best Practices for Manual Testing with SeaLights
Isolate Test Runs: As mentioned, focus on one test at a time. Avoid mixing actions from multiple logical test cases within a single test reporting for SeaLights.
Verify Data Capture: After completing a manual test run and submitting its results, verify on the SeaLights dashboard that the results and coverage data have been captured as expected.
Training: Ensure all manual testers are trained on the chosen method (Chrome extension or SeaLights UI) and understand the importance of correctly starting and submitting test results for test stages.
By following these guidelines, your manual testing efforts will seamlessly integrate with SeaLights, providing valuable insights into the quality and coverage of your application.
A CSV file for example:
Last updated
Was this helpful?