> For the complete documentation index, see [llms.txt](https://docs.sealights.io/knowledgebase/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.sealights.io/knowledgebase/knowledge-base/sealights-for-abap/how-it-works/day-one-onward-learning-from-execution.md). # Day One Onward: Learning from Execution ### High-Level Description #### Core Capability Day One Onward is the phase that begins after tests execute and report results back to SeaLights. Where Day Zero predicts which tests cover which executables based on structure, Day One Onward observes what each test actually touched at runtime — confirming correct predictions, removing incorrect ones, and building an ever-more-accurate model of the real relationship between tests and SAP objects. Every cycle that reports execution results makes every future cycle more precise. #### Interface Experience and Navigation Day One Onward is not a separate workspace — its effects are visible in two places. During an active cycle, the **Coverage Analysis** section reflects which most-at-risk executables have been verified by executed tests, refreshing approximately every 15 minutes. Across future cycles, improved execution links are expressed through tighter test recommendations and a more accurate optimal set. *** ### Business Value & Key Use Cases * **Proving coverage, not just predicting it** — Structural analysis tells you which tests *should* cover a change. Execution observation tells you which tests *did*. The distinction matters for audit sign-off, release governance, and any situation where "we ran the recommended tests" is not sufficient — only "we confirmed those tests exercised the changed objects" is. * **Progressively eliminating false positives** — Structural links between tests and executables are not always real at runtime. A test may be associated with an executable in the repository, but never actually trigger the changed object during execution. Day One Onward identifies and removes these phantom links, so they stop inflating future recommendation lists. * **Self-improving recommendations** — Teams do not need to maintain or curate test-to-object mappings. The model updates itself from execution data, becoming more accurate with each cycle without any manual effort. #### Impact Metrics In the first cycle, recommendations are built entirely from structural analysis — a strong approximation but not a verified one. By the third or fourth cycle on the same landscape, execution history has confirmed or corrected the most common structural links, producing a recommendation that is grounded in observed behavior. The optimal test set becomes smaller and more reliable over time without any change to the test repository or configuration. *** ### Detailed Functionality #### What Happens After Tests Execute Three things occur in sequence when test results are reported to SeaLights: **1. SeaLights observes what was hit** For each test that executed, SeaLights identifies exactly which underlying ABAP objects were exercised during that run — not which objects the test was predicted to cover, but which objects it demonstrably touched at runtime. **2. Predicted links are confirmed or dropped** Every structural link between a test and an object that was established in Day Zero is evaluated against the execution evidence: * If the test exercised the predicted object → the link is **confirmed** and strengthened in the model * If the test did not exercise the predicted object → the link is **dropped** and will not appear in future recommendations for that test-object pair **3. The model updates for future cycles** Confirmed links become the preferred signal in future cycles — they take precedence over structural predictions for the same test-object pairs. The dependency model is continuously updated as execution data accumulates, so each new cycle starts from a better-informed baseline than the previous one. #### Coverage Reporting During the Cycle As execution results arrive, the cycle's coverage picture updates in near real-time (approximately every 15 minutes). The **Coverage Analysis** section shows which most-at-risk executables have been verified and which remain open — giving release owners a live view of release readiness without waiting for a manual status update. Coverage is reported at three levels, from broadest to most specific: * **App Area** (e.g., Finance, Purchasing) — which functional modules are covered * **Executable** (e.g., a specific transaction or program) — which runnable artifacts have been verified * **Object** — which individual changed ABAP objects have been exercised This hierarchy allows different stakeholders to engage at the level of detail relevant to them: business owners at App Area, QA leads at Executable, and architects or auditors at Object. #### How Execution Is Reported The mechanism for reporting test results back to SeaLights differs by test repository: | Repository | How results reach SeaLights | | -------------------------- | --------------------------------------------------------------------------------------------------------------------- | | **Tricentis Tosca** | Automatically — results are reported in real time with no manual action required | | **All other repositories** | Via the repository's integration mechanism \[Configuration detail pending validation — specific steps per repository] | For non-Tosca repositories, coverage will not update until results are actively reported. Teams using these repositories should ensure their execution pipeline includes the SeaLights reporting step to avoid a gap between tests run and coverage reflected. #### The Compounding Effect Across Cycles Day One Onward does not reset between cycles. Execution links confirmed in one cycle carry forward and inform the next. This means: * **Cycle 1** — recommendations built from structural analysis only * **Cycle 2** — structural links for previously executed tests are confirmed or corrected; recommendations are more precise * **Cycle 3+** — execution history for the most common test-object pairs is well-established; recommendations are grounded in observed behavior, not prediction Teams running regular cycles on a stable landscape typically see the recommendation accuracy stabilize and the optimal test set tighten over the first several cycles. *** ### FAQ **Q: If a test covered an executable in a past cycle but not this one, is the link removed?** A: No. Links are not dropped based on a single cycle's absence. SeaLights uses the accumulated execution history to assess link strength — a confirmed link from past cycles is not immediately discarded because a test didn't run in the current one. \[Configuration detail pending validation — the specific logic for link decay or removal over time should be confirmed.] **Q: Does Day One Onward affect the current cycle's recommendations, or only future ones?** A: It affects the current cycle's **coverage reporting** — as tests execute, the coverage picture updates and previously unverified executables move to verified. It does not retroactively change the current cycle's test recommendations (the Tests to Run list is fixed at cycle creation). Future cycles benefit from improved recommendations based on the current cycle's execution data. **Q: What if we execute tests but never report them back to SeaLights?** A: SeaLights has no visibility into unreported executions. Coverage will not update, the most-at-risk executables will remain unverified in the cycle record, and no execution links will be confirmed or dropped. The Day One Onward learning loop depends entirely on execution results being reported. **Q: Can Day One Onward reduce the Test Gaps list during an active cycle?** A: Yes. If a test executes and observes coverage of a previously gapped object, that object's gap status updates in the Test Gaps tab. Coverage refreshes approximately every 15 minutes, so gap closure is visible quickly after results are reported. *** *Part of the* [*How SeaLights Works*](https://claude.ai/chat/how-00-how-sealights-works.md) *section | SeaLights for ABAP* --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.sealights.io/knowledgebase/knowledge-base/sealights-for-abap/how-it-works/day-one-onward-learning-from-execution.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.