Incremental Reporting

Incremental reporting ensures that each run of npm run report only sends test executions that have not already been reported, preventing duplicate submissions to SeaLights.

How It Works

Each test stage has its own last-reported timestamp tracked in the tool's local state. When a report runs:

1. For each test stage, the tool reads the last-reported timestamp

2. It fetches only test executions from qTest that occurred after that timestamp

3. After a successful report, the timestamp is updated to the current time

On the first run for a new test stage (no timestamp yet), the tool falls back to the --days window as the starting point.

First run (stage "Regression"):
  No timestamp → use --days fallback (default: 7 days)
  Report executions from: [now - 7 days] to [now]
  Save timestamp: [now]

Second run (next day):
  Timestamp: [yesterday at time X]
  Report executions from: [yesterday at time X] to [now]
  Save timestamp: [now]

The --days Flag

The --days flag controls two different things depending on whether incremental or full mode is active:

Mode	--days behavior
Incremental (default)	Sets the fallback window for test stages that have never been reported
Full (--full)	Sets the date range for all stages, ignoring saved timestamps

# Incremental: new stages fall back to 14-day window
npm run report -- --days 14

# Full: report all stages for the last 30 days
npm run report -- --full --days 30

When to Use Full Mode

Use --full in these situations:

• Re-reporting: A previous report failed or was incomplete and you need to re-send data

• Configuration change: You changed test stage names or mapping and want to re-report with the new structure

• Data correction: SeaLights data for a period needs to be refreshed

npm run report -- --full --days 30

Full mode re-sends all test executions in the date range, which may result in duplicate data in SeaLights if the same executions have already been processed. Use it intentionally, not as a routine option.

Why This Matters for Test Optimization

The SeaLights Test Optimization model improves over time as more test execution data is reported. Incremental reporting ensures:

• Consistent daily feedback loop — SeaLights receives fresh execution data regularly, allowing the model to refine its recommendations based on recent testing activity

• No noise from duplicates — Duplicate test submissions would distort the coverage data that drives Test Optimization decisions

• Correct "last reported" state — The recommendations wizard uses the last-reported timestamp to validate that the model has recent enough data before applying recommendations

For best results, schedule npm run report to run at least daily. See CI/CD Integration.