Running and Managing Tests

Running a test

Click the run icon next to any scenario-NNN.json in the left sidebar’s Tests panel.

AutoReg executes the test and streams output in real-time. (Under the hood, tests run on the Playwright browser-automation engine.) When complete, a run folder is created:

myworkflow/
└── runs/
    └── 2026-06-23T14-32/
        ├── meta.json
        ├── execution.log
        └── artifacts/
            ├── screenshots/
            └── videos/

Click a run in the Runs panel to view:

Step-by-step execution log
Screenshots at each step
Videos (if video recording is enabled)

Running the full suite

In the Plans panel, open e2e-plan.json and click Run All. AutoReg runs every scenario in the plan in sequence.

Stopping a test run

Click Stop in the toolbar (or type "stop tests" in chat). The stop_tests tool aborts the running test process cleanly.

Viewing run artifacts

For each run:

Execution log (execution.log) — line-by-line step output with pass/fail per step
Screenshots — captured at key steps and on failure
Videos — full recording of the browser session

To view: click the run folder in the left sidebar → artifacts load in the main panel.

Editing a scenario manually

Every test is a plain JSON file (tests/scenario-NNN.json). Click the file in the sidebar, edit the JSON in the main panel, and save.

JSON Step DSL — action reference

Action	Required fields	Optional fields
`navigate`	`value` or `selector` (URL)	—
`click`	`selector`	—
`fill`	`selector`, `value`	—
`select`	`selector`, `value` (string or array)	—
`hover`	`selector`	—
`press`	`selector`, `value` (key, e.g. `Enter`)	—
`scroll`	—	`selector` (scrolls element into view; without it, scrolls the page)
`assert`	`selector`	`assertionType` (default `visible`), `expected`
`wait`	one of: `waitFor` (text), `selector`, or `value`/`timeout` (ms pause)	`assertionType` (wait state)
`evaluate`	`value` (JS code)	—
`screenshot`	—	—

assertionType values:

Value	Meaning
`visible` (default)	Element or `expected` text is visible
`hidden` / `not_visible` / `not_exists`	Element is hidden or absent
`exists`	Element is in the DOM (not necessarily visible)
`equals`	Element text/value equals `expected`; a numeric `expected` asserts element count
`contains`	Element text/value contains `expected`

Every step also accepts description (shown in logs), timeout (ms), and screenshot: true (capture after the step).

Example: adding an assertion step

{
  "action": "assert",
  "selector": "[data-testid='success-banner']",
  "assertionType": "visible"
}

Adding, updating, and deleting scenarios via the agent

Type in the chat panel:

"Add a test scenario for password reset"
"Update scenario-005 to use the new modal selector"
"Delete scenario-012"

The agent calls add_test_scenario, update_test_scenario, or delete_test_scenario and updates the .ar manifest and plan automatically.

Generating a single new scenario

To add a test for a feature not in the original spec:

"Generate a test for the export CSV button on the reports page"

The agent calls generate_scenario_from_page, which:

Navigates to the relevant page
Captures a live DOM snapshot
Generates a new scenario-NNN.json with real selectors

Reorganizing the plan

If your test structure needs cleanup:

"Organize the plan into logical sections"

The agent calls auto_organize, which re-groups tests by feature area and rewrites e2e-plan.json.

Capturing a page snapshot

To debug selector issues or inspect what the agent sees:

"Take a snapshot of https://myapp.example.com/dashboard"

The agent calls capture_page_snapshot, which navigates to the URL, takes an accessibility snapshot, and shows it in chat.

Generating Tests Autohealing