Skip to main content
| Every Verify run ends with an outcome for the run and for each scenario or user journey in it, backed by screenshots, session recordings, and text evidence. Findings from pull request runs stay with the pull request’s scenarios in Change Stack. Bugs and blockers found by user journey runs are tracked per repository, where you can copy them, hand them to Coding Agent, or turn them into Jira or Linear tickets.

Outcomes

Each run and each scenario or journey has one primary outcome. The app uses these labels: When a run has several kinds of results, the outcome follows this order: Bugs first, then Blocked, then Blockers, then the test verdict. Other findings stay visible in the scenario’s details.

Comment labels and app outcomes

The Verify comment on a pull request, the CodeRabbit app, and the Activity page’s filters name some results differently. The Activity page’s Outcomes filter matches runs that contain at least one scenario with that result. In the comment’s scenario rows, Error corresponds to the app’s Blocked. See Run labels.

Find results

  • Pull request runs: in the Verify comment on the pull request, and on the pull request’s Verify tab in Change Stack. See Test pull requests.
  • One repository: on the repository’s page, Pull Requests > PR runs lists pull request runs, and Journey Testing > Run history lists user journey runs.
  • All repositories: the Activity page at app.coderabbit.ai/verify/activity. It isn’t in the sidebar; the same dashboard also appears below the repository list when you select Verify.
On the Activity page, filter by Repository, by a date range of up to 90 days, by Run type (Pull request or Repository), by Status, and by Outcomes, where blockers are called Testability blockers. The run list shows each run’s Agent minutes, Status, and Outcomes; select View run to open one.

Findings on a pull request

A pull request run’s findings stay with its scenarios on the pull request’s Verify tab in Change Stack; they don’t appear on the repository’s Bugs and Blockers pages. Select a scenario to see its result, its findings, and its evidence. From there you can:
  • Hand findings to a coding agent: select Fix issues with AI (or Investigate with AI) for the run, or Fix with AI (or Investigate with AI) on a scenario. The Handoff to coding agent dialog lets you choose findings and evidence, then Copy markdown or Open in Codex.
  • Change the scenarios: select one scenario and choose Edit manually or Edit with a prompt, select scenarios and choose Revise or Delete, or choose Add scenario, so later runs on the pull request test what you intend.
Pull request findings have no Create ticket action. To file one in your tracker, copy it with Copy markdown in the handoff dialog.

Judging a finding

A finding is what Verify observed, checked against what it expected. Before you act on one, watch the recording and compare the steps and the expected behavior with what your change intends. Verify has no action to dismiss a finding or mark it as not a bug, and findings don’t block merging. If a scenario expects behavior that your change intentionally alters, edit or delete the scenario so later runs on the pull request don’t report it again.

Evidence

A run can attach these kinds of evidence to a scenario, journey, or finding:
  • Screenshots that the tester takes as it works.
  • Session recordings of the browser. When both exist, Watch shortened recording plays a version with idle stretches cut out and Watch full recording plays the whole session. The recording’s timeline marks where assertions passed or failed and where bugs and blockers were found; select a marker to jump to it.
  • Text files, such as each scenario’s execution result and log. Text files up to 2 MiB are previewed in the app; download larger ones.
  • API requests, for pull request runs, on the scenario in Change Stack: the method, path, response status and duration, and the request and response bodies.
If a recording is incomplete, the app says so, for example “Session video is partial.” or “No browser session was started.” followed by the reason. Viewing evidence requires the Coding Agent read permission. Change Stack also checks your access to the repository.
CodeRabbit replaces the environment’s variable and secret values with [REDACTED] in text evidence, logs, and findings. Values shorter than 8 characters aren’t matched, so don’t rely on redaction for short secrets. Screenshots and session recordings aren’t redacted: they show whatever the app displayed. Use test accounts and non-production secrets in the environment, and avoid real personal data in test fixtures.

Bugs and blockers from user journeys

This section and the next cover findings from user journey runs. For pull request runs, see Findings on a pull request. Each repository has Journey Testing > Bugs and Journey Testing > Blockers pages for findings from user journey runs. The counts next to them show open findings on the default branch across all environments. Filter the list by Branch, Environment, and Status (Open or Closed). Its columns are Finding, Journey, Type, Status, and Tickets. Open a finding to see its details:
  • A bug shows Description, Actual behavior, Expected behavior, the Journey, Reproduction steps, and its evidence.
  • A blocker shows what blocked testing, its Impact, the Blocked tasks, Suggested remediation, and its evidence.
Verify closes a finding when a later run of the same journey on the same branch and environment verifies the fix, and reopens it when the problem reproduces. It links a bug to the same bug found in earlier runs; select View occurrence history on a bug to see each time it was observed, with the branch and environment of each observation.

Copy a finding or fix it with Coding Agent

On an open finding:
  • Copy copies the finding as Markdown, with its repository, branch, and environment.
  • Fix with Coding Agent opens a new Coding Agent task, prefilled with the finding, the repository, the tested branch, and a link back to the finding in Verify. Review the task and start it yourself; nothing runs until you do. Evidence files stay in Verify and aren’t attached to the task.
Fix with Coding Agent appears on open findings on the repository’s Bugs and Blockers pages when your organization has Coding Agent and you have the Coding Agent write permission.

Create Jira or Linear tickets

Verify creates tickets in Jira and Linear. It needs a Jira or Linear connection in your organization.
1

Save the repository's ticket defaults

On the repository’s page, select Journey Testing > Settings and open Issue tracker settings. Choose the Issue tracker type, the Connection, the Linear team or Jira project, and the Default status. Optionally set Default assignee (optional), Default priority (optional), Linear project (optional), and Default labels. Then select Save changes.
2

Create a ticket

On a finding, select Create ticket. Verify creates the ticket with your saved defaults and confirms with its key, and the ticket appears in the Tickets column.
If no defaults are saved yet, Create ticket on a finding opens the Issue tracker settings dialog first; select Save and create ticket. Without a Jira or Linear connection, Create ticket is disabled. Once a ticket exists, the button links to it.

Automatic tickets from schedules

A schedule can create tickets for you. In the schedule editor, under Issue tracker, turn on Auto create issues and choose Bugs only or Bugs & blockers. Only new findings create issues. The repository must have saved issue tracker settings first; until it does, the schedule editor links to the issue tracker settings instead of showing Auto create issues. When a scheduled retest closes a finding, Verify moves its linked issue to Done in Jira or to the team’s completed state in Linear, whether the issue was created automatically or by hand.

What’s next

Troubleshooting

Fix blocked, inconclusive, and failed runs and missing evidence.

Coding Agent

Review and deliver a fix started from a Verify finding.

User journeys

Schedule journeys so that fixed bugs get retested automatically.