Start a run with a comment
Post this as a new comment on the pull request:verify, on the same line or on the following lines:
- It must start a line, outside a fenced code block.
- Each comment starts at most one run. Editing a comment that already started a run doesnât start another.
- You need write access to the repository in your Git provider. Otherwise Verify replies: âThis command requires write access to the repository. Only users with write or admin permissions can trigger CodeRabbit to commit or create pull requests.â
- On GitLab, Verify also checks your membership in the GitLab organization. On GitHub, if the repositoryâs configuration sets
chat.allow_non_org_memberstofalse, you must be a member of the GitHub organization. - If a run is already testing the pull requestâs current commit, Verify replies âVerify is already testing the current PR head.â with a View running test link instead of starting another. To run with different instructions, wait for that run to finish or stop it, then comment again.
Automatic runs
When Auto Verify is on for the repository, Verify considers each pull request when it is opened. With Also verify new commits also on, it considers the pull request again whenever new commits are pushed. Automatic runs start alongside CodeRabbitâs review. Automatic runs skip draft pull requests, pull requests opened by bots, closed and merged pull requests, and public repositories on GitHub. Editing a comment or the title, reopening a pull request, or marking a draft ready for review doesnât start one either. To test any of these except a pull request closed without merging, comment@coderabbitai verify.
To turn automatic runs on, see Turn on automatic runs.
Selection rules
Before an automatic run, Verify checks whether the pull requestâs changes qualify. By default it includes:- PRs that change product behavior, including UI, APIs, validation, permissions, data, or background workflows.
- PRs that mix functional and non-functional changes.
- PRs where itâs unclear whether product behavior changes.
- PRs limited to documentation, test maintenance, CI, build tooling, infrastructure, or internal logging changes.
- Refactoring and performance improvements that preserve product behavior.
@coderabbitai verify is never filtered.
When the check excludes a pull request, Verify posts a CodeRabbit Verify comment that says âVerify testing was intentionally skipped under the PR testing criteria. This is not a failed test result.â and names the commits it evaluated, followed by âNo Verify tests were run.â No run is created, so you can still start one with @coderabbitai verify or, on GitHub, with the Run a verifier button in that comment.
The Verify comment
Verify posts one comment headed CodeRabbit Verify and edits it at three points:- When the run starts, with the label Planning scenarios.
- When the scenarios are ready, with the label Scenarios ready and a list of the scenarios Verify will test.
- When the run ends, with the final label and a row per scenario.
- A link to the pull requestâs Verify tab in Change Stack, labeled View progress before scenarios exist, View scenarios once they do, and Review evidence once thereâs evidence.
- A bold status line with the run label, how many scenarios are complete, and counts per result. On GitHub repositories with the comment header, the run label appears in the heading instead, such as CodeRabbit Verify ¡ All scenarios passed, and the bold line starts with the scenario counts.
- The commit the run covers, labeled Target commit until a scenario passes or fails and Tested commit after that (Analyzed commit when scenarios were only generated), and the Total time.
- A table with the columns Result, Scenario, and Run summary. Each row links to the scenarioâs Evidence or Details, and to Watch recording when a recording is available. With more than five scenarios, passed scenarios are collapsed.
- Footer links: View test results (on GitHub), Revise scenarios, and Turn off automatic Verify. Revise scenarios opens the pull requestâs Verify tab in Change Stack, where you can edit or delete scenarios and run the tests again. See Review results in Change Stack.
Run labels
Each scenario row shows a result: Bugs, Blockers, Passed, Failed, Error, or Inconclusive. The CodeRabbit app and the Activity page use other names for some run labels and results. See Comment labels and app outcomes.
Review results in Change Stack
The pull requestâs Verify tab in Change Stack shows the runâs progress, each scenario with its evidence and recording, and earlier runs under Recent Verify runs. There you can edit, revise, add, or delete scenarios, and start a new run with Run tests, which also requires write access to the repository. To act on findings, see Findings on a pull request. If the run youâre viewing tested an older commit, Change Stack says âThese Verify results belong to a different commit.â and offers Run tests for latest commit.New commits and outdated runs
When a new run starts for a newer commit, automatically or with@coderabbitai verify, Verify stops any run still testing an older commit. A run that finishes after you push shows Outdated run and a note that it doesnât cover the current pull request head, and its results apply only to the commit it tested. To test the new commit, turn on Also verify new commits or comment @coderabbitai verify.
With Also verify new commits off, an automatic run that hasnât started testing when you push is cancelled, and the comment says âVerify was cancelled before testing started because a new commit was pushed. Automatic runs on new commits are disabled for this repository.â Comment @coderabbitai verify to test the latest commit.
Later runs on the same pull request reuse its saved scenarios, including the ones you add or revise in Change Stack. If you add instructions after @coderabbitai verify, Verify can add scenarios or refocus saved ones to cover them, and every saved scenario still runs. To drop a scenario, delete it on the pull requestâs Verify tab in Change Stack.
Stop a run
You canât stop a run from the pull request. Open the pull requestâs Verify tab in Change Stack from the Verify comment, select View agent run, and then select Stop in the page header. You need the Coding Agent write permission. Editing or deleting the@coderabbitai verify comment doesnât stop the run. A stopped run ends as superseded, like a run interrupted by a new commit.
Verify doesnât block merges
Verify posts no check run and no commit status, so a failing or unfinished Verify run never blocks a merge. Read the comment before you merge.Whatâs next
Results and bugs
Read outcomes and evidence, and turn bugs into tickets or coding tasks.
User journeys
Test critical flows on a branch, on demand or on a schedule.
Troubleshooting
Fix runs that donât start, get skipped, or end without a result.