Skip to main content
| Verify needs two things for each repository: an environment that can start your app, and a registration that tells Verify which environment and branch to use. You set up both in the CodeRabbit app. Verify has no organization-wide switch and no .coderabbit.yaml settings.

Before you begin

  • Your organization uses CodeRabbit Cloud with GitHub Cloud, GitLab.com, or Bitbucket Cloud, and Verify appears in the CodeRabbit app’s sidebar.
  • The repository is connected to CodeRabbit.
  • To add the repository and change its Verify settings, you need the Coding Agent write permission, which Admins and Members have by default. See Permissions.
  • To create and prepare environments, you need the Admin role and an assigned Review seat or an active trial, and Coding Agent must be enabled for your organization, including data retention in the organization settings. See Coding environments.

Prepare an environment

Verify runs your app from a coding environment, the same environments that Coding Agent and runtime verification use. Create one, assign the repository to it, and prepare it until its status is Ready. Verify only offers an environment that is ready and prepared for the repository at its current settings, so prepare it again after you change its setup. Verify uses the environment like this:
  • Setup runs once, at preparation. Each run restores the environment’s prepared snapshot and checks out the code under test. Verify doesn’t run the setup again.
  • The startup script starts your app. Verify uses the environment’s Startup script for the repository to start your app before testing. The tester also gets the environment’s Startup instructions.
  • The app runs inside the sandbox. The startup script must start the app inside the sandbox. Put the starting URL, such as http://localhost:3000, in the test instructions.
  • Variables and secrets are injected. The environment’s variables and secrets are set in the environment of the startup script and the tester, so the app can read them and the tester can use them to sign in.
The tester can read the environment’s variables and secrets during a run. Give the environment test accounts and non-production values only. CodeRabbit redacts these values from text evidence, but screenshots and session recordings show whatever is on screen. See Evidence.

Write the startup script for Verify

Keep dependency installs and builds in the environment’s setup script. Use the startup script for what each run needs: apply migrations, seed test data, and start every service the app needs, such as the database, the API, and the frontend. Before it tests, the tester reads the startup log and checks that the app responds. End the script with a readiness check, so the log shows when the app is up. For example, with your project’s own commands in place of these:
When you open Environments from Verify, a Return to Verify button takes you back, and the Add repository form keeps your selections for 30 minutes.

Add a repository

1

Open the repository list

In the CodeRabbit app, select Verify in the sidebar, then select Add repository.
2

Choose the repository and branch

In Repository, select a connected repository. A repository you’ve already added shows Already added to Test suites. In Default branch, keep the repository default or choose the branch that user journey runs should test.
3

Choose the environment

In Environment, select a ready environment that is prepared for this repository. If none is listed, select Manage environments to prepare one, then Refresh environments.
4

Add it

Select Add repository. The repository’s page opens.
A repository can be added once per organization. Its settings apply to every Verify run on it.

Repository settings

A repository’s Verify settings are on two pages:
  • Test suite settings: on the repository’s page, select Journey Testing > Settings.
  • Verify, for pull request testing: select Review > Verify in the sidebar and choose the repository, or select Pull Requests > Settings on the repository’s page. The Turn off automatic Verify link in a Verify comment opens the same page.
Select Save changes to apply your edits. Settings apply to future runs.

Write test instructions

Test instructions tell every tester in the repository what it can’t learn from the code: where the app starts, which account to use, and what matters most. The field’s hint says: “Include the test account to use, the starting URL, and the behavior to verify. Reference configured credentials instead of pasting secrets here.” For example:
Keep secret values in the environment’s variables and secrets, and refer to them by name.

Turn on automatic runs

Automatic runs are off for a repository you add. To turn them on:
1

Open the Verify settings

Select Review > Verify in the sidebar and choose the repository, or select Pull Requests > Settings on the repository’s page. Check that Enable PR testing is on.
2

Choose when to run

Expand Automatic runs. Turn on Auto Verify to test pull requests when they are opened. To also test new pushes, select Also verify new commits. Turning on Auto Verify alone doesn’t test new commits.
3

Adjust the selection rules

Optionally, add Custom selection rules to change which pull requests qualify. See Selection rules.
4

Confirm and save

In the Enable automatic Verify runs? dialog, select Enable automatic runs, then select Save changes.

Who can set up Verify

What’s next

Test pull requests

Start a run with a comment or automatically, and read the Verify comment.

User journeys

Build a library of critical flows and test them on demand or on a schedule.

Access and billing

Supported platforms, permissions, and what the beta costs.