Skip to main content

​
What does Critical, Major or Minor mean?

Each finding shows its severity before its title: Critical, Major or Minor. Rapid Review judges every finding with the same rubric:
  • Critical: data loss, an outage, or an exploitable security issue. Data loss means users’ or production data. A tool losing output that a re-run restores doesn’t count.
  • Major: significant impact on normal use.
  • Minor: narrow impact, such as an edge case, a confusing result or a rarely used path.
A finding Rapid Review didn’t judge shows no level. The Rapid Review app calls it “Not judged”.

​
What is a finding’s area?

Each finding has one area:
  • Correctness: the code likely behaves wrongly at run time.
  • Security: the code introduces or weakens a security boundary.
  • Reliability: the code can cause a crash, race, leak, deadlock or poor failure recovery.
  • Compatibility: the code can break a caller, a stored format, a protocol or public behaviour.
  • Test gap: important behaviour lacks targeted tests.

​
What kinds of problems does a review look for?

The reviewer reports defects the change introduces: a concrete input or state that makes the changed code behave wrongly, fail, leak or break a caller. It doesn’t report style, naming, formatting, missing documentation or problems in unchanged code. Alongside the main review, every review also looks specifically for:
  • wrong or missing conditions on a value, such as null, empty or zero values, off-by-one errors, boundaries, wrong comparisons and wrong defaults;
  • text that is built, read, quoted or escaped wrongly for some inputs, such as markup, URLs, paths, headers, shell commands, SQL, regular expressions, format strings and encodings.

​
What is a finding marked Refactor?

Some repositories also get comments labelled Refactor. They point out a function a pull request adds that repeats other functions in the repository, where one refactoring could remove the repetition. A Refactor comment has no severity, and it ranks after every finding that has one.

​
Why were only some findings posted as comments?

A review posts a limited number of inline comments, most severe first. When it found more, the check says how many lower-ranked findings exceeded the comment limit. Refactor comments rank after every finding with a severity. Findings that aren’t tied to a changed line aren’t inline comments, so the limit doesn’t apply to them. They’re listed in the review’s text and in the check.

​
What does “not tied to a changed line” mean?

A finding about a line that GitHub’s diff doesn’t show can’t be an inline comment. Rapid Review lists such findings in the review’s text, each with its explanation, and in the check under “Not tied to a changed line”, with the line the reviewer cited where it gave one. They count as findings: the check’s title then reads, for example, “1 finding not tied to a changed line” rather than “No findings”.

​
What does “Below the severity floor” mean?

A repository, or the account that owns it, can have a severity floor: minor, major or critical. A finding judged below the floor is held back:
  • it isn’t posted as a comment or listed in the review;
  • it isn’t counted in the summary comment or the check’s title;
  • the check lists it under “Below the severity floor”.
Where the check’s title would otherwise say “No findings”, it says how many findings it held back, for example “No findings at or above Major; 2 below the severity floor”. Some findings are posted whatever the floor: a finding without a severity, a Refactor comment, and a finding that’s already on the pull request. Without a floor, findings of every severity are posted. A repository whose floor is minor posts every severity, whatever its account’s floor.

​
Can I set a minimum severity for my repository?

A severity floor, for a repository or for a whole account, is set on Rapid Review’s side. You can’t set one in .rapid-review.yml, and the Rapid Review app has no setting for it. Without a floor, findings of every severity are posted.