How to review AI-generated pull requests
AI-written PRs fail in different ways than human ones. People forget edge cases; agents invent packages, stub the hard part, and swallow errors so the demo runs. This is the order we'd review in.
1. Read the deletions first
Look for large deletions next to placeholder comments like // ... existing code .... That combination means code was dropped, not refactored. More on placeholders.
2. Check every new import
Is each new package in the manifest, and does it really exist? Look up anything unfamiliar before installing it. Hallucinated imports and slopsquatting.
3. Search for stubs
TODO, "Not implemented", pass, empty functions. The PR description may say a feature is done while one path throws. TODO stubs.
4. Read every catch block
Empty catches and catches that log a vague string hide the failures you most need to see. Swallowed exceptions.
5. Look for code nobody calls
New helpers that aren't referenced, and blocks pasted twice. Delete or reuse. Unused helpers, duplicated blocks.
6. Check the types and the calls
as any, @ts-ignore, and methods that don't exist (Math.clamp, fs.promises.exists). Type escape hatches, hallucinated APIs.
7. Ask for a split if it's huge
Nobody reviews 900 new lines in one file well. Giant generated changes.
8. Then review the logic
Only after the mechanical problems are gone is it worth a person's time (or an AI reviewer's credits) to check whether the approach is right.
Automate steps 1-7
Steps 1 to 7 are mechanical, so a script can do them. SlopScore for Code runs them on every PR as a GitHub Action, annotates the lines, and fails the check above a threshold. It's free and open source; paste a public PR on the home page to try it.