SlopScore for Code

Patterns / Hallucinated APIs

Hallucinated APIs

Besides inventing packages, models invent methods on real ones, usually by borrowing a name from another language or an older version. TypeScript catches some of these; plain JavaScript and Python don't until the line runs.

What it looks like

Math.clamp(x, 0, 1)  // not in JavaScript
await fs.promises.exists(p)  // doesn't exist
fs.readFileAsync(p)  // Bluebird, not Node
os.path.exist(p)  # it's exists()
d.has_key(k)  # removed in Python 3
JSON.tryParse(s)

Why it matters

These read naturally, so reviewers skim past them. The bug shows up as 'x is not a function' in production.

Before and after

Before
if (await fs.promises.exists(path)) {
  return Math.clamp(size, 0, MAX);
}
After
if (existsSync(path)) {
  return Math.min(Math.max(size, 0), MAX);
}

How to fix it

  1. Turn on type checking for JS (checkJs) and Python (mypy/pyright) where you can.
  2. Keep a deny-list of known hallucinations in CI; SlopScore for Code ships one for JS and Python.
  3. Run the code path at least once: a smoke test catches most of these.

When it's fine

Your own utility that happens to share the name (for example a custom Math.clamp polyfill). Silence the line with slopscore-ignore.

Catch this automatically. SlopScore for Code checks every pull request for this pattern (rule nonexistent-api) and 11 others. Score a public PR or add the free GitHub Action.

Related