cogloom · Free lesson

Your AI suggested `pip install` for a package that doesn't exist. Check before you install.

Result first: a made-up but plausible package name, fastjson-schema-validator-pro-xq7, returns "No matching distribution found" from pip and E404 from npm. A real one, requests, lists its versions. That one-second check catches a hallucinated dependency before install ever runs.

pip index versions <name>                          # lists versions if the package exists on PyPI
curl -s -o /dev/null -w '%{http_code}\n' https://pypi.org/pypi/<name>/json   # 200 = exists, 404 = not on PyPI
npm view <name> version                            # E404 if the package isn't on the npm registry

Why this matters (peer-reviewed, USENIX Security 2025): Spracklen et al. generated 576,000 code samples with 16 code-generating LLMs. They found the average share of hallucinated packages was "at least 5.2% for commercial models and 21.7% for open-source models," with 205,474 unique hallucinated names. The authors' repository explains the risk: an attacker can publish a malicious package under a name that models tend to hallucinate.

So "it exists" is not the same as "it's safe." A hallucinated name may already have been registered by someone. Before you install a package an AI suggested: 1. Check that it exists (the commands above). 2. Check that it is the package you meant: open its project page and source repository from the registry, not from the AI's answer. 3. Prefer packages you or your team already use, and pin versions in your lockfile.

Do this now: add a rule to your agent's instructions: "Before suggesting an install, run the registry check and show its output."

Sources

USENIX Security 2025 paper page, Spracklen et al., "We Have a Package for You!": https://www.usenix.org/conference/usenixsecurity25/presentation/spracklen
arXiv version: https://arxiv.org/abs/2406.10279
Authors' code and data repository (risk explanation): https://github.com/Spracks/PackageHallucination/

Also on dev.to: read this lesson on dev.to. More free lessons: cogloom lessons index.