Beyond the package name: we checked the code behind our dependencies, and fixed what it found

Most of what we have written here is about names. Does this package exist? Is it a name language models invent? Is there an advisory against this version? Those checks answer one question well: is this package safe to install?

They cannot answer the next one: is the way my code uses it safe? A dependency with a clean record can still be where your service breaks, because of what your own code hands it. That kind of problem has no CVE. It is not in the package; it is in the call.

That second question is what vdb harden is for. It follows the arguments your code passes into a dependency, through that package and the ones it depends on, and flags the ones that reach something dangerous (a parser, a shell, a file path, a network request) without waiting for anyone to file an advisory. To find out what that is worth today, we pointed it at our own service and checked every result by hand.

The short version: it led us to a way to stall our API that no advisory will ever describe, and we fixed it the same day. It also produced a lot of noise, and we counted that too.

Call sites in the service	1,173
Flagged for review	11
Flagged sites that led to a real problem	1, unreported, fixed the same day
Problems found by reading near the flagged lines	2 more
Reported paths whose description was right	0 of 39

So: useful today as a way to decide where to read, not yet as a verdict. Here is what that looked like.

What the name check says

Start with the dependency at the centre of this story. Our own gate, asked about it, gives the right answer:

POST /v1/ai/check-packages
{"packages": ["pkg:pypi/openpyxl@3.1.5"]}

"registry":     {"exists": true, "rationale": "Established package (registered 5895 days ago)."}
"vulnerabilities_total": 0
"agent_action": "PROCEED"
"because":      "no known advisory or slop signal"

openpyxl is a well-maintained library with no known vulnerability in that version. Every name-level check passes, correctly. The problem was never in openpyxl. It was in what we did with it:

The one lead that was real

VDB is 52 Python files with 1,173 call sites, 315 of which take an argument that could plausibly come from outside. Of its 18 dependencies, 17 had a summary: the record of where each of a package's arguments can go. The analysis came back with 11 call sites and 39 paths.

One of those call sites takes input that really does come from outside: the spreadsheet parser behind SBOM scan, where raw is the file a user uploaded.

wb = openpyxl.load_workbook(io.BytesIO(raw), read_only=True, data_only=True)
for ws in wb.worksheets:
    rows = list(ws.iter_rows(values_only=True))   # the whole sheet, in memory
    ...
# then the API keeps the first 20,000 components

The analysis was right about where and wrong about what. It said the upload reached a file-path operation, a regular expression and global state. None of those is the problem. Reading the line was what found it.

The obvious attack on a spreadsheet parser is an XML bomb, since an .xlsx is a zip of XML. That one was already handled: defusedxml is installed and openpyxl uses it. What was not handled is plainer. One repeated row compresses to about eleven bytes, so the 20 MB upload limit holds nearly two million rows. All of them were read into memory, and only then did the API keep the twenty thousand it scans.

Nobody had reported this, and there was no advisory to match, because nothing was wrong with any package. We measured it on the real function, fixed it, and measured again:

Case	Parse time	Extra memory
Before, 20 MB upload	≈ 54 s	≈ 900 MB	estimated from measured runs
After, largest file accepted	3.4 s	≈ 60 MB	measured
After, anything larger	0.1 s	—	refused, measured

On a two-worker API, two of the "before" uploads at once would have left every other request waiting. The fix took two steps, because the first was not enough:

# 1. Measure what the archive really expands to, by decompressing it,
#    not by trusting the sizes its own directory declares.
if _xlsx_expanded_bytes(raw, limit=64 * MB) > 64 * MB:
    raise ParseError("spreadsheet expands to more than 64 MB; "
                     "export the SBOM as CycloneDX or SPDX JSON instead")

# 2. Stream the sheet and stop one row past the component cap.
for row in ws.iter_rows(values_only=True):
    if len(purls) >= XLSX_MAX_ROWS:      # 20,001
        break
    ...

Stopping at the row cap made reading the sheet a fixed cost. But opening the workbook still grew with the file (0.42 s at fifty thousand rows, 1.8 s at two hundred thousand) however few rows were then read. So the archive is now expanded and measured first, by actually decompressing it: the sizes a zip declares are written by whoever made the file, and a check that trusted them would let a file that understates them straight through.

Two more, found by a person, next to the leads

The other two problems were not flagged. The analysis pointed at a network call; the problems were in the lines around it, and a person found them because they had opened the file. We count them as a side effect of reading, not as detections.

• An API key in a URL: fixed. One of the language-model calls behind our slopsquatting collector put the provider's key in the query string, the part of a request that proxy logs, access logs and error messages keep. It now goes in a header.
• Visitor addresses sent in the clear: not fixed yet. The geolocation lookup behind our traffic dashboard sends visitor IP addresses to its provider over plain HTTP, the only scheme that provider's free tier offers. The fix we want is a local database, so the addresses never leave the server. Until it ships, this is the open item from this review.

The other ten leads, and why they were wrong

None of the 39 reported paths described a real risk as stated. By the kind of sink they were said to reach:

Sink	Paths	On review
regex_eval	25	all misclassified
path_op	7	not real path operations; input was configuration or the upload above
net_request	6	plausible kind; input was our own configuration, not a request
global_state	1	the upload above, misdescribed

Two mistakes account for almost all of it.

Library-internal regexes. httpx validates URLs and header names with regular expressions it wrote itself. That fact was attached to every httpx call we make, including this one, where the argument traced "into a regular expression" is a number of seconds:

# flagged as two regex_eval paths
r = httpx.get(RANKING_URL, timeout=timeout, follow_redirects=True)
#                                  ^^^^^^^

A fixed pattern matched against our string is not the dangerous direction for a regex. Teaching the analysis that is the next change to it.

Configuration treated as outside input. The analysis treats every function parameter as something an attacker might control. Our storage client is built from environment settings, and it came back as nine paths, twice, once for each place we create one.

What it does not look at

The analysis follows arguments into dependencies. It does not examine what your own code does directly with the standard library. With a tainted argument, every one of these comes back empty:

def run(cmd, expr, blob):
    os.system(cmd)                       # 0 paths
    subprocess.run(cmd, shell=True)      # 0 paths
    eval(expr)                           # 0 paths
    pickle.loads(blob)                   # 0 paths

That is deliberate. Finding eval(user_input) in your own code is what static analysers such as Bandit and Semgrep do well, and you should run one alongside. What we got wrong was not saying so: a clean result used to read as a clean result for those lines too. Every report now states its scope and names the tool that covers the rest. (VDB has none of those calls. We know that from grep, not from harden.)

Why it was worth doing

Three things in this run would not have happened without it.

• It found a class of problem nothing else we run can see. Every check we had before this was about the package: its name, its age, its advisories. Each of them passes openpyxl, correctly, today and forever. An upload that stalls the API is a fact about one line of our code, and no advisory will ever be written about it, because there is nothing wrong with the package. The only way to find it is to follow our own arguments into the dependency, which is what this does.
• It made the review an afternoon instead of a project. Nobody reads 1,173 call sites. Eleven, we read in full, and one of them was the problem. Reading next to the leads found two more.
• Its mistakes are specific. Almost all of the 39 wrong descriptions come from two rules we can name: regexes inside a library treated as dangerous, and our own configuration treated as outside input. Those are fixable, and we know how we will measure the fix.

What we can and cannot claim

What this run supports:

• A problem with no CVE, in a dependency with a clean record, was found before anyone reported it, starting from a lead the analysis produced.
• The analysis cut 1,173 call sites to 11 worth opening.

What it does not support:

• That the analysis detects such problems on its own. One lead in eleven pointed at a real problem, and even that one described it wrongly.
• That it beats the alternatives. We did not run Bandit, Semgrep or a plain manual review of the same code for comparison, so we cannot say whether those would have found the upload issue too.
• That this generalises. On ten popular open-source repositories, more than half of the reported paths rested on assumptions rather than traced flow. Libraries, where every public parameter is outside input by definition, are the worst case.

Names and code, together

A name check tells you a package is what it claims to be. A code check is meant to tell you what your service does with it, and to do that before there is an advisory, because there is nothing wrong with the package to write one about. On our own service it did that once, with a lot of noise around it. Next: stop flagging fixed-pattern regexes and stop treating our own configuration as outside input, then run this again and publish the numbers either way.

To try it on your own code, see Hardening. Each change above is dated in the changelog.