Why following the call is the hard part of supply-chain security

Most supply-chain security checks the package. Does it exist, or did a model invent the name? Was it registered last week? Is there a CVE against this version? Those checks answer one question well: is this package safe to install?

They cannot answer the next one: is the way my code uses it safe? A clean library can still be where your service breaks, because of what your own code hands it:

• a URL from a request passed to an HTTP client, reaching an internal address
• an uploaded archive handed to a parser that expands it without limit
• a user-supplied path joined into a file operation
• input that becomes a shell command three packages down the chain

Nothing is wrong with the library in any of these. It does what it was asked. So no advisory will ever be filed, and every check that looks at the package will keep saying "safe", correctly and uselessly.

The answer sounds simple: follow the data. Start where input enters your code, follow it through the call into the dependency, and see where it lands. That is what vdb harden does:

We have been building it for Python, measuring it on our own service, on ten popular open-source projects, and on real advisories. This post is about what made it hard: seven problems, each with the case that exposed it, what we changed, and what is still open.

Where the numbers stand

Before the problems, the scoreboard. These are the same measurements before and after the changes described below.

	Before	After
Our own service: call sites flagged	11	3
Our own service: paths reported	39	15
20 real advisories, calls into dependencies: right line, right reason	5	10
19 real advisories, the package's own standard-library calls	0 (not examined)	15
Benchmark written from library docs	21 / 21	21 / 21

The last row is in the table on purpose. It is problem 7.

1. Who controls this argument?

Everything depends on knowing whether a value comes from outside. Statically, you mostly can't tell. The safe assumption is that every function parameter might be attacker-controlled, and that assumption is where most of the noise comes from.

On our own service, a storage client built from environment settings came back as nine dangerous paths, twice, once for each place we create one:

def _minio_client(s):
    return Minio(
        s.minio_endpoint,          # "tainted": s is a parameter
        access_key=s.minio_user,
        secret_key=s.minio_password,
        secure=False,
    )

s is our configuration. No request ever reaches it. But to the analysis it is a parameter like any other.

It is worse for libraries, where every public parameter is outside input by definition: the caller is the outside. When we ran the analysis over ten widely used open-source Python projects, more than half of the reported paths rested on that assumption rather than on a flow we could trace from an actual entry point.

What we changed. Nothing that solves it yet. It is the biggest single source of noise, and it is why two classes of standard-library sink stay switched off (problem 5).

Still open. Telling a request from configuration statically. Entry points (web handlers, CLI arguments, queue consumers) are the obvious anchor, and frameworks each declare them differently.

2. Danger has a direction

httpx validates URLs and header names with regular expressions it wrote itself. Our first rule was simple: outside input reaching a regular expression is a ReDoS risk. So every httpx call we make was flagged, including this one:

# flagged as two regex_eval paths
r = httpx.get(RANKING_URL, timeout=timeout, follow_redirects=True)
#                                  ^^^^^^^
# the argument traced "into a regular expression" is a number of seconds

On our service, 25 of 39 reported paths were this one mistake. On real advisories it did something worse than add noise: three server-side request forgery cases were at the right line but reported only as regex_eval. The real network sink was hidden underneath it.

A regular expression is dangerous when the attacker writes the pattern. When their string is matched against your fixed pattern, that is ordinary validation.

What we changed.

• regex_eval only when the pattern argument is tainted (position 0, or pattern=), not the subject or the replacement.
• re.escape(x) is a sanitizer: its result is a literal for every sink.
• re.compile(CONST).match(x) used to be named after the inner call and judged as compiling x. Sink classification now sees the method actually called.
• Sink names resolve import aliases: from zipfile import ZipFile, import subprocess as sp.

The same lesson applies to every sink, not just regexes. Which argument, which direction, which mode. ZipFile(upload) for reading is a decompression risk. ZipFile(target, "w") for writing is not; jinja2 builds a zip that way. To tell those apart, the client now sends file-open modes from a closed vocabulary (MODE<w>), while every other literal in your code stays masked.

Effect. On our service, 11 flagged call sites became 3 and 39 paths became 15. The three hidden SSRF advisories now read net_request.

3. Right place, wrong reason

The one real problem the first run led us to was a spreadsheet upload. Our parser read the whole sheet into memory before keeping the rows it needed, and a 20 MB .xlsx holds about two million rows. The analysis pointed at the correct line:

wb = openpyxl.load_workbook(io.BytesIO(raw), read_only=True, data_only=True)

and described it as a file-path operation, a regular expression and global state. None of those was the problem. The real category, decompression amplification, did not exist in our list of sinks. A person reading the line found it.

That matters more than it looks. A wrong explanation is not a smaller version of a right one; it sends the reviewer looking for the wrong thing.

What we changed. A new sink class, decompress (zipfile, tarfile, gzip, zlib, bz2, lzma, shutil.unpack_archive), on the archive argument only, and not when the mode writes. The fix it suggests measures the expansion by actually decompressing, never by trusting the sizes the archive declares about itself. The same line now reads decompress and path_op.

Still open. Sink lists are written by people who already know the bug class. The next unknown class will be found the same way this one was: by someone reading a line the analysis pointed at.

4. Every package, at every version

To follow an argument into a dependency, you need to know where that package sends it, for the exact version in the lockfile. We build a summary per package version: for each public API and argument, the dangerous operations it can reach, inside the package or onward into its own dependencies.

Python resists this:

• Calls on objects. client.get(url) needs to know what client is. A summary keyed by function name loses it.
• Wrapper modules. A project's own secure_popen() around subprocess hides the sink one module away.
• Native code. Packages like torch do most of their work in compiled extensions, which a source summary cannot read.
• Taint lost in the middle. A value written to a file and read back, or carried through a loop, often loses its mark before the call that matters.
• Sources that are not parameters. Input read inside a function, from a request object or the environment, rather than passed in.

Between them, these explain most of the advisories we still miss.

When a package on the path has no summary yet, the analysis assumes it passes the argument through, and the report says so: conservative pass-through assumed at 1 unsummarized hop. That keeps a missing summary from reading as a safe path. Of the ten advisories we now find, five go through such an assumed hop: a lead worth reading, not a proof.

There is an operational cost too. Every change to the summariser invalidates every summary, so a version bump means rebuilding summaries for every package version anyone has used, and analyses built on the old ones can come out differently. We learned that when a rebuild re-decided paths for projects analysed days earlier, and the difference looked like new findings. Results now record which engine produced them, and a change of engine is not a change in your code.

5. The standard library is not a dependency

subprocess.run(cmd), eval, pickle.loads are among the most dangerous calls in Python, and none of them is in your lockfile. An analysis built on the dependency graph dropped them, even though the client had already recorded them with their tainted arguments. Every report had to say "we do not examine your own standard-library calls; use Bandit or Semgrep for those."

What we changed. Those calls are now judged on the spot, one hop, with the argument positions that carry the danger: a tainted cwd= or timeout= is not command injection. Covered: commands and os.exec*/spawn*, eval/exec, pickle/marshal/shelve, zlib/gzip/bz2/lzma, archive reads, urlopen, socket connections.

Effect. On real advisories in this class, 0 of 19 became 15 of 19: commands 4/5, eval 4/5, pickle 2/4, decompression 5/5.

Deliberately left off. Path operations and re.*. With every parameter treated as a source (problem 1), they were 140 of 283 findings across 39 real releases, for the lowest-severity class. They come back when sources can be told from configuration. The cost of what is on: 3.7 more call sites per release on average.

6. Paths explode

A tainted value can reach a sink along many routes through a dependency tree, and the routes multiply with every hop. Two kinds of limit are needed, and both turned out to be wrong the first time.

A limit on work. Composition used to run inside the API's request processes with a budget of 20,000 steps. On the twenty dependency advisories, that budget cut eight analyses short before they reached the answer, one of them a release with only two call sites. We raised it twenty-fold and moved composition onto its own workers with a hard time limit. Two more advisories were found; the slowest composition takes 2.5 seconds. Only the largest project in the set, with about 30,000 call sites, still hits it.

A limit on output. Of the ten open-source projects, four finished in the first run. One returned 1,744 paths across 65 call sites, 264 of them from a single line. Nobody reads the 264th. We capped the total at 500, and that was worse: the budget is spent in the order paths are found, so on airflow one line used all 500 and no other call site was reported. On fastapi, 65 call sites became 18, and the note said "stopped after 500 paths", true and silent about the 47 nobody looked at.

What we changed. A cap of 20 paths per call site, so every call site stays in the answer with a bounded list under it, and a total backstop of 2,000 that says, if it fires, which call sites were not examined. A partial answer is labelled partial. Absence is never a safety determination, and neither is a prefix.

7. What counts as "right"?

Our first benchmark was calls we wrote from library documentation, each labelled with the sink it should reach. The analysis scored 21 of 21. That measured almost nothing: the cases were written by the people who wrote the rules.

The product question is different: when an application really misused a dependency and someone filed an advisory, would the analysis have pointed at that line, for that reason? So we built that test. For each advisory, download the newest release still in the affected range, abstract it exactly as the client does, compose against real summaries, and check whether the blamed call is reported with the right sink.

That is the 5 → 10 of 20 in the table, and the 0 → 15 of 19. It also caught the benchmark lying: two of its cases listed regex_eval as the right answer for httpx calls. That was the misclassification from problem 2, written into the answer key. They are corrected, with the reason next to them.

Locations had to be checked by hand too. Two first-pass "hits" were a different call in the same file: a web search instead of a web fetch, and a different send_from_directory. A recall test that counts the file instead of the call flatters itself.

Where that leaves it

Following the call is where supply-chain security has to go, because it is the only way to see risk that no advisory will ever describe. Today, vdb harden is a way to decide where to look: it narrows thousands of call sites to a few worth reading, and on real misuse it points at the right line for the right reason about half the time.

It is not yet a verdict, and the reasons are the problems above. The open ones:

• Telling outside input from configuration, starting from framework entry points.
• Following calls on objects, so client.get(url) reaches the right summary.
• Keeping taint through files, loops and sources that are not parameters.

We will rerun the same three measurements after each change (our own service, the open-source set, the advisories) and publish them either way.

If you have worked on any of these, especially telling a request from configuration statically in Python, we would like to hear how you approached it. The comments are below.