PYSEC-2026-2562

## Server-Side Request Forgery (SSRF) in ChatOpenAI Image Token Counting

### Summary The `ChatOpenAI.get_num_tokens_from_messages()` method fetches arbitrary `image_url` values without validation when computing token counts for vision-enabled models. This allows attackers to trigger Server-Side Request Forgery (SSRF) attacks by providing malicious image URLs in user input.

### Severity **Low** - The vulnerability allows SSRF attacks but has limited impact due to: - Responses are not returned to the attacker (blind SSRF) - Default 5-second timeout limits resource exhaustion - Non-image responses fail at PIL image parsing

### Impact An attacker who can control image URLs passed to `get_num_tokens_from_messages()` can: - Trigger HTTP requests from the application server to arbitrary internal or external URLs - Cause the server to access internal network resources (private IPs, cloud metadata endpoints) - Cause minor resource consumption through image downloads (bounded by timeout)

**Note:** This vulnerability occurs during token counting, which may happen outside of model invocation (e.g., in logging, metrics, or token budgeting flows).

### Details The vulnerable code path: 1. `get_num_tokens_from_messages()` processes messages containing `image_url` content blocks 2. For images without `detail: "low"`, it calls `_url_to_size()` to fetch the image and compute token counts 3. `_url_to_size()` performs `httpx.get(image_source)` on any URL without validation 4. Prior to the patch, there was no SSRF protection, size limits, or explicit timeout

**File:** `libs/partners/openai/langchain_openai/chat_models/base.py`

### Patches The vulnerability has been patched in `langchain-openai==1.1.9` (requires `langchain-core==1.2.11`).

The patch adds: 1. **SSRF validation** using `langchain_core._security._ssrf_protection.validate_safe_url()` to block: - Private IP ranges (RFC 1918, loopback, link-local) - Cloud metadata endpoints (169.254.169.254, etc.) - Invalid URL schemes 2. **Explicit size limits** (50 MB maximum, matching OpenAI's payload limit) 3. **Explicit timeout** (5 seconds, same as `httpx.get` default) 4. **Allow disabling image fetching** via `allow_fetching_images=False` parameter

### Workarounds If you cannot upgrade immediately:

1. **Sanitize input:** Validate and filter `image_url` values before passing messages to token counting or model invocation 2. **Use network controls:** Implement egress filtering to prevent outbound requests to private IPs