---
title: Requests HTTP Pattern
category: patterns
---
# Requests HTTP Pattern

## Rules

- Reuse requests.Session when making multiple calls to the same host to benefit from connection pooling.
- Wrap outbound HTTP calls with retry/backoff logic and respect Retry-After on 429.
- Treat 5xx as transient and retry; surface 4xx as configuration/client errors (do not retry unless 429).
- Raise or wrap non-OK responses into domain ProviderError to make behavior consistent across the codebase.

## Examples

### ai_provider.py - 429 handling with Retry-After

```python
resp = requests.post(url, json=json, headers=headers, timeout=10)
...
if getattr(resp, "status_code", 0) == 429:
    if attempt == retries:
        raise ProviderError(f"Provider returned HTTP {resp.status_code}")
    retry_after = None
    raw = resp.headers.get("Retry-After") if getattr(resp, "headers", None) else None
    if raw:
        try:
            retry_after = int(raw)
        except Exception:
            ...
    if retry_after is not None:
        time.sleep(retry_after)
        continue
```

### api_client.py - Session + raise_for_status

```python
response = self.session.get(
    base_url, params=params, timeout=config.API_TIMEOUT
)
response.raise_for_status()
data = response.json()
```

### pipeline/ai_provider_wrapper.py - Retry/backoff wrapper

```python
def _attempt_batch(chunk_texts, start_index):
    backoff = 0.5
    for attempt in range(1, retries + 1):
        try:
            emb_chunk = _embedder(
                chunk_texts, model=model, batch_size=len(chunk_texts)
            )
            return emb_chunk, None
        except Exception as exc:
            if attempt == retries:
                break
            sleep = backoff * (2 ** (attempt - 1))
            time.sleep(sleep)
            continue
```

## Anti-Patterns

### Bad: Silent exception swallowing

**Problem**: Blindly catching all requests exceptions and returning empty response.

**Remediation**: Map network exceptions to retryable vs terminal (ProviderError) and log details.

### Bad: Using print() for errors

**Problem**: Using print() for network errors instead of structured logging.

**Remediation**: Use `_logger.exception()` instead (see api_client.py needs fixing).