motief/.mindmodel/patterns/requests-http.md

title	category
Requests HTTP Pattern	patterns

Requests HTTP Pattern

Rules

• Reuse requests.Session when making multiple calls to the same host to benefit from connection pooling.
• Wrap outbound HTTP calls with retry/backoff logic and respect Retry-After on 429.
• Treat 5xx as transient and retry; surface 4xx as configuration/client errors (do not retry unless 429).
• Raise or wrap non-OK responses into domain ProviderError to make behavior consistent across the codebase.

Examples

ai_provider.py - 429 handling with Retry-After

resp = requests.post(url, json=json, headers=headers, timeout=10)
...
if getattr(resp, "status_code", 0) == 429:
    if attempt == retries:
        raise ProviderError(f"Provider returned HTTP {resp.status_code}")
    retry_after = None
    raw = resp.headers.get("Retry-After") if getattr(resp, "headers", None) else None
    if raw:
        try:
            retry_after = int(raw)
        except Exception:
            ...
    if retry_after is not None:
        time.sleep(retry_after)
        continue

api_client.py - Session + raise_for_status

response = self.session.get(
    base_url, params=params, timeout=config.API_TIMEOUT
)
response.raise_for_status()
data = response.json()

pipeline/ai_provider_wrapper.py - Retry/backoff wrapper

def _attempt_batch(chunk_texts, start_index):
    backoff = 0.5
    for attempt in range(1, retries + 1):
        try:
            emb_chunk = _embedder(
                chunk_texts, model=model, batch_size=len(chunk_texts)
            )
            return emb_chunk, None
        except Exception as exc:
            if attempt == retries:
                break
            sleep = backoff * (2 ** (attempt - 1))
            time.sleep(sleep)
            continue

Anti-Patterns

Bad: Silent exception swallowing

Problem: Blindly catching all requests exceptions and returning empty response.

Remediation: Map network exceptions to retryable vs terminal (ProviderError) and log details.

Bad: Using print() for errors

Problem: Using print() for network errors instead of structured logging.

Remediation: Use _logger.exception() instead (see api_client.py needs fixing).