You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
65 lines
2.4 KiB
65 lines
2.4 KiB
name: requests_http
|
|
|
|
rules:
|
|
- Reuse requests.Session when making multiple calls to the same host to benefit from connection pooling.
|
|
- Wrap outbound HTTP calls with retry/backoff logic and respect Retry-After on 429.
|
|
- Treat 5xx as transient and retry; surface 4xx as configuration/client errors (do not retry unless 429).
|
|
- Raise or wrap non-OK responses into domain ProviderError to make behavior consistent across the codebase.
|
|
|
|
examples:
|
|
- path: ai_provider.py
|
|
excerpt: |
|
|
```python
|
|
resp = requests.post(url, json=json, headers=headers, timeout=10)
|
|
...
|
|
if getattr(resp, "status_code", 0) == 429:
|
|
if attempt == retries:
|
|
raise ProviderError(f"Provider returned HTTP {resp.status_code}")
|
|
retry_after = None
|
|
raw = resp.headers.get("Retry-After") if getattr(resp, "headers", None) else None
|
|
if raw:
|
|
try:
|
|
retry_after = int(raw)
|
|
except Exception:
|
|
...
|
|
if retry_after is not None:
|
|
time.sleep(retry_after)
|
|
continue
|
|
```
|
|
note: explicit handling of 429 and Retry-After
|
|
|
|
- path: api_client.py
|
|
excerpt: |
|
|
```python
|
|
response = self.session.get(
|
|
base_url, params=params, timeout=config.API_TIMEOUT
|
|
)
|
|
response.raise_for_status()
|
|
data = response.json()
|
|
```
|
|
note: uses session + raise_for_status() to surface HTTP errors
|
|
|
|
- path: pipeline/ai_provider_wrapper.py
|
|
excerpt: |
|
|
```python
|
|
def _attempt_batch(chunk_texts, start_index):
|
|
backoff = 0.5
|
|
for attempt in range(1, retries + 1):
|
|
try:
|
|
emb_chunk = _embedder(
|
|
chunk_texts, model=model, batch_size=len(chunk_texts)
|
|
)
|
|
return emb_chunk, None
|
|
except Exception as exc:
|
|
if attempt == retries:
|
|
break
|
|
sleep = backoff * (2 ** (attempt - 1))
|
|
time.sleep(sleep)
|
|
continue
|
|
```
|
|
note: wrapper adds retry/backoff and per-item fallback
|
|
|
|
anti_patterns:
|
|
- Bad: Blindly catching all requests exceptions and returning empty response
|
|
remediation: map network exceptions to retryable vs terminal (ProviderError) and log details.
|
|
- Bad: Using print() for network errors instead of structured logging (see api_client.py where print() is used; prefer logging).
|
|
|