Repository navigation
[P3][bug] URL guard and retry hardening: raw ValueError escapes, scheme-less base_url no-ops the guard, jitter exceeds the 60s cap #123
Description
Follow-ups from a Python-expert review of #113 (cc7fd9b) and #115 (fcc4edf) on 2026-09-13. All four confirmed by execution against main (CPython 3.14, httpx 0.28.1). None is exploitable; each is a robustness gap rather than a live customer failure.
For context, what I could not break is worth recording: I probed 27 path forms through resolve_api_url — fullwidth solidus, U+2100 (NFKC-expands to a/c), U+FF1A, U+2028, NBSP, soft hyphen, bidi override, lone surrogate, a 100 KB path, %2F%2F, double-encoding, .. traversal, https:/host single-slash, @-in-path, // after ? and #, punycode-shaped segments, bytes. None changed the origin. I also confirmed by execution that httpx 0.28.1 strips Authorization on a cross-origin redirect, so follow_redirects=True does not bypass the guard. The origin pin holds.
1. _origin() leaks a raw ValueError — _url.py:41-46
urlsplit(...).port raises, and _origin is called on base_url on every request, so a bad base URL produces an exception type the docstring says cannot happen (Raises: ValidationError).
from oilpriceapi._url import resolve_api_url
resolve_api_url("https://api.oilpriceapi.com:99999", "/v1/prices")
# ValueError: Port out of range 0-65535
resolve_api_url("https://℀evil.example", "/v1/prices")
# ValueError: netloc '...' contains invalid characters under NFKC normalizationThe second is the unicode case an ASCII probe cannot reach: CPython's urlsplit refuses a non-ASCII netloc whose NFKC normalisation introduces /?#@:.
Fix: wrap _origin and re-raise as ConfigurationError, and validate the base URL once in __init__ rather than on every call.
2. A scheme-less base_url makes the guard a tautology — _url.py:104
resolve_api_url("api.oilpriceapi.com", "/v1/prices") # -> '/v1/prices'
resolve_api_url("", "/v1/prices") # -> '/v1/prices'Both sides of _origin(url) != _origin(base_url) are ("", "", 0), so the comparison always passes and the function returns a relative URL. End-to-end that surfaces as ValueError: unknown url type: 'api.oilpriceapi.com/v1/test' from the stdlib, via httpx.
Fix: require an absolute http/https base — reject at construction with ConfigurationError if urlsplit(base_url).scheme is not in {"http", "https"} or the host is empty.
3. calculate_wait_time is not bound by the new MAX_WAIT_SECONDS — retry.py:246-278
RetryStrategy(max_retries=20).calculate_wait_time(10) # -> 63.42min(2 ** attempt, 60) plus up to 30% jitter exceeds 60. Its own docstring says "Wait time in seconds (capped at 60 seconds)". #115 added bounded_wait and applied it only to the 429/Retry-After path; the 5xx and transport-error paths (client.py:322, 345, 363, 455, 478, 495 and async_client.py:279, 302, 319) call calculate_wait_time raw.
Fix: return cls.bounded_wait(base_wait + jitter_amount, base_wait), or clamp at the call sites. Also correct the docstring's stated examples.
4. ValidationError hard-codes status_code=422 — exceptions.py:332
A purely local guard failure (no request sent) reports .status_code == 422 and .is_client_error is True. Anything that logs or aggregates by status code will record a 422 for a request that never reached the network.
Fix: let _url._reject pass status_code=None, or expose an OilPriceAPIError subclass for client-side refusals that carries no HTTP status.
Minor, while in the area
resources/diesel.py:118 takes location["state_code"] verbatim while the fallback uppercases state, so DieselPrice.state comes back 'ca' on one branch and 'CA' on the other for the same caller input. Uppercase both.
Activity
fix(config): wrap the raw ValueError urlsplit raises on a bad base_url (
fix(config): refuse a scheme-less base_url so the origin guard stays …
fix(url,retry,errors): typed errors, a real 60s cap, no status on a l…
fix(diesel): give the async client the state fallback, and uppercase …
karlwaldman commentedon Sep 13, 2026
All five items verified on
mainat4318dee(2026-09-13) by reading the merged source:_origin()rawValueError→ re-raised asConfigurationError(_url.py:61-62,:131-132) — fix(url,retry,errors): typed errors, a real 60s cap, no status on a local refusal (#123) #134base_url→ rejected at construction byvalidated_base_url(retry.py:118, scheme check:167), wired into bothclient.py:132andasync_client.py:111— fix(config): refuse a scheme-less base_url so the origin guard stays real (#123) #131calculate_wait_time→ now returnsbounded_wait(base_wait + jitter_amount, base_wait), capped atMAX_WAIT_SECONDS(retry.py:421-427) — fix(url,retry,errors): typed errors, a real 60s cap, no status on a local refusal (#123) #134ValidationErrorhard-coded 422 →_rejectpassesstatus_code=None(_url.py) — fix(url,retry,errors): typed errors, a real 60s cap, no status on a local refusal (#123) #134(state_code or state).upper()on both branches (resources/diesel.py:122), and the async client gained the missingstate_codefallback — fix(diesel): give the async client the state fallback, and uppercase both branches (#123) #138Merged, not yet released: none of this is on PyPI until the next python-sdk release.
chore(release): 1.15.0 (#141)
fix(spreads,indicators): raise ValidationError for local refusals, no…
feat(spreads,indicators): typed sync+async Spreads and Indicators res…