Skip to content

Reuse HTTPS connections, negotiate HTTP/2, add new response fields - #4

Merged
jonathan-kosgei merged 2 commits into
mainfrom
http-reuse-and-new-fields
Nov 3, 2026
Merged

jonathan-kosgei merged 2 commits into
mainfrom
http-reuse-and-new-fields

Conversation

@jonathan-kosgei

@jonathan-kosgei jonathan-kosgei commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Two commits, additive only (no public symbol renamed or removed, every existing constructor and call signature keeps working, error types unchanged):

  1. 8b694e8 Reuse HTTPS connections and negotiate HTTP/2. Enables reqwest's http2 feature (ALPN h2 with automatic HTTP/1.1 fallback, never h2-only); extracts build_http_client() with the User-Agent default header and a pool tuned for reuse (60 s idle timeout, HTTP/2 keep-alive PING every 30 s with a 10 s timeout, also while idle, TCP keep-alive every 30 s; no total request timeout); adds IpData::with_client(api_key, base_url, client) and routes with_base_url through it; retries POST /bulk exactly once when a pooled connection dies before any response byte arrives; adds the crate's first test suite (tests/client.rs: wiremock plus a small scripted HTTP/1.1 keep-alive server, which shows three sequential lookups on one IpData, and a lookup on a clone, arriving on a single accepted TCP connection, a connection the server closed while idle being replaced transparently, and the bulk retry happening exactly once, two connections and three requests seen, and never on an HTTP status); adds .github/workflows/ci.yml (fmt, clippy -D warnings, test on pull requests and pushes to main; publish.yml untouched); adds the README section "Connection reuse and HTTP/2"; bumps the version to 0.1.2. rustfmt was also applied to two pre-existing spots (examples/lookup.rs and the lookup_field signature) because the new cargo fmt --check gate fails on main without it.
  2. ff05a6e Add is_satellite, is_anycast and dated blocklist entries. IpInfo::is_satellite and IpInfo::is_anycast (#[serde(default)] bool, false when the key is absent); Blocklist::first_seen and last_seen (Option<String>, YYYY-MM-DD UTC strings) and days_seen (Option<i32>); fixtures built from the three API sample responses; decoding and wiremock tests; README fields documentation. Also lets TimeZone decode explicit nulls through a null_as_default deserializer on its five fields: the API's complete sample response for an anycast address, and a real lookup("8.8.8.8"), fail to decode with the 0.1.1 model (invalid type: null, expected a string).

What changes for users

  • An IpData keeps its connection pool alive between requests and multiplexes over HTTP/2 against api.ipdata.co, so a lookup on a warm pool skips the TCP and TLS handshake. Build one client per process (or per API key or endpoint), keep it and reuse it. Clone is cheap and shares the pool; dropping the last clone releases it (there is no close method). IpData is Send + Sync.
  • IpData::with_client(api_key, base_url, reqwest::Client) injects your own reqwest::Client (a pool shared with the rest of the application, timeouts, proxy, TLS roots). The client is used as given, so the SDK User-Agent and keep-alive tuning are yours to set on that builder. The default client still sets no total request timeout.
  • POST /bulk is retried once if the pooled connection turns out dead before any response byte arrives (reset, EOF, broken pipe); never on an HTTP status, never after response bytes were received. The retry triggers only on reqwest request errors that are neither connect nor timeout errors (no status, no response bytes); reqwest's error API cannot tell a reused connection from a fresh one, so a request that dies on a brand-new connection is retried once as well. GET lookups are not retried by the SDK; hyper itself re-sends only a request it never wrote to a dead reused connection.
  • New fields: IpInfo.is_satellite, IpInfo.is_anycast (bool, false when absent), Blocklist.first_seen, Blocklist.last_seen (Option<String>), Blocklist.days_seen (Option<i32>, matching the crate's other integers such as ThreatScores and IpInfo::status). Bulk responses carry the same shape per item. lookup_fields and lookup_field take is_satellite and is_anycast like any other name (the Rust client passes field names through without validation).
  • TimeZone now decodes a time_zone object whose values are all null, as the API returns for anycast addresses such as 8.8.8.8, instead of failing the whole lookup with a decode error. Field types are unchanged: such fields read "" or false.
  • Code that builds IpInfo or Blocklist with an exhaustive struct literal (without ..Default::default()) needs the new fields; deserialization and field access are unaffected.
  • Dependencies: reqwest gains the http2 feature (adds h2 to the tree). Dev-dependencies only: wiremock 0.6 and tokio's time feature (used by the connection-reuse tests to pace requests so that hyper returns the connection to the pool between them, as happens between real lookups).

Release notes

Version 0.1.2. After merging, create the GitHub release v0.1.2 (tag v0.1.2 on main); publish.yml then runs cargo publish.

ipdata 0.1.2

Connection reuse and HTTP/2
- IpData keeps a persistent HTTPS connection pool and negotiates HTTP/2 through ALPN
  with automatic HTTP/1.1 fallback. Pooled connections idle for 60 s are closed, HTTP/2
  connections are kept alive with a PING every 30 s and TCP keep-alive runs every 30 s.
  Build one client, keep it and reuse it for every lookup; Clone is cheap and shares the
  pool, dropping the last clone releases it.
- New IpData::with_client(api_key, base_url, reqwest::Client) to supply your own reqwest
  client (shared pool, timeouts, proxy, TLS settings).
- POST /bulk is retried exactly once when a pooled connection dies before any response
  byte arrives; never on an HTTP status.

New response fields
- IpInfo::is_satellite and IpInfo::is_anycast (bool). The API sends each key only when
  it is true; the fields read false otherwise.
- Blocklist::first_seen and Blocklist::last_seen (Option<String>, YYYY-MM-DD, UTC) and
  Blocklist::days_seen (Option<i32>); None for feeds that report no dates, such as
  ipdata's own vpn feed.
- TimeZone decodes the all-null time_zone object the API returns for anycast addresses
  instead of failing the lookup.

Tooling
- First test suite (tests/client.rs, tests/types.rs) and a CI workflow running
  cargo fmt --check, cargo clippy --all-targets -- -D warnings and cargo test on pull
  requests and pushes to main.

Verification

Local, rustc/cargo 1.97.0, run before each push:

  • cargo fmt --check: clean. (main fails this check today on examples/lookup.rs and the lookup_field signature; both were reformatted in commit 1 so the new gate passes.)
  • cargo clippy --all-targets -- -D warnings: clean, no warnings; this also compiles examples/lookup.rs.
  • cargo test: tests/client.rs 28 passed, tests/types.rs 8 passed, doc-tests 3 passed, 0 failed. The integration suite was also run 12 times in a row to check the connection-level tests for flakiness: 12 of 12 green.

Requests to api.ipdata.co from the library's client negotiated HTTP/2 and later requests on the same pool reused the connection.

CI: the new CI workflow ran on this pull request, run 41621500327, job test: cargo fmt --check, cargo clippy --all-targets -- -D warnings and cargo test all succeeded on the first attempt (conclusion success).

Noticed, not changed

  • Other typed models still reject an explicit null for String, bool and f64 fields (Asn, Company, Carrier, Currency, Language.name/native, the Threat booleans, ThreatScores, IpInfo.latitude/longitude); #[serde(default)] only covers a missing key. Only TimeZone, which the API returns all-null for anycast addresses, was changed. If the API can send nulls elsewhere, the same null_as_default treatment fits.
  • lookup_field duplicates get_typed (same request and error handling, serde_json::Value result).
  • The response structs are not #[non_exhaustive], so every added field is technically breaking for exhaustive struct literals; the crate has grown fields this way since 0.1.0.
  • publish.yml runs cargo test and cargo publish on release without fmt or clippy; left alone, ci.yml covers pull requests and pushes to main.
  • The library does not validate fields= names (the API does), so there was no list to extend for is_satellite and is_anycast.
  • With default-features = false reqwest's system-proxy feature is off, yet HTTPS_PROXY is still honoured by the default client: hyper-util reads the proxy environment variables unconditionally and the feature only adds macOS and Windows system settings. Noted for anyone debugging proxies; no change needed.

🤖 Generated with Claude Code

Every lookup used to be sent through a reqwest::Client built with its
defaults, so a client instance did pool connections but had no idle
policy, no keep-alive and no HTTP/2 (the crate disabled reqwest's default
features and never re-enabled "http2"). A first request to api.ipdata.co
costs a TCP plus TLS 1.3 handshake (roughly 660 to 980 ms); a request on a
reused connection costs about 220 to 300 ms, so the pool should be tuned
to keep connections alive and multiplex over HTTP/2 where the server
negotiates it.

* Enable reqwest's "http2" feature: HTTP/2 is offered through ALPN with
  automatic HTTP/1.1 fallback (never h2-only).
* Extract build_http_client(), which sets the User-Agent default header
  and tunes the pool: 60 s idle timeout, HTTP/2 keep-alive PING every
  30 s with a 10 s timeout (also while idle) and TCP keep-alive every
  30 s. No total request timeout is added.
* Add IpData::with_client(api_key, base_url, client) to inject a caller-
  supplied reqwest::Client (shared pool, timeouts, proxy, TLS roots) and
  route with_base_url through it. Document that the client is built
  once, that Clone is cheap and shares the pool, that each constructor
  call creates a new pool, and that dropping releases it (no close).
* Retry POST /bulk exactly once when a pooled connection dies before any
  response byte arrives. hyper only re-sends requests it had not yet
  written to a dead reused connection; a written request fails with a
  transport error, which this change retries once on a fresh connection.
  HTTP statuses and body-read failures are never retried.
* Add tests/client.rs: wiremock-backed tests for every public method,
  the api-key query parameter, the User-Agent header, error mapping and
  the injected client, plus a small scripted HTTP/1.1 keep-alive server
  that proves sequential lookups (and clones) reuse one TCP connection,
  that a connection closed while idle is replaced, and that the bulk
  retry happens exactly once and never on an HTTP status.
* Add .github/workflows/ci.yml running cargo fmt --check, cargo clippy
  --all-targets -D warnings and cargo test on pushes to main and on pull
  requests. publish.yml is unchanged.
* README: new "Connection reuse and HTTP/2" section.
* Bump the version to 0.1.2; add wiremock as a dev-dependency and the
  tokio "time" feature used by the tests to pace requests.
* Apply rustfmt to two pre-existing spots (examples/lookup.rs and the
  lookup_field signature) so the new fmt check passes.

Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The API now returns two more top-level booleans next to is_eu and three
details on each threat.blocklists entry. Expose them in the typed models
without changing any existing field.

* IpInfo gains `is_satellite` and `is_anycast` as `#[serde(default)]
  bool`. The API sends each key only when it is true, so the fields read
  false when the key is absent.
* Blocklist gains `first_seen` and `last_seen` (Option<String>, calendar
  dates as YYYY-MM-DD in UTC, deliberately kept as strings) and
  `days_seen` (Option<i32>, the crate's integer style). Some feeds, for
  example ipdata's own vpn feed, omit all three, so they are optional and
  decode as None when absent.
* TimeZone fields now accept an explicit JSON null and decode it as their
  default (empty string or false). The API answers with a time_zone
  object of nulls for addresses without a location, such as the anycast
  8.8.8.8, and the sample anycast response did not decode at all with the
  previous model: serde rejects null for String and bool fields even with
  #[serde(default)], which only covers a missing key. Field types are
  unchanged.
* No decoder uses deny_unknown_fields, so unknown keys keep being
  ignored; a test now pins that at every level.
* Tests: fixtures built from the three API samples (satellite ISP,
  anycast address with dated blocklist entries, VPN exit with an entry
  lacking dates) under tests/fixtures, decoding tests in tests/types.rs,
  and wiremock tests covering lookup, lookup_fields with the new field
  names, lookup_field, threat and bulk responses carrying the new shape.
* README: the Response Types section documents the five new names.

Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jonathan-kosgei
jonathan-kosgei merged commit 55bb5c5 into main Nov 3, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants