Repository navigation
Reuse HTTPS connections, negotiate HTTP/2, add new response fields - #4
Merged
Merged
Conversation
Every lookup used to be sent through a reqwest::Client built with its defaults, so a client instance did pool connections but had no idle policy, no keep-alive and no HTTP/2 (the crate disabled reqwest's default features and never re-enabled "http2"). A first request to api.ipdata.co costs a TCP plus TLS 1.3 handshake (roughly 660 to 980 ms); a request on a reused connection costs about 220 to 300 ms, so the pool should be tuned to keep connections alive and multiplex over HTTP/2 where the server negotiates it. * Enable reqwest's "http2" feature: HTTP/2 is offered through ALPN with automatic HTTP/1.1 fallback (never h2-only). * Extract build_http_client(), which sets the User-Agent default header and tunes the pool: 60 s idle timeout, HTTP/2 keep-alive PING every 30 s with a 10 s timeout (also while idle) and TCP keep-alive every 30 s. No total request timeout is added. * Add IpData::with_client(api_key, base_url, client) to inject a caller- supplied reqwest::Client (shared pool, timeouts, proxy, TLS roots) and route with_base_url through it. Document that the client is built once, that Clone is cheap and shares the pool, that each constructor call creates a new pool, and that dropping releases it (no close). * Retry POST /bulk exactly once when a pooled connection dies before any response byte arrives. hyper only re-sends requests it had not yet written to a dead reused connection; a written request fails with a transport error, which this change retries once on a fresh connection. HTTP statuses and body-read failures are never retried. * Add tests/client.rs: wiremock-backed tests for every public method, the api-key query parameter, the User-Agent header, error mapping and the injected client, plus a small scripted HTTP/1.1 keep-alive server that proves sequential lookups (and clones) reuse one TCP connection, that a connection closed while idle is replaced, and that the bulk retry happens exactly once and never on an HTTP status. * Add .github/workflows/ci.yml running cargo fmt --check, cargo clippy --all-targets -D warnings and cargo test on pushes to main and on pull requests. publish.yml is unchanged. * README: new "Connection reuse and HTTP/2" section. * Bump the version to 0.1.2; add wiremock as a dev-dependency and the tokio "time" feature used by the tests to pace requests. * Apply rustfmt to two pre-existing spots (examples/lookup.rs and the lookup_field signature) so the new fmt check passes. Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The API now returns two more top-level booleans next to is_eu and three details on each threat.blocklists entry. Expose them in the typed models without changing any existing field. * IpInfo gains `is_satellite` and `is_anycast` as `#[serde(default)] bool`. The API sends each key only when it is true, so the fields read false when the key is absent. * Blocklist gains `first_seen` and `last_seen` (Option<String>, calendar dates as YYYY-MM-DD in UTC, deliberately kept as strings) and `days_seen` (Option<i32>, the crate's integer style). Some feeds, for example ipdata's own vpn feed, omit all three, so they are optional and decode as None when absent. * TimeZone fields now accept an explicit JSON null and decode it as their default (empty string or false). The API answers with a time_zone object of nulls for addresses without a location, such as the anycast 8.8.8.8, and the sample anycast response did not decode at all with the previous model: serde rejects null for String and bool fields even with #[serde(default)], which only covers a missing key. Field types are unchanged. * No decoder uses deny_unknown_fields, so unknown keys keep being ignored; a test now pins that at every level. * Tests: fixtures built from the three API samples (satellite ISP, anycast address with dated blocklist entries, VPN exit with an entry lacking dates) under tests/fixtures, decoding tests in tests/types.rs, and wiremock tests covering lookup, lookup_fields with the new field names, lookup_field, threat and bulk responses carrying the new shape. * README: the Response Types section documents the five new names. Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two commits, additive only (no public symbol renamed or removed, every existing constructor and call signature keeps working, error types unchanged):
8b694e8Reuse HTTPS connections and negotiate HTTP/2. Enables reqwest'shttp2feature (ALPNh2with automatic HTTP/1.1 fallback, never h2-only); extractsbuild_http_client()with theUser-Agentdefault header and a pool tuned for reuse (60 s idle timeout, HTTP/2 keep-alive PING every 30 s with a 10 s timeout, also while idle, TCP keep-alive every 30 s; no total request timeout); addsIpData::with_client(api_key, base_url, client)and routeswith_base_urlthrough it; retriesPOST /bulkexactly once when a pooled connection dies before any response byte arrives; adds the crate's first test suite (tests/client.rs: wiremock plus a small scripted HTTP/1.1 keep-alive server, which shows three sequential lookups on oneIpData, and a lookup on a clone, arriving on a single accepted TCP connection, a connection the server closed while idle being replaced transparently, and the bulk retry happening exactly once, two connections and three requests seen, and never on an HTTP status); adds.github/workflows/ci.yml(fmt, clippy-D warnings, test on pull requests and pushes tomain;publish.ymluntouched); adds the README section "Connection reuse and HTTP/2"; bumps the version to 0.1.2. rustfmt was also applied to two pre-existing spots (examples/lookup.rsand thelookup_fieldsignature) because the newcargo fmt --checkgate fails onmainwithout it.ff05a6eAdd is_satellite, is_anycast and dated blocklist entries.IpInfo::is_satelliteandIpInfo::is_anycast(#[serde(default)] bool, false when the key is absent);Blocklist::first_seenandlast_seen(Option<String>,YYYY-MM-DDUTC strings) anddays_seen(Option<i32>); fixtures built from the three API sample responses; decoding and wiremock tests; README fields documentation. Also letsTimeZonedecode explicitnulls through anull_as_defaultdeserializer on its five fields: the API's complete sample response for an anycast address, and a reallookup("8.8.8.8"), fail to decode with the 0.1.1 model (invalid type: null, expected a string).What changes for users
IpDatakeeps its connection pool alive between requests and multiplexes over HTTP/2 against api.ipdata.co, so a lookup on a warm pool skips the TCP and TLS handshake. Build one client per process (or per API key or endpoint), keep it and reuse it.Cloneis cheap and shares the pool; dropping the last clone releases it (there is no close method).IpDataisSend + Sync.IpData::with_client(api_key, base_url, reqwest::Client)injects your ownreqwest::Client(a pool shared with the rest of the application, timeouts, proxy, TLS roots). The client is used as given, so the SDKUser-Agentand keep-alive tuning are yours to set on that builder. The default client still sets no total request timeout.POST /bulkis retried once if the pooled connection turns out dead before any response byte arrives (reset, EOF, broken pipe); never on an HTTP status, never after response bytes were received. The retry triggers only on reqwest request errors that are neither connect nor timeout errors (no status, no response bytes); reqwest's error API cannot tell a reused connection from a fresh one, so a request that dies on a brand-new connection is retried once as well. GET lookups are not retried by the SDK; hyper itself re-sends only a request it never wrote to a dead reused connection.IpInfo.is_satellite,IpInfo.is_anycast(bool, false when absent),Blocklist.first_seen,Blocklist.last_seen(Option<String>),Blocklist.days_seen(Option<i32>, matching the crate's other integers such asThreatScoresandIpInfo::status). Bulk responses carry the same shape per item.lookup_fieldsandlookup_fieldtakeis_satelliteandis_anycastlike any other name (the Rust client passes field names through without validation).TimeZonenow decodes atime_zoneobject whose values are allnull, as the API returns for anycast addresses such as 8.8.8.8, instead of failing the whole lookup with a decode error. Field types are unchanged: such fields read""orfalse.IpInfoorBlocklistwith an exhaustive struct literal (without..Default::default()) needs the new fields; deserialization and field access are unaffected.http2feature (addsh2to the tree). Dev-dependencies only:wiremock 0.6and tokio'stimefeature (used by the connection-reuse tests to pace requests so that hyper returns the connection to the pool between them, as happens between real lookups).Release notes
Version
0.1.2. After merging, create the GitHub releasev0.1.2(tagv0.1.2onmain);publish.ymlthen runscargo publish.Verification
Local, rustc/cargo 1.97.0, run before each push:
cargo fmt --check: clean. (mainfails this check today onexamples/lookup.rsand thelookup_fieldsignature; both were reformatted in commit 1 so the new gate passes.)cargo clippy --all-targets -- -D warnings: clean, no warnings; this also compilesexamples/lookup.rs.cargo test:tests/client.rs28 passed,tests/types.rs8 passed, doc-tests 3 passed, 0 failed. The integration suite was also run 12 times in a row to check the connection-level tests for flakiness: 12 of 12 green.Requests to api.ipdata.co from the library's client negotiated HTTP/2 and later requests on the same pool reused the connection.
CI: the new
CIworkflow ran on this pull request, run 41621500327, jobtest:cargo fmt --check,cargo clippy --all-targets -- -D warningsandcargo testall succeeded on the first attempt (conclusionsuccess).Noticed, not changed
nullforString,boolandf64fields (Asn,Company,Carrier,Currency,Language.name/native, theThreatbooleans,ThreatScores,IpInfo.latitude/longitude);#[serde(default)]only covers a missing key. OnlyTimeZone, which the API returns all-null for anycast addresses, was changed. If the API can send nulls elsewhere, the samenull_as_defaulttreatment fits.lookup_fieldduplicatesget_typed(same request and error handling,serde_json::Valueresult).#[non_exhaustive], so every added field is technically breaking for exhaustive struct literals; the crate has grown fields this way since 0.1.0.publish.ymlrunscargo testandcargo publishon release without fmt or clippy; left alone,ci.ymlcovers pull requests and pushes tomain.fields=names (the API does), so there was no list to extend foris_satelliteandis_anycast.default-features = falsereqwest'ssystem-proxyfeature is off, yetHTTPS_PROXYis still honoured by the default client: hyper-util reads the proxy environment variables unconditionally and the feature only adds macOS and Windows system settings. Noted for anyone debugging proxies; no change needed.🤖 Generated with Claude Code