Repository navigation
Reuse HTTPS connections, negotiate HTTP/2, add new response fields - #23
Merged
Merged
Conversation
A Client already kept one *http.Client for its lifetime, but the custom DialContext in newHTTPClient() switches off the automatic HTTP/2 setup of net/http, so every request to api.ipdata.co went over HTTP/1.1. Set ForceAttemptHTTP2 so h2 is offered through ALPN (with the HTTP/1.1 fallback net/http provides) and configure HTTP/2 keepalive pings every 30 seconds with a 15 second answer timeout, so idle multiplexed connections are health checked instead of dying silently. The 60 second idle timeout and the 30 second TCP keepalive stay as they were. Add Client.CloseIdleConnections (nil-guarded) so programs can release the pooled connections at shutdown, and document the lifecycle: one Client per process (or per API key or endpoint), safe for concurrent use, reused for every lookup. The bulk endpoint is a non-idempotent POST that net/http only replays when nothing was written to the connection yet. When a pooled connection the server had already dropped is picked, the request bytes go out and the read fails before any response byte arrives; the request is now replayed exactly once in that situation (detected with net/http/httptrace), never after response bytes arrived, on an HTTP status, or after a cancellation or timeout. While there: close the response body of non-200 answers in RawLookup and RawBulkLookup, set Version to 0.8.2 (the const still said 0.8.0 while the last tag is v0.8.1), name github.com/ipdata/go instead of the old theckman/go-ipdata path in the User-Agent, and fix the stale package example in doc.go (import path, two return values, non-empty key). Tests cover the negotiated protocol against an httptest TLS server with HTTP/2 enabled, connection reuse across sequential lookups counted through ConnState over both HTTP/1.1 and HTTP/2, CloseIdleConnections, and the bulk replay against a scripted raw server (retried once, at most once, never after response bytes, never on an HTTP status, never on a fresh connection). Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The API now reports two more top-level booleans next to is_eu, and each element of threat.blocklists may carry the dates on which the address was seen. IP and bulkIP gain IsSatellite and IsAnycast, copied through bulkToIP. The API includes each key only when it is true, so a plain bool reads false whenever the key is absent, which is the documented meaning. Blocklist gains FirstSeen and LastSeen (calendar dates as YYYY-MM-DD strings, UTC, deliberately not parsed into time.Time) and DaysSeen as *int: some feeds, for example ipdata's own vpn feed, report none of the three, and nil keeps "not reported" apart from a zero count. encoding/json ignores unknown keys, so responses carrying further additions keep decoding. The package has no field-name validation for LookupFields, so there is no list to extend; the README field tables document the five new names. Tests decode the three sample responses (satellite ISP, anycast address with dated blocklist entries, VPN exit whose third entry has no dates) through decodeIP, Lookup and BulkLookup, assert the false-when-absent behaviour and the unknown-key tolerance, extend the bulkToIP test, and check the omitempty encoding of a Blocklist without dates. gofmt also normalises the whitespace of Threat, Blocklist and Scores in types.go, which were indented with spaces. Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two logical commits, additive only (no public symbol renamed or removed; every constructor, option and method signature unchanged; the
Errortype users see is unchanged):Clientalready kept one*http.Clientfor its lifetime, but the customDialContextinnewHTTPClient()switches off net/http's automatic HTTP/2 setup, so every request went over HTTP/1.1. The transport now setsForceAttemptHTTP2: true(h2 through ALPN, with net/http's HTTP/1.1 fallback) andHTTP2: &http.HTTP2Config{SendPingTimeout: 30 * time.Second, PingTimeout: 15 * time.Second}(Go 1.24 API;go.modalready requires 1.24.7); the 60 s idle timeout and 30 s TCP keepalive stay. Newfunc (c Client) CloseIdleConnections()(nil-guarded). The bulkPOSTis replayed exactly once when a pooled connection died before any response byte arrived (details below). While there:RawLookup/RawBulkLookupclose the body of non-200 responses,Versionis0.8.2(the const said 0.8.0 while the last tag is v0.8.1), the User-Agent namesgithub.com/ipdata/goinstead oftheckman/go-ipdata, and the staledoc.goexample is fixed (import path, two return values, non-empty key). README gains a "Connection reuse and HTTP/2" section after Basic Lookup.IP.IsSatelliteandIP.IsAnycast(bool, siblings ofIsEU; the API sends each key only when true, so they read false when absent), and onBlocklistthe optionalFirstSeen,LastSeen(string,YYYY-MM-DD, UTC, deliberately kept as strings) andDaysSeen(*int, nil when a feed such as ipdata's own vpn feed does not report it). Bulk responses carry the same shape per item.encoding/jsonignores unknown keys and the package has no strict decoder, so further additions keep decoding. gofmt also normalisedThreat,BlocklistandScoresintypes.go, which were indented with spaces.Tests added: the negotiated protocol against an
httptestTLS server withEnableHTTP2, using the transport fromnewHTTPClient()given the test CA (resp.Proto == "HTTP/2.0", ALPNh2); connection reuse across 11 sequential lookups, bulk lookups and error responses counted throughConnState(1 connection) over both plain HTTP/1.1 and HTTPS+h2, thenCloseIdleConnectionsforcing a second dial; nil-guard and custom-transport cases ofCloseIdleConnections; the bulk replay against a scripted raw keep-alive server (replayed once, at most once, never after response bytes, never on an HTTP status, never on a fresh connection); the three API sample responses decoded throughdecodeIP,LookupandBulkLookupwith the typed values, false-when-absent and unknown-key tolerance (region_type) asserted;bulkToIPand theomitemptyencoding of aBlocklistwithout dates.What changes for users
Clientper process (or per API key or endpoint) and reuse it for every lookup; it is safe for concurrent use from multiple goroutines. Idle HTTP/2 connections are health-checked with a PING every 30 s and closed if unanswered within 15 s; pooled connections are closed after 60 s idle.client.CloseIdleConnections()releases the pooled connections at shutdown (or before a long idle period); the client stays usable afterwards. It is a no-op for a zeroClientor a custom transport without that method.WithHTTPClientkeeps working unchanged. Connection reuse then follows that transport's settings; a transport with a customDialContextorTLSClientConfigmust setForceAttemptHTTP2: trueitself to speak HTTP/2 (documented in the README).RawBulkLookup/BulkLookupnow replay the request exactly once in that case, detected withnet/http/httptrace(connection reused, no response byte received, error is neither a cancellation nor a timeout). Never on an HTTP status, never after response bytes arrived, never on a connection that was not reused, and at most one extra attempt.RawLookup/RawBulkLookupclose the body of non-200 responses instead of leaving it to the garbage collector.IP.IsSatellite,IP.IsAnycast,Blocklist.FirstSeen,Blocklist.LastSeen,Blocklist.DaysSeen; the README field tables document them.LookupFieldshas no field-name validation, so there was no list to extend.go-ipdata/0.8.2 (github.com/ipdata/go) Go-http-client/<goversion> (<os> <arch>).Release notes
Version in the repository:
0.8.2(Versionconst). Tag to push after merge:v0.8.2.Verification
Local, go1.24.7 linux/amd64, all run on the final tree before the push:
gofmt -l .prints nothing.go vet ./...clean.go test -race -covermode atomic -cover -v ./...:ok github.com/ipdata/gowith coverage 88.8% of statements (baseline 88.3%); 25 top-level tests and 54 subtests PASS, 0 FAIL.go run honnef.co/go/tools/cmd/staticcheck@latest ./...(staticcheck v0.8.2): no findings.Test_newHTTPClient_http2,Test_client_connectionReuse,TestClient_CloseIdleConnections,TestClient_BulkLookup_staleConnectionRetry,TestClient_BulkLookup_freshConnectionNotRetried) were additionally run with-race -count=15: ok, no flakes.Three sequential
LookupWithContextcalls on oneClientto api.ipdata.co negotiated HTTP/2 (ALPNh2) and the second and third reused the connection.CI: the workflow already has a
pull_requesttrigger. Run https://github.com/ipdata/go/actions/runs/41621314653 (CI #5, head1ce679c) completed with conclusionsuccess:Test (Go 1.21),Test (Go 1.22),Test (Go 1.23),Test (Go 1.24)andStatic Analysisall succeeded (the 1.21–1.23 jobs switch to the go1.24.7 toolchain, see "Noticed, not changed").Noticed, not changed
LookupFieldsdrops thefieldsquery.LookupFieldsWithContextappends?fields=...to the URL, butnewGetRequestWithContextthen assignsreq.URL.RawQuery = q.Encode()with onlyapi-key, which overwrites it, so the API always returns the full response.Test_client_LookupFieldspasses only because the test handler's full response also satisfies the assertions. Fix: putfieldsinto the sameurl.Values(out of scope for this PR).go.modsaysgo 1.24.7, so every job switches to the 1.24.7 toolchain throughGOTOOLCHAIN=auto(which is also whyhttp.HTTP2Configbuilds in all four jobs). The matrix could be reduced to 1.24 and later.doc.gostill sends issue reports togithub.com/theckman/go-ipdata/issues.ThreatandScoreshave no doc comments, andScoresis missing from the README's "Nested Types".newHTTPClient()sets no overall request timeout (http.Client.TimeoutorResponseHeaderTimeout); callers rely on contexts.errors.Wrapf(err, "failed to read body ...: %s", resp.Status, err)formats the error into the message it also wraps, so it appears twice inError().🤖 Generated with Claude Code