Repository navigation
Reuse HTTPS connections, add new response fields - #7
Merged
Merged
Conversation
Every request built a fresh Net::HTTP object, so each lookup paid a TCP and TLS handshake (about 660 to 980 ms against api.ipdata.co, versus 220 to 300 ms on a reused connection). A Client now holds one Net::HTTP that is created lazily on the first request, started and kept: HTTP/1.1 keep-alive (Net::HTTP has no HTTP/2, so the saving is the handshake, not multiplexing), open/read/write timeouts from timeout:, max_retries = 1, and a client-side idle timeout from the new keyword keep_alive_timeout: (default 60 s; Net::HTTP's own default is 2 s). While there, use_ssl now follows the base_url scheme instead of being unconditionally true. Requests are serialised with a Mutex, so a client can be shared between threads. The process id is recorded when connecting; a connection inherited across fork is dropped without finish (no TLS close_notify on the socket shared with the parent) and a new one opened. POST /bulk, which Net::HTTP never retries, is retried exactly once on a fresh connection when a reused socket dies before any response byte arrives (EOFError, IOError, Errno::ECONNRESET, Errno::EPIPE, OpenSSL::SSL::SSLError); GETs are retried by Net::HTTP itself. Nothing is retried on an HTTP status. New public method Client#close finishes the connection; the client stays usable and reconnects on demand. get, post and the error mapping are unchanged. The Net::HTTP stand-in in the tests now provides start, started?, finish and the new setters, the tests that counted Net::HTTP.new calls count requests instead, and new tests cover one connection across calls, close, reconnect after a pid change, the POST retry (once, not after response bytes, not on a status) and GET not being retried by the client. README gains a "Connection reuse" section documenting thread and fork safety, close and keep_alive_timeout:. Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The API now returns two more top-level booleans, is_satellite and is_anycast (siblings of is_eu), and each threat.blocklists entry may carry first_seen and last_seen (calendar dates as "YYYY-MM-DD" strings, UTC) and days_seen (an integer). Both flags are included in a response only when they are true, and some feeds (ipdata's own vpn feed, for example) report no dates. The library passes the JSON through unchanged. Because Response resolves dot-notation through method_missing, response.is_satellite would have raised NoMethodError for every non-satellite address, so Response gains explicit is_satellite and is_anycast readers that return false when the key is absent; the raw hash (to_h, [], key?, each, to_json) is untouched. The client does not validate fields= names, so the new names already pass through to the API; a test pins that down. Blocklist dates stay strings (no parsing, no new dependency) and are reached as before through the nested Response objects, with [] giving nil when a feed omits them. Tests add fixtures built from the API samples (a satellite ISP, an anycast address with dated blocklist entries, and a VPN exit whose third entry has no dates) and cover the typed values, false-when-absent, the untouched raw hash, unknown keys passing through, the new names in fields= and select_field, and the same shape per item in bulk responses. The README fields documentation gains the five names and the "present only when true" rule. Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
IPData::Clientused to build a newNet::HTTPfor every request, paying a TCP + TLS handshake each time. Two logical commits:Net::HTTP, created lazily on the first request,started and kept: HTTP/1.1 keep-alive with a 60 s client-side idle timeout (new keywordkeep_alive_timeout:; Net::HTTP's own default is 2 s), open/read/write timeouts fromtimeout:,max_retries = 1. Requests are serialised with aMutex. A connection inherited acrossforkis detected by process id and dropped withoutfinish(no TLS close_notify on the socket shared with the parent) before reconnecting.POST /bulkis retried once on a fresh connection when a reused socket dies before any response byte arrives (EOFError,IOError,Errno::ECONNRESET,Errno::EPIPE,OpenSSL::SSL::SSLError); GETs are retried by Net::HTTP itself; nothing is retried on an HTTP status. New publicClient#close. Fix while there:use_sslnow follows thebase_urlscheme instead of being unconditionally true. Net::HTTP has no HTTP/2, so this is the keep-alive-only variant: the saving is the TLS handshake, not multiplexing.is_satelliteandis_anycast(top-level booleans the API sends only when true) andfirst_seen/last_seen/days_seenonthreat.blocklistsentries (optional; some feeds omit them). The JSON passes through unchanged;Responsegains explicitis_satellite/is_anycastreaders that returnfalsewhen the key is absent (dot-notation would otherwise raiseNoMethodErrorfor every non-satellite address), the raw hash is untouched, and the dates stay strings.No new runtime dependencies (
opensslis stdlib and was already loaded by Net::HTTP for HTTPS). CI already runs on pull requests and is unchanged.lib/ipdata/version.rbis not edited (see Release notes).What changes for users
IPData::Clientreuse a single TLS connection. Create one client per process (or per API key / endpoint), keep it, and reuse it; each new client pays a handshake.keep_alive_timeout:(seconds, default 60): how long an idle connection is kept before the next request opens a new one.timeout:now also sets the write timeout (previously only open and read; Net::HTTP's write default was 60 s).Client#close: finishes the connection; the client stays usable and reconnects on the next request. Call it at shutdown.bulk_lookupis retried once when a reused connection turns out to be dead before any response byte arrived.lookupwas already retried once by Net::HTTP in that case.Response#is_satelliteandResponse#is_anycastreadfalsewhen the API omitted the key;response["is_satellite"],key?andto_hreflect the raw response. Blocklist entries exposefirst_seen,last_seen("YYYY-MM-DD"strings, UTC) anddays_seenwhen present;entry["first_seen"]isnilwhen a feed omits them.base_url:with anhttp://scheme now connects without TLS (it always used TLS before, which could not work for plain-HTTP URLs).lookup,bulk_lookup,Client.eu, caching, error classes, User-Agent format.Release notes
Version: the Release workflow (
workflow_dispatch, bump =minor) bumpslib/ipdata/version.rbfrom 0.1.5 to 0.2.0, commits, tagsv0.2.0and publishes to RubyGems.version.rbis deliberately not edited in this PR.Verification
Local (Ruby 3.3.6, Bundler 4.0.18), run before each push, exactly as CI does:
ruby -Ilib -Itest -e "Dir['test/test_*.rb'].each { |f| require File.expand_path(f) }"main: 79 runs, 210 assertions, 0 failures, 0 errorsgem build ipdata.gemspec: "Successfully built RubyGem" (ipdata-0.1.5) after each commit. A pre-existing gemspec warning about identicalhomepage_uri/source_code_uriis untouched.ruby -w -Ilib -e 'require "ipdata"': no warnings from the library. (-won the test suite reports one pre-existing unused localuriinassert_request, kept as is.)Two
IPData::Client#lookupcalls on one client against api.ipdata.co were served over the same HTTP/1.1 keep-alive connection (Net::HTTP does not negotiate HTTP/2).CI: the CI workflow already has a
pull_requesttrigger onmain(Ruby 3.0, 3.1, 3.2 and 3.3: tests +gem build), so no workflow change was needed. CI run 13 (https://github.com/ipdata/ruby/actions/runs/41620699728) on head 21f8e43 completed with conclusion success. All four matrix jobs passed, each running the test suite andgem build: test (3.0), test (3.1), test (3.2), test (3.3). No follow-up commits were needed.Noticed, not changed
ipdata.gemspecchangelog_uripoints toCHANGELOG.md, which does not exist in the repository (the release notes for this change are above).gem buildwarns thathomepage_uriandsource_code_uriare identical ("Only the first one will be shown on rubygems.org")..github/workflows/ci.ymlstarts with a Ruby magic comment (# frozen_string_literal: true) that means nothing in YAML.release.ymlruns itsreleasejob on every push tomain(github.event_name == 'push'matches branch pushes as well as tags), so a plain merge tomainattempts a RubyGems publish of the current version.lookup(nil, select_field: "city")builds the path//city(double slash) becauseipis nil.build_urireplaces the whole path ofbase_url:, so a base URL with a path prefix loses it.ruby-ipdata/<version> ...format, which differs from theipdata-<language>/<version>shape used by the other ipdata client libraries.test/test_client.rbassert_requestassigns an unused localuri(pre-existing-wwarning).bundle exec rake test, while CI runsruby -Ilib -Itest -e ...; both work in a normal setup.🤖 Generated with Claude Code