Reuse HTTPS connections, add new response fields

README.md

@@ -118,9 +118,32 @@ client.lookup("8.8.8.8") # served from cache
 ### Custom Timeout
 
 ```ruby
-client = IPData::Client.new("YOUR_API_KEY", timeout: 10) # 10 seconds
+client = IPData::Client.new("YOUR_API_KEY", timeout: 10) # 10 seconds to connect, read and write
 ```
 
+## Connection reuse
+
+An `IPData::Client` keeps one persistent HTTPS connection (HTTP/1.1 keep-alive) for as long as it lives. Create one client per process (or one per API key or endpoint), keep it, and reuse it for every lookup: each new client pays a fresh TLS handshake. HTTP/2 is not available with Ruby's `Net::HTTP`, so the saving is the TLS handshake, not multiplexing.
+
+A client is safe to share between threads, but it serialises requests on its single connection, so use one client per thread when you need lookups to run in parallel. Create clients after forking (for example in your web server's after-fork hook): a client inherited from the parent process notices the new process id and opens its own connection instead of reusing the parent's socket, which it never closes. Call `close` at shutdown to finish the connection; the client stays usable and reconnects on the next request.
+
+```ruby
+client = IPData::Client.new("YOUR_API_KEY")
+
+client.lookup("8.8.8.8") # opens the connection (TCP + TLS handshake)
+client.lookup("1.1.1.1") # reuses it
+
+at_exit { client.close }
+```
+
+The connection is opened on the first request. An idle connection is kept for `keep_alive_timeout:` seconds (default 60); after that the next request opens a new one. `timeout:` applies to connecting, reading and writing.
+
+```ruby
+client = IPData::Client.new("YOUR_API_KEY", keep_alive_timeout: 30)
+```
+
+If the server has silently dropped an idle connection, `lookup` (a `GET`) is retried once by `Net::HTTP` and `bulk_lookup` (a `POST`) once by the client, in both cases on a fresh connection and only when the failure happened before any response byte arrived. Nothing is retried on an HTTP error status.
+
 ## Response Fields
 
 All responses are wrapped in `IPData::Response` objects that support both dot-notation and hash-style access:
@@ -138,6 +161,8 @@ response.continent_name
 response.latitude
 response.longitude
 response.is_eu
+response.is_satellite # satellite ISP; sent only when true, reads false otherwise
+response.is_anycast   # anycast address; sent only when true, reads false otherwise
 response.postal
 response.calling_code
 response.flag
@@ -182,7 +207,10 @@ response.threat.is_known_attacker
 response.threat.is_known_abuser
 response.threat.is_threat
 response.threat.is_bogon
-response.threat.blocklists
+response.threat.blocklists                  # array of entries: name, site, type and, when the feed reports them:
+response.threat.blocklists.first.first_seen # first day listed, "YYYY-MM-DD" (UTC); absent for some feeds
+response.threat.blocklists.first.last_seen  # last day listed, "YYYY-MM-DD" (UTC); absent for some feeds
+response.threat.blocklists.first.days_seen  # number of days listed (integer); absent for some feeds
 response.threat.scores
 
 # Languages (array)
@@ -198,6 +226,8 @@ response[:country_code]
 response.to_h
 ```
 
+`is_satellite` and `is_anycast` are included in the API response only when they are true. `response.is_satellite` and `response.is_anycast` read `false` when the key is absent, while `response["is_satellite"]`, `response.key?("is_satellite")` and `response.to_h` reflect the raw response (`nil`, `false` and no key). Each `threat.blocklists` entry carries `first_seen` and `last_seen` (calendar dates as `"YYYY-MM-DD"` strings, UTC) and `days_seen` (an integer) when the feed reports them; some feeds, for example ipdata's own vpn feed, omit all three, so read them with `entry["first_seen"]` (`nil` when absent) or check `entry.key?("first_seen")`. The dates are kept as strings; use `Date.iso8601(entry.first_seen)` if you need `Date` objects.
+
 ## Error Handling
 
 ```ruby

lib/ipdata/client.rb

@@ -1,12 +1,25 @@
 # frozen_string_literal: true
 
 require "net/http"
+require "openssl"
 require "json"
 require "uri"
 
 module IPData
   # HTTP client for the ipdata.co API.
   #
+  # A client keeps one persistent HTTPS connection (HTTP/1.1 keep-alive) for
+  # as long as it lives: the connection is opened on the first request and
+  # reused for every later one, so only the first lookup pays the TCP and TLS
+  # handshake. Create one client per process (or per API key or endpoint),
+  # keep it, and call {#close} at shutdown.
+  #
+  # A client is safe to share between threads; requests are serialised on its
+  # single connection, so use one client per thread for parallel lookups.
+  # Create clients after forking: a client inherited from a parent process
+  # notices the new process id and opens its own connection instead of
+  # reusing (or closing) the parent's socket.
+  #
   # @example Basic usage
   #   client = IPData::Client.new("YOUR_API_KEY")
   #   response = client.lookup("8.8.8.8")
@@ -22,17 +35,27 @@ class Client
     BASE_URL = "https://api.ipdata.co"
     EU_BASE_URL = "https://eu-api.ipdata.co"
 
+    # Failures that mean a reused connection died before the server answered.
+    RETRYABLE_ERRORS = [EOFError, IOError, Errno::ECONNRESET, Errno::EPIPE, OpenSSL::SSL::SSLError].freeze
+    private_constant :RETRYABLE_ERRORS
+
     # @param api_key [String] Your ipdata API key
     # @param cache [IPData::Cache, nil] Optional LRU cache instance
-    # @param timeout [Integer] HTTP timeout in seconds (default: 30)
+    # @param timeout [Integer] HTTP open, read and write timeout in seconds (default: 30)
     # @param base_url [String, nil] Override the base URL (default: global endpoint)
-    def initialize(api_key, cache: nil, timeout: 30, base_url: nil)
+    # @param keep_alive_timeout [Integer] Seconds an idle connection is kept before
+    #   the next request opens a new one (default: 60)
+    def initialize(api_key, cache: nil, timeout: 30, base_url: nil, keep_alive_timeout: 60)
       raise ArgumentError, "api_key is required" if api_key.nil? || api_key.empty?
 
       @api_key = api_key
       @cache = cache
       @timeout = timeout
+      @keep_alive_timeout = keep_alive_timeout
       @base_uri = URI.parse(base_url || BASE_URL)
+      @mutex = Mutex.new
+      @connection = nil
+      @connection_pid = nil
     end
 
     # Create a client configured for the EU endpoint.
@@ -91,6 +114,16 @@ def bulk_lookup(ips, fields: nil)
       data.map { |entry| Response.new(entry) }
     end
 
+    # Close the persistent connection, if one is open.
+    #
+    # The client stays usable: the next request opens a new connection.
+    #
+    # @return [nil]
+    def close
+      @mutex.synchronize { discard_connection }
+      nil
+    end
+
     private
 
     def user_agent
@@ -118,7 +151,7 @@ def get(path, params = {})
       request["Accept"] = "application/json"
       request["User-Agent"] = user_agent
 
-      execute(uri, request)
+      execute(request)
     end
 
     def post(path, body, params = {})
@@ -130,22 +163,73 @@ def post(path, body, params = {})
       request["User-Agent"] = user_agent
       request.body = JSON.generate(body)
 
-      execute(uri, request)
+      execute(request)
     end
 
-    def execute(uri, request)
-      http = Net::HTTP.new(uri.host, uri.port)
-      http.use_ssl = true
-      http.open_timeout = @timeout
-      http.read_timeout = @timeout
-
-      response = http.request(request)
+    def execute(request)
+      response = @mutex.synchronize { perform(request) }
 
       unless response.is_a?(Net::HTTPSuccess)
         raise Error.from_response(response.code.to_i, response.body)
       end
 
       JSON.parse(response.body)
     end
+
+    # Send +request+ on the pooled connection. Must be called with @mutex held.
+    #
+    # The server may drop an idle connection without notice. Net::HTTP retries
+    # idempotent requests (GET) on such a failure itself (max_retries); POST
+    # /bulk is not idempotent, so Net::HTTP leaves it alone and it is retried
+    # here instead: exactly once, on a fresh connection, and only when the
+    # failure happened before any response byte arrived. An HTTP error status
+    # is a response and is never retried.
+    def perform(request)
+      retried = false
+      loop do
+        http = connection
+        responded = false
+        begin
+          return http.request(request) { responded = true }
+        rescue *RETRYABLE_ERRORS
+          discard_connection
+          raise if retried || responded || request.method != "POST"
+
+          retried = true
+        end
+      end
+    end
+
+    # The pooled Net::HTTP, opened on first use. A connection inherited from a
+    # parent process is dropped without finish (the socket is shared with the
+    # parent, so no TLS close_notify may be sent on it) and a new one opened.
+    def connection
+      discard_connection if @connection && @connection_pid != Process.pid
+      @connection ||= connect
+    end
+
+    def connect
+      http = Net::HTTP.new(@base_uri.host, @base_uri.port)
+      http.use_ssl = @base_uri.scheme == "https"
+      http.open_timeout = @timeout
+      http.read_timeout = @timeout
+      http.write_timeout = @timeout
+      http.keep_alive_timeout = @keep_alive_timeout
+      http.max_retries = 1
+      http.start
+      @connection_pid = Process.pid
+      http
+    end
+
+    # Forget the pooled connection, finishing it only if this process opened it.
+    def discard_connection
+      http = @connection
+      @connection = nil
+      return unless http && @connection_pid == Process.pid
+
+      http.finish if http.started?
+    rescue IOError, SystemCallError
+      nil
+    end
   end
 end

lib/ipdata/response.rb

@@ -14,6 +14,9 @@ module IPData
   #
   # Nested hashes are recursively wrapped as Response objects.
   # Arrays of hashes (e.g. languages, blocklists) are mapped to Response arrays.
+  #
+  # Keys the API sends only when they are true (+is_satellite+, +is_anycast+)
+  # have explicit readers that return false when the key is absent.
   class Response
     def initialize(data)
       @data = data || {}
@@ -24,6 +27,23 @@ def [](key)
       wrap(@data[key.to_s])
     end
 
+    # Whether the address belongs to a satellite internet provider.
+    #
+    # The API includes the +is_satellite+ key only when it is true, so this
+    # reads +false+ when the key is absent instead of raising NoMethodError.
+    # The raw hash (+to_h+, +[]+, +key?+) is left untouched.
+    def is_satellite
+      @data.fetch("is_satellite", false)
+    end
+
+    # Whether the address is anycast (announced from several locations at once).
+    #
+    # The API includes the +is_anycast+ key only when it is true, so this
+    # reads +false+ when the key is absent. The raw hash is left untouched.
+    def is_anycast
+      @data.fetch("is_anycast", false)
+    end
+
     # Returns the underlying raw Hash.
     def to_h
       @data