Reuse HTTPS connections, add new response fields

.github/workflows/publish.yml

@@ -2,16 +2,28 @@ name: Publish to Hex.pm
 
 on:
   push:
+    branches:
+      - main
     tags:
       - 'v*'
+  pull_request:
 
 jobs:
   test:
     name: Test
-    runs-on: ubuntu-latest
+    runs-on: ${{ matrix.os }}
     strategy:
       matrix:
-        otp: ['25', '26', '27']
+        # ubuntu-latest resolves to Ubuntu 26.04, for which no OTP 25 builds
+        # are published (erlef/setup-beam supports OTP 26+ there and 24.3+ on
+        # ubuntu-24.04), so the OTP 25 job runs on ubuntu-24.04.
+        include:
+          - otp: '25'
+            os: ubuntu-24.04
+          - otp: '26'
+            os: ubuntu-latest
+          - otp: '27'
+            os: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
 
@@ -42,6 +54,7 @@ jobs:
   publish:
     name: Publish to Hex.pm
     needs: test
+    if: startsWith(github.ref, 'refs/tags/v')
     runs-on: ubuntu-latest
     permissions:
       contents: read

README.md

@@ -14,6 +14,7 @@ Official Erlang client library for the [ipdata.co](https://ipdata.co) IP geoloca
 - Bulk lookups (up to 100 IPs per request)
 - Field filtering to reduce response size
 - EU endpoint support for data residency
+- Persistent HTTPS connections (HTTP/1.1 keep-alive) through a dedicated `httpc` profile
 - Production-ready with proper TLS verification and error handling
 
 ## Requirements
@@ -27,14 +28,14 @@ Add `ipdata` to your `rebar.config` dependencies:
 
 ```erlang
 {deps, [
-    {ipdata, "~> 1.0"}
+    {ipdata, "~> 1.1"}
 ]}.
 ```
 
 ## Quick Start
 
 ```erlang
-%% Start the application
+%% Start the application (it owns the pool of persistent connections)
 application:ensure_all_started(ipdata).
 
 %% Create a client with your API key
@@ -89,6 +90,10 @@ maps:get(<<"name">>, ASN).            %% => <<"Google LLC">>
 Threat = maps:get(<<"threat">>, Result).
 maps:get(<<"is_vpn">>, Threat).       %% => false
 
+%% Flags the API includes only when they are true: read them with a default
+maps:get(<<"is_anycast">>, Result, false).   %% => true
+maps:get(<<"is_satellite">>, Result, false). %% => false
+
 TimeZone = maps:get(<<"time_zone">>, Result).
 maps:get(<<"name">>, TimeZone).       %% => <<"America/New_York">>
 ```
@@ -137,6 +142,28 @@ case ipdata:lookup(Client, <<"8.8.8.8">>) of
 end.
 ```
 
+## Connection reuse
+
+The `ipdata` application owns a dedicated `httpc` profile, also named `ipdata`, that keeps persistent HTTPS connections (HTTP/1.1 keep-alive) to the API for as long as the application runs. Start the application once per node and keep it running: every lookup from every process then goes out on an already established connection instead of paying a new TCP and TLS handshake (roughly 660 to 980 ms for a cold connection to api.ipdata.co against 220 to 300 ms for a request on a reused one). Idle connections are closed after 60 seconds. HTTP/2 is not available with `httpc`, so the saving is the TLS handshake, not multiplexing; an HTTP/2 client such as `gun` would need a connection-owner process and a new client lifecycle, which is out of scope for this library. TLS 1.3 session tickets are enabled so that a connection opened after the idle timeout can resume the previous TLS session.
+
+A client map returned by `ipdata:new/1,2` is plain immutable data: it holds no process, socket or other resource, so it can be shared freely between processes, kept in application state or rebuilt at any time, and there is nothing to close. `application:stop(ipdata)` stops the profile and closes its connections. If a request is made before the application has been started, the profile is started on demand (`inets` and `ssl` must be running).
+
+The profile is tuned through the `httpc_options` key of the `ipdata` application environment, for example in `sys.config`:
+
+```erlang
+[{ipdata, [
+    {httpc_options, [
+        {max_sessions, 16},          %% persistent connections per host (default 16)
+        {max_keep_alive_length, 4},  %% outstanding requests per connection (default 4)
+        {keep_alive_timeout, 60000}  %% close a connection idle for this long, in ms (default 60000)
+    ]}
+]}].
+```
+
+Each entry overrides the default of the same name, and any other `httpc:set_options/2` option (for example `https_proxy`) can be added. The options are applied when the application starts.
+
+A bulk lookup is a POST, so it is retried exactly once if the server closed the connection before answering (`socket_closed_remotely`); `httpc` has already discarded that connection, so the retry goes out on a fresh one. HTTP error statuses are never retried.
+
 ## API Reference
 
 ### `ipdata:new/1,2`
@@ -178,6 +205,8 @@ Look up multiple IP addresses in a single request. Maximum 100 IPs. Requires a p
 |-------|------|-------------|
 | `ip` | binary | IP address |
 | `is_eu` | boolean | Whether the country is in the EU |
+| `is_satellite` | boolean | Satellite internet provider; present only when `true` |
+| `is_anycast` | boolean | Anycast address; present only when `true` |
 | `city` | binary | City name |
 | `region` | binary | Region/state name |
 | `region_code` | binary | Region code |
@@ -203,6 +232,8 @@ Look up multiple IP addresses in a single request. Maximum 100 IPs. Requires a p
 | `threat` | map | Threat data (see below) |
 | `count` | binary | API request count |
 
+The API includes `is_satellite` and `is_anycast` only when they are `true` and omits them otherwise. The library returns the decoded JSON unchanged, so read them with a default: `maps:get(<<"is_satellite">>, Result, false)`. Both names can be requested through the fields parameter like any other field.
+
 ### Threat Fields
 
 | Field | Type | Description |
@@ -217,9 +248,24 @@ Look up multiple IP addresses in a single request. Maximum 100 IPs. Requires a p
 | `is_known_abuser` | boolean | Known abuser |
 | `is_threat` | boolean | Any threat detected |
 | `is_bogon` | boolean | Bogon IP |
-| `blocklists` | list | Blocklist entries (`name`, `site`, `type`) |
+| `blocklists` | list | Blocklist entries (see below) |
 | `scores` | map | Reputation scores (`vpn_score`, `proxy_score`, `threat_score`, `trust_score`) |
 
+### Blocklist Entry Fields
+
+Each element of `blocklists` is a map:
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `name` | binary | Blocklist name |
+| `site` | binary | Blocklist website |
+| `type` | binary | Blocklist type (for example `vpn`, `cloud`, `general`) |
+| `first_seen` | binary | First day the IP was listed, `YYYY-MM-DD` (UTC); optional |
+| `last_seen` | binary | Last day the IP was listed, `YYYY-MM-DD` (UTC); optional |
+| `days_seen` | integer | Number of days the IP was listed; optional |
+
+Some feeds, for example ipdata's own `vpn` feed, carry none of `first_seen`, `last_seen` and `days_seen`, so read them with a default such as `maps:get(<<"days_seen">>, Entry, undefined)`. The dates are returned as binaries and are not parsed.
+
 ## Testing
 
 ```bash

src/ipdata.app.src

@@ -1,6 +1,6 @@
 {application, ipdata, [
     {description, "Erlang client library for the ipdata.co IP geolocation API"},
-    {vsn, "1.0.1"},
+    {vsn, "1.1.0"},
     {registered, [ipdata_sup]},
     {mod, {ipdata_app, []}},
     {applications, [
@@ -11,7 +11,7 @@
         public_key,
         jsx
     ]},
-    {env, []},
+    {env, [{httpc_options, []}]},
     {modules, []},
     {licenses, ["MIT"]},
     {links, [

src/ipdata_app.erl

@@ -1,8 +1,12 @@
 %%%-------------------------------------------------------------------
 %%% @doc Application behaviour callback for the ipdata library.
 %%%
-%%% Ensures that required OTP applications (`inets', `ssl') are
-%%% started and launches the top-level supervisor.
+%%% The application depends on `inets', `ssl' and `public_key', which
+%%% `application:ensure_all_started(ipdata)' starts first. On start it
+%%% brings up the `ipdata' httpc profile that holds the persistent
+%%% connections used by every request (see
+%%% `ipdata_http:start_profile/0'), then launches the top-level
+%%% supervisor. On stop the profile and its connections are closed.
 %%% @end
 %%%-------------------------------------------------------------------
 -module(ipdata_app).
@@ -13,17 +17,29 @@
 
 %%--------------------------------------------------------------------
 %% @doc Start the ipdata application.
+%%
+%% Starts the `ipdata' httpc profile (accepting one that is already
+%% running) and applies `ipdata_http:profile_options/0' before the
+%% supervisor is started.
 %% @end
 %%--------------------------------------------------------------------
 -spec start(application:start_type(), term()) ->
     {ok, pid()} | {error, term()}.
 start(_StartType, _StartArgs) ->
-    ipdata_sup:start_link().
+    case ipdata_http:start_profile() of
+        ok ->
+            ipdata_sup:start_link();
+        {error, Reason} ->
+            {error, {httpc_profile, Reason}}
+    end.
 
 %%--------------------------------------------------------------------
 %% @doc Stop the ipdata application.
+%%
+%% Stops the `ipdata' httpc profile, closing its connections.
 %% @end
 %%--------------------------------------------------------------------
 -spec stop(term()) -> ok.
 stop(_State) ->
+    _ = inets:stop(httpc, ipdata),
     ok.