Reuse HTTPS connections, negotiate HTTP/2, add new response fields

README.md

@@ -11,6 +11,7 @@ An 100% compliant [ipdata.co](https://ipdata.co) API java client.
   - [Configuration](#configuration)
   - [API](#create-an-instance)
     - [Basic Usage](#basic-usage)
+    - [Connection reuse and HTTP/2](#connection-reuse-and-http2)
     - [Single Field Selection](#single-field-selection)
     - [Multiple Field Selection](#multiple-field-selection)
     - [Bulk data](#bulk-data)
@@ -22,7 +23,7 @@ You can have the library from Maven Central.
 <dependency>
     <groupId>co.ipdata.client</groupId>
     <artifactId>ipdata-java-client</artifactId>
-    <version>0.2.0</version>
+    <version>0.3.0</version>
 </dependency>
 ```
 
@@ -143,12 +144,49 @@ Output:
 }
 ```
 
+The following fields were added to the API responses recently and are mapped by the model (every other field is described in the [API reference](https://docs.ipdata.co/api-reference/response-fields)):
+
+- `is_satellite` – `model.isSatellite()`: the address belongs to a satellite ISP. The API sends the key only when it is `true`; the model reads `false` when it is absent.
+- `is_anycast` – `model.isAnycast()`: the address is an anycast address (`8.8.8.8` for example). Same present-only-when-true behaviour.
+- `threat.blocklists[].first_seen` – `Blocklist.getFirstSeen()`: the first day the address was seen on that blocklist, as a `YYYY-MM-DD` (UTC) string, or `null` when the feed does not report it.
+- `threat.blocklists[].last_seen` – `Blocklist.getLastSeen()`: the most recent such day, same format, or `null`.
+- `threat.blocklists[].days_seen` – `Blocklist.getDaysSeen()`: the number of days the address was seen on that blocklist, as an `Integer`, or `null`.
+
+#### Connection reuse and HTTP/2
+An `IpdataService` keeps a persistent HTTPS connection pool for as long as it lives. Create one service per process (or one per API key or endpoint), keep it, and reuse it for every lookup: each new service pays a fresh TLS handshake. Requests to api.ipdata.co are multiplexed over HTTP/2 when the platform negotiates it, and fall back to HTTP/1.1 keep-alive otherwise.
+
+`IpdataService` is thread-safe: share one instance between threads. The full-model, single-field and bulk calls all go through the same pool. There is nothing to close: the default transport is the JDK's `java.net.http.HttpClient` (through `feign-java11`), which has no close method on Java 11 and drops idle connections on its own (it has no per-instance idle setting; the `jdk.httpclient.keepalive.timeout` system property, 1200 seconds by default, applies). A request whose transport call fails with an `IOException`, for example on a pooled connection the server had already closed, is re-sent by Feign's default `Retryer`, exactly as before; HTTP error statuses are never retried.
+
+Timeouts default to 10 seconds to connect and 60 seconds to read and are set on the builder, either one by one or as any `feign.Request.Options`:
+```java
+IpdataService ipdataService = Ipdata.builder().url(url)
+      .key("MY_KEY")
+      .connectTimeout(5, TimeUnit.SECONDS)
+      .readTimeout(30, TimeUnit.SECONDS)
+      .get();
+
+IpdataService ipdataService = Ipdata.builder().url(url)
+      .key("MY_KEY")
+      .options(new Request.Options(5, TimeUnit.SECONDS, 30, TimeUnit.SECONDS, true))
+      .get();
+```
+
+To use another HTTP stack, pass a `feign.Client` to `feignClient(...)`, for example Apache HttpClient (`io.github.openfeign:feign-httpclient`) or OkHttp (`io.github.openfeign:feign-okhttp`). The injected client is shared by all calls of that service, so configure its connection pooling there; the timeouts above are handed to it on every request:
+```java
+IpdataService ipdataService = Ipdata.builder().url(url)
+      .key("MY_KEY")
+      .feignClient(new ApacheHttpClient(HttpClientBuilder.create().build()))
+      .get();
+```
+
 #### Single Field Selection
 If you're interested in only one field from the model capturing an IP address information, The service interface
 exposes a method on each available field:
 
 ```java
 boolean isEu = ipdataService.isEu("1.1.1.1");
+boolean isSatellite = ipdataService.isSatellite("1.1.1.1");
+boolean isAnycast = ipdataService.isAnycast("1.1.1.1");
 AsnModel asn = ipdataService.asn("1.1.1.1");
 TimeZone tz  = ipdataService.timeZone("1.1.1.1");
 ThreatModel threat = ipdataService.threat("1.1.1.1");
@@ -164,6 +202,7 @@ import io.ipdata.client.service.IpdataService;
 
 /* The model will be hydrated by the selected fields only */
 IpdataModel model = ipdataService.getFields("1.1.1.1", IpdataField.ASN, IpdataField.CURRENCY);
+IpdataModel flags = ipdataService.getFields("1.1.1.1", IpdataField.IS_SATELLITE, IpdataField.IS_ANYCAST);
 
 ```
 

pom.xml

@@ -3,7 +3,7 @@
   <modelVersion>4.0.0</modelVersion>
   <groupId>co.ipdata.client</groupId>
   <artifactId>ipdata-java-client</artifactId>
-  <version>0.2.1</version>
+  <version>0.3.0</version>
   <description>A java client for ipdata.co</description>
   <name>Ipdata java client</name>
   <url>https://github.com/ipdata/java</url>
@@ -23,7 +23,7 @@
     <connection>scm:git:git@github.com:ipdata/java.git</connection>
     <developerConnection>scm:git:git@github.com:ipdata/java.git</developerConnection>
     <url>https://github.com/ipdata/java</url>
-    <tag>0.2.1</tag>
+    <tag>0.3.0</tag>
   </scm>
   <distributionManagement>
     <repository>
@@ -52,6 +52,11 @@
       <artifactId>feign-jackson</artifactId>
       <version>${version.client.feign}</version>
     </dependency>
+    <dependency>
+      <groupId>io.github.openfeign</groupId>
+      <artifactId>feign-java11</artifactId>
+      <version>${version.client.feign}</version>
+    </dependency>
     <dependency>
       <groupId>io.github.openfeign</groupId>
       <artifactId>feign-httpclient</artifactId>

src/main/java/io/ipdata/client/Ipdata.java

@@ -1,6 +1,7 @@
 package io.ipdata.client;
 
 import feign.Client;
+import feign.Request;
 import io.ipdata.client.service.CacheConfig;
 import io.ipdata.client.service.IpdataService;
 import io.ipdata.client.service.IpdataServiceBuilder;
@@ -47,10 +48,27 @@ public static class Builder {
     @Setter
     private Logger logger;
     /**
-     * (Optional) Configures a custom Feign client (If you wanna use this with Ribbon or Hystrix for example).
+     * (Optional) Configures a custom Feign client, for example Apache HttpClient ({@code feign-httpclient}),
+     * OkHttp ({@code feign-okhttp}) or a Ribbon/Hystrix wrapper.
+     * <p>
+     * The default is the JDK HTTP/2 client ({@code feign.http2client.Http2Client} from {@code feign-java11}),
+     * which keeps one persistent connection pool for the lifetime of the service and multiplexes requests over
+     * HTTP/2 when the server negotiates it, falling back to HTTP/1.1 keep-alive otherwise. An injected client
+     * is shared by all calls of the built service (full model, single field and bulk), so configure its
+     * connection pooling there; the {@code options(Request.Options)} are passed to it on every request.
      */
     @Setter
     private Client feignClient;
+    /**
+     * (Optional) Connect and read timeouts and the redirect policy of the HTTP transport, shared by all calls
+     * of the built service. Defaults to {@code new Request.Options()}: 10 seconds to connect, 60 seconds to
+     * read, follow redirects. With the default transport both timeouts must be positive.
+     *
+     * @see #connectTimeout(long, TimeUnit)
+     * @see #readTimeout(long, TimeUnit)
+     */
+    @Setter
+    private Request.Options options;
 
     /**
      * Overrides current cache config with null (no caching).
@@ -86,6 +104,52 @@ public CacheConfigBuilder withCache() {
       return new CacheConfigBuilder(this);
     }
 
+    /**
+     * Sets the connect timeout of the HTTP transport (default: 10 seconds), keeping the other request options.
+     * Shorthand for {@code options(Request.Options)} with only the connect timeout changed.
+     *
+     * @param timeout the connect timeout, greater than 0
+     * @param unit    the unit of the timeout
+     * @return this builder
+     * @see Request.Options
+     */
+    public Builder connectTimeout(long timeout, TimeUnit unit) {
+      Request.Options current = currentOptions();
+      this.options = new Request.Options(checkTimeout(timeout, unit), unit,
+        current.readTimeout(), current.readTimeoutUnit(), current.isFollowRedirects());
+      return this;
+    }
+
+    /**
+     * Sets the read timeout of the HTTP transport (default: 60 seconds), keeping the other request options.
+     * Shorthand for {@code options(Request.Options)} with only the read timeout changed.
+     *
+     * @param timeout the read timeout, greater than 0
+     * @param unit    the unit of the timeout
+     * @return this builder
+     * @see Request.Options
+     */
+    public Builder readTimeout(long timeout, TimeUnit unit) {
+      Request.Options current = currentOptions();
+      this.options = new Request.Options(current.connectTimeout(), current.connectTimeoutUnit(),
+        checkTimeout(timeout, unit), unit, current.isFollowRedirects());
+      return this;
+    }
+
+    private Request.Options currentOptions() {
+      return options == null ? new Request.Options() : options;
+    }
+
+    private static long checkTimeout(long timeout, TimeUnit unit) {
+      if (timeout <= 0) {
+        throw new IllegalArgumentException("timeout must be greater than 0");
+      }
+      if (unit == null) {
+        throw new IllegalArgumentException("timeout unit should not be null");
+      }
+      return timeout;
+    }
+
     /**
      * @return An Ipdata api facade
      * @see io.ipdata.client.service.IpdataService
@@ -97,7 +161,7 @@ public IpdataService get() {
       if (url == null) {
         throw new IllegalArgumentException("Target URL can't be null");
       }
-      return IpdataServiceBuilder.of(url, key, cacheConfig, logger, feignClient).build();
+      return IpdataServiceBuilder.of(url, key, cacheConfig, logger, feignClient, options).build();
     }
 
   }

src/main/java/io/ipdata/client/model/Blocklist.java

@@ -1,5 +1,6 @@
 package io.ipdata.client.model;
 
+import com.fasterxml.jackson.annotation.JsonProperty;
 import lombok.Getter;
 import lombok.ToString;
 
@@ -8,4 +9,21 @@ public class Blocklist {
   private String name;
   private String site;
   private String type;
+  /**
+   * First day the address was seen on this blocklist, as a {@code YYYY-MM-DD} (UTC) string, or {@code null}
+   * when the feed does not report it.
+   */
+  @JsonProperty("first_seen")
+  private String firstSeen;
+  /**
+   * Most recent day the address was seen on this blocklist, as a {@code YYYY-MM-DD} (UTC) string, or
+   * {@code null} when the feed does not report it.
+   */
+  @JsonProperty("last_seen")
+  private String lastSeen;
+  /**
+   * Number of days the address was seen on this blocklist, or {@code null} when the feed does not report it.
+   */
+  @JsonProperty("days_seen")
+  private Integer daysSeen;
 }

src/main/java/io/ipdata/client/model/IpdataModel.java

@@ -14,6 +14,18 @@ public class IpdataModel {
   private String ip;
   @JsonProperty("is_eu")
   private boolean eu;
+  /**
+   * The address belongs to a satellite ISP. The API sends the key only when it is true, so the field reads
+   * {@code false} when it is absent.
+   */
+  @JsonProperty("is_satellite")
+  private boolean satellite;
+  /**
+   * The address is an anycast address. The API sends the key only when it is true, so the field reads
+   * {@code false} when it is absent.
+   */
+  @JsonProperty("is_anycast")
+  private boolean anycast;
   private String city;
   private String organisation;
 
@@ -44,4 +56,12 @@ public class IpdataModel {
   public boolean isEu() {
     return eu;
   }
+
+  public boolean isSatellite() {
+    return satellite;
+  }
+
+  public boolean isAnycast() {
+    return anycast;
+  }
 }

src/main/java/io/ipdata/client/service/IpdataField.java

@@ -14,6 +14,8 @@
 public class IpdataField<T> {
   public static final IpdataField<String> IP = new IpdataField<String>("ip", String.class);
   public static final IpdataField<Boolean> IS_EU = new IpdataField<Boolean>("is_eu", Boolean.class);
+  public static final IpdataField<Boolean> IS_SATELLITE = new IpdataField<Boolean>("is_satellite", Boolean.class);
+  public static final IpdataField<Boolean> IS_ANYCAST = new IpdataField<Boolean>("is_anycast", Boolean.class);
   public static final IpdataField<String> CITY = new IpdataField<String>("city", String.class);
   public static final IpdataField<String> REGION = new IpdataField<String>("region", String.class);
   public static final IpdataField<String> REGION_CODE = new IpdataField<String>("region_code", String.class);

src/main/java/io/ipdata/client/service/IpdataInternalSingleFieldClient.java

@@ -16,6 +16,12 @@ interface IpdataInternalSingleFieldClient {
   @RequestLine("GET /{ip}/is_eu")
   boolean isEu(@Param(value = "ip", encoded = true) String ip) throws IpdataException;
 
+  @RequestLine("GET /{ip}/is_satellite")
+  boolean isSatellite(@Param(value = "ip", encoded = true) String ip) throws IpdataException;
+
+  @RequestLine("GET /{ip}/is_anycast")
+  boolean isAnycast(@Param(value = "ip", encoded = true) String ip) throws IpdataException;
+
   @RequestLine("GET /{ip}/city")
   String getCity(@Param(value = "ip", encoded = true) String ip) throws IpdataException;
 

src/main/java/io/ipdata/client/service/IpdataService.java

@@ -14,6 +14,11 @@
  * <p>
  * Also exposes single-field accessors (e.g. {@code getCountryName}, {@code getCity})
  * inherited from {@link IpdataInternalSingleFieldClient}.
+ * <p>
+ * A service owns one persistent HTTP connection pool, shared by all of its methods: requests are multiplexed
+ * over HTTP/2 when the server negotiates it and use HTTP/1.1 keep-alive otherwise. Create one instance per
+ * API key or endpoint, keep it for the lifetime of the application and share it between threads (it is
+ * thread-safe); every new instance pays a fresh TLS handshake. There is nothing to close.
  *
  * @see io.ipdata.client.Ipdata#builder()
  */

src/main/java/io/ipdata/client/service/IpdataServiceBuilder.java

@@ -9,6 +9,8 @@
 import com.google.common.cache.CacheLoader;
 import feign.Client;
 import feign.Feign;
+import feign.Request;
+import feign.http2client.Http2Client;
 import feign.jackson.JacksonDecoder;
 import feign.jackson.JacksonEncoder;
 import io.ipdata.client.model.*;
@@ -20,6 +22,15 @@
 
 import static com.fasterxml.jackson.databind.PropertyNamingStrategies.SNAKE_CASE;
 
+/**
+ * Assembles an {@link IpdataService}.
+ * <p>
+ * The full-model proxy ({@code ipdata}, {@code bulk}, {@code getFields}) and the single-field proxy
+ * ({@code asn}, {@code threat}, ...) share one HTTP transport, so every call of the resulting service goes
+ * through the same persistent connection pool. The default transport is the JDK HTTP/2 client
+ * ({@link Http2Client} from {@code feign-java11}): HTTP/2 multiplexing when the server negotiates it,
+ * HTTP/1.1 keep-alive otherwise.
+ */
 @RequiredArgsConstructor(staticName = "of")
 public class IpdataServiceBuilder {
 
@@ -28,6 +39,27 @@ public class IpdataServiceBuilder {
   private final CacheConfig cacheConfig;
   private final Logger logger;
   private final Client httpClient;
+  /**
+   * Connect and read timeouts and the redirect policy, applied to both Feign proxies and used to build the
+   * default transport. {@code null} means {@code new Request.Options()}: 10 seconds connect, 60 seconds read,
+   * follow redirects.
+   */
+  private final Request.Options options;
+
+  /**
+   * Same as the generated {@code of(url, key, cacheConfig, logger, httpClient, options)} factory with default
+   * request options.
+   *
+   * @param url         the API endpoint
+   * @param key         the API key
+   * @param cacheConfig the cache configuration, or {@code null} for no caching
+   * @param logger      the logger to use, or {@code null} for the default one
+   * @param httpClient  a custom Feign client shared by all calls, or {@code null} for the JDK HTTP/2 client
+   * @return a builder
+   */
+  public static IpdataServiceBuilder of(URL url, String key, CacheConfig cacheConfig, Logger logger, Client httpClient) {
+    return of(url, key, cacheConfig, logger, httpClient, null);
+  }
 
   public IpdataService build() {
     final ObjectMapper mapper = new ObjectMapper();
@@ -40,16 +72,25 @@ public IpdataService build() {
     final Logger customLogger = this.logger == null ? LoggerFactory.getLogger(IpdataService.class) : this.logger;
     final ApiErrorDecoder apiErrorDecoder = new ApiErrorDecoder(mapper, customLogger);
 
+    final Request.Options requestOptions = this.options == null ? new Request.Options() : this.options;
+    // One transport for both proxies: the java.net.http client behind Http2Client owns a single connection
+    // pool that lives as long as the service does. The very same options instance must be handed to
+    // Http2Client and to Feign (.options below): Http2Client silently builds a new HttpClient, and thus a new
+    // connection, for every request whose options differ from the ones it was constructed with.
+    final Client transport = new Ipv6SafeClient(httpClient != null ? httpClient : new Http2Client(requestOptions));
+
     final IpdataInternalClient client = Feign.builder()
-      .client(new Ipv6SafeClient(httpClient == null ? new Client.Default(null, null) : httpClient))
+      .client(transport)
+      .options(requestOptions)
       .decoder(new JacksonDecoder(mapper))
       .encoder(new JacksonEncoder(mapper))
       .requestInterceptor(keyRequestInterceptor)
       .errorDecoder(apiErrorDecoder)
       .target(IpdataInternalClient.class, url.toString());
 
     final IpdataInternalSingleFieldClient singleFieldClient = Feign.builder()
-      .client(new Ipv6SafeClient(httpClient == null ? new Client.Default(null, null) : httpClient))
+      .client(transport)
+      .options(requestOptions)
       .decoder(new FieldDecoder(mapper))
       .encoder(new JacksonEncoder(mapper))
       .requestInterceptor(keyRequestInterceptor)