Skip to content

[Integration Spec] Update integration-client.md #3170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
chernser wants to merge 11 commits into
base: main
Choose a base branch
from
+727 −172
Open

[Integration Spec] Update integration-client.md #3170

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
c139eec
Updated integration-jdbc with error handling and some additional info…
chernser Sep 6, 2026
abb7085
Merge branch 'main' into 09/06/26/int_spec_jdbc_changes
chernser Sep 21, 2026
eeb1124
Merge branch 'main' into 09/06/26/int_spec_jdbc_changes
chernser Sep 26, 2026
dd2cf91
updated integration-jdbc: corrections, restructured some sections. mo…
chernser Sep 26, 2026
6b56c3c
Added documentation for FINAL and sequential consistency. Fixed some …
chernser Sep 27, 2026
4876482
Updated part of integration-client.md
chernser Sep 30, 2026
3eec9b0
Merge branch 'main' into 09/26/26/int_spec_client_update
chernser Oct 14, 2026
8f781f5
Made general cleanup of integration spec for client
chernser Oct 14, 2026
7751b3f
Added type mapping for integration spec
chernser Oct 15, 2026
01a1597
Added Optimization section and words about compression
chernser Oct 15, 2026
727a60a
Added about JVM and memory
chernser Oct 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 100 additions & 0 deletions docs/clickhouse-docs/client.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -523,6 +523,79 @@ This client version supports:
| [Markdown](/reference/formats/Markdown) | - | raw |
| [Form](/reference/formats/Form) | raw | - |

## Data Types {#data-types}

Database data types are mapped to Java types by the reader or writer and generally follow similar rules across formats (though custom or format-specific readers and writers may use different mappings).

**ClickHouse / Java Type Mapping**

The following table summarizes typical mappings between ClickHouse database and Java types. Conversion depends on data
format and reader/writer implementation. Following table represents supported conversions when used with ClickHouse binary formats
like `RowBinary` and `Native`.

Reader Interface: `com.clickhouse.client.api.data_formats.ClickHouseBinaryFormatReader`
Writer Interface: `com.clickhouse.client.api.data_formats.ClickHouseBinaryFormatWriter`

| ClickHouse Type | Java Type (Reader) | Java Type (Writer) |
|--------------------------------------|---------------------------------------------------------------|-------------------------------------------------------------------------------------|
| `Bool` | `boolean` | `boolean` |
| `Int8` | `byte`, `short`, `int`, `long` | `byte`, `short`, `int`, `long` |
| `UInt8` | `short`, `int`, `long` | `byte`, `short`, `int`, `long` |
| `Int16` | `short`, `int`, `long` | `short`, `int`, `long` |
| `UInt16` | `int`, `long` | `short`, `int`, `long` |
| `Int32` | `int`, `long` | `int`, `long` |
| `UInt32` | `long` | `int`, `long` |
| `Int64` | `long`, `BigInteger` | `long`, `BigInteger` |
| `UInt64` | `BigInteger`, `long` | `long`, `BigInteger` |
| `Int128`, `UInt128` | `BigInteger` | `BigInteger`, `long` |
| `Int256`, `UInt256` | `BigInteger` | `BigInteger`, `long` |
| `BFloat16` | `float`, `double` | `float` |
| `Float32` | `float`, `double` | `float`, `double` |
| `Float64` | `double` | `float`, `double` |
| `Decimal`, `Decimal32`..`Decimal256` | `BigDecimal` | `BigDecimal`, `String` |
| `String`, `FixedString` | `String`, `byte[]` | `String`, `byte[]` |
| `Date` | `LocalDate` | `LocalDate` |
| `Date32` | `LocalDate` | `LocalDate` |
| `DateTime`, `DateTime32` | `ZonedDateTime`, `OffsetDateTime`, `Instant`, `LocalDateTime` | `ZonedDateTime`, `OffsetDateTime`, `Instant`, `LocalDateTime`, `java.sql.Timestamp` |
| `DateTime64` | `ZonedDateTime`, `OffsetDateTime`, `Instant`, `LocalDateTime` | `ZonedDateTime`, `OffsetDateTime`, `Instant`, `LocalDateTime`, `java.sql.Timestamp` |
| `Time`, `Time64` | `LocalTime`, `LocalDateTime`, `Duration`, `TemporalAmount` | `int` (`Time`), `long`, `BigInteger`, `Instant` (`Time64`) |
| `Interval*` | `TemporalAmount` (`Period` / `Duration`), `Duration` | `Period` (date intervals), `Duration` (time/day intervals) |
| `Enum8`, `Enum16` | `String`, `byte` (`Enum8`), `short` (`Enum16`), `int` | `String`, `byte`, `short`, `int` |
| `UUID` | `UUID`, `String` | `UUID`, `String` |
| `IPv4`, `IPv6` | `Inet4Address`, `Inet6Address`, `String` | `Inet4Address` (`IPv4`), `Inet6Address` (`IPv6`) |
| `Nullable(T)` | `T` or `null` | `T` or `null` |
| `LowCardinality(T)` | `T` | `T` |
| `Array(T)` | `List<T>`, primitive `T[]`, `String[]`, `Object[]` | `List<T>`, `T[]` |
| `Tuple` | `Object[]`, `List<?>` (POJO) | `Object[]`, `List<?>` |
| `Map(K, V)` | `Map<K, V>` | `Map<K, V>` |
| `Nested` | `List<Object[]>`, `Object[]` | `List<Object[]>`, `Object[][]` |
| `Point` | `double[]` | `double[]` |
| `Ring`, `LineString`, `MultiPoint` | `double[][]` | `double[][]` |
| `Polygon`, `MultiLineString` | `double[][][]` | `double[][][]` |
| `MultiPolygon` | `double[][][][]` | `double[][][][]` |
| `Geometry` | `double[]` … `double[][][][]` | `double[]` … `double[][][][]` |
| `QBit(T, N)` | `float[]`, `double[]`, `List<T>` | `float[]`, `double[]`, `List<T>` (length `N`) |
| `Variant`, `Dynamic` | `Object` (resolved runtime value) | `Object` (supported scalar/array/map types) |
| `JSON` | `Map<String, Object>`, `String` | `String` |
| `SimpleAggregateFunction(f, T)` | `T` | `T` |

**Notes:**
- **Signed vs. unsigned integers:** Unsigned types (`UInt8`, `UInt16`, `UInt32`) are widened to the next larger signed Java primitive (`short`, `int`, `long`) on read so values above the signed max do not overflow; `UInt64`, `Int128`, `UInt128`, `Int256`, and `UInt256` use `BigInteger`. Narrower getters (e.g., `getByte` on `UInt8 > 127` or `getLong` on `UInt64 > Long.MAX_VALUE`) throw `ArithmeticException`.
- **Binary strings (`String`, `FixedString`):** By default, strings are decoded as UTF-8 `String` on read. To read raw binary bytes via `getByteArray(...)` (or POJO `byte[]` fields) without UTF-8 transcoding, enable `Client.Builder#binaryStringSupport(true)` (or per-query `binary_string_support`).
- **Dates (`Date`, `Date32`):** Use `java.time.LocalDate` (timezone-agnostic calendar date). Note that `java.sql.Date` is not supported by the `RowBinary` writer.
- **Timestamps (`DateTime`, `DateTime64`):** **Prefer `ZonedDateTime`, `OffsetDateTime`, or `Instant`** over `LocalDateTime`. ClickHouse stores `DateTime` / `DateTime64` as UTC epoch values. Because `LocalDateTime` carries no timezone offset, using it can easily cause timezone confusion:
- On **write**, a `LocalDateTime` is interpreted in the column's declared timezone (e.g., `DateTime('Asia/Tokyo')`), or in **`UTC`** if the column has no timezone parameter — *not* the JVM default timezone.
- On **read**, the reader decodes the timestamp into a `ZonedDateTime` using the column's timezone (or the client's configured timezone), and `getLocalDateTime(...)` simply strips that zone (`toLocalDateTime()`), which may not match the application's local wall clock.
- Using `Instant` or `ZonedDateTime` / `OffsetDateTime` keeps the exact point in time unambiguous across client, server, and column timezones.
- **Nullability:** Reading a `NULL` value through a primitive getter (`getInt`, `getLong`, `getDouble`, `getBoolean`, `getEnum8`, etc.) or into a primitive POJO field throws `NullValueException`. Check `hasValue(...)` first or use boxed/object types.
- **`LowCardinality(T)`:** Treated as a transparent storage modifier by the client. A `LowCardinality(T)` (or `LowCardinality(Nullable(T))`) column is unwrapped to its underlying type `T` and follows the exact same Java type conversion rules on both read and write.
- **Containers (`Array`, `Tuple`, `Map`, `Nested`):** `getList(...)` reads any `Array` as `List<T>`, primitive array getters (`getIntArray`, `getDoubleArray`, etc.) read 1D primitive arrays, and `getObjectArray(...)` reads any 1D or multidimensional array as `Object[]`. `Tuple` is read via `getTuple(...)` as `Object[]` (and accepts `Object[]` or `List<?>` on write and in POJO mapping).
- **Enums:** `Enum8` and `Enum16` can be read/written by label (`String`) or numeric value (`getEnum8`/`getEnum16`/`getInteger`).
- **Decimals:** `Decimal` and `Decimal32`/`64`/`128`/`256` map to `BigDecimal` to preserve scale and precision.

When reading or writing data programmatically, favor the canonical Java type above for portability.


## Insert API {#insert-api}

### insert(String tableName, InputStream data, ClickHouseFormat format) {#insertstring-tablename-inputstream-data-clickhouseformat-format}
Expand Down Expand Up @@ -1028,6 +1101,33 @@ public Client.Builder configureGlobalTelemetry(Client.Builder builder) {
}
```

### Optimization {#v2-optimization}

#### Compression modes & supported algorithms {#v2-compression}

The client supports two wire-level compression mechanisms and an application passthrough mode:

| Mode | How it works on the wire | Supported algorithms | Read (Server → Client) | Write (Client → Server) |
|------|--------------------------|----------------------|------------------------|-------------------------|
| **ClickHouse Native Block Compression** *(default when `useHttpCompression` is `false`)* | Uses ClickHouse's native 25-byte framed block format with `CityHash128` checksums via query parameters `compress=1` (read) and `decompress=1` (write). | `LZ4`, `ZSTD` ([`CompressionMethod`](../client-v2/src/main/java/com/clickhouse/client/api/CompressionMethod.java)) | **Enabled by default** (`compress=true`). Automatically decompresses both `LZ4` (`0x82`) and `ZSTD` (`0x90`) blocks returned by the server. | **Disabled by default** (`decompress=false`). When enabled, compresses blocks using `ZSTD` by default (configurable to `LZ4` via `compressionMethod(...)`). |
| **HTTP Compression** *(`useHttpCompression(true)`)* | Uses standard HTTP headers (`Accept-Encoding` for reads, `Content-Encoding` for writes) with query parameter `enable_http_compression=1`. | `lz4` *(default)*, `zstd`, `gzip`, `deflate` | Sends `Accept-Encoding: lz4` by default; override per-client or per-query with `httpHeader("Accept-Encoding", "<algo>")`. | Sends `Content-Encoding: lz4` by default; override per-client or per-insert with `httpHeader("Content-Encoding", "<algo>")`. |
| **Application-Compressed Data** *(`appCompressedData`)* | Streams pre-compressed payloads directly to the server with `enable_http_compression=1` and `Content-Encoding: <algo>` without re-compressing on the client. | `gzip`, `lz4`, `zstd`, `deflate`, `br`, `xz`, `bz2`, `snappy` *(any format supported by the ClickHouse server)* | N/A | Enabled via `InsertSettings.appCompressedData(true, "<algo>")`. |

#### Configuration properties reference

Compression settings can be configured globally on `Client.Builder` or overridden per operation via `QuerySettings` and `InsertSettings`:

| Property key (`ClientConfigProperties`) | `Client.Builder` method | Operation-level (`QuerySettings` / `InsertSettings`) | Default | Applies to | Description |
|-----------------------------------------|-------------------------|------------------------------------------------------|---------|------------|-------------|
| `compress` (`COMPRESS_SERVER_RESPONSE`) | `compressServerResponse(boolean)` | `QuerySettings.setOption("compress", boolean)` | `true` | Read | Requests the server to compress query responses. |
| `decompress` (`COMPRESS_CLIENT_REQUEST`) | `compressClientRequest(boolean)` | `InsertSettings.compressClientRequest(boolean)` | `false` | Write | Compresses client request payloads before sending to the server. |
| `compression.method` (`COMPRESSION_METHOD`) | `compressionMethod(CompressionMethod)` | `InsertSettings.setOption("compression.method", CompressionMethod)` | `CompressionMethod.ZSTD` | Write (Native) | Selects `LZ4` or `ZSTD` for native block compression of client requests. |
| `client.use_http_compression` (`USE_HTTP_COMPRESSION`) | `useHttpCompression(boolean)` | `InsertSettings.useHttpCompression(boolean)` / `QuerySettings.setOption("client.use_http_compression", boolean)` | `false` | Read & Write | Switches from ClickHouse native block compression to standard HTTP header-based compression (`enable_http_compression=1`). |
| `app_compressed_data` (`APP_COMPRESSED_DATA`) | `appCompressedData(boolean)` | `InsertSettings.appCompressedData(boolean, String)` | `false` | Write | Indicates the insert stream is already compressed by the application; sets `useHttpCompression(true)` and `Content-Encoding`. |
| `compression.lz4.uncompressed_buffer_size` (`COMPRESSION_LZ4_UNCOMPRESSED_BUF_SIZE`) | `setLZ4UncompressedBufferSize(int)` | `setOption("compression.lz4.uncompressed_buffer_size", int)` | `65536` (64 KB) | Read & Write | Uncompressed block buffer size (in bytes) for native block compression and decompression. |
| `disable_native_compression` (`DISABLE_NATIVE_COMPRESSION`) | `disableNativeCompression(boolean)` | Client-level only | `false` | Read & Write | Disables JNI/native `lz4-java` bindings (`LZ4Factory.fastestInstance()`) and falls back to the pure Java implementation (`LZ4Factory.fastestJavaInstance()`). |


### Metrics Reported {#v2-o11y-metrics}

#### Operational Metrics
Expand Down
Loading
Loading