Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .github/jwks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"keys": [
{
"e": "AQAB",
"kid": "ZoA8VvkIi_fBxd5DEx9xekutcCztcanNFIHOTQp5aDo",
"kty": "RSA",
"n": "q-qV7VPWHe0-Pj-sgmq0_szurrPPwAsq3BPTllfFNQcQq5967SdS5jiCKhpVKYeXng468e9yimOMPzLIVBoCkV9ul9ruZDDJiPBoQ-_Pp5IuQQWTFUmnJ-ONkczVdb-FdkfQz4z1NVz_blCdOVyVPmeE9gh1S1V4g3kQazznwxKgOlJM4mO23j8xOmhGIL0w1IktP09vqicZoOsq3-RgN0x6jXdiv5BnTAh1yDKIFRRqIa7idz55Bv_AGosH7hsKHYoCwfiPSCeVc4StYIhokj9HZswnMyAStELfQWen2Ev3pDuiANI4TuE34TOx0LsinVoWeEtiMaRi67mYFk5JIQ",
"use": "sig",
"x5c": [
"MIIDYjCCAkqgAwIBAgIUH0aS9zvv7N9L9UR/HmOaA5k9l/4wDQYJKoZIhvcNAQELBQAwazELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExFjAUBgNVBAcMDVNhbiBGcmFuY2lzY28xGDAWBgNVBAoMD015IE9yZ2FuaXphdGlvbjEVMBMGA1UEAwwMbXlkb21haW4uY29tMB4XDTI2MDkwMzIxMDgxOFoXDTI3MDkwMzIxMDgxOFowazELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExFjAUBgNVBAcMDVNhbiBGcmFuY2lzY28xGDAWBgNVBAoMD015IE9yZ2FuaXphdGlvbjEVMBMGA1UEAwwMbXlkb21haW4uY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAq+qV7VPWHe0+Pj+sgmq0/szurrPPwAsq3BPTllfFNQcQq5967SdS5jiCKhpVKYeXng468e9yimOMPzLIVBoCkV9ul9ruZDDJiPBoQ+/Pp5IuQQWTFUmnJ+ONkczVdb+FdkfQz4z1NVz/blCdOVyVPmeE9gh1S1V4g3kQazznwxKgOlJM4mO23j8xOmhGIL0w1IktP09vqicZoOsq3+RgN0x6jXdiv5BnTAh1yDKIFRRqIa7idz55Bv/AGosH7hsKHYoCwfiPSCeVc4StYIhokj9HZswnMyAStELfQWen2Ev3pDuiANI4TuE34TOx0LsinVoWeEtiMaRi67mYFk5JIQIDAQABMA0GCSqGSIb3DQEBCwUAA4IBAQCD5HRitEaurh0QOv/sqH0eFZ5KWAWbr4gP3EU9tgEAGAo0DQz2v7ov+wey4aTplXKtXIxB7aPobXFqTBEI6yD+to2UgmC35U+0tHZTgDG+BSmDEERoj+/Guk55LuUwODvQJviFUzEM9q7I9fLANPVkhY/SJ2/hl1ftxqxNFAH8mVzUbBNb2DMDc30eicHvsJUg0bqKoB5pLciyYqFsyA9z0x8ZaVpPVbXnVOHbzilp0jnhHeyLr+JCYH2UKfH/zFjxvcMP/q3M+HHCYyHXgdhMIFGBRE7xUMZjRnir28aUyet6hB5iV4wM8Yjd2mVaAsYtPqY8GnQxISzhUJ9rugie"
]
}
]
}
5 changes: 5 additions & 0 deletions .github/scripts/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
__pycache__
.venv
*.token
*.json
*.pem
60 changes: 60 additions & 0 deletions .github/scripts/generate_jwt.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
import os
import sys
import time

from jwcrypto.jwk import JWK
from jwcrypto.jwt import JWT


def get_private_key_pem(private_key_pem=None):
"""Retrieve private key PEM string from parameter or JWT_PKEY environment variable."""
pkey = private_key_pem or os.getenv("JWT_PKEY")
if pkey:
return pkey

raise ValueError(
"Private key not found. Please set JWT_PKEY environment variable."
)


def generate_jwt_token(
private_key_pem=None,
audience="ci-test-service",
issuer="mydomain.com",
subject="ci-test",
expiration_seconds=600, # Valid for 10 minutes by default
output_file=None,
):
"""Generate a signed JWT token valid for a few minutes using the private key."""
private_key_pem = get_private_key_pem(private_key_pem)
jwk_key = JWK.from_pem(private_key_pem.encode("utf-8"))

now = int(time.time())
payload = {
"iss": issuer,
"sub": subject,
"aud": audience,
"iat": now,
"exp": now + expiration_seconds,
}

header = {
"alg": "RS256",
"typ": "JWT",
"kid": jwk_key.key_id,
}

token = JWT(header=header, claims=payload)
token.make_signed_token(jwk_key)
jwt_str = token.serialize()

if output_file:
with open(output_file, "w", encoding="utf-8") as f:
f.write(jwt_str)

return jwt_str


if __name__ == "__main__":
out_file = sys.argv[1] if len(sys.argv) > 1 else "jwt.token"
generate_jwt_token(output_file=out_file)
2 changes: 2 additions & 0 deletions .github/scripts/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
jwcrypto==1.6.0

15 changes: 12 additions & 3 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@

test-with-cloud:
runs-on: ubuntu-latest
needs: test-jdbc-driver
needs: compile
strategy:
matrix:
# most recent LTS releases as well as latest stable builds
Expand Down Expand Up @@ -267,13 +267,22 @@
run: |
echo "Creating database: \`$TEST_DB_NAME\`"
curl --fail-with-body -s -u "default:$CLICKHOUSE_CLOUD_PASSWORD" "https://$CLICKHOUSE_CLOUD_HOST:8443" --data-binary "CREATE DATABASE IF NOT EXISTS $TEST_DB_NAME"
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install dependencies
run: python3 -m pip install -r .github/scripts/requirements.txt

Check warning on line 275 in .github/workflows/build.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Using dependencies without locking resolved versions is security-sensitive.

See more on https://sonarcloud.io/project/issues?id=ClickHouse_clickhouse-java&issues=AaCmVRhrizdXRJ_ii6U8&open=AaCmVRhrizdXRJ_ii6U8&pullRequest=3099
Comment thread
chernser marked this conversation as resolved.
Dismissed
- name: Test http client
env:
CLICKHOUSE_CLOUD_HOST: ${{ secrets.INTEGRATIONS_TEAM_TESTS_CLOUD_HOST_SMT_PROD }}
CLICKHOUSE_CLOUD_PASSWORD: ${{ secrets.INTEGRATIONS_TEAM_TESTS_CLOUD_PASSWORD_SMT_PROD }}
CLIENT_JWT: ${{ secrets.INTEGRATIONS_TEAM_TESTS_CLOUD_JWT_DESERT_VM_43 }}
JWT_TEST_HOST: ${{ secrets.INTEGRATIONS_TEAM_TESTS_CLOUD_HOST_SMT }}
JWT_PKEY: ${{ secrets.JWT_PKEY }}
run: |
python3 .github/scripts/generate_jwt.py jwt.token
Comment thread
cursor[bot] marked this conversation as resolved.
JWT_TOKEN="$(cat jwt.token)"
rm -f jwt.token
export JWT_TOKEN
mvn --batch-mode --no-transfer-progress --projects ${{ matrix.project }} -DclickhouseVersion=${{ matrix.clickhouse }} -Dprotocol=http -Dmaven.javadoc.skip=true verify
- name: Upload test results
uses: actions/upload-artifact@v4
Expand Down
27 changes: 27 additions & 0 deletions client-v2/src/test/java/com/clickhouse/client/ClientTests.java
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.testng.Assert;
import org.testng.SkipException;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
import org.testng.util.Strings;
Expand Down Expand Up @@ -823,6 +824,32 @@
}
}

@Test(groups = { "integration" })
public void testJWTWithCloud() throws Exception {
String jwt = System.getenv("JWT_TOKEN");
if (jwt == null || jwt.trim().isEmpty()) {
throw new SkipException("JWT_TOKEN environment variable is not set. Skipping JWT test.");
}
ClickHouseNode node = getServer(ClickHouseProtocol.HTTP);
Assert.assertFalse(jwt.contains("\n") || jwt.contains("-----"), "JWT should be single string ready for HTTP header");
try (Client client = new Client.Builder()
.addEndpoint(Protocol.HTTP, node.getHost(), node.getPort(), isCloud())
.compressClientRequest(false)
.setDefaultDatabase(ClickHouseServerForTest.getDatabase())
.serverSetting(ServerSettings.WAIT_END_OF_QUERY, "1")
.useBearerTokenAuth(jwt).build()) {
try {
List<GenericRecord> response = client.queryAll("SELECT currentUser()");
String username = response.get(0).getString(1);
Assert.assertTrue(username != null && username.matches("^JWT::.+::.+$"),

Check warning on line 844 in client-v2/src/test/java/com/clickhouse/client/ClientTests.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Simplify this regular expression to reduce its runtime, as it has super-linear performance due to backtracking.

See more on https://sonarcloud.io/project/issues?id=ClickHouse_clickhouse-java&issues=AaBpR5gOy_01LXV_QFv7&open=AaBpR5gOy_01LXV_QFv7&pullRequest=3099
"Expected username in format JWT::<subject>::<claims_hash>, but actual username was: '" + username + "'");
} catch (Exception e) {
e.printStackTrace();
throw e;
}
}
}

protected Client.Builder newClient() {
ClickHouseNode node = getServer(ClickHouseProtocol.HTTP);
boolean isSecure = isCloud();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1562,33 +1562,6 @@ public void testSetCredentialsAfterClientCreation() throws Exception {
}
}

@Test(groups = { "integration" })
public void testJWTWithCloud() throws Exception {
if (!isCloud()) {
return; // only for cloud
}
final String jwt = System.getenv("CLIENT_JWT");
final String host = System.getenv("JWT_TEST_HOST");
Assert.assertTrue(jwt != null && !jwt.trim().isEmpty(), "CLIENT_JWT is not set.");
Assert.assertTrue(host != null && !host.trim().isEmpty(), "JWT_TEST_HOST is not set");
Assert.assertFalse(jwt.contains("\n") || jwt.contains("-----"), "JWT should be single string ready for HTTP header");
try (Client client = new Client.Builder()
.addEndpoint(Protocol.HTTP, host, 8443, true)
.setUsername("default")
.compressClientRequest(false)
.setDefaultDatabase("default")
.serverSetting(ServerSettings.WAIT_END_OF_QUERY, "1")
.useBearerTokenAuth(jwt).build()) {
try {
List<GenericRecord> response = client.queryAll("SELECT user(), now()");
System.out.println("response: " + response.get(0).getString(1) + " time: " + response.get(0).getString(2));
} catch (Exception e) {
e.printStackTrace();
throw e;
}
}
}

@Test(groups = { "integration" })
public void testWithDefaultTimeouts() {
if (isCloud()) {
Expand Down
9 changes: 9 additions & 0 deletions docs/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,15 @@ client.updateAccessToken("Bearer new_access_token"); // -> Authorization: Bearer
> Note: as with username/password, a client built without an access token cannot
> be promoted to token-based authentication at runtime.

### JWT Authentication (ClickHouse Cloud)

> **Main Documentation:** See [JWT Authentication in ClickHouse Cloud](https://clickhouse.com/docs/concepts/features/security/external-authenticators/jwt) and [clickhouse-docs/client.mdx#jwt-authentication](clickhouse-docs/client.mdx#jwt-authentication).

- **Cloud-Only Feature:** JWT authentication is exclusive to **ClickHouse Cloud**. ClickHouse Cloud dynamically creates **ephemeral users** derived from claims embedded in each token.
- **User-to-Service Authentication:** JWT authentication is designed for **user-to-service** authentication mainly.
- **Service-to-Service Recommendation:** Using JWT for **service-to-service** communication is **not recommended** because JWT tokens have a short lifespan and require frequent refreshing.
- **Runtime Refresh:** In Java Client V2, tokens can be updated at runtime on an existing `Client` instance using `client.updateBearerToken(newJwtToken)`. In JDBC, there is no standard JDBC mechanism to update tokens on active connections.

---

## 3. Mutual TLS (mTLS) / Client Certificates
Expand Down
53 changes: 47 additions & 6 deletions docs/clickhouse-docs/client.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -85,11 +85,11 @@ Authentication by a password requires setting user name password by calling `set
.build();
```

Authentication by an access token requires setting access token by calling `setAccessToken(String)`:
Authentication by an access token or Bearer token (such as a JWT) requires setting the token by calling `useBearerTokenAuth(String)` (or `setAccessToken(String)` for raw access tokens):
```java showLineNumbers
Client client = new Client.Builder()
.addEndpoint("https://clickhouse-cloud-instance:8443/")
.setAccessToken(userAccessToken)
.useBearerTokenAuth(userAccessToken)
.build();
```

Expand Down Expand Up @@ -281,7 +281,7 @@ Configuration is defined during client creation. See `com.clickhouse.client.api.
| `setSharedOperationExecutor(ExecutorService executorService)` | `executorService` - executor service instance | Sets executor service for operation tasks. | `none` | `none` |
| `setQueryIdGenerator(Supplier<String> supplier)` | `supplier` - a `Supplier<String>` that generates query IDs | Sets a custom query ID generator used when no query ID is specified in operation settings (`InsertSettings`, `QuerySettings`). | - | - |
| `setClientNetworkBufferSize(int size)` | `size` - size in bytes | Sets size of a buffer in application memory space that is used to copy data between socket and application. | `300000` | `client_network_buffer_size` |
| `allowBinaryReaderToReuseBuffers(boolean reuse)` | `reuse` - flag to enable/disable | If enabled, reader will use preallocated buffers to do numbers transcoding. Reduces GC pressure for numeric data. | - | - |
| `allowBinaryReaderToReuseBuffers(boolean reuse)` | `reuse` - flag to enable/disable | If enabled, reader will use pre-allocated buffers to do numbers transcoding. Reduces GC pressure for numeric data. | - | - |
| `columnToMethodMatchingStrategy(ColumnToMethodMatchingStrategy strategy)` | `strategy` - matching strategy implementation | Sets custom strategy to be used for matching DTO class fields and DB columns when registering DTO. | `none` | `none` |
| `setClientName(String clientName)` | `clientName` - application name string | Sets additional information about calling application. Will be passed as `User-Agent` header. | - | `client_name` |
| `registerClientMetrics(Object registry, String name)` | `registry` - Micrometer registry instance<br/>`name` - metrics group name | Registers sensors with Micrometer (https://micrometer.io/) registry instance. | - | - |
Expand Down Expand Up @@ -406,6 +406,47 @@ try (Client client = new Client.Builder()

To configure per operation, set the header on `QuerySettings`, `InsertSettings`, or `CommandSettings` instead of the client builder.

### JWT Authentication (CLOUD) {#jwt-authentication}

[JWT Authentication](/concepts/features/security/external-authenticators/jwt) allows authenticating users with JSON Web Tokens (JWTs) in ClickHouse Cloud. Instead of pre-existing database users, ClickHouse dynamically creates **ephemeral users** derived from claims embedded in each token.

#### Initial Configuration

To configure initial JWT authentication on the client, use `useBearerTokenAuth(token)` (this automatically adds `Bearer_` prefix.
(see [HTTP interface docs](/concepts/features/security/external-authenticators/jwt#http-interface) ) when building the `Client` instance:

```java showLineNumbers
import com.clickhouse.client.api.Client;

public Client createJwtClient(String jwtToken) {
return new Client.Builder()
.addEndpoint("https://your-service.clickhouse.cloud:8443")
.useBearerTokenAuth(jwtToken)
.build();
}
```

<Note>
`useBearerTokenAuth(token)` automatically prepends the `Bearer ` prefix to the `Authorization` HTTP header.
</Note>

#### Updating Token at Runtime

Because JWTs are short-lived and ephemeral users exist only for the token's lifetime, new JWT tokens should be set **before** the current token expires so no operations fail.

You can update the bearer token on an existing `Client` instance at runtime without recreating the client:

```java showLineNumbers
import com.clickhouse.client.api.Client;

public void refreshJwtToken(Client client, String newJwtToken) {
// Set the new JWT before the current token expires
client.updateBearerToken(newJwtToken);
}
```

All subsequent requests on the `Client` instance will use the updated JWT token.

## Common Definitions {#common-definitions}

### ClickHouseFormat {#clickhouseformat}
Expand Down Expand Up @@ -627,7 +668,7 @@ There are differences in behavior depending on ClickHouse and Client version:
- Client &lt; `0.11.0` &amp; ClickHouse &lt; `26.8` - `FORMAT` clause has priority over request format header.
- Client &gt; `0.11.0` &amp; ClickHouse &gt; `26.8` - request header has priority over `FORMAT` clause.
- Client sets header when no format is specified in `QuerySettings` - this doesn't allow to override format in query.
- Client starting `0.11.0` - sets default `format` value on client level and not on operation level. This allows existig code
- Client starting `0.11.0` - sets default `format` value on client level and not on operation level. This allows existing code
work without changes and new code to use `FORMAT` clause by settings `format` on client to `null`.

### query(String sqlQuery) {#querystring-sqlquery}
Expand Down Expand Up @@ -901,7 +942,7 @@ can be used with any other custom reader implementation. `QueryResponse` also pr
- `getStringArray(...)` - for `Array(String)` (and enum values represented as names).
- `getObjectArray(...)` - generic option for any `Array(...)` element type, including nested arrays. Use to read arrays with nullable values and nested arrays.

Index-based and name-based overloads are available for all methods. Index is 1-based. Index-based do dirrect access to a column.
Index-based and name-based overloads are available for all methods. Index is 1-based. Index-based do direct access to a column.
Name-based methods require index lookup each time.

```java
Expand All @@ -926,7 +967,7 @@ try (QueryResponse response = client.query("SELECT * FROM my_table").get()) {
- `getStringArray(...)` - for `Array(String)` (and enum values represented as names).
- `getObjectArray(...)` - generic option for any `Array(...)` element type, including nested arrays. Use to read arrays with nullable values and nested arrays.

Index-based and name-based overloads are available for all methods. Index is 1-based. Index-based do dirrect access to a column.
Index-based and name-based overloads are available for all methods. Index is 1-based. Index-based do direct access to a column.
Name-based methods require index lookup each time.

```java
Expand Down
49 changes: 49 additions & 0 deletions docs/clickhouse-docs/jdbc.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,55 @@ Connection conn = Driver.connect("jdbc:ch:https://your-service.clickhouse.cloud:
// jdbc:ch:https://your-service.clickhouse.cloud:8443?http_header_X-CLICKHOUSE-REPLICA-TAG=my-app-session-1
```

### JWT Authentication (CLOUD) {#jwt-authentication}

[JWT Authentication](/concepts/features/security/external-authenticators/jwt) allows authenticating with JSON Web Tokens (JWTs) in ClickHouse Cloud. ClickHouse dynamically creates **ephemeral users** derived from token claims.

#### Initial Configuration

To configure initial JWT authentication in JDBC, pass the `bearer_token` property in connection properties or directly in the JDBC URL:

```java showLineNumbers
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.util.Properties;

public Connection createJwtConnection(String jwtToken) throws SQLException {
Properties properties = new Properties();
properties.setProperty("bearer_token", jwtToken);

return DriverManager.getConnection("jdbc:ch:https://your-service.clickhouse.cloud:8443", properties);
}
```

URL equivalent:
```plaintext
jdbc:ch:https://your-service.clickhouse.cloud:8443?bearer_token=<your_jwt_token>
```

#### Updating Token at Runtime

Because JWTs have a limited validity period, new JWT tokens should be set **before** the current token expires so no operations fail.

To update the JWT token on an active JDBC connection at runtime, unwrap the connection to access the underlying `com.clickhouse.client.api.Client` instance and call `updateBearerToken`:

```java showLineNumbers
import com.clickhouse.jdbc.ConnectionImpl;
import java.sql.Connection;
import java.sql.SQLException;

public void refreshJwtToken(Connection connection, String newJwtToken) throws SQLException {
// Unwrap the connection to get the underlying Client instance
ConnectionImpl connImpl = connection.unwrap(ConnectionImpl.class);

// Set the new JWT before the current token expires
connImpl.getClient().updateBearerToken(newJwtToken);
}
```

Subsequent statements executed on this JDBC connection will use the updated JWT token.

## Supported data types {#supported-data-types}

JDBC driver supports the same data formats as the underlying [java client](/integrations/language-clients/java/index#supported-data-types).
Expand Down
Loading
Loading