Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,20 @@

### New Features

- **[client-v2]** Added an OpenTelemetry implementation of the observability SPI.
`Client.Builder.setSpanRecorder(new OpenTelemetrySpanRecorder(openTelemetry))`
reports every client operation and every transport request as an OpenTelemetry `CLIENT` span: an operation span is
started as a child of the current OpenTelemetry context, so it joins the application's own trace, and each request
span - including one per retry - is a child of its operation span. Span names and attribute keys are the standard
ones of the SPI (the recorder derives them through `SpanSupport`), every value is recorded with the OpenTelemetry
attribute type that matches it, and a failure sets the span status to `ERROR` and is recorded as an OpenTelemetry
exception event next to the `error.type` and `db.response.status_code` attributes. The recorder reports to a
supplied `OpenTelemetry` instance, to a `Tracer` given to `new OpenTelemetrySpanRecorder(Tracer)`, or to
`GlobalOpenTelemetry` - read when a span is started - when constructed without arguments. Previously an application that wanted
OpenTelemetry spans had to write that mapping itself. The OpenTelemetry API is a compile-only dependency of
`client-v2`: the recorder is used only by an application that already provides `opentelemetry-api` at runtime, so
nothing is added to the classpath of a client that does not use it.
(https://github.com/ClickHouse/clickhouse-java/issues/2974)
- **[client-v2]** Added an observability SPI that lets an application observe client operations as spans.
`Client.Builder.setSpanRecorder(SpanRecorder)` registers a backend-agnostic recorder from the new
`com.clickhouse.client.api.observability` package: each operation (a query, a command or an insert - including
Expand All @@ -25,7 +39,7 @@
for every operation that starts. Previously the client exposed no hook for tracing, so an
application could not attribute a query or a retried request to its own trace. When no recorder is registered
nothing is recorded and no span-related work is done, so the default path is unchanged. An OpenTelemetry
implementation of the SPI follows in a separate module.
implementation of the SPI is available as `OpenTelemetrySpanRecorder`.
(https://github.com/ClickHouse/clickhouse-java/issues/2974)
- **[client-v2, jdbc-v2]** Added support for the `BFloat16` data type (ClickHouse `24.11+`). `BFloat16` columns are read as
Java `float` values (widening is lossless) and written from `float`/`Float` values, including through generic records, POJO
Expand Down
21 changes: 21 additions & 0 deletions client-v2/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,28 @@
<version>${guava.version}</version>
</dependency>

<!-- Compile-only: OpenTelemetrySpanRecorder is used only when the
application already provides the OpenTelemetry API at runtime. -->
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-api</artifactId>
<version>${opentelemetry.version}</version>
<scope>provided</scope>
</dependency>

<!-- Test Dependencies -->
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk</artifactId>
<version>${opentelemetry.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk-testing</artifactId>
<version>${opentelemetry.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,266 @@
package com.clickhouse.client.api.observability.otel;

import com.clickhouse.client.api.insert.InsertSettings;
import com.clickhouse.client.api.metrics.OperationMetrics;
import com.clickhouse.client.api.observability.Span;
import com.clickhouse.client.api.observability.SpanAttribute;
import com.clickhouse.client.api.observability.SpanRecorder;
import com.clickhouse.client.api.observability.SpanSupport;
import com.clickhouse.client.api.query.QuerySettings;
import com.clickhouse.client.api.transport.Endpoint;
import io.opentelemetry.api.GlobalOpenTelemetry;
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.api.trace.StatusCode;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.context.Context;

import java.util.concurrent.atomic.AtomicBoolean;

/**
* {@link SpanRecorder} that reports client operations and transport requests as OpenTelemetry spans.
* <p>
* It is registered like any other recorder:
* <pre>{@code
* Client client = new Client.Builder()
* .addEndpoint("http://localhost:8123")
* .setSpanRecorder(new OpenTelemetrySpanRecorder(openTelemetry))
* .build();
* }</pre>
* Every span is a {@link SpanKind#CLIENT} span and carries the client's standard name and
* attributes, which are derived by {@link SpanSupport} - so the recorded keys are the ones listed in
* {@link SpanAttribute} and mean the same as for every other recorder.
* <p>
* An operation span is started as a child of the {@linkplain Context#current() current context}, so
* it appears under the application's own span when the operation is started on a thread that has
* one. A request span is a child of the operation span it was started for. The recorder does not
* make any span current: the client hands the response to the caller before the response body is
* read, so a span is ended on a thread the recorder does not control.
* <p>
* Instances are thread-safe and can be shared by several clients.
*/
public class OpenTelemetrySpanRecorder implements SpanRecorder {

/**
* Default instrumentation scope name. It is reported for the spans of a recorder created by the
* no-argument constructor or by {@link #OpenTelemetrySpanRecorder(OpenTelemetry)}. A recorder
* created by {@link #OpenTelemetrySpanRecorder(Tracer)} reports the scope of the given tracer
* instead.
*/
public static final String INSTRUMENTATION_SCOPE_NAME = "com.clickhouse.client";

private final SpanSupport spanSupport = SpanSupport.DEFAULT;

Check warning on line 53 in client-v2/src/main/java/com/clickhouse/client/api/observability/otel/OpenTelemetrySpanRecorder.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Make this final field static too.

See more on https://sonarcloud.io/project/issues?id=ClickHouse_clickhouse-java&issues=AaAmbVdmnqQ_OWuWyFgc&open=AaAmbVdmnqQ_OWuWyFgc&pullRequest=3065

/**
* Tracer the spans are created with, or {@code null} when they are created with the tracer of the
* global OpenTelemetry instance, which is then read every time a span is started.
*/
private final Tracer tracer;

/**
* Creates a recorder that reports to the {@linkplain GlobalOpenTelemetry#get() global}
* OpenTelemetry instance. Use it when the application configures OpenTelemetry globally, for
* example through the OpenTelemetry Java agent or the autoconfigure SDK extension.
* <p>
* The global instance is read when a span is started, not here, so a client may be created before
* the application installs its OpenTelemetry SDK.
*/
public OpenTelemetrySpanRecorder() {
Comment thread
polyglotAI-bot marked this conversation as resolved.
Comment thread
polyglotAI-bot marked this conversation as resolved.
this.tracer = null;
}

/**
* Creates a recorder that reports to the given OpenTelemetry instance.
*
* @param openTelemetry - OpenTelemetry instance to report to; must not be {@code null}
*/
public OpenTelemetrySpanRecorder(OpenTelemetry openTelemetry) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be a base constructor.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ddce5b6OpenTelemetrySpanRecorder(OpenTelemetry) no longer has its own body: it delegates with this(tracerOf(openTelemetry)). The constructor that only assigns the field is OpenTelemetrySpanRecorder(Tracer), because the no-argument constructor must NOT read GlobalOpenTelemetry eagerly - it is documented (and pinned by testGlobalInstanceIsReadWhenSpanStartsNotWhenRecorderIsCreated) that a client may be built before the application installs its SDK. So the no-argument constructor leaves the tracer unset and getTracer() reads the global instance when a span starts. Tell me if you would rather have this(GlobalOpenTelemetry.get()) and drop the lazy behaviour.

this(tracerOf(openTelemetry));
}

/**
* Creates a recorder that reports to the given tracer. Use it to report the client's spans under
* an instrumentation scope of the application's choice.
*
* @param tracer - tracer to create spans with; must not be {@code null}
*/
public OpenTelemetrySpanRecorder(Tracer tracer) {
if (tracer == null) {
throw new IllegalArgumentException("tracer must not be null");
}
this.tracer = tracer;
}

private static Tracer tracerOf(OpenTelemetry openTelemetry) {
if (openTelemetry == null) {
throw new IllegalArgumentException("openTelemetry must not be null");
}
return openTelemetry.getTracer(INSTRUMENTATION_SCOPE_NAME);
}

/**
* Returns the tracer the next span is created with - the one given to this recorder, or the tracer
* of the global OpenTelemetry instance as it is installed now.
*
* @return tracer; never {@code null}
*/
protected Tracer getTracer() {
return tracer != null ? tracer : GlobalOpenTelemetry.get().getTracer(INSTRUMENTATION_SCOPE_NAME);
}

@Override
public Span startQuerySpan(QuerySettings settings, String sqlQuery, Endpoint endpoint) {
OpenTelemetrySpan span = startSpan(spanSupport.querySpanName(settings), Context.current());
spanSupport.fillQueryAttributes(span, settings, sqlQuery, endpoint);
return span;
}

@Override
public Span startInsertSpan(InsertSettings settings, String tableName, int batchSize, Endpoint endpoint) {
OpenTelemetrySpan span = startSpan(spanSupport.insertSpanName(settings, tableName), Context.current());
spanSupport.fillInsertAttributes(span, settings, tableName, batchSize, endpoint);
return span;
}

@Override
public Span startRequestSpan(Span operationSpan, String host, int port) {
OpenTelemetrySpan span = startSpan(spanSupport.requestSpanName(), parentContextOf(operationSpan));
spanSupport.fillRequestAttributes(span, host, port);
return span;
}

@Override
public void recordHttpStatus(Span requestSpan, int statusCode) {
spanSupport.recordHttpStatus(requestSpan, statusCode);
}

@Override
public void recordSuccess(Span operationSpan, OperationMetrics metrics) {
spanSupport.recordSuccess(operationSpan, metrics);
}

@Override
public void recordFailure(Span operationSpan, Throwable t) {
spanSupport.recordFailure(operationSpan, t);
recordException(operationSpan, t);
}

@Override
public void recordRequestFailure(Span requestSpan, Throwable t) {
spanSupport.recordRequestFailure(requestSpan, t);
recordException(requestSpan, t);
}

/**
* Records the failure itself as an OpenTelemetry exception event, so that its message and stack
* trace are reported next to the {@link SpanAttribute#ERROR_TYPE} attribute.
*
* @param span - span the failure was reported on
* @param t - failure, may be {@code null}
*/
protected void recordException(Span span, Throwable t) {
if (t != null && span instanceof OpenTelemetrySpan) {
((OpenTelemetrySpan) span).getSpan().recordException(t);
}
}

/**
* Starts a client span with the given name under the given parent context.
*
* @param spanName - name of the span
* @param parentContext - context the span is started under
* @return new span
*/
protected OpenTelemetrySpan startSpan(String spanName, Context parentContext) {
io.opentelemetry.api.trace.Span span = getTracer().spanBuilder(spanName)
.setSpanKind(SpanKind.CLIENT)
.setParent(parentContext)
.startSpan();
return new OpenTelemetrySpan(span, parentContext.with(span));
}

/**
* Returns the context a request span is started under - the context of its operation span, or the
* current context when the operation span was not created by this recorder.
*/
private static Context parentContextOf(Span operationSpan) {
return operationSpan instanceof OpenTelemetrySpan
? ((OpenTelemetrySpan) operationSpan).getContext()
: Context.current();
}

/**
* {@link Span} backed by an OpenTelemetry span.
*/
public static class OpenTelemetrySpan implements Span {

private final io.opentelemetry.api.trace.Span span;

private final Context context;

private final AtomicBoolean ended = new AtomicBoolean();

OpenTelemetrySpan(io.opentelemetry.api.trace.Span span, Context context) {
this.span = span;
this.context = context;
}

/**
* Returns the OpenTelemetry span this span records on.
*
* @return OpenTelemetry span
*/
public io.opentelemetry.api.trace.Span getSpan() {
return span;
}

/**
* Returns the context that holds this span. It is the parent context of the spans started for
* the same operation.
*
* @return context holding this span
*/
public Context getContext() {
return context;
}

@Override
public void setAttribute(String key, Object value) {
if (key == null || value == null) {
return;
}
if (value instanceof String) {
span.setAttribute(AttributeKey.stringKey(key), (String) value);
} else if (value instanceof Boolean) {
span.setAttribute(AttributeKey.booleanKey(key), (Boolean) value);
} else if (value instanceof Double || value instanceof Float) {
span.setAttribute(AttributeKey.doubleKey(key), ((Number) value).doubleValue());
} else if (value instanceof Number) {
span.setAttribute(AttributeKey.longKey(key), ((Number) value).longValue());
} else {
span.setAttribute(AttributeKey.stringKey(key), String.valueOf(value));
}
}

@Override
public void setError(String errorType) {
span.setStatus(StatusCode.ERROR);
if (errorType != null) {
span.setAttribute(AttributeKey.stringKey(SpanAttribute.ERROR_TYPE.getKey()), errorType);
}
}

@Override
public void end() {
if (ended.compareAndSet(false, true)) {
span.end();
}
}

@Override
public String toString() {
return "OpenTelemetrySpan[" + span.getSpanContext().getSpanId() + "]";
}
}
}
Loading
Loading