Repository navigation
Conversation
github-actions
Bot
added
dependencies
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #5376 +/- ##
============================================
- Coverage 92.59% 92.52% -0.07%
- Complexity 5043 5061 +18
============================================
Files 1252 1261 +9
Lines 53583 53955 +372
Branches 6672 6743 +71
============================================
+ Hits 49617 49924 +307
- Misses 2319 2353 +34
- Partials 1647 1678 +31
☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 379 | 0.232 | 25,290/33,182/33,182 us | 🟢 -13.3% / 🔴 +145.5% |
| 🔴 | bs=100 sw=10 sl=64 | 773 | 0.472 | 128,073/171,626/171,626 us | 🔴 +18.9% / 🔴 +83.0% |
| ⚪ | bs=1000 sw=10 sl=64 | 903 | 0.551 | 1,108,779/1,168,693/1,168,693 us | ⚪ within ±5% / 🔴 +29.2% |
Baseline details
Latest main 9d768b7 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 379 tuples/sec | 408 tuples/sec | 963.37 tuples/sec | -7.1% | -60.7% |
| bs=10 sw=10 sl=64 | MB/s | 0.232 MB/s | 0.249 MB/s | 0.588 MB/s | -6.8% | -60.5% |
| bs=10 sw=10 sl=64 | p50 | 25,290 us | 22,652 us | 10,998 us | +11.6% | +129.9% |
| bs=10 sw=10 sl=64 | p95 | 33,182 us | 38,262 us | 13,515 us | -13.3% | +145.5% |
| bs=10 sw=10 sl=64 | p99 | 33,182 us | 38,262 us | 16,850 us | -13.3% | +96.9% |
| bs=100 sw=10 sl=64 | throughput | 773 tuples/sec | 831 tuples/sec | 1,235 tuples/sec | -7.0% | -37.4% |
| bs=100 sw=10 sl=64 | MB/s | 0.472 MB/s | 0.507 MB/s | 0.754 MB/s | -6.9% | -37.4% |
| bs=100 sw=10 sl=64 | p50 | 128,073 us | 118,048 us | 87,833 us | +8.5% | +45.8% |
| bs=100 sw=10 sl=64 | p95 | 171,626 us | 144,387 us | 93,795 us | +18.9% | +83.0% |
| bs=100 sw=10 sl=64 | p99 | 171,626 us | 144,387 us | 103,718 us | +18.9% | +65.5% |
| bs=1000 sw=10 sl=64 | throughput | 903 tuples/sec | 916 tuples/sec | 1,273 tuples/sec | -1.4% | -29.1% |
| bs=1000 sw=10 sl=64 | MB/s | 0.551 MB/s | 0.559 MB/s | 0.777 MB/s | -1.4% | -29.1% |
| bs=1000 sw=10 sl=64 | p50 | 1,108,779 us | 1,095,722 us | 861,707 us | +1.2% | +28.7% |
| bs=1000 sw=10 sl=64 | p95 | 1,168,693 us | 1,113,056 us | 904,523 us | +5.0% | +29.2% |
| bs=1000 sw=10 sl=64 | p99 | 1,168,693 us | 1,113,056 us | 931,473 us | +5.0% | +25.5% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,527.28,200,128000,379,0.232,25290.08,33181.81,33181.81
1,100,10,64,20,2587.09,2000,1280000,773,0.472,128072.53,171625.91,171625.91
2,1000,10,64,20,22144.59,20000,12800000,903,0.551,1108779.26,1168693.39,1168693.39
Contributor
Automated Reviewer SuggestionsBased on the
|
zuozhiw
requested changes
Jul 14, 2026
Contributor
|
I left some comments in this PR, specifically in one of these comments I put a link to otel official demo app written in Java and we should follow official guidelines m. |
Ma77Ball
added a commit
to Ma77Ball/texera
that referenced
this pull request
Jul 16, 2026
…WorkflowMetrics rename Follow the OTel Java demo patterns instead of custom wrappers (zuozhiw review): - WorkflowService: start the run-level span with the standard OTel API, name it WorkflowService.initExecutionService, and drop the initExecutionServiceSpanned split so no span is passed as an argument. The real execution failure is now recorded onto the span from errorHandler, where it is actually caught. - TexeraTracer: drop the withSpan wrapper, keeping only the tracer accessor (single instrumentation scope) and currentContext. - SpanAttrs: drop the awkward with*/set* setter helpers, keep the standard label keys and sanitizeFreeText; callers set attributes via the OTel API. - Rename TexeraMetrics to WorkflowMetrics, document that this facade is only for the workflow-execution cluster, and add an example of calling the OTel meter API directly at a call site. Update SpanAttrsSpec and WorkflowMetricsSpec to match.
zuozhiw
approved these changes
Jul 30, 2026
zuozhiw
left a comment
zuozhiw
left a comment
Contributor
There was a problem hiding this comment.
changes look good overall, left a few more comments, after fixing them you can go ahead and merge
| ): Unit = { | ||
| val span = TexeraTracer.tracer | ||
| .spanBuilder("WorkflowService.initExecutionService") | ||
| .setAttribute("texera.workflow.id", workflowId.id.toString) |
Contributor
There was a problem hiding this comment.
minor: use standard keys instead of plain string
| state.withExecutionId(workflowContext.executionId) | ||
| ) | ||
| val errorHandler: Throwable => Unit = { t => | ||
| { |
Contributor
There was a problem hiding this comment.
this error handler needs to be extra careful becaue it's passed around as an argument to WorkflowExecutionService, and it will be called beyond this function's lifecycle, we really need to check who calls it. when they call it this specific span object we get here might already ended after this function ends. this handler is also used locally, that is fine, but if other places call it then we have troubles and the error is not attached to the correct span
this one is a bit tricky, as we need to inspect the places that invokes this error handler and see what spans they have, then we either call span.current() again inside this span (but need to make sure the callers has good spans...), or has to pass in a span as optinal argument, please check how we are using this handler and see what's the best way to handle this, maybe passing such error handler around is not a good error handling method
Ma77Ball
marked this pull request as ready for review
August 4, 2026 19:18
Contributor
|
This pull request has been automatically marked as stale because it has If this PR is still relevant, please:
To exempt this PR from auto-close, a maintainer can add the |
Introduce the OpenTelemetry foundation for Texera: an SDK bootstrap, a Logback-to-OTel log bridge, and a log sanitizer, wired into every service entry point. This is PR 1 of the observability stack; it ships logging only, with the trace and metric exporters wired but not yet emitted (those arrive in follow-up PRs). New module (common/observability): - OtelInit: one-call SDK bootstrap per service. Reads OTEL_* settings from observability.conf, validates the OTLP endpoint (scheme + host allowlist) before any exporter is built, wires the span/log/metric providers explicitly (no sdk-extension-autoconfigure), clamps the metric export interval, and attaches the log appender to the Logback ROOT logger. The whole init body is guarded so a missing or malformed config returns None with one WARN instead of throwing into a service's run(), even when telemetry is disabled. - TexeraOtelLogAppender: bridges Logback events to OTel log records. Maps severity, forwards MDC, sets exception.type/message/stacktrace semantic attributes, and drops records from io.opentelemetry loggers so export-failure diagnostics are not fed back to the collector that just failed. - LogSanitizer: strips C0 control characters, redacts secrets, and caps body length (MaxBodyChars) to keep individual records bounded. Config (common/config): - observability.conf with the OTEL_* defaults, ObservabilityConfig to read it, and ENV_OTEL_* entries in EnvironmentalVariable. Wiring: - build.sbt defines the Observability module and adds dependsOn(Observability) to the eight Dropwizard services. - Each service entry point calls OtelInit.init with its own service name, and each service config gains a logging block. Deployment: - OTEL_* env entries in bin/single-node/.env, bin/k8s/values.yaml, and bin/k8s/values-development.yaml (kept a name-for-name mirror). - LICENSE-binary manifests updated with the pinned OTel 1.50.0 jars. Tests: OtelInitSpec, TexeraOtelLogAppenderSpec, LogSanitizerSpec, and ObservabilityConfigSpec cover endpoint validation, interval clamping, severity/MDC/exception mapping, self-diagnostic filtering, redaction, and truncation. Rebased onto current main.
Build on the logging foundation (PR 1) with the backend emit path: workflow lifecycle metrics and a run-level setup trace span, plus the tracing/metrics primitives they use. This is PR 2 of the observability stack. Primitives (common/observability): - TexeraTracer: lazy accessor for the process tracer off GlobalOpenTelemetry. - SpanAttrs: typed AttributeKey constants so span attributes use standard keys rather than ad hoc strings. - WorkflowMetrics: the OTel instruments (start/completion/failure/cancellation counters and run-duration histogram) keyed by workflow kind. - TraceparentValidator: validates W3C traceparent headers before use. Emit path (amber): - WorkflowMetricsRecorder: single owner of the metric instruments. init() wires them once; onStart stamps a run's start; onStateChange records terminal counters and duration exactly once on the first transition into a terminal state (idempotent, safe to call on every transition). - WorkflowService.initExecutionService runs inside a run-level setup span so setup-path logs carry its trace id. The span covers only the synchronous setup; synchronous setup failures are recorded on it in the catch block, and the span is ended in finally. The async errorHandler deliberately does not touch the span: it is invoked after setup returns and the span has ended, so the failure is surfaced through the metadata store instead. - ExecutionStateStore.updateWorkflowState is the single chokepoint that feeds every state transition to WorkflowMetricsRecorder.onStateChange. - ComputingUnitMaster initializes the recorder at startup. All observability sources live under common/observability (the module from PR 1); the tracing/metrics classes are not duplicated into common/config. Tests: WorkflowMetricsSpec, SpanAttrsSpec, and TraceparentValidatorSpec. Review follow-ups addressed: span attributes use SpanAttrs keys instead of plain strings; the error handler no longer records onto a span that may have already ended (documented and handled via the metadata store). Stacked on PR 1 (obs/pr1/foundations).
Ma77Ball
force-pushed
the
obs/pr2/backend-emit
branch
from
October 7, 2026 11:04
2dee2bd to
b372b82
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
Adds the libraries services use to emit metrics and distributed traces. These stay dormant until the SDK is enabled in PR1.
TexeraMetrics: metric instruments for workflow lifecycle and throughput. The active-executions metric is modeled as an observable gauge read from the live registry, correcting the phantom-active count a manual up/down counter produced.TexeraTracerandSpanAttrs: helpers for creating spans and attaching standardized attributes.TraceparentValidator: parses and validates W3Ctraceparentheaders for context propagation.Configmodule; no behavior changes when telemetry is disabled.Any related issues, documentation, or discussions?
Closes: #5368
Part of #4070. Stacked on #5375.
How was this PR tested?
sbt scalafmtCheckAllpasses; compile and tests run in this PR's CI.Was this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Opus 4.8 in compliance with ASF