Skip to content

Add number of affected items to Exporter error logging - #8780

Open
gniadeck wants to merge 4 commits into
open-telemetry:mainfrom
gniadeck:exporter-log-numitems
Open

Add number of affected items to Exporter error logging#8780
gniadeck wants to merge 4 commits into
open-telemetry:mainfrom
gniadeck:exporter-log-numitems

Conversation

@gniadeck

@gniadeck gniadeck commented Sep 8, 2026

Copy link
Copy Markdown

when debugging exported traces, it was hard for me to asses impact the issue, because it was not known how many spans weren't exported. this is similar to what was added to BatchSpanProcessor in #8167 . here the change was even simpler - we just push down the value we already had in exporter

@gniadeck
gniadeck requested a review from a team as a code owner September 8, 2026 10:41
@linux-foundation-easycla

linux-foundation-easycla Bot commented Sep 8, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: gniadeck / name: gniadeck (c3c2864)

@opentelemetry-pr-dashboard

opentelemetry-pr-dashboard Bot commented Sep 8, 2026

Copy link
Copy Markdown

Pull request dashboard status

Waiting on the author · refreshed 2026-09-09 09:45 UTC

Wait for the required status checks to report; this pull request moves to reviewers once the results are clean.

Status above doesn't look right?
  • Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
  • Should this be with reviewers? Comment /dashboard route:reviewers to route it to them.
  • Anything wrong — including the routing? Report it with what you expected; it helps us improve the dashboard.

@gniadeck
gniadeck force-pushed the exporter-log-numitems branch from c3c2864 to d37eaf7 Compare September 8, 2026 10:47
@codecov

codecov Bot commented Sep 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 91.28%. Comparing base (c87b50e) to head (da59cbe).
⚠️ Report is 17 commits behind head on main.

Files with missing lines Patch % Lines
...telemetry/exporter/otlp/internal/GrpcExporter.java 66.66% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #8780      +/-   ##
============================================
- Coverage     91.29%   91.28%   -0.01%     
- Complexity    10498    10516      +18     
============================================
  Files          1006     1007       +1     
  Lines         28338    28386      +48     
  Branches       3581     3591      +10     
============================================
+ Hits          25870    25911      +41     
- Misses         1675     1682       +7     
  Partials        793      793              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@psx95 psx95 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of nits. Additionally, just confirming - IIUC, this change won't account for partial failures - did you mean to explicitly handle the case for partial failures?

Your change description mentioned assessing the impact of failures. if the failure logs appear, it means all the spans were rejected - is it helpful to know the number of spans being rejected in that case?

Let me know if you think I am missing something.

logger.log(
Level.SEVERE,
"Failed to export "
+ numItems

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I would prefer that the number of items are appended to towards the end of the log.
This could avoid breaking any user queries who might be relying on existing log message regex.

Failed to export spans. Server is UNAVAILABLE. Make sure your collector is running and reachable from this network. Full error message: Some status description. Failed to export 10 spans."

Doesn't necessarily have to be at the end, but IMO adding it as a separate sentence might be better.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point with the regexes, added it as separate sentences :)

@gniadeck

gniadeck commented Sep 9, 2026

Copy link
Copy Markdown
Author

@psx95 thank you for your review! yes, here i only target full failures - the scenario i was debugging was observing a high number of incomplete traces, and i wanted a way to tell, just from the logs, whether the cause was in the transport layer, or in code implementation. high number of failed items would indicate transport layer, low number would mean i need to look into the implementation instead

i see there is an issue to provide better handling for partial success - #4706 - i'll be happy to take that as a followup :)

@psx95

psx95 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

@psx95 thank you for your review! yes, here i only target full failures - the scenario i was debugging was observing a high number of incomplete traces, and i wanted a way to tell, just from the logs, whether the cause was in the transport layer, or in code implementation. high number of failed items would indicate transport layer, low number would mean i need to look into the implementation instead

i see there is an issue to provide better handling for partial success - #4706 - i'll be happy to take that as a followup :)

The change you are making here - only adds one additional piece of information - the number of failed export items, it does not give any other (new) information about the cause/location of error.
It is an extra piece of information, but I don't see how this information helps your debugging use-case.

Let me know if this makes sense or if I misunderstood your use-case here.

logger.log(
Level.SEVERE, "Failed to export " + type + "s. The request could not be executed.", e);
Level.SEVERE,
"Failed to export "

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just add the number of failed items here?

Suggested change
"Failed to export "
"Failed to export " + numItems + " "

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a minor concern about this - #8780 (comment), but I'm curious to hear your thoughts on this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its a slippery slope to try to consider log messages part of our public API surface area.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, I did not have a strong opinion on this.

cc: @gniadeck

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reverted to previous approach

@gniadeck

gniadeck commented Sep 9, 2026

Copy link
Copy Markdown
Author

The change you are making here - only adds one additional piece of information - the number of failed export items, it does not give any other (new) information about the cause/location of error. It is an extra piece of information, but I don't see how this information helps your debugging use-case.

Let me know if this makes sense or if I misunderstood your use-case here.

@psx95 you're right that it doesn't point to the cause or location of the error. however, the goal here is just to surface the magnitude of the data loss

knowing the count allows us to correlate a specific export error with the volume of missing data on the tracing server. as i wrote in the description, it follows the approach already set by BatchSpanProcessor, which logs the amount of dropped data. given that this variable is already available and costs nothing to include, i think it's just a good logging practice to surface it

@psx95

psx95 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

The change you are making here - only adds one additional piece of information - the number of failed export items, it does not give any other (new) information about the cause/location of error. It is an extra piece of information, but I don't see how this information helps your debugging use-case.
Let me know if this makes sense or if I misunderstood your use-case here.

@psx95 you're right that it doesn't point to the cause or location of the error. however, the goal here is just to surface the magnitude of the data loss

knowing the count allows us to correlate a specific export error with the volume of missing data on the tracing server. as i wrote in the description, it follows the approach already set by BatchSpanProcessor, which logs the amount of dropped data. given that this variable is already available and costs nothing to include, i think it's just a good logging practice to surface it

Yup, I don't see any issues in committing this change, just wanted to verify the use-case here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants