Add number of affected items to Exporter error logging - #8780
Conversation
|
|
Pull request dashboard statusWaiting on the author · refreshed 2026-09-09 09:45 UTC Wait for the required status checks to report; this pull request moves to reviewers once the results are clean. Status above doesn't look right?
|
bcc967c to
c3c2864
Compare
c3c2864 to
d37eaf7
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8780 +/- ##
============================================
- Coverage 91.29% 91.28% -0.01%
- Complexity 10498 10516 +18
============================================
Files 1006 1007 +1
Lines 28338 28386 +48
Branches 3581 3591 +10
============================================
+ Hits 25870 25911 +41
- Misses 1675 1682 +7
Partials 793 793 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
psx95
left a comment
There was a problem hiding this comment.
A couple of nits. Additionally, just confirming - IIUC, this change won't account for partial failures - did you mean to explicitly handle the case for partial failures?
Your change description mentioned assessing the impact of failures. if the failure logs appear, it means all the spans were rejected - is it helpful to know the number of spans being rejected in that case?
Let me know if you think I am missing something.
| logger.log( | ||
| Level.SEVERE, | ||
| "Failed to export " | ||
| + numItems |
There was a problem hiding this comment.
nit: I would prefer that the number of items are appended to towards the end of the log.
This could avoid breaking any user queries who might be relying on existing log message regex.
Failed to export spans. Server is UNAVAILABLE. Make sure your collector is running and reachable from this network. Full error message: Some status description. Failed to export 10 spans."
Doesn't necessarily have to be at the end, but IMO adding it as a separate sentence might be better.
There was a problem hiding this comment.
good point with the regexes, added it as separate sentences :)
|
@psx95 thank you for your review! yes, here i only target full failures - the scenario i was debugging was observing a high number of incomplete traces, and i wanted a way to tell, just from the logs, whether the cause was in the transport layer, or in code implementation. high number of failed items would indicate transport layer, low number would mean i need to look into the implementation instead i see there is an issue to provide better handling for partial success - #4706 - i'll be happy to take that as a followup :) |
The change you are making here - only adds one additional piece of information - the number of failed export items, it does not give any other (new) information about the cause/location of error. Let me know if this makes sense or if I misunderstood your use-case here. |
| logger.log( | ||
| Level.SEVERE, "Failed to export " + type + "s. The request could not be executed.", e); | ||
| Level.SEVERE, | ||
| "Failed to export " |
There was a problem hiding this comment.
Why not just add the number of failed items here?
| "Failed to export " | |
| "Failed to export " + numItems + " " |
There was a problem hiding this comment.
I had a minor concern about this - #8780 (comment), but I'm curious to hear your thoughts on this.
There was a problem hiding this comment.
I think its a slippery slope to try to consider log messages part of our public API surface area.
There was a problem hiding this comment.
Sounds good, I did not have a strong opinion on this.
cc: @gniadeck
@psx95 you're right that it doesn't point to the cause or location of the error. however, the goal here is just to surface the magnitude of the data loss knowing the count allows us to correlate a specific export error with the volume of missing data on the tracing server. as i wrote in the description, it follows the approach already set by |
Yup, I don't see any issues in committing this change, just wanted to verify the use-case here. |
when debugging exported traces, it was hard for me to asses impact the issue, because it was not known how many spans weren't exported. this is similar to what was added to
BatchSpanProcessorin #8167 . here the change was even simpler - we just push down the value we already had in exporter