[Bug][Bitbucket] Extract Pull Requests fails on microsecond timestamp parsing #8708
Description
Search before asking
- I had searched in the issues and found no similar issues.
What happened
After upgrading to v1.0.3-beta9, Bitbucket PR sync fails during the Extract Pull Requests subtask with a time.ParseError.
The issue started occurring after merging PR:
It seems related to timestamp parsing changes introduced in that PR.
What do you expect to happen
Bitbucket timestamps with microseconds should be parsed correctly.
How to reproduce
Use DevLake v1.0.3-beta9
Configure a Bitbucket datasource (cloud)
Add a Bitbucket repository to a project
From the Dashboard, trigger Sync (or run the pipeline manually)
Anything else
No response
Version
v1.0.3-beta9
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Reacted by dosubot, vlxbot, Michael T Halligan and Jaime I.
Activity
dosubot commentedon Feb 11, 2026
The Bitbucket PR extraction failure is caused by Go's default time.Time parsing, which doesn't handle microsecond precision timestamps returned by Bitbucket Cloud (e.g.,
2025-12-01T12:34:56.123456Z). The extractor currently unmarshals thecreated_onandupdated_onfields directly into time.Time, leading to time.ParseError when microseconds are present.To fix this, update the BitbucketApiPullRequest struct to use the existing
common.Iso8601Timetype for these fields instead of time.Time. TheIso8601Timehelper provides flexible ISO8601 parsing and can be easily extended to support microseconds by adding a matcher/format for six-digit fractional seconds (or by falling back totime.RFC3339Nano) source.In summary: replace
with
and update downstream usage to call
.ToNullableTime()or.ToTime()as needed. This will make the extractor robust to all ISO8601 timestamp variants Bitbucket emits, including those with microseconds source.If you encounter any timestamp formats not currently handled, you can add a new regex/format pair to the
DateTimeFormatslist iniso8601time.gofor full coverage.To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.
EmyysJanaK commentedon Feb 28, 2026
@oguzgu Hi, I would like to take this
klesh commentedon Mar 2, 2026
@EmyysJanaK I just assigned the issue to you. Happy coding.
fix(bitbucket): handle microsecond precision timestamps in PR extractor
test(common): add test cases for millisecond and microsecond timestamps
JakubHelis commentedon Mar 13, 2026
I needed this fix for my own use case and decided to contribute it back rather than use it as private workaround. Feel free to use or discard as you see fit.
v1bh0r commentedon Apr 8, 2026
RCA Report — [Bug][Bitbucket] Extract Pull Requests fails on microsecond timestamp parsing
Generated using Flowcraft Skills
backend/plugins/bitbucket/tasks/pr_extractor.go,backend/core/models/common/iso8601time.go1. Bug Recap
Observed: After upgrading to v1.0.3-beta9, the Extract Pull Requests subtask in the Bitbucket plugin fails with a
time.ParseError. The entire PR sync pipeline is blocked.Expected: PRs with all timestamp precisions (seconds, milliseconds, microseconds) should be extracted without error.
Impact: 100% of Bitbucket PR syncs fail for any repository whose PRs have participant records with microsecond-precision
participated_ontimestamps (Bitbucket Cloud regularly returns 6-digit fractional seconds).When started: After merging PR #8668 (Dec 31, 2025), shipped in v1.0.3-beta9.
Suspected components at intake:
pr_extractor.go— theBitbucketApiPullRequeststruct fieldsBitbucketCreatedAt/BitbucketUpdatedAtiso8601time.go—ConvertStringToTimeregex/format matchingBitbucketParticipant.ParticipatedOn— newly-added field in PR feat: Collect Pull Request Reviewer Status #8668Linked PR (quarantine): GitHub #8765 — quarantined until PASS 5 per anti-bias protocol.
Prior case-files: No case-files directory existed in the repo —
No prior case-files found for this component.2. Hypothesis Table (Ranked)
Iso8601Time.ConvertStringToTimefails for microsecond timestamps with+HH:MMtimezone because the[\d]{3}[+-][\d]{2}:[\d]{2}$regex incorrectly matches the last 3 of 6 fractional digits, then attempts the 3-decimal-place format which failstime.Parserejects the 6-digit input against"2006-01-02T15:04:05.000-07:00""2025-12-01T12:34:56.123456+00:00"; format parse failsBitbucketCreatedAt time.Time/BitbucketUpdatedAt time.Timeuse Go's built-intime.Time.UnmarshalJSONwhich cannot handle microsecond timestamps from Bitbucketjson.Unmarshalcallstime.Parse(RFC3339, s)which may fail for 6-digit fractional stampsparticipants.participated_on(type*common.Iso8601Time) receives a microsecond timestamp → H1's bug triggers → entirejson.UnmarshalofBitbucketApiPullRequestfails → extractor errors outParticipantsfieldparticipated_onvaluesparticipantsflow+HH:MMtimezone stampsDateTimeFormatsDateTimeFormatsorderHypothesis completeness gate — all covered:
time.Timevs*Iso8601Time(H2)3. Evidence Ledger
BitbucketParticipant.ParticipatedOndeclared as*common.Iso8601Timeinpr_extractor.go— introduced in PR #8668DateTimeFormatsentry:[\d]{3}[+-][\d]{2}:[\d]{2}$→"2006-01-02T15:04:05.000-07:00". For input"2025-12-01T12:34:56.123456+00:00", the regex matches456+00:00(last 3 of 6 fractional digits + timezone). Format only handles 3 fractional digits;time.Parsefails.ConvertStringToTimefallback istime.Parse(time.RFC3339, s). ForZ-timezone microsecond stamps, this fallback succeeds via Go's implicit fractional-second handling — but it is NEVER reached for+HH:MMstamps because regex entry #2 matches first and fails.Iso8601Time.UnmarshalJSONreturns the error fromConvertStringToTimedirectly:if err != nil { return err }— error propagates tojson.Unmarshalpr_extractor.goline 106:err := errors.Convert(json.Unmarshal(row.Data, rawL))thenif err != nil { return nil, err }— fatal bail-out on any unmarshal errorBitbucketCreatedAt time.TimeandBitbucketUpdatedAt time.Timein the API struct — Go usestime.Time.UnmarshalJSONwhich callstime.Parse(RFC3339, ...). Go docs confirm RFC3339 handles implicit fractional seconds, so 6-digit Bitbucketcreated_on/updated_onstamps may succeed without error.timedocspipeline_extractor.gouses*common.Iso8601Timeforcreated_on/completed_oncorrectly — confirming the intended codebase pattern[\d]{3}[+-][\d]{2}:[\d]{2}$and its format string were present in the codebase since commit4250e11ab5(Jun 24, 2023, Liang Zhang). No 6-digit fractional format was ever added.pr_collector.goafter PR #8668 requestsvalues.participants.participated_onfrom Bitbucket API — this field was not requested before and triggers microsecond data. Changing this field request is what activated the latentIso8601Timebug.participated_ondata). The reporter's framing "started after merging PR #8668" suggests participant-only scenarios trigger it — this contradicts H2 as the root trigger.Mandatory falsification: At least one
Centry per hypothesis — satisfied above.4. Root Cause Analysis
Most Likely Root Cause
The bug is a latent defect in
ConvertStringToTimetriggered by PR #8668.Toyota 5 Whys (Leading Hypothesis H1/H3)
WHY 1 (Symptom): The
Extract Pull Requestssubtask fails withtime.ParseError.json.Unmarshalfails.WHY 2 (Context):
json.Unmarshalfails becauseIso8601Time.UnmarshalJSONreturns an error for aparticipated_onfield value like"2025-12-01T12:34:56.123456+00:00".UnmarshalJSONpropagates theConvertStringToTimeerror.WHY 3 (System):
ConvertStringToTimefails because the regex[\d]{3}[+-][\d]{2}:[\d]{2}$matches a microsecond timestamp (hitting the last 3 of 6 fractional digits), and the corresponding format"2006-01-02T15:04:05.000-07:00"only accepts exactly 3 decimal places.WHY 4 (Design): The
DateTimeFormatslist was designed only for millisecond (3-digit) fractional seconds with+HH:MMtimezone. No entry exists for 4–9 digit fractional seconds. The matching is positional (regex against the whole string), so the 3-digit regex incorrectly matches the tail of a 6-digit string.WHY 5 (Root Cause):
DateTimeFormatswas built to handle timestamps returned by the plugins in use at the time. Bitbucket Cloud's API was not exercising the microsecond path until PR #8668 addedparticipants.participated_onto the field projection. TheConvertStringToTimefunction lacks a format entry for 4–9 digit fractional seconds with±HH:MMtimezone, and the earlier, narrower regex causes a false match that blocks the RFC3339 fallback.Validation: Root cause → forward trace:
DateTimeFormatsmissing microsecond entryIso8601Time.UnmarshalJSONreturns errorjson.Unmarshal(row.Data, rawL)fails(nil, err)for every PR with microsecondparticipated_on4b. Authorship & Change Timeline
Implicated Files and Commits
iso8601time.goL59-62[\d]{3}regex +.000format4250e11ab5pr_extractor.goL88ParticipatedOn *common.Iso8601Time07bc58370pr_extractor.goL60-61BitbucketCreatedAt/UpdatedAt time.Timef527fb5689Introduction Hypothesis
The structural defect was most likely introduced in commit
4250e11ab5on June 24, 2023 by Liang Zhang, as part of adding a millisecond-precision timestamp format for TAPD/ZenTao timestamps using the+HH:MMtimezone notation. The intent was correct for 3-digit inputs, but the regex pattern[\d]{3}was not anchored to match EXACTLY three digits — it matches any occurrence of 3 consecutive digits just before the+HH:MMsuffix. This made it a false-positive matcher for 6-digit microsecond strings.The defect remained latent for over 2 years because no Bitbucket API field exercised the
+HH:MM+ microseconds combination until PR #8668 (Warren Chen, Dec 31, 2025) addedparticipants.participated_onto the field projection.Author Roster for Implicated Code
07bc58370)4250e11ab5)time.Timefields)f527fb5689)Primary contact: Warren Chen (most recent change to
pr_extractor.go, author of PR #8668).Likely introducer (forensic context only, not blame): Liang Zhang — introduced the 3-digit-only format that couldn't handle 6-digit inputs.
4c. Root Cause Category
Selected category:
Code Quality— A regex pattern with insufficient specificity caused incorrect matching, and the corresponding format string silently excluded a valid ISO8601 variant (microseconds). The defect went undetected due to absent test coverage for sub-second variants beyond milliseconds.
5a. 🔴 Must Have Corrective Actions
CA-1: Add microsecond-precision format to
DateTimeFormats(IMPLEMENTED)Change: In
backend/core/models/common/iso8601time.go, insert a newDateTimeFormatItemimmediately before the[\d]{3}[+-][\d]{2}:[\d]{2}$entry:{ // Microsecond / nanosecond precision (4+ fractional digits) with ±HH:MM timezone. // Must appear before the 3-digit millisecond entry so the longer match wins. Matcher: regexp.MustCompile(`[\d]{4,}[+-][\d]{2}:[\d]{2}$`), Format: "2006-01-02T15:04:05.999999999-07:00", },Why:
[\d]{4,}matches 4 or more fractional digits before+/-HH:MM. The"2006-01-02T15:04:05.999999999-07:00"format uses.9*which in Go means optional fractional digits up to 9. This handles millisecond (4+ digit edge cases), microsecond (6), and nanosecond (9) timestamps correctly.Why ordered before the 3-digit entry: Regex matching is first-match-wins. Without this ordering, the 3-digit entry would still fire on the tail of a 6-digit string.
Validation:
TestConvertStringToTime_FractionalSecondstest suite added and all 4 cases pass.Rollback: Remove the new entry; the behaviour reverts to the (broken) previous state.
CA-2: Change
BitbucketCreatedAt/BitbucketUpdatedAtto*common.Iso8601Timein PR extractor (IMPLEMENTED)Change: In
backend/plugins/bitbucket/tasks/pr_extractor.go:Update
convertBitbucketPullRequestto use nil-safe conversion:Why: Using raw
time.Timebypasses the codebase's standard ISO8601 handling and is inconsistent with the established pattern (seepipeline_extractor.goE7). If CA-1 alone is applied,created_on/updated_onwould still route throughtime.Time.UnmarshalJSONwhich relies on Go's implicit fractional-second behavior — version-dependent and fragile. Aligning with*common.Iso8601Timeis the correct, consistent pattern.Validation:
go build ./plugins/bitbucket/tasks/...succeeds. No existing tests broken.5b. 🟡 Good to Have Corrective Actions
CA-3: Apply same
*common.Iso8601Timefix to other Bitbucket extractorsThe following extractors also use
time.Timefor Bitbucketcreated_on/updated_onfields and are at risk if Bitbucket returns microsecond precision on those API endpoints:issue_extractor.go:52-53BitbucketCreatedAt time.Time,BitbucketUpdatedAt time.Timepr_comment_extractor.go:41-42CreatedOn time.Time,UpdatedOn *time.Timeissue_comment_extractor.go:33-34CreatedOn time.Time,UpdatedOn *time.TimeThese should be migrated to
*common.Iso8601Timefollowing the same pattern as CA-2 to prevent the same class of bug occurring in other subtasks.CA-4: Add test coverage gating
DateTimeFormatsbreadthAdd a CI-enforced test table in
iso8601time_test.gocovering all combinations of fractional precision (0, 1, 3, 6, 9 digits) × timezone format (+HH:MM,-HH:MM,+HHMM,Z). This prevents future regressions when new format entries are added.5c. Blast-Radius Analysis
CA-1 — Add microsecond entry to
DateTimeFormatsDependency scan:
ConvertStringToTimeis called fromIso8601Time.UnmarshalJSON(shared codebase-wide). All plugins that use*common.Iso8601Timeare affected. Search:Result: 60+ usages across all plugins (see
backend/plugins/*/tasks/).Behavioral delta before/after:
+HH:MM→ parse fails → error+HH:MM→ parse succeeds → correcttime.Timevalue+HH:MM): unchanged — the 3-digit entry still fires for those (new entry only fires for 4+)created_onfields also have 6-digit timestamps that were previously being skipped*common.Iso8601Timefields across all plugins.999999999produces unexpected precision for MarshalJSONIso8601Time.MarshalJSONuses the storedformat, it might re-serialize with 9 decimal placesIso8601Time.MarshalJSON→String()usesjt.formatjt.formatis set by the original match — the microsecond entry setsformat = "2006-01-02T15:04:05.999999999-07:00"which would serialize with trailing zeros stripped; acceptableConvertStringToTimecallsinit(); runtime evaluation is one extraMatchStringcallNet assessment for CA-1: Safe to ship. Change is additive (new list entry), does not alter existing format handling, and resolves a blocking bug.
CA-2 — Change
time.Timeto*common.Iso8601Timein PR extractorDependency scan:
Result:
pr_convertor.go:78readspr.BitbucketCreatedAt(typetime.Timefrom the modelmodels.BitbucketPullRequest) — not from the API struct. The model field type is unchanged; only the API/extractor struct changes. The convertor uses the model, not the API struct, so it is unaffected.created_onis missing from Bitbucket response*common.Iso8601Timepointer is nil;convertBitbucketPullRequestguards withif pull.BitbucketCreatedAt != nilcreated_onis possibly absenttime.Time{}(zero value)pr.BitbucketCreatedAtzero time stored for previously-valid PRstime.Time.UnmarshalJSONwas actually succeeding before (via RFC3339 implicit fractions), the switch to*Iso8601Timevia CA-1's new format still produces the correct value — no regressionbackend/plugins/bitbucket/e2e/pr_test.gotime.Timecolumns doesn't change at DB levelNet assessment for CA-2: Safe to ship. The change is scoped to the API response struct only; model types, DB schema, and downstream convertor are unchanged.
6. Preventive Actions
TestConvertStringToTime_FractionalSecondstoiso8601time_test.gocovering millisecond, microsecond, nanosecond × Z/+HH:MM timezones (IMPLEMENTED)time.Timefor timestamp fields — use*common.Iso8601Timeissue_extractor.go,pr_comment_extractor.go,issue_comment_extractor.gofor the sametime.TimeissueDateTimeFormatscomprehensive test matrix covering all precision × timezone combinations to prevent regression when the format list is extendedtasks/package README that all API timestamp fields must use*common.Iso8601Time8. PR Alignment Review
Activating PASS 5 — linked PR #8765 by JakubHelis.
PR Summary
PR #8765 "Fix/bitbucket microsecond timestamp" contains two commits:
6727b47—fix(bitbucket): handle microsecond precision timestamps in PR extractor4931204—test(common): add test cases for millisecond and microsecond timestampsIntent: Fix the
time.Timeusage inpr_extractor.goforcreated_on/updated_onby changing to*common.Iso8601Time, and add microsecond test cases.Alignment Assessment
DateTimeFormatsiso8601time.gochanges; exact diff not seen — if only the PR extractor struct changes without theiso8601time.gofix,participated_onwould still failtime.Time→*common.Iso8601Timein PR extractorOverall verdict: The independent RCA identifies the root trigger as
Iso8601Time.ConvertStringToTime's missing microsecond format, activated viaparticipated_on. PR #8765 appears to address the PR extractor struct change (CA-2). Whether it also fixes CA-1 (theiso8601time.goformat gap) depends on the actual diff. If PR #8765 only addressescreated_on/updated_onwithout fixing CA-1, theparticipated_onparsing path would remain broken.Recommendation: Verify PR #8765 includes both:
iso8601time.gomicrosecond format entry (CA-1), ORtime.RFC3339Nanoas the fallback inConvertStringToTimeThe fix implementation in this RCA addresses both CA-1 and CA-2 for complete coverage.
ROI Summary
iso8601time.goregex forensics + Go time format researchHuman Under Duress Counterfactual: A developer under production pressure, debugging a
time.ParseErrorwithout knowing about the regex first-match ordering issue or the trigger fromparticipated_on, would likely spend 4–6 hours before finding the correctDateTimeFormatsentry. They might also fix onlytime.Time→*Iso8601Time(CA-2) without realising theConvertStringToTimebug (CA-1) is the actual root cause, leaving the issue partially unresolved.fix(bitbucket): handle microsecond precision timestamps in PR extractor
fix(bitbucket): handle microsecond precision timestamps in PR extract…
fix(bitbucket): handle microsecond precision timestamps in PR extract…
Sync upstream Apache DevLake (v1.0.3-beta9 → beta15) (#22)