Skip to content

feat(go): add stream/topic purge API with BDD tests and fix server purge bug - #3608

Open
chengxilo wants to merge 10 commits into
apache:masterfrom
chengxilo:feat/go-sdk-purge-stream-topic
Open

feat(go): add stream/topic purge API with BDD tests and fix server purge bug#3608
chengxilo wants to merge 10 commits into
apache:masterfrom
chengxilo:feat/go-sdk-purge-stream-topic

Conversation

@chengxilo

Copy link
Copy Markdown
Contributor

Which issue does this PR address?

Relates to #2148

Rationale

Go SDK doesn't have the purge topic/ purge stream API, so this PR implement them. To test it, I added an new BDD test scenario, and implemented both Go and Rust BDD tests.

What changed?

The Go SDK was missing PurgeStream and PurgeTopic methods. Added them along with a shared BDD scenario (stream_topic_purge.feature) and implementations in both Go and Rust.

While testing, discovered that purge_all_segments only cleared on-disk segments but left stale messages in the journal and in-flight tiers. Polling after purge would return messages that should have been deleted. Fixed by calling journal.reset() and clear_in_flight() during partition re-initialization. I am not famliar with Rust server code so during the code review this part worth more attention

Local Execution

  • Passed
  • Pre-commit hooks ran

AI Usage

  1. DeepSeek pro v4, Claude Opus 4.6
  2. DeepSeek pro v4 for BDD test implementation, Claude Opus 4.6 for server bug fixing.
  3. I reviewed them with my best effert, the bdd scripts are executed and actually works, and they found that there is a bug in the server so I believe they works.
  4. I hope so.

@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

Thanks for the PR. It is labeled S-waiting-on-review and queued for review.

Slash commands (own line, regular comment) move it around the queue:

  • /ready - back to S-waiting-on-review after addressing feedback
  • /author - flip to S-waiting-on-author while you finish changes
  • /request-review @user-or-team - request a reviewer

See CONTRIBUTING.md for details.

@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Jul 4, 2026
@codecov

codecov Bot commented Jul 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 30.00000% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 40.18%. Comparing base (0604cca) to head (2064d1d).

Files with missing lines Patch % Lines
foreign/go/internal/command/stream.go 0.00% 4 Missing ⚠️
foreign/go/internal/command/topic.go 0.00% 4 Missing ⚠️
foreign/go/client/tcp/tcp_stream_management.go 0.00% 3 Missing ⚠️
foreign/go/client/tcp/tcp_topic_management.go 0.00% 3 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             master    #3608       +/-   ##
=============================================
- Coverage     76.40%   40.18%   -36.23%     
  Complexity     1046     1046               
=============================================
  Files          1334     1332        -2     
  Lines        165228   141379    -23849     
  Branches     137583   113798    -23785     
=============================================
- Hits         126242    56807    -69435     
- Misses        35271    81692    +46421     
+ Partials       3715     2880      -835     
Components Coverage Δ
Rust Core 30.83% <100.00%> (-45.58%) ⬇️
Java SDK 63.67% <ø> (ø)
C# SDK 71.14% <ø> (-1.13%) ⬇️
Python SDK 88.14% <ø> (ø)
PHP SDK 82.97% <ø> (ø)
Node SDK 96.36% <ø> (+0.08%) ⬆️
Go SDK 42.96% <0.00%> (-0.13%) ⬇️
Files with missing lines Coverage Δ
core/server/src/shard/system/segments.rs 89.43% <100.00%> (+0.54%) ⬆️
core/server/src/streaming/partitions/journal.rs 86.04% <100.00%> (+0.68%) ⬆️
foreign/go/client/tcp/tcp_stream_management.go 76.31% <0.00%> (-6.55%) ⬇️
foreign/go/client/tcp/tcp_topic_management.go 76.78% <0.00%> (-4.35%) ⬇️
foreign/go/internal/command/stream.go 83.78% <0.00%> (-10.16%) ⬇️
foreign/go/internal/command/topic.go 87.17% <0.00%> (-4.72%) ⬇️

... and 455 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@chengxilo chengxilo changed the title test(sdk): add stream/topic purge scenario, go/rust implemented, one server purge bug fixed feat(go): add stream/topic purge API with BDD tests and fix server purge bug Jul 4, 2026
@hubcio

hubcio commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

@chengxilo i'll check this during the weekend. could you please rebase?

@hubcio hubcio added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Jul 9, 2026
@numinnex

numinnex commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

I've checked whether there is potential concurrency issue due to interleaving of flush + purge commands as this is what I've reckon from the changes could potentially be an issue and did not find any problems with that. Let's wait for @hubcio review and we can merge it.

@chengxilo
chengxilo force-pushed the feat/go-sdk-purge-stream-topic branch from 8bc60d8 to f5d74d9 Compare July 14, 2026 02:08
@chengxilo

Copy link
Copy Markdown
Contributor Author

/ready

@github-actions github-actions Bot added S-waiting-on-review PR is waiting on a reviewer and removed S-waiting-on-author PR is waiting on author response labels Jul 14, 2026
@chengxilo

Copy link
Copy Markdown
Contributor Author

nothing has been changed, just because I rebased onto master locally so it was forced pushed() I shouldn't do that tho.

@chengxilo
chengxilo force-pushed the feat/go-sdk-purge-stream-topic branch from 9b5e79c to a604653 Compare July 25, 2026 22:29
@hubcio

hubcio commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

@chengxilo could you please rebase again? i will review this today

@chengxilo

chengxilo commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

@chengxilo could you please rebase again? i will review this today

done. Currently, we get the pre-purge messages back for roughly > a second. I'm not sure if we should justify it in BDD test or fix it. Previously for the non-VSR it was fixed. But not for the VSR enabled one. (since according to the comments, it was intended)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review PR is waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants