Repository navigation
Conversation
Resource center deletion passes recursive=true, but OssStorageOperator only removed the directory marker and left child objects in the bucket.
SbloodyS
left a comment
There was a problem hiding this comment.
Please follow the pull request template first.
|
Thanks for the review. I've updated the PR description to follow the pull request template. |
SbloodyS
left a comment
There was a problem hiding this comment.
Allow Resource Center to delete OSS directories without a marker
The old implementation can leave child objects behind after deleting the
directory marker. Resource Center still lists the directory as a common
prefix, but DeleteResourceDtoValidator calls OssStorageOperator.exists(),
which checks only the marker. After upgrading, the delete request still fails
before reaching the new recursive deletion code.
For recursive OSS deletes, treat a prefix with child objects as an existing
directory. Add a regression test through the Resource Center delete path with
a child object and no directory marker.
Surface partial OSS batch deletion failures
OssStorageOperator.deleteObjectsByPrefix() enables quiet mode and discards
the batch result. If OSS leaves a child object undeleted, Resource Center can
report success while the directory remains visible.
Use verbose mode, compare the confirmed keys with the requested keys, and
report a failure that identifies the unconfirmed keys. Add a regression test
where OSS confirms only a subset of the batch.
Was this PR generated or assisted by AI?
YES. Both the code and the PR description were AI-assisted.
Purpose of the pull request
Deleting a non-empty directory in the resource center with
resource.storage.type=OSSonly removed the directory marker. Child objects stayed in the bucket, so the directory prefix was still listed.Fixes #18780.
Brief change log
recursiveis true,OssStorageOperator#deletelists every object under<path>/and deletes them withdeleteObjectsin pages of 1000.recursive=falsestill delete a single object.recursiveindelete.Verify this pull request
This change added tests and can be verified as follows:
OssStorageOperatorTestwith a mocked OSS client (5 tests passed)../mvnw -pl dolphinscheduler-storage-plugin/dolphinscheduler-storage-oss spotless:check.Pull Request Notice
Pull Request Notice
If your pull request contains incompatible change, you should also add it to
docs/docs/en/guide/upgrade/incompatible.md