Skip to content

[Fix-17813][ApiServer][Part1] Treat the mask value echoed back by the frontend as an unmodified password when updating or testing the connection - #18737

Open
njnu-seafish wants to merge 13 commits into
apache:devfrom
njnu-seafish:Improvement-17813
Open

njnu-seafish wants to merge 13 commits into
apache:devfrom
njnu-seafish:Improvement-17813

Conversation

@njnu-seafish

@njnu-seafish njnu-seafish commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Was this PR generated or assisted by AI?

Yes, generate test cases by AI

Purpose of the pull request

Fix part of #17813 (the password-overwrite half; also reported by #18004).

The query APIs (queryDataSource, queryDataSourceListPaging) always return the password masked as Constants.XXXXXX (******). The edit form echoes this value back unchanged on save, and on dev updateDataSource only treats a blank
password as "not updated" — so the literal mask gets stored as the real password:

expected: <oldSecret> but was: <******>

That is exactly the #18004 failure mode on dev: edit a datasource, save without touching the password, and the stored password is silently overwritten; SQL tasks on that datasource stop connecting afterwards.

Brief change log

  • DataSourceServiceImpl#updateDataSource: "password not updated" now means blank OR equals the mask value returned by the query APIs, reusing the existing old-password backfill branch.

  • The comparison happens on the raw DTO value (dataSourceParam.getPassword()) before buildConnectionParams, so it also holds when DATASOURCE_ENCRYPTION_ENABLE is on (the connection param carries the Base64-encoded password in that case — comparing there would silently break).

  • The mask reference is the same one the query APIs use to mask, so the guard and the read path can never drift apart.

  • No config switch: this value can only originate from our own echo and never deserves to be written as a password. Semantically this extends the existing "blank keeps the old password" rule from {blank} to {blank, mask}; the only
    input lost is setting the password to a literal ******, the same class as the existing "cannot set an empty password".

  • POST /datasources/connect (new DataSourceService#checkConnection overload): when the request targets an existing datasource (id present) and the password echoes the mask value, the connectivity check runs against the credentials already stored for that datasource instead of the mask value — the same rule applied to the other path that receives the echoed edit form.

  • Resolving the stored password requires read access to sensitive fields, so the fallback goes through the same permission check as connectionTest (RESOURCE_NOT_EXIST / USER_NO_OPERATION_PERM on failure).

  • Creating a datasource (no id in the form) is unaffected: the submitted password is used as-is.

Verify this pull request

Added 4 contract tests to DataSourceServiceTest, each asserting the persisted connectionParams.password byte-for-byte:

  • blank password → old stored value kept (existing rule, regression coverage)
  • mask ******, default config → old stored value kept
  • mask ******, DATASOURCE_ENCRYPTION_ENABLE=true → old stored value kept
    (guards the compare-on-raw-DTO decision; comparing on the encoded connection
    param differs in exactly this case)
  • new password → new value stored

Added 5 tests to DataSourceServiceTest for the connection-test path, asserting the credentials handed to the connectivity check byte-for-byte:

  • editing with the mask uses the stored password;
  • creating with the mask / editing with a new password use the submitted value as-is with no DB access;
  • editing with the mask without permission fails with USER_NO_OPERATION_PERM and no connectivity check;
  • editing the mask of a non-existent datasource fails with RESOURCE_NOT_EXIST.

DataSourceServiceTest: 23/23 green.

Mutation-checked: removing the backfill turns the edit-with-mask case red (expected: but was: <******>), forcing the fallback on unconditionally turns both non-fallback cases red, and removing the permission check flips the no-permission case to a connection failure.

Pull Request Notice

Pull Request Notice

If your pull request contains incompatible change, you should also add it to docs/docs/en/guide/upgrade/incompatible.md

@njnu-seafish

Copy link
Copy Markdown
Contributor Author

@ruanwenjun @SbloodyS
The user never changed the password, yet the system silently persisted ****** as the real password — and the SQL tasks started failing at runtime with authentication errors.
I've seen many customers run into problems caused by this default behavior, and it happened again just today.
That's why I'm opening a PR once again to fix this.

@njnu-seafish njnu-seafish changed the title [Fix-17813][ApiServer] Treat the mask value echoed back by the frontend as an unmodified password [Fix-17813-Part1][ApiServer] Treat the mask value echoed back by the frontend as an unmodified password Oct 23, 2026
@njnu-seafish njnu-seafish changed the title [Fix-17813-Part1][ApiServer] Treat the mask value echoed back by the frontend as an unmodified password [Fix-17813][ApiServer][Part1] Treat the mask value echoed back by the frontend as an unmodified password Oct 26, 2026
@ruanwenjun

Copy link
Copy Markdown
Member

Please add e2e test case.

@njnu-seafish njnu-seafish changed the title [Fix-17813][ApiServer][Part1] Treat the mask value echoed back by the frontend as an unmodified password [Fix-17813][ApiServer][Part1] Treat the mask value echoed back by the frontend as an unmodified password when updating or testing the connection Oct 26, 2026
@njnu-seafish

Copy link
Copy Markdown
Contributor Author

Please add e2e test case.

Hi @ruanwenjun, thanks for the review.

I've added an e2e test case in d0d81aa:

  • PostgresDataSourceEditE2ETest runs against a real PostgreSQL datasource. It creates a datasource, then opens the edit form without changing the password and runs the connection test from the form — this succeeds only if the backend tests the connectivity against the stored password instead of the echoed mask (******).
  • After saving, the form is opened once more and the connection test is run again, which fails if the update had persisted the mask as the real password. The connection test inside the edit form used to connect with the echoed mask directly, that's fixed in 60a573c the same way as the update path.

@github-actions github-actions Bot added CI&CD e2e e2e test labels Oct 26, 2026
Comment thread .github/workflows/e2e.yml Outdated
- name: PostgresDataSourceE2ETest
class: org.apache.dolphinscheduler.e2e.cases.PostgresDataSourceE2ETest
- name: PostgresDataSourceEditE2ETest
class: org.apache.dolphinscheduler.e2e.cases.PostgresDataSourceEditE2ETest

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's best to add verification of success and failure of modification in existing test cases.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's best to add verification of success and failure of modification in existing test cases.

Hi @SbloodyS, thanks for the review.

I folded the two edit checks into the existing PostgresDataSourceE2ETest (67739ea) and removed the separate class and its matrix entry. Success is verified explicitly (the first edit's connection test only succeeds against the stored password), and the failure path is locked implicitly: the second edit re-opens the form and asserts the connection test still succeeds — if the previous save had persisted the mask, it would necessarily fail. The e2e UI exposes no failure signal (a failed connection is silent), so an explicit failure case isn't assertable; the two rounds cover it.

@njnu-seafish
njnu-seafish requested a review from SbloodyS October 30, 2026 08:13
@github-actions github-actions Bot removed the CI&CD label Oct 30, 2026
@github-actions github-actions Bot added the UI ui and front end related label Nov 2, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend e2e e2e test test UI ui and front end related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants