Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions docs/docs/en/guide/installation/standalone.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,9 +76,18 @@ bash ./bin/dolphinscheduler-daemon.sh stop standalone-server
bash ./bin/dolphinscheduler-daemon.sh status standalone-server
```

> Note: Python gateway service is disabled by default. If you want to start the Python gateway
> service please enable it by changing the yaml config `python-gateway.enabled : true` in api-server's configuration
> path `api-server/conf/application.yaml`
> Note: Python gateway service is disabled by default. To enable it, configure both settings in
> `standalone-server/conf/application.yaml` before starting or restarting Standalone:

```yaml
api:
python-gateway:
enabled: true
auth-token: "<your-auth-token>"
```

Replace `<your-auth-token>` with your own non-blank token and configure the Python client to use the same token.
Standalone fails to start if the gateway is enabled without a non-blank token.

[jdk]: https://www.oracle.com/technetwork/java/javase/downloads/index.html

Expand Down
2 changes: 2 additions & 0 deletions docs/docs/en/guide/upgrade/incompatible.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ This document records the incompatible updates between each version. You need to

## 3.5.0

* Python Gateway no longer provides a default `api.python-gateway.auth-token`. Set a non-blank token when enabling the gateway. API, Master, Worker, Alert and Standalone no longer provide default Actuator credentials; set both `management.security.username` and `management.security.password` to non-blank values before enabling `management.security.enabled`. Missing or blank required credentials cause startup to fail. Disabled features do not require credentials. ([#18812](https://github.com/apache/dolphinscheduler/issues/18812))

* Internal RPC authentication is disabled by default. Upgrade all services before coordinating `rpc.authentication.enabled=true` with the same `rpc.authentication.secret` in each service's `common.properties`; authenticated and unauthenticated services cannot communicate. Credentials are cached for the process lifetime, so enablement and rotation require restarting services. See [RPC authentication](../security/authentication-type.md#inter-service-rpc-authentication).

* Master, Worker and Alert Server now generate heartbeat snapshots on RPC requests and share a thread-safe system-metrics cache with a one-second refresh interval. `BaseHeartBeatTask` and the Master/Worker/Alert heartbeat task subclasses have been removed; custom integrations that extend them must manage scheduling explicitly. Alert Server no longer schedules registry heartbeat writes. It registers stable process identity at startup and restores registration on reconnect. `alert.max-heartbeat-interval` is removed; API queries Alert heartbeat data directly over RPC when requested.
Expand Down
15 changes: 12 additions & 3 deletions docs/docs/zh/guide/installation/standalone.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,18 @@ bash ./bin/dolphinscheduler-daemon.sh stop standalone-server
bash ./bin/dolphinscheduler-daemon.sh status standalone-server
```

> **_注意_**: Python 网关服务默认为关闭状态,如果您想启动 Python 网关服务,
> 请修改 YAML 配置文件,将 `python-gateway.enabled` 设置为 `true`,
> 配置文件路径为 `api-server/conf/application.yaml`
> **_注意_**: Python 网关服务默认为关闭状态。若要启用,请在启动或重启 Standalone 前,
> 在 `standalone-server/conf/application.yaml` 中配置以下两项:

```yaml
api:
python-gateway:
enabled: true
auth-token: "<your-auth-token>"
```

请将 `<your-auth-token>` 替换为自行生成的非空白 token,并为 Python 客户端配置相同的 token。
启用网关但未提供非空白 token 时,Standalone 将启动失败。

[jdk]: https://www.oracle.com/technetwork/java/javase/downloads/index.html

Expand Down
2 changes: 2 additions & 0 deletions docs/docs/zh/guide/upgrade/incompatible.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@

## 3.5.0

* Python Gateway 不再提供默认的 `api.python-gateway.auth-token`,启用时必须显式配置非空白 token。API、Master、Worker、Alert 和 Standalone 不再提供默认的 Actuator 账号密码;开启 `management.security.enabled` 前,必须为 `management.security.username` 和 `management.security.password` 配置非空白值。缺少必需凭据或凭据仅含空白字符时,服务启动失败;未启用的功能无需配置凭据。([#18812](https://github.com/apache/dolphinscheduler/issues/18812))

* 内部 RPC 认证默认关闭。升级全部服务后,再协调所有服务在各自的 `common.properties` 中开启 `rpc.authentication.enabled=true` 并配置相同的 `rpc.authentication.secret`,认证开启与关闭的服务不能互通。凭据在进程内缓存,启用认证和轮换密钥需要重启服务。参见 [RPC 认证](../security/authentication-type.md#服务间-rpc-认证)。

* Master、Worker 和 Alert Server 现在按 RPC 请求生成心跳快照,并发请求共享一秒刷新间隔的线程安全系统指标缓存。`BaseHeartBeatTask` 及 Master、Worker、Alert 的心跳任务子类已移除,自定义扩展如继承这些类,需要自行管理调度。Alert Server 取消定期写注册心跳,只在启动时注册稳定进程身份,并在重连时恢复注册。`alert.max-heartbeat-interval` 已移除,API 在收到查询请求时直接通过 RPC 获取 Alert 心跳。
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,9 @@ server:
management:
security:
enabled: false
username: username
password: password
# Username and password are required when enabled is true.
username: ""
password: ""
exclude: [health,metrics]
endpoints:
web:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,8 @@ public void validate(Object target, Errors errors) {
}

private void validatePythonGateway(Errors errors) {
if (pythonGateway.isEnabled() && StringUtils.isEmpty(pythonGateway.getAuthToken())) {
errors.rejectValue("pythonGateway.auth-token", null, "should not be empty when enabled is true");
if (pythonGateway.isEnabled() && StringUtils.isBlank(pythonGateway.getAuthToken())) {
errors.rejectValue("pythonGateway.auth-token", null, "should not be blank when enabled is true");
}
}

Expand Down Expand Up @@ -109,7 +109,7 @@ public static class PythonGatewayConfiguration {
private int pythonPort = 25334;
private int connectTimeout = 0;
private int readTimeout = 0;
private String authToken = "jwUDzpLsNKEFER4*a8gruBH_GsAurNxU7A@Xc";
private String authToken;
}

}
7 changes: 4 additions & 3 deletions dolphinscheduler-api/src/main/resources/application.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,9 @@ mybatis-plus:
management:
security:
enabled: false
username: username
password: password
# Username and password are required when enabled is true.
username: ""
password: ""
exclude: [health,metrics]
endpoints:
web:
Expand Down Expand Up @@ -160,7 +161,7 @@ api:
python-gateway:
# Weather enable python gateway server or not. The default value is false.
enabled: false
# Authentication token for connection from python api to python gateway server. Should not be empty when enabled
# Authentication token for connection from python api to python gateway server. Should not be blank when enabled
auth-token:
# The address of Python gateway server start. Set its value to `0.0.0.0` if your Python API run in different
# between Python gateway server. It could be be specific to other address like `127.0.0.1` or `localhost`
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.dolphinscheduler.api.configuration;

import static org.assertj.core.api.Assertions.assertThat;

import java.util.Collections;

import org.junit.jupiter.api.Test;
import org.springframework.boot.autoconfigure.AutoConfigurations;
import org.springframework.boot.autoconfigure.context.ConfigurationPropertiesAutoConfiguration;
import org.springframework.boot.test.context.runner.ApplicationContextRunner;
import org.springframework.boot.test.util.TestPropertyValues;

class ApiConfigValidationTest {

private final ApplicationContextRunner contextRunner = new ApplicationContextRunner()
.withConfiguration(AutoConfigurations.of(ConfigurationPropertiesAutoConfiguration.class))
.withUserConfiguration(ApiConfig.class);

@Test
void shouldNotRequireTokenWhenPythonGatewayIsDisabled() {
contextRunner.run(context -> {
assertThat(context).hasNotFailed();
assertThat(context.getBean(ApiConfig.class).getPythonGateway().getAuthToken()).isNull();
});
}

@Test
void shouldRejectMissingTokenWhenPythonGatewayIsEnabled() {
contextRunner.withPropertyValues("api.python-gateway.enabled=true").run(context -> {
assertThat(context).hasFailed();
assertThat(context.getStartupFailure()).hasStackTraceContaining("pythonGateway.auth-token");
});
}

@Test
void shouldRejectEmptyTokenWhenPythonGatewayIsEnabled() {
contextRunner.withPropertyValues("api.python-gateway.enabled=true", "api.python-gateway.auth-token=")
.run(context -> {
assertThat(context).hasFailed();
assertThat(context.getStartupFailure()).hasStackTraceContaining("pythonGateway.auth-token");
});
}

@Test
void shouldRejectBlankTokenWhenPythonGatewayIsEnabled() {
contextRunner.withPropertyValues("api.python-gateway.enabled=true")
.withInitializer(context -> TestPropertyValues.of(Collections.singletonMap(
"api.python-gateway.auth-token", " \t ")).applyTo(context))
.run(context -> {
assertThat(context).hasFailed();
assertThat(context.getStartupFailure()).hasStackTraceContaining("pythonGateway.auth-token");
});
}

@Test
void shouldAcceptExplicitTokenWhenPythonGatewayIsEnabled() {
contextRunner
.withPropertyValues("api.python-gateway.enabled=true", "api.python-gateway.auth-token=gateway-token")
.run(context -> {
assertThat(context).hasNotFailed();
assertThat(context.getBean(ApiConfig.class).getPythonGateway().getAuthToken())
.isEqualTo("gateway-token");
});
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,12 @@
package org.apache.dolphinscheduler.authentication.actuator.config;

import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;

import java.net.URI;
import java.util.Collections;

import org.junit.jupiter.api.Test;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.SpringBootConfiguration;
import org.springframework.boot.WebApplicationType;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
Expand Down Expand Up @@ -76,31 +77,57 @@ void shouldAuthenticateActuatorPathsWithoutContextPath() {

@Test
void shouldPermitActuatorEndpointsWhenAuthenticationIsDisabled() {
try (ConfigurableApplicationContext context = startApplication(false)) {
try (ConfigurableApplicationContext context = startApplication(false, CONTEXT_PATH, null, null)) {
assertThat(get(context, "/actuator/prometheus", null, null).getStatusCode())
.isEqualTo(HttpStatus.OK);
assertThat(get(context, "/%61ctuator/prometheus", null, null).getStatusCode())
.isEqualTo(HttpStatus.OK);
}
}

@Test
void shouldRejectMissingOrBlankUsernameWhenAuthenticationIsEnabled() {
for (String username : new String[]{null, "", " \t "}) {
assertThatThrownBy(() -> startApplication(true, CONTEXT_PATH, username, PASSWORD))
.hasRootCauseInstanceOf(IllegalArgumentException.class)
.hasStackTraceContaining("management.security.username must be configured when enabled.");
}
}

@Test
void shouldRejectMissingOrBlankPasswordWhenAuthenticationIsEnabled() {
for (String password : new String[]{null, "", " \t "}) {
assertThatThrownBy(() -> startApplication(true, CONTEXT_PATH, USERNAME, password))
.hasRootCauseInstanceOf(IllegalArgumentException.class)
.hasStackTraceContaining("management.security.password must be configured when enabled.");
}
}

private ConfigurableApplicationContext startApplication(boolean securityEnabled) {
return startApplication(securityEnabled, CONTEXT_PATH);
}

private ConfigurableApplicationContext startApplication(boolean securityEnabled, String contextPath) {
SpringApplication application = new SpringApplicationBuilder(TestApplication.class)
return startApplication(securityEnabled, contextPath, USERNAME, PASSWORD);
}

private ConfigurableApplicationContext startApplication(boolean securityEnabled, String contextPath,
String username, String password) {
SpringApplicationBuilder application = new SpringApplicationBuilder(TestApplication.class)
.web(WebApplicationType.SERVLET)
.properties(
"server.port=0",
"server.servlet.context-path=" + contextPath,
"management.security.enabled=" + securityEnabled,
"management.security.username=" + USERNAME,
"management.security.password=" + PASSWORD,
"management.security.exclude=health,metrics",
"management.endpoints.web.exposure.include=health,metrics,prometheus",
"spring.main.banner-mode=off")
.build();
"spring.main.banner-mode=off");
if (username != null) {
application.properties(Collections.singletonMap("management.security.username", username));
}
if (password != null) {
application.properties(Collections.singletonMap("management.security.password", password));
}
return application.run();
}

Expand Down
5 changes: 3 additions & 2 deletions dolphinscheduler-master/src/main/resources/application.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -149,8 +149,9 @@ server:
management:
security:
enabled: false
username: username
password: password
# Username and password are required when enabled is true.
username: ""
password: ""
exclude: [health,metrics]
endpoints:
web:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ api:
python-gateway:
# Weather enable python gateway server or not. The default value is false.
enabled: false
# Authentication token for connection from python api to python gateway server. Should not be empty when enabled
# Authentication token for connection from python api to python gateway server. Should not be blank when enabled
auth-token:
# The address of Python gateway server start. Set its value to `0.0.0.0` if your Python API run in different
# between Python gateway server. It could be be specific to other address like `127.0.0.1` or `localhost`
Expand Down Expand Up @@ -314,8 +314,9 @@ server:
management:
security:
enabled: false
username: username
password: password
# Username and password are required when enabled is true.
username: ""
password: ""
exclude: [health,metrics]
endpoints:
web:
Expand Down
5 changes: 3 additions & 2 deletions dolphinscheduler-worker/src/main/resources/application.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,9 @@ server:
management:
security:
enabled: false
username: username
password: password
# Username and password are required when enabled is true.
username: ""
password: ""
exclude: [health,metrics]
endpoints:
web:
Expand Down
Loading