Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .vitepress/config/cn.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ export const cn = defineConfig({
{ text: 'P2P 优化', link: '/guide/network/p2p-optimize' },
{ text: '魔法 DNS', link: '/guide/network/magic-dns' },
{ text: 'ACL', link: '/guide/config/acl' },
{ text: 'EasyTier CNI(Kubernetes)', link: '/guide/network/easytier-cni' },
] },
{ text: '开机自启(注册服务)', collapsed: true, items: [
{ text: '一键安装服务', link: '/guide/network/oneclick-install-as-service' },
Expand Down
1 change: 1 addition & 0 deletions .vitepress/config/en.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ export const en = defineConfig({
{ text: 'P2P Optimization', link: '/en/guide/network/p2p-optimize' },
{ text: 'Magic DNS', link: '/en/guide/network/magic-dns' },
{ text: 'ACL', link: '/en/guide/config/acl' },
{ text: 'EasyTier CNI (Kubernetes)', link: '/en/guide/network/easytier-cni' },
] },
{ text: 'Autostart (Register Service)', collapsed: true, items: [
{ text: 'One-Click Install Service', link: '/en/guide/network/oneclick-install-as-service' },
Expand Down
1 change: 1 addition & 0 deletions en/guide/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ The software can be used via command line or graphical interface. It is ready to
- **KCP / QUIC Proxy**: Converts TCP traffic to KCP / QUIC protocol, improving transmission latency and stability in high UDP packet loss environments.
- **Non-Privileged Mode**: Supports running under non-privileged users, avoiding the need for root permissions (only as an accessed endpoint).
- **WireGuard Access**: Supports WireGuard client access to the EasyTier network.
- **Kubernetes Pod Networking**: Adds an EasyTier secondary network to Pods through [EasyTier CNI](/en/guide/network/easytier-cni).

## Graphical Interface (GUI)

Expand Down
145 changes: 145 additions & 0 deletions en/guide/network/easytier-cni.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
# EasyTier CNI (Kubernetes)

EasyTier CNI uses Multus to attach an EasyTier TUN as a secondary Pod interface. The existing primary CNI interface continues to provide Kubernetes Services, DNS, and the EasyTier underlay. Only traffic for the secondary virtual subnet uses EasyTier.

```mermaid
flowchart LR
A[Pod A net1] --> B[EasyTier Overlay]
B --> C[Pod B net1]
A -. eth0 .-> D[Primary CNI / Kubernetes network]
C -. eth0 .-> D
```

::: warning Current scope
The initial release supports CNI `1.0.0`, one IPv4 address, and a standalone Multus delegate. It does not change the Pod default route or DNS and does not support a primary CNI, IPv6, or chained `prevResult`.
:::

## Prerequisites

- Linux Kubernetes nodes with `/dev/net/tun`.
- [Multus CNI](https://github.com/k8snetworkplumbingwg/multus-cni).
- [Whereabouts](https://github.com/k8snetworkplumbingwg/whereabouts), or another IPAM plugin that returns one IPv4 address and no extra routes.
- At least one EasyTier peer reachable through every Pod's primary network.
- An EasyTier image containing both `easytier-core` and `easytier-cni`.

## Coexisting with Flannel

EasyTier CNI does not replace or modify Flannel. Flannel continues to provide the Pod's primary `eth0` interface, while Multus invokes EasyTier for the secondary `net1` interface.

An existing Flannel cluster does not need Flannel reinstalled. For a deployment created from the upstream Flannel manifest, use these commands to confirm that Flannel and all nodes are healthy. Use the actual namespace and DaemonSet name for distribution-managed or Helm deployments:

```sh
kubectl -n kube-flannel rollout status daemonset/kube-flannel-ds --timeout=5m
kubectl wait node --all --for=condition=Ready --timeout=5m
```

Flannel still requires standard CNI plugins such as `bridge`, `host-local`, and `portmap`, together with the `br_netfilter` kernel module. EasyTier CNI does not install these Flannel prerequisites.

After the primary network is healthy, install Multus, Whereabouts, and EasyTier CNI in that order. For a new cluster, [install Flannel](https://github.com/flannel-io/flannel#deploying-flannel-manually) and wait for the nodes to become Ready first. Multus auto-configuration selects the existing Flannel configuration as its default network; for a manual Multus configuration, set Flannel as the default delegate. Do not add EasyTier to the Flannel conflist. Attach EasyTier only through the `NetworkAttachmentDefinition` created later in this guide.

When Flannel uses its default `10.244.0.0/16`, the example EasyTier range `10.200.0.0/24` can be used, provided it does not overlap any other network in the environment.

::: tip Validated combination
The three-node Kind test disables the default kindnet and uses Flannel `v0.28.9` as the only primary CNI, with Multus `v4.3.0` and Whereabouts `v0.9.4`. It verifies cross-node `net1` ping and HTTP between Pods on different workers, Service/DNS access through the Flannel primary network, MTU, deletion, and IPAM address release.
:::

::: warning Security
The node DaemonSet must enter Pod network namespaces and create TUN devices, so it uses privileged mode, hostPID, and the host `/run`. Pin production deployments to an EasyTier release that contains CNI support. Do not deploy a mutable `unstable` tag in production.
:::

## 1. Create the network secret

Do not put the EasyTier network secret in a `NetworkAttachmentDefinition`:

```sh
kubectl -n kube-system create secret generic easytier-cni \
--from-literal=network-secret="$EASYTIER_NETWORK_SECRET"
```

The DaemonSet installs the secret as a root-only node file. Management RPC uses a root-only Unix socket on the node and does not expose a TCP management port.

## 2. Deploy the node DaemonSet

Download the [official DaemonSet manifest](https://github.com/EasyTier/EasyTier/blob/main/easytier-contrib/easytier-cni/deploy/daemonset.yaml), pin both EasyTier image references to the same release, and deploy it:

```sh
kubectl apply -f daemonset.yaml
kubectl -n kube-system rollout status daemonset/easytier-cni
```

Confirm that the plugin and management process are running on every node:

```sh
kubectl -n kube-system get pods -l app.kubernetes.io/name=easytier-cni -o wide
```

## 3. Create the secondary network

Start with the [official NAD example](https://github.com/EasyTier/EasyTier/blob/main/easytier-contrib/easytier-cni/deploy/network-attachment-definition.yaml):

```json
{
"cniVersion": "1.0.0",
"name": "easytier",
"type": "easytier-cni",
"rpcPortal": "unix:///run/easytier-cni/rpc.sock",
"networkName": "kubernetes",
"networkSecretFile": "/etc/easytier-cni/network-secret",
"peers": ["tcp://192.0.2.10:11010"],
"mtu": 1380,
"timeoutSeconds": 30,
"ipam": {
"type": "whereabouts",
"range": "10.200.0.0/24",
"range_start": "10.200.0.10",
"range_end": "10.200.0.250"
}
}
```

- `peers` must be reachable through the Pod's primary interface. Prefer IP addresses so CNI setup does not depend on cluster DNS.
- The EasyTier range must not overlap node, Pod, Service, LAN, VPN, or container-runtime networks.
- `mtu` is the EasyTier packet MTU. The TUN MTU excludes protocol overhead; for example, `1380` produces a TUN MTU of `1360`.

Apply the completed NAD:

```sh
kubectl apply -f network-attachment-definition.yaml
```

## 4. Attach EasyTier to a Pod

Add a Multus annotation to the Pod:

```yaml
apiVersion: v1
kind: Pod
metadata:
name: easytier-client
annotations:
k8s.v1.cni.cncf.io/networks: easytier
spec:
containers:
- name: client
image: busybox:1.37.0
command: ["/bin/sh", "-c", "sleep 3600"]
```

The default secondary interface name is `net1`. Inspect its address and the Multus network status:

```sh
kubectl exec easytier-client -- ip address show net1
kubectl get pod easytier-client \
-o jsonpath='{.metadata.annotations.k8s\.v1\.cni\.cncf\.io/network-status}'
```

After two Pods join the same EasyTier network, they can communicate directly through their `net1` IPv4 addresses.

## Operations

- Deleting a Pod removes its EasyTier instance and releases its Whereabouts address.
- The node daemon persists attachments under `/var/lib/easytier-cni/configs`. This directory contains network credentials and must remain root-only.
- After updating the Secret, restart the `easytier-cni` DaemonSet and recreate Pods that use the EasyTier network.
- After uninstalling, remove `/etc/easytier-cni` and `/var/lib/easytier-cni` from nodes when appropriate.

Complete configuration, automated netns tests, and the three-node Kind test are available under [`easytier-contrib/easytier-cni`](https://github.com/EasyTier/EasyTier/tree/main/easytier-contrib/easytier-cni).
1 change: 1 addition & 0 deletions guide/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ EasyTier 是一款简单、安全、去中心化的内网穿透和异地组网
- **KCP / QUIC 代理**: 支持将 TCP 流量转为 KCP / QUIC 协议,提升高 UDP 丢包环境下的传输延迟和稳定性。
- **非特权模式**:支持在非特权用户下运行,避免需要 root 权限。(仅可作为被访问端)
- **WireGuard 接入**: 支持 WireGuard 客户端接入 EasyTier 网络。
- **Kubernetes Pod 组网**:通过 [EasyTier CNI](/guide/network/easytier-cni) 为 Pod 添加 EasyTier 辅助网络。


## 图形界面(GUI)
Expand Down
145 changes: 145 additions & 0 deletions guide/network/easytier-cni.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
# EasyTier CNI(Kubernetes)

EasyTier CNI 可以通过 Multus 为 Kubernetes Pod 添加一个 EasyTier TUN 辅助网卡。Pod 原有的主 CNI 网卡继续负责集群 Service、DNS 和 EasyTier 底层连接,EasyTier 网络只承载分配给辅助网卡的虚拟网段流量。

```mermaid
flowchart LR
A[Pod A net1] --> B[EasyTier Overlay]
B --> C[Pod B net1]
A -. eth0 .-> D[主 CNI / Kubernetes 网络]
C -. eth0 .-> D
```

::: warning 当前范围
首版仅支持 CNI `1.0.0`、单个 IPv4 地址和 Multus standalone delegate。不修改 Pod 默认路由和 DNS,也不支持 Primary CNI、IPv6 或 chained `prevResult`。
:::

## 前置条件

- Linux Kubernetes 节点,且存在 `/dev/net/tun`。
- 已安装 [Multus CNI](https://github.com/k8snetworkplumbingwg/multus-cni)。
- 已安装 [Whereabouts](https://github.com/k8snetworkplumbingwg/whereabouts),或其他能返回单个 IPv4 且不返回额外路由的 IPAM 插件。
- 每个 Pod 的主网络都能访问至少一个 EasyTier peer。
- EasyTier 镜像包含 `easytier-core` 和 `easytier-cni`。

## 与 Flannel 共存

EasyTier CNI 不会替换或修改 Flannel。Flannel 继续提供 Pod 的 `eth0` 主网卡,Multus 将 EasyTier 作为 `net1` 辅助网卡调用。

已有 Flannel 集群不需要重新安装 Flannel。使用 Flannel 上游清单部署时,可通过以下命令确认 Flannel 和节点正常;发行版内置或 Helm 部署请使用实际的 namespace 和 DaemonSet 名称:

```sh
kubectl -n kube-flannel rollout status daemonset/kube-flannel-ds --timeout=5m
kubectl wait node --all --for=condition=Ready --timeout=5m
```

Flannel 自身仍需要 `bridge`、`host-local`、`portmap` 等标准 CNI plugins 以及 `br_netfilter` 内核模块;EasyTier CNI 不负责安装这些 Flannel 前置依赖。

确认主网络正常后,依次安装 Multus、Whereabouts 和 EasyTier CNI。新集群必须先按照 [Flannel 文档](https://github.com/flannel-io/flannel#deploying-flannel-manually)完成安装并等待节点 Ready。Multus 自动配置模式会选择已有的 Flannel 配置作为默认网络;手工配置 Multus 时也要把 Flannel 设为默认 delegate。不要把 EasyTier 配置加入 Flannel conflist,EasyTier 只通过本页后续创建的 `NetworkAttachmentDefinition` 接入。

Flannel 默认使用 `10.244.0.0/16` 时,可以使用本页示例中的 EasyTier `10.200.0.0/24`,但仍需检查它是否与实际环境的其他网络重叠。

::: tip 已验证组合
三节点 Kind 测试关闭了默认 kindnet,以 Flannel `v0.28.9` 作为唯一主 CNI,并安装 Multus `v4.3.0` 和 Whereabouts `v0.9.4`。测试覆盖不同 worker 上 Pod 的 `net1` 跨节点 ping 和 HTTP、Flannel 主网络 Service/DNS、MTU、删除及 IPAM 地址回收。
:::

::: warning 安全提示
节点 DaemonSet 需要进入 Pod network namespace 并创建 TUN,因此会使用 privileged、hostPID 和宿主 `/run`。生产环境必须固定到包含 CNI 的正式 EasyTier 版本,不要直接使用可变的 `unstable` 标签。
:::

## 1. 创建网络密钥

不要把 EasyTier 网络密钥写进 `NetworkAttachmentDefinition`:

```sh
kubectl -n kube-system create secret generic easytier-cni \
--from-literal=network-secret="$EASYTIER_NETWORK_SECRET"
```

DaemonSet 会把密钥以 root-only 文件安装到节点。管理 RPC 使用节点上的 root-only Unix socket,不会开放 TCP 管理端口。

## 2. 部署节点 DaemonSet

下载[官方 DaemonSet 清单](https://github.com/EasyTier/EasyTier/blob/main/easytier-contrib/easytier-cni/deploy/daemonset.yaml),把两个 EasyTier 镜像都固定到相同的正式版本,然后部署:

```sh
kubectl apply -f daemonset.yaml
kubectl -n kube-system rollout status daemonset/easytier-cni
```

确认每个节点都安装了插件并启动管理进程:

```sh
kubectl -n kube-system get pods -l app.kubernetes.io/name=easytier-cni -o wide
```

## 3. 创建辅助网络

以[官方 NAD 示例](https://github.com/EasyTier/EasyTier/blob/main/easytier-contrib/easytier-cni/deploy/network-attachment-definition.yaml)为基础修改:

```json
{
"cniVersion": "1.0.0",
"name": "easytier",
"type": "easytier-cni",
"rpcPortal": "unix:///run/easytier-cni/rpc.sock",
"networkName": "kubernetes",
"networkSecretFile": "/etc/easytier-cni/network-secret",
"peers": ["tcp://192.0.2.10:11010"],
"mtu": 1380,
"timeoutSeconds": 30,
"ipam": {
"type": "whereabouts",
"range": "10.200.0.0/24",
"range_start": "10.200.0.10",
"range_end": "10.200.0.250"
}
}
```

- `peers` 必须能通过 Pod 主网卡访问。建议使用 IP 地址,避免 CNI 阶段依赖集群 DNS。
- EasyTier 地址段不能与节点、Pod、Service、局域网、VPN 或容器运行时网段重叠。
- `mtu` 是 EasyTier packet MTU;TUN MTU 会扣除协议开销,例如 `1380` 对应 TUN MTU `1360`。

完成修改后应用 NAD:

```sh
kubectl apply -f network-attachment-definition.yaml
```

## 4. 为 Pod 添加 EasyTier 网络

在 Pod 上添加 Multus 注解:

```yaml
apiVersion: v1
kind: Pod
metadata:
name: easytier-client
annotations:
k8s.v1.cni.cncf.io/networks: easytier
spec:
containers:
- name: client
image: busybox:1.37.0
command: ["/bin/sh", "-c", "sleep 3600"]
```

默认辅助网卡名为 `net1`。检查地址和 Multus 网络状态:

```sh
kubectl exec easytier-client -- ip address show net1
kubectl get pod easytier-client \
-o jsonpath='{.metadata.annotations.k8s\.v1\.cni\.cncf\.io/network-status}'
```

两个 Pod 都连接到相同 EasyTier 网络后,可以直接通过各自的 `net1` IPv4 地址通信。

## 运维注意事项

- 删除 Pod 时,CNI 会删除对应 EasyTier 实例并释放 Whereabouts 地址。
- 节点 daemon 会在 `/var/lib/easytier-cni/configs` 持久化 attachment;目录包含网络凭据,只允许 root 访问。
- 更新 Secret 后,需要重启 `easytier-cni` DaemonSet,并重建已挂载 EasyTier 网络的 Pod。
- 卸载后按需删除节点上的 `/etc/easytier-cni` 和 `/var/lib/easytier-cni`。

完整配置、自动化 netns 测试和 Kind 三节点测试位于 [`easytier-contrib/easytier-cni`](https://github.com/EasyTier/EasyTier/tree/main/easytier-contrib/easytier-cni)。