diff --git a/.vitepress/config/cn.ts b/.vitepress/config/cn.ts index dd9eb9e..b9413b1 100644 --- a/.vitepress/config/cn.ts +++ b/.vitepress/config/cn.ts @@ -36,6 +36,7 @@ export const cn = defineConfig({ { text: 'P2P 优化', link: '/guide/network/p2p-optimize' }, { text: '魔法 DNS', link: '/guide/network/magic-dns' }, { text: 'ACL', link: '/guide/config/acl' }, + { text: 'EasyTier CNI(Kubernetes)', link: '/guide/network/easytier-cni' }, ] }, { text: '开机自启(注册服务)', collapsed: true, items: [ { text: '一键安装服务', link: '/guide/network/oneclick-install-as-service' }, diff --git a/.vitepress/config/en.ts b/.vitepress/config/en.ts index 6391759..c31b978 100644 --- a/.vitepress/config/en.ts +++ b/.vitepress/config/en.ts @@ -35,6 +35,7 @@ export const en = defineConfig({ { text: 'P2P Optimization', link: '/en/guide/network/p2p-optimize' }, { text: 'Magic DNS', link: '/en/guide/network/magic-dns' }, { text: 'ACL', link: '/en/guide/config/acl' }, + { text: 'EasyTier CNI (Kubernetes)', link: '/en/guide/network/easytier-cni' }, ] }, { text: 'Autostart (Register Service)', collapsed: true, items: [ { text: 'One-Click Install Service', link: '/en/guide/network/oneclick-install-as-service' }, diff --git a/en/guide/introduction.md b/en/guide/introduction.md index 4d0ca92..a4a3816 100644 --- a/en/guide/introduction.md +++ b/en/guide/introduction.md @@ -30,6 +30,7 @@ The software can be used via command line or graphical interface. It is ready to - **KCP / QUIC Proxy**: Converts TCP traffic to KCP / QUIC protocol, improving transmission latency and stability in high UDP packet loss environments. - **Non-Privileged Mode**: Supports running under non-privileged users, avoiding the need for root permissions (only as an accessed endpoint). - **WireGuard Access**: Supports WireGuard client access to the EasyTier network. +- **Kubernetes Pod Networking**: Adds an EasyTier secondary network to Pods through [EasyTier CNI](/en/guide/network/easytier-cni). ## Graphical Interface (GUI) diff --git a/en/guide/network/easytier-cni.md b/en/guide/network/easytier-cni.md new file mode 100644 index 0000000..908bc6a --- /dev/null +++ b/en/guide/network/easytier-cni.md @@ -0,0 +1,145 @@ +# EasyTier CNI (Kubernetes) + +EasyTier CNI uses Multus to attach an EasyTier TUN as a secondary Pod interface. The existing primary CNI interface continues to provide Kubernetes Services, DNS, and the EasyTier underlay. Only traffic for the secondary virtual subnet uses EasyTier. + +```mermaid +flowchart LR + A[Pod A net1] --> B[EasyTier Overlay] + B --> C[Pod B net1] + A -. eth0 .-> D[Primary CNI / Kubernetes network] + C -. eth0 .-> D +``` + +::: warning Current scope +The initial release supports CNI `1.0.0`, one IPv4 address, and a standalone Multus delegate. It does not change the Pod default route or DNS and does not support a primary CNI, IPv6, or chained `prevResult`. +::: + +## Prerequisites + +- Linux Kubernetes nodes with `/dev/net/tun`. +- [Multus CNI](https://github.com/k8snetworkplumbingwg/multus-cni). +- [Whereabouts](https://github.com/k8snetworkplumbingwg/whereabouts), or another IPAM plugin that returns one IPv4 address and no extra routes. +- At least one EasyTier peer reachable through every Pod's primary network. +- An EasyTier image containing both `easytier-core` and `easytier-cni`. + +## Coexisting with Flannel + +EasyTier CNI does not replace or modify Flannel. Flannel continues to provide the Pod's primary `eth0` interface, while Multus invokes EasyTier for the secondary `net1` interface. + +An existing Flannel cluster does not need Flannel reinstalled. For a deployment created from the upstream Flannel manifest, use these commands to confirm that Flannel and all nodes are healthy. Use the actual namespace and DaemonSet name for distribution-managed or Helm deployments: + +```sh +kubectl -n kube-flannel rollout status daemonset/kube-flannel-ds --timeout=5m +kubectl wait node --all --for=condition=Ready --timeout=5m +``` + +Flannel still requires standard CNI plugins such as `bridge`, `host-local`, and `portmap`, together with the `br_netfilter` kernel module. EasyTier CNI does not install these Flannel prerequisites. + +After the primary network is healthy, install Multus, Whereabouts, and EasyTier CNI in that order. For a new cluster, [install Flannel](https://github.com/flannel-io/flannel#deploying-flannel-manually) and wait for the nodes to become Ready first. Multus auto-configuration selects the existing Flannel configuration as its default network; for a manual Multus configuration, set Flannel as the default delegate. Do not add EasyTier to the Flannel conflist. Attach EasyTier only through the `NetworkAttachmentDefinition` created later in this guide. + +When Flannel uses its default `10.244.0.0/16`, the example EasyTier range `10.200.0.0/24` can be used, provided it does not overlap any other network in the environment. + +::: tip Validated combination +The three-node Kind test disables the default kindnet and uses Flannel `v0.28.9` as the only primary CNI, with Multus `v4.3.0` and Whereabouts `v0.9.4`. It verifies cross-node `net1` ping and HTTP between Pods on different workers, Service/DNS access through the Flannel primary network, MTU, deletion, and IPAM address release. +::: + +::: warning Security +The node DaemonSet must enter Pod network namespaces and create TUN devices, so it uses privileged mode, hostPID, and the host `/run`. Pin production deployments to an EasyTier release that contains CNI support. Do not deploy a mutable `unstable` tag in production. +::: + +## 1. Create the network secret + +Do not put the EasyTier network secret in a `NetworkAttachmentDefinition`: + +```sh +kubectl -n kube-system create secret generic easytier-cni \ + --from-literal=network-secret="$EASYTIER_NETWORK_SECRET" +``` + +The DaemonSet installs the secret as a root-only node file. Management RPC uses a root-only Unix socket on the node and does not expose a TCP management port. + +## 2. Deploy the node DaemonSet + +Download the [official DaemonSet manifest](https://github.com/EasyTier/EasyTier/blob/main/easytier-contrib/easytier-cni/deploy/daemonset.yaml), pin both EasyTier image references to the same release, and deploy it: + +```sh +kubectl apply -f daemonset.yaml +kubectl -n kube-system rollout status daemonset/easytier-cni +``` + +Confirm that the plugin and management process are running on every node: + +```sh +kubectl -n kube-system get pods -l app.kubernetes.io/name=easytier-cni -o wide +``` + +## 3. Create the secondary network + +Start with the [official NAD example](https://github.com/EasyTier/EasyTier/blob/main/easytier-contrib/easytier-cni/deploy/network-attachment-definition.yaml): + +```json +{ + "cniVersion": "1.0.0", + "name": "easytier", + "type": "easytier-cni", + "rpcPortal": "unix:///run/easytier-cni/rpc.sock", + "networkName": "kubernetes", + "networkSecretFile": "/etc/easytier-cni/network-secret", + "peers": ["tcp://192.0.2.10:11010"], + "mtu": 1380, + "timeoutSeconds": 30, + "ipam": { + "type": "whereabouts", + "range": "10.200.0.0/24", + "range_start": "10.200.0.10", + "range_end": "10.200.0.250" + } +} +``` + +- `peers` must be reachable through the Pod's primary interface. Prefer IP addresses so CNI setup does not depend on cluster DNS. +- The EasyTier range must not overlap node, Pod, Service, LAN, VPN, or container-runtime networks. +- `mtu` is the EasyTier packet MTU. The TUN MTU excludes protocol overhead; for example, `1380` produces a TUN MTU of `1360`. + +Apply the completed NAD: + +```sh +kubectl apply -f network-attachment-definition.yaml +``` + +## 4. Attach EasyTier to a Pod + +Add a Multus annotation to the Pod: + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: easytier-client + annotations: + k8s.v1.cni.cncf.io/networks: easytier +spec: + containers: + - name: client + image: busybox:1.37.0 + command: ["/bin/sh", "-c", "sleep 3600"] +``` + +The default secondary interface name is `net1`. Inspect its address and the Multus network status: + +```sh +kubectl exec easytier-client -- ip address show net1 +kubectl get pod easytier-client \ + -o jsonpath='{.metadata.annotations.k8s\.v1\.cni\.cncf\.io/network-status}' +``` + +After two Pods join the same EasyTier network, they can communicate directly through their `net1` IPv4 addresses. + +## Operations + +- Deleting a Pod removes its EasyTier instance and releases its Whereabouts address. +- The node daemon persists attachments under `/var/lib/easytier-cni/configs`. This directory contains network credentials and must remain root-only. +- After updating the Secret, restart the `easytier-cni` DaemonSet and recreate Pods that use the EasyTier network. +- After uninstalling, remove `/etc/easytier-cni` and `/var/lib/easytier-cni` from nodes when appropriate. + +Complete configuration, automated netns tests, and the three-node Kind test are available under [`easytier-contrib/easytier-cni`](https://github.com/EasyTier/EasyTier/tree/main/easytier-contrib/easytier-cni). diff --git a/guide/introduction.md b/guide/introduction.md index eba8d61..d502bba 100644 --- a/guide/introduction.md +++ b/guide/introduction.md @@ -31,6 +31,7 @@ EasyTier 是一款简单、安全、去中心化的内网穿透和异地组网 - **KCP / QUIC 代理**: 支持将 TCP 流量转为 KCP / QUIC 协议,提升高 UDP 丢包环境下的传输延迟和稳定性。 - **非特权模式**:支持在非特权用户下运行,避免需要 root 权限。(仅可作为被访问端) - **WireGuard 接入**: 支持 WireGuard 客户端接入 EasyTier 网络。 +- **Kubernetes Pod 组网**:通过 [EasyTier CNI](/guide/network/easytier-cni) 为 Pod 添加 EasyTier 辅助网络。 ## 图形界面(GUI) diff --git a/guide/network/easytier-cni.md b/guide/network/easytier-cni.md new file mode 100644 index 0000000..7ca342c --- /dev/null +++ b/guide/network/easytier-cni.md @@ -0,0 +1,145 @@ +# EasyTier CNI(Kubernetes) + +EasyTier CNI 可以通过 Multus 为 Kubernetes Pod 添加一个 EasyTier TUN 辅助网卡。Pod 原有的主 CNI 网卡继续负责集群 Service、DNS 和 EasyTier 底层连接,EasyTier 网络只承载分配给辅助网卡的虚拟网段流量。 + +```mermaid +flowchart LR + A[Pod A net1] --> B[EasyTier Overlay] + B --> C[Pod B net1] + A -. eth0 .-> D[主 CNI / Kubernetes 网络] + C -. eth0 .-> D +``` + +::: warning 当前范围 +首版仅支持 CNI `1.0.0`、单个 IPv4 地址和 Multus standalone delegate。不修改 Pod 默认路由和 DNS,也不支持 Primary CNI、IPv6 或 chained `prevResult`。 +::: + +## 前置条件 + +- Linux Kubernetes 节点,且存在 `/dev/net/tun`。 +- 已安装 [Multus CNI](https://github.com/k8snetworkplumbingwg/multus-cni)。 +- 已安装 [Whereabouts](https://github.com/k8snetworkplumbingwg/whereabouts),或其他能返回单个 IPv4 且不返回额外路由的 IPAM 插件。 +- 每个 Pod 的主网络都能访问至少一个 EasyTier peer。 +- EasyTier 镜像包含 `easytier-core` 和 `easytier-cni`。 + +## 与 Flannel 共存 + +EasyTier CNI 不会替换或修改 Flannel。Flannel 继续提供 Pod 的 `eth0` 主网卡,Multus 将 EasyTier 作为 `net1` 辅助网卡调用。 + +已有 Flannel 集群不需要重新安装 Flannel。使用 Flannel 上游清单部署时,可通过以下命令确认 Flannel 和节点正常;发行版内置或 Helm 部署请使用实际的 namespace 和 DaemonSet 名称: + +```sh +kubectl -n kube-flannel rollout status daemonset/kube-flannel-ds --timeout=5m +kubectl wait node --all --for=condition=Ready --timeout=5m +``` + +Flannel 自身仍需要 `bridge`、`host-local`、`portmap` 等标准 CNI plugins 以及 `br_netfilter` 内核模块;EasyTier CNI 不负责安装这些 Flannel 前置依赖。 + +确认主网络正常后,依次安装 Multus、Whereabouts 和 EasyTier CNI。新集群必须先按照 [Flannel 文档](https://github.com/flannel-io/flannel#deploying-flannel-manually)完成安装并等待节点 Ready。Multus 自动配置模式会选择已有的 Flannel 配置作为默认网络;手工配置 Multus 时也要把 Flannel 设为默认 delegate。不要把 EasyTier 配置加入 Flannel conflist,EasyTier 只通过本页后续创建的 `NetworkAttachmentDefinition` 接入。 + +Flannel 默认使用 `10.244.0.0/16` 时,可以使用本页示例中的 EasyTier `10.200.0.0/24`,但仍需检查它是否与实际环境的其他网络重叠。 + +::: tip 已验证组合 +三节点 Kind 测试关闭了默认 kindnet,以 Flannel `v0.28.9` 作为唯一主 CNI,并安装 Multus `v4.3.0` 和 Whereabouts `v0.9.4`。测试覆盖不同 worker 上 Pod 的 `net1` 跨节点 ping 和 HTTP、Flannel 主网络 Service/DNS、MTU、删除及 IPAM 地址回收。 +::: + +::: warning 安全提示 +节点 DaemonSet 需要进入 Pod network namespace 并创建 TUN,因此会使用 privileged、hostPID 和宿主 `/run`。生产环境必须固定到包含 CNI 的正式 EasyTier 版本,不要直接使用可变的 `unstable` 标签。 +::: + +## 1. 创建网络密钥 + +不要把 EasyTier 网络密钥写进 `NetworkAttachmentDefinition`: + +```sh +kubectl -n kube-system create secret generic easytier-cni \ + --from-literal=network-secret="$EASYTIER_NETWORK_SECRET" +``` + +DaemonSet 会把密钥以 root-only 文件安装到节点。管理 RPC 使用节点上的 root-only Unix socket,不会开放 TCP 管理端口。 + +## 2. 部署节点 DaemonSet + +下载[官方 DaemonSet 清单](https://github.com/EasyTier/EasyTier/blob/main/easytier-contrib/easytier-cni/deploy/daemonset.yaml),把两个 EasyTier 镜像都固定到相同的正式版本,然后部署: + +```sh +kubectl apply -f daemonset.yaml +kubectl -n kube-system rollout status daemonset/easytier-cni +``` + +确认每个节点都安装了插件并启动管理进程: + +```sh +kubectl -n kube-system get pods -l app.kubernetes.io/name=easytier-cni -o wide +``` + +## 3. 创建辅助网络 + +以[官方 NAD 示例](https://github.com/EasyTier/EasyTier/blob/main/easytier-contrib/easytier-cni/deploy/network-attachment-definition.yaml)为基础修改: + +```json +{ + "cniVersion": "1.0.0", + "name": "easytier", + "type": "easytier-cni", + "rpcPortal": "unix:///run/easytier-cni/rpc.sock", + "networkName": "kubernetes", + "networkSecretFile": "/etc/easytier-cni/network-secret", + "peers": ["tcp://192.0.2.10:11010"], + "mtu": 1380, + "timeoutSeconds": 30, + "ipam": { + "type": "whereabouts", + "range": "10.200.0.0/24", + "range_start": "10.200.0.10", + "range_end": "10.200.0.250" + } +} +``` + +- `peers` 必须能通过 Pod 主网卡访问。建议使用 IP 地址,避免 CNI 阶段依赖集群 DNS。 +- EasyTier 地址段不能与节点、Pod、Service、局域网、VPN 或容器运行时网段重叠。 +- `mtu` 是 EasyTier packet MTU;TUN MTU 会扣除协议开销,例如 `1380` 对应 TUN MTU `1360`。 + +完成修改后应用 NAD: + +```sh +kubectl apply -f network-attachment-definition.yaml +``` + +## 4. 为 Pod 添加 EasyTier 网络 + +在 Pod 上添加 Multus 注解: + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: easytier-client + annotations: + k8s.v1.cni.cncf.io/networks: easytier +spec: + containers: + - name: client + image: busybox:1.37.0 + command: ["/bin/sh", "-c", "sleep 3600"] +``` + +默认辅助网卡名为 `net1`。检查地址和 Multus 网络状态: + +```sh +kubectl exec easytier-client -- ip address show net1 +kubectl get pod easytier-client \ + -o jsonpath='{.metadata.annotations.k8s\.v1\.cni\.cncf\.io/network-status}' +``` + +两个 Pod 都连接到相同 EasyTier 网络后,可以直接通过各自的 `net1` IPv4 地址通信。 + +## 运维注意事项 + +- 删除 Pod 时,CNI 会删除对应 EasyTier 实例并释放 Whereabouts 地址。 +- 节点 daemon 会在 `/var/lib/easytier-cni/configs` 持久化 attachment;目录包含网络凭据,只允许 root 访问。 +- 更新 Secret 后,需要重启 `easytier-cni` DaemonSet,并重建已挂载 EasyTier 网络的 Pod。 +- 卸载后按需删除节点上的 `/etc/easytier-cni` 和 `/var/lib/easytier-cni`。 + +完整配置、自动化 netns 测试和 Kind 三节点测试位于 [`easytier-contrib/easytier-cni`](https://github.com/EasyTier/EasyTier/tree/main/easytier-contrib/easytier-cni)。