Document batch upgrade boundaries and S3 readiness checks

content/administration/batch-framework/_index.md

@@ -18,7 +18,7 @@ A batch file defines one job task.
 
 Once started, MinIO starts processing the job. Time to completion depends on the resources available to the deployment.
 
-Ordinary job failures use the retry limits in the definition. The next SILO release distinguishes authorization refusal from temporary pauses as described below.
+Ordinary job failures use the retry limits in the definition. SILO 20261109 distinguishes authorization refusal from temporary pauses as described below.
 
 The MinIO Batch Framework supports the following job types:
 
@@ -29,7 +29,7 @@ The MinIO Batch Framework supports the following job types:
 | [expire](/administration/batch-framework-job-expire/#minio-batch-framework-expire-job) | Perform a one-time immediate expiration of objects in a bucket. |
 {.wide .expand}
 
-## Next SILO release: authorization and retries {#silo-lifecycle}
+## SILO 20261109: authorization and retries {#silo-lifecycle}
 
 SILO binds local batch operations to the authenticated submitter at job creation. The credentials in a definition are used for its remote connections; they do not grant local server authority. IAM changes are checked again when actions are taken. An explicit denial, expired or revoked credential, or changed submitter binding ends the job with `failureReason`. Statements that need request context unavailable to a background job refuse the action.
 
@@ -41,6 +41,15 @@ Startup discovery also retries failed manifest listings and storage reads with b
 
 Completed jobs remain completed after restart, including legacy jobs and jobs whose credentials later expire or change. Unfinished jobs from releases without submitter binding require resubmission. New bound definitions and authorized progress reports use formats rejected by older executors. This fences those records on rollback, but does not make a mixed cluster safe: an old node can still accept and execute its own old-format jobs. Stop submissions and drain or cancel old jobs before changing versions, and do not downgrade with pending authorized jobs.
 
+### Upgrading and rolling back 20261109 {#upgrade-20261109}
+
+This release deliberately introduces a batch-record format boundary: new job manifests use the `SILO-BATCH` v2 envelope and authorized progress reports also use v2. The 20261026 executor cannot execute or resume these records. Ordinary S3 object formats are unaffected, but batch jobs cannot follow a binary downgrade transparently.
+
+1. Stop submissions to every node before upgrading. Inventory jobs with `mcli batch list ALIAS` and `mcli batch status ALIAS JOBID`. Drain running, pending and paused jobs, or cancel them with `mcli batch cancel ALIAS JOBID`; preserve their original YAML definitions and records of completed work.
+2. Resume submissions only after every node is upgraded. Unfinished legacy jobs do not acquire an authorization binding automatically: resubmit with an identity that has the required data permissions. Reconcile completed work before resubmission to avoid unwanted repeated work. Completed legacy jobs need no resubmission.
+3. Do not run batch jobs during a mixed-version upgrade: old nodes can still accept and execute their own old-format jobs without the new submitter authorization checks.
+4. Before rollback, stop submissions again and drain or cancel all new jobs, including paused jobs. Do not expect 20261026 to resume their progress, strip the format envelope, or edit authorization records. Preserve the records and reconcile results on a compatible release before deciding whether to resubmit.
+
 Use [mcli batch status](/reference/minio-mc/mc-batch-status/#silo-job-outcomes) (`20261109`) to distinguish pauses, terminal failures, and completion with protected work left in place. `infraAdmin` still has other infrastructure administration capabilities; these job checks do not turn that role into a strict data-isolation boundary.
 
 ## MinIO Batch CLI {#minio-batch-cli}
@@ -83,7 +92,7 @@ The [`mc batch`](/reference/minio-mc/mc-batch/#command-mc.batch) commands includ
 
 ## Access to `mc batch` {#access-to-mc-batch}
 
-The submitting identity needs the relevant administration action and the local [data permissions](/administration/identity-access-management/policy-based-access-control/#minio-policy) for the job. Credentials in the definition must separately permit its remote operations. See the next-release rules above.
+The submitting identity needs the relevant administration action and the local [data permissions](/administration/identity-access-management/policy-based-access-control/#minio-policy) for the job. Credentials in the definition must separately permit its remote operations. See the 20261109 rules above.
 
 The user executing the batch job must have the following permissions. You can alternatively restrict users from accessing these functions by blocking or limiting access to these actions:
 

content/administration/batch-framework/_index.zh.md

@@ -18,7 +18,7 @@ MinIO 批处理框架允许使用 YAML 格式的作业定义文件（“batch fi
 
 启动后，MinIO 会开始处理该作业。 完成所需时间取决于部署可用的资源。
 
-普通作业故障遵循定义中的重试次数上限。下一个 SILO 版本会区分授权拒绝与临时暂停，详见下文。
+普通作业故障遵循定义中的重试次数上限。SILO 20261109 区分授权拒绝与临时暂停，详见下文。
 
 MinIO 批处理框架支持以下作业类型：
 
@@ -29,7 +29,7 @@ MinIO 批处理框架支持以下作业类型：
 | [expire](/zh/administration/batch-framework-job-expire/#minio-batch-framework-expire-job) | 对存储桶中的对象执行一次性立即过期操作。 |
 {.wide .expand}
 
-## SILO 下一个版本：授权与重试 {#silo-lifecycle}
+## SILO 20261109：授权与重试 {#silo-lifecycle}
 
 SILO 创建作业时，把本地批处理操作绑定到经过认证的提交者。定义中的凭据用于远端连接，不会授予本地服务器权限；执行动作时再次检查当时的 IAM 权限。显式拒绝、凭据过期或撤销、提交者绑定发生变化，都会以 `failureReason` 结束作业。策略依赖后台作业无法取得的请求上下文时，也会拒绝相应动作。
 
@@ -41,6 +41,15 @@ Push、Pull 和 Snowball 复制作业在必要的授权依赖或报告读取暂
 
 已完成作业在重启后仍保持完成，包括旧格式作业和完成后凭据过期或变更的作业。来自不记录提交者绑定的旧版本、且尚未完成的作业需要重新提交。新绑定定义和已授权进度报告采用旧执行器会拒绝的格式。这可以阻止旧程序接管这些记录，但不能保证混合版本集群安全：旧节点仍能接收和执行自己的旧格式作业。切换版本前停止提交，并排空或取消旧作业；仍有等待执行的授权作业时不要降级。
 
+### 20261109 升级与回退 {#upgrade-20261109}
+
+此版本对批处理记录有意设置了格式边界：新作业清单使用 `SILO-BATCH` v2 封装，已授权的进度报告也使用 v2；20261026 不能执行或恢复这些记录。这不改变普通 S3 对象的格式，但批处理作业不能随二进制无缝降级。
+
+1. 升级前停止向所有节点提交新作业，用 `mcli batch list ALIAS` 和 `mcli batch status ALIAS JOBID` 盘点作业。等待运行中、等待中和暂停中的作业结束，或用 `mcli batch cancel ALIAS JOBID` 取消；保留原始 YAML 定义和已完成工作的记录。
+2. 升级全部节点后再恢复提交。旧格式未完成作业不会自动获得新授权绑定，需要用有相应数据权限的身份重新提交。先核对已完成的部分，避免重新扫描产生不需要的重复工作；旧作业已完成则无需重交。
+3. 混合版本期间不要运行批处理：旧节点仍能接收并执行自己的旧格式作业，不受新版本的提交者授权约束。
+4. 回退前同样停止提交，排空或取消全部新作业，包括暂停的作业。不要指望 20261026 接续新作业的进度，也不要手工剥除格式头或编辑授权记录。保留证据，在兼容版本上核对结果后再决定是否重交。
+
 使用 [mcli batch status](/zh/reference/minio-mc/mc-batch-status/#silo-job-outcomes)（`20261109`）区分暂停、终态失败和仍有受保护工作未处理的完成结果。`infraAdmin` 仍有其他基础设施管理能力，这些作业检查不把该角色变成严格的数据隔离边界。
 
 <a id="minio-cli" data-td-print-local></a>
@@ -85,7 +94,7 @@ Push、Pull 和 Snowball 复制作业在必要的授权依赖或报告读取暂
 
 ## `mc batch` 的访问权限 {#access-to-mc-batch}
 
-提交身份需要相应的管理动作权限，以及作业需要的本地 [数据权限](/zh/administration/identity-access-management/policy-based-access-control/#minio-policy)。定义中的凭据还必须具备远端操作所需的权限。下一个版本的具体规则见上文。
+提交身份需要相应的管理动作权限，以及作业需要的本地 [数据权限](/zh/administration/identity-access-management/policy-based-access-control/#minio-policy)。定义中的凭据还必须具备远端操作所需的权限。20261109 的具体规则见上文。
 
 执行批处理作业的用户必须具有以下权限。 也可以通过阻止或限制对这些操作的访问，来限制用户使用这些功能：
 

content/compatibility/feature/healthcheck.md

@@ -103,7 +103,7 @@ The table below is verified against the handler source, not quoted from document
 | `/minio/health/cluster/read` | the read-quorum version of the above | as above | |
 {.wide .expand}
 
-Consequences worth spelling out: `live` and `ready` are *liveness*-grade signals — they do not tell you the node can serve objects; only the `cluster` pair does. That is exactly why the cluster pair must stay out of per-container probes (scope, log noise, cascading restarts) and why it is the right tool for operational questions like "may I take this node down?" (`--maintenance`, where 200 means safe and 412 means you would lose HA).
+`live` and `ready` are process-level signals and do not prove object access. The `cluster` pair checks initialization and reported erasure-set quorum, but does not guarantee that the receiving node has mounted its remote drives; see the [restart window and data-path checks](/operations/monitoring/healthcheck-probe/#startup-readiness-window). Use them for operational quorum and maintenance checks, not per-container liveness or prerequisites for peer discovery. Resuming applications also requires a bounded signed S3 write/read check.
 
 ## The distroless variant {#distroless}
 
@@ -130,7 +130,7 @@ The decisions folded into that sketch:
 
 - **`ENTRYPOINT` is the binary itself.** `docker run pgsty/silo:distroless server /data` — no argv-translation script, because there is no shell to run one. The classic image's `MINIO_USERNAME`/`MINIO_GROUPNAME` privilege-drop path (which needs GNU `chroot` and a writable `/etc/passwd`) is **not supported** in this variant; the supported mechanism is `--user` / Kubernetes `runAsUser`.
 - **`/data` is created in the layer, mode 0777, and the default user stays root for the pilot.** Issue [#55](https://github.com/pgsty/silo/issues/55) demonstrated that declaring `VOLUME ["/data"]` without creating it breaks *every* non-root invocation, and that no entrypoint can repair it after the fact — in distroless there is no entrypoint at all. Creating it world-writable in the layer is the one option that makes all privilege modes work (`--user` included), keeps drop-in parity with the classic image, and its exposure is bounded by the image running a single process. A nonroot-by-default posture (uid 65532) was considered and deferred: it would break the documented bind-mount workflow on UID mismatch, and the pilot's job is to measure friction, not maximize it. Revisit at promotion time, possibly as a `-nonroot` tag.
-- **The health check is baked in, exec-form.** Shell-form `HEALTHCHECK` strings need `/bin/sh` and are impossible here; the JSON-array form is mandatory. Compose inherits an image's `HEALTHCHECK` automatically (with `disable: true` as the escape hatch), so compose users of this variant get working `depends_on: condition: service_healthy` with zero configuration. `ready` rather than `live` because Docker's health status feeds *gating* (start ordering), which is readiness semantics — and the two are identical anyway unless KMS/etcd are configured.
+- **The health check is baked in, exec-form.** Distroless has no shell, so it invokes `silo healthcheck ready` as a JSON array. Compose inherits this check, but `depends_on: condition: service_healthy` confirms only the process/configured-dependency probe. It does not prove storage, IAM or S3 readiness; dependent applications need the separate data-path gate in the [operations guide](/operations/monitoring/healthcheck-probe/#native-probe). Keeping `ready`'s bootstrap semantics avoids making peer discovery depend on completed cluster initialization.
 - **One release-blocking verification**: `HEALTHCHECK` is a Docker extension, absent from the OCI image spec (opencontainers/image-spec#749 is still open), and OCI-media-type builds drop it silently. The publish pipeline must assert `docker inspect` shows the `Health` config on the pushed manifest, or adjust the build's media types until it does.
 - **Naming**: `docker.io/pgsty/silo:<RELEASE>-distroless`, plus a rolling `distroless` tag. A new `Dockerfile.distroless` in the server repo — which, having no download stages, is fully offline and can therefore be built and asserted in CI on every release, closing the "the gate tests a synthetic image, not the shipped one" coverage gap that #55 documented for the classic Dockerfile.
 - **What a user gives up in the variant, stated honestly in its docs**: no `docker exec <container> sh` debugging (use `docker debug` / `kubectl debug` ephemeral containers), no in-image `mc` (use the `pgsty/mc` image or a host-installed `mcli`), no `MINIO_USERNAME` path (use `--user`).

content/compatibility/feature/healthcheck.zh.md

@@ -103,7 +103,7 @@ FLAGS：
 | `/minio/health/cluster/read` | 上一行的读 quorum 版本 | 同上 | |
 {.wide .expand}
 
-值得点破的推论：`live` 和 `ready` 是 *存活* 级别的信号——它们不能告诉你节点能否服务对象，只有 `cluster` 这一对能。这正是 cluster 端点必须远离单容器探针的原因（尺度错位、日志噪音、级联重启），也正是它适合回答运维问题的原因——"我现在可以把这个节点下线吗？"（`--maintenance`：200 表示安全，412 表示会失去高可用）。
+`live` 和 `ready` 是进程级信号，不证明对象可读写。`cluster` 这一对检查初始化和各纠删集上报的仲裁，但也不保证收到请求的节点已挂载全部远端磁盘；详见[重启窗口与数据面检查](/zh/operations/monitoring/healthcheck-probe/#startup-readiness-window)。它们适合运维仲裁与维护检查，不应作为单容器存活探针或节点发现的前置条件。应用恢复还需有界的签名 S3 写入与回读校验。
 
 ## Distroless 变体 {#distroless}
 
@@ -130,7 +130,7 @@ ENTRYPOINT ["/usr/bin/silo"]
 
 - **`ENTRYPOINT` 就是二进制本身。**`docker run pgsty/silo:distroless server /data`——没有 argv 翻译脚本，因为没有 shell 来跑脚本。经典镜像的 `MINIO_USERNAME`/`MINIO_GROUPNAME` 降权路径（依赖 GNU `chroot` 和可写的 `/etc/passwd`）在此变体中 **不受支持**；受支持的机制是 `--user` / Kubernetes `runAsUser`。
 - **`/data` 在镜像层内创建、mode 0777，试点期默认用户保持 root。** Issue [#55](https://github.com/pgsty/silo/issues/55) 证明了：只声明 `VOLUME ["/data"]` 而不创建它，会让 *所有* 非 root 运行方式失败，而且事后没有任何入口脚本能修补——Distroless 里更是压根没有入口脚本。在层内以全员可写方式创建它，是唯一让全部权限模式（包括 `--user`）都能工作的选项，同时与经典镜像保持即插即用的对等；其暴露面被"镜像只运行一个进程"这一事实所限定。nonroot 默认（uid 65532）的姿态经过考虑后推迟：它会在 UID 不匹配时破坏文档记载的 bind-mount 工作流，而试点的任务是测量摩擦，不是制造摩擦。晋升为默认推荐时再议，可能以 `-nonroot` 标签的形式出现。
-- **健康检查内置、exec 形式。** Shell 形式的 `HEALTHCHECK` 字符串需要 `/bin/sh`，在这里不可能存在；JSON 数组形式是唯一选择。Compose 会自动继承镜像的 `HEALTHCHECK`（逃生门是 `disable: true`），所以这个变体的 compose 用户零配置就能得到可用的 `depends_on: condition: service_healthy`。选 `ready` 而不是 `live`，是因为 Docker 健康状态的主要用途是 *门控*（启动顺序），那是就绪语义——况且在没有 KMS/etcd 时两者本就相同。
+- **健康检查内置、exec 形式。** Distroless 没有 shell，所以使用 JSON 数组调用 `silo healthcheck ready`。Compose 会继承该检查，但 `depends_on: condition: service_healthy` 只证明进程及已配置依赖的探针成功。它不证明存储、IAM 或 S3 路径已就绪；依赖应用须按[操作说明](/zh/operations/monitoring/healthcheck-probe/#native-probe)做独立数据面检查。保留 `ready` 的启动语义，避免节点发现依赖尚未完成的集群初始化。
 - **一项发布前必须完成的验证**：`HEALTHCHECK` 是 Docker 扩展，不在 OCI 镜像规范里（opencontainers/image-spec#749 至今开放），OCI 媒体类型的构建会静默丢弃它。发布流水线必须断言推送后的 manifest 上 `docker inspect` 能看到 `Health` 配置，否则就调整构建的媒体类型直到能看到为止。
 - **命名**：`docker.io/pgsty/silo:<RELEASE>-distroless`，外加滚动的 `distroless` 标签。服务端仓库新增 `Dockerfile.distroless`——它没有任何下载阶段，完全离线，因此可以在每次发布的 CI 里真实构建并断言，顺带堵上 #55 记录的那个"门禁测的是合成镜像而非发布镜像"的覆盖缺口。
 - **变体文档里要诚实写明用户失去了什么**：没有 `docker exec <container> sh` 式调试（改用 `docker debug` / `kubectl debug` 临时容器）；镜像内没有 `mc`（改用 `pgsty/mc` 镜像或宿主机安装的 `mcli`）；没有 `MINIO_USERNAME` 路径（改用 `--user`）。