Optional CI / scheduled GPU smoke tests

[tetsuo-cpp]

Summary

Add a non-blocking or scheduled CI path that runs a small subset of GPU tests on real CUDA hardware (Vast.ai or a self-hosted runner), so device regressions are not only caught manually.

Motivation

.github/workflows/ci.yml builds on ubuntu and runs LIT + clang-tidy + format + ruff. It never executes PTX. Shared-memory, address-space, and runner bugs have historically needed manual Vast runs.

Full gpu_test on every PR may be too slow/costly; a smoke subset on a schedule or workflow_dispatch is enough.

Proposed design

1. Workflow gpu-smoke.yml:
   • workflow_dispatch + weekly schedule
   • Secrets: VASTAI_API_KEY (or self-hosted GPU runner)
   • Run a short pytest marker, e.g. @pytest.mark.gpu_smoke
2. Smoke cases: addition, one shared+barrier kernel, one multi-param store (keep under ~N minutes)
3. Status: report as a separate check; do not hard-gate canon until stable
4. Document cost controls (instance type, idle teardown — reuse existing conftest patterns)

Acceptance criteria

• Workflow file exists and is documented in README/AGENTS
• Smoke marker selects a tiny subset of tests
• Failures upload logs as CI artifacts
• Teardown of rented instances is reliable (no orphan cost)
• Clear skip behavior when secrets are absent (fork PRs)

Related

• Algorithm-level GPU tests (separate; smoke is a subset)
• SSH reliability (Fix SSH timing issues in GPU test infrastructure #48 closed)
• Runner redesign (Replace warpforth-runner C++ CLI with Python cuda-python + JSON protocol #50)