Skip to content

docs(readme): make the top-level README sample runnable and safe - #695

Open
deanq wants to merge 7 commits into
mainfrom
deanq/sls-1006-readme-sample
Open

deanq wants to merge 7 commits into
mainfrom
deanq/sls-1006-readme-sample

Conversation

@deanq

@deanq deanq commented Nov 7, 2026

Copy link
Copy Markdown
Member

What

The ## Example sample in the top-level README is the first code most users copy. As written, it doesn't work, and it can leave a pod billing after the script exits:

Problem Fix
The sync entrypoint called finetune.spawn(...) and returned. A spawned task keeps its pod until the job is awaited or cancelled, and TaskJob.wait()/cancel() are async, so the sync entrypoint could not clean up. Use finetune.remote(...), which terminates the pod when it returns. A short paragraph shows spawn.aio() with wait()/cancel() from an async entrypoint.
finetune returned final_loss, which was never defined. A small real training loop that saves a checkpoint to the mounted volume.
chat returned vLLM RequestOutput objects. Remote results must be JSON values. Return the generated text.
Secret("hf-token") with no step to create it. Add rp secret add hf-token.
The models volume was mounted on chat but never used. Nothing said the volume persists. Use the volume only for finetune checkpoints, and note how to delete it.

The task now runs on one H100 instead of two, since the sample loop doesn't need more.

Tests

New tests/test_readme_example.py extracts the README's first python block and checks that:

  • every name it uses is defined (this failed on final_loss before the fix)
  • app discovery accepts it (spec validation passes)
  • a sync @local_entrypoint doesn't call .spawn() on a task (this failed on finetune before the fix)

Test plan

  • New tests failed against the old README and pass against the new one
  • make quality-check (2776 passed)
  • Checked rp secret add, rp volume delete VOLUME_ID, Invoker.aio, and TaskJob.wait/cancel against the CLI and source
  • Not run live: it provisions H100s and pulls a gated model

- the sync entrypoint called finetune.spawn() and exited, which
  leaves the task's pod running; use .remote(), which terminates
  the pod on return, and document spawn.aio() plus wait()/cancel()
- finetune returned an undefined final_loss; give it a small real
  training loop that writes a checkpoint to the volume
- chat returned vLLM RequestOutput objects, which are not JSON
  values; return the generated text
- add the `rp secret add hf-token` step and volume cleanup
- add tests that the example defines every name it uses, passes
  app discovery, and does not spawn tasks from a sync entrypoint
@celine-runpod celine-runpod changed the title docs(readme): make the example runnable and safe to copy docs(readme): make the example runnable and safe to copy (SLS-1006) Nov 9, 2026
@celine-runpod celine-runpod changed the title docs(readme): make the example runnable and safe to copy (SLS-1006) docs(readme): make the top-level README sample runnable and safe (SLS-1006) Nov 9, 2026
@celine-runpod

Copy link
Copy Markdown
Contributor

bugbot run

@deanq
deanq requested a balanced review from Copilot November 9, 2026 18:03
@deanq deanq changed the title docs(readme): make the top-level README sample runnable and safe (SLS-1006) docs(readme): make the top-level README sample runnable and safe Nov 9, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Non-positive step counts leave loss undefined, causing an UnboundLocalError after the persisted checkpoint has already been overwritten with an untrained model.

1 open issue
What changed in this PR

Makes the README’s introductory SDK example runnable and safer to copy, with explicit task cleanup and persistent-volume guidance.

Changes:

  • Returns generated text from inference and adds a training loop that saves a checkpoint.
  • Uses blocking task execution and documents async cleanup, secret setup, and volume deletion.
  • Adds regression checks for undefined names, app discovery, and synchronous task spawning.
File Description
tests/​test_readme_example.py Adds validation checks for the README example.
README.md Updates the example and documents resource setup and cleanup.

Review effort: Balanced

Comment thread README.md Resolved
deanq and others added 3 commits November 9, 2026 10:11
With steps <= 0 the training loop never ran, so loss was unbound: the
example overwrote the checkpoint with an untrained model and then raised
UnboundLocalError on return. Guard the step count first so a bad call
fails before touching the checkpoint.
Comment thread README.md Resolved

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants