Add custom train-recipe support (describe schema/template, train_recipe on create_training, CLI) - #510
Conversation
…explicit recipes Addresses two Jarbas review warnings on #510 plus trivia: Non-object JSON flag values (e.g. --hyperparameters '5', --train-recipe '[1]') previously passed the syntax-only parse and either crashed with a raw TypeError in build_train_recipe (after a wasted recipe fetch) or shipped a malformed trainRecipe to the server. _parse_json_flag now requires a JSON object and exits with a structured error before any network call. epochs was still silently ignored when combined with an explicit train_recipe: the server dense-fills recipe hyperparameters, so recipe epochs always beats body epochs, and a template-derived recipe + --epochs 5 smoke run would train the full default epochs at GPU cost. Explicit recipes are now run through the same build_train_recipe fold (setdefault semantics: an epochs set inside the recipe still wins; the fold creates the hyperparameters key when a hand-written recipe omits it). Docstrings and --train-recipe help updated from "recipe epochs takes precedence" to the fold semantics. Trivia: sweep docstring example now imports time (it calls time.sleep); CHANGELOG.md gains the Unreleased entry for the v2 trainings SDK methods, CLI command, and flags (#510). Tests: +7 (util fold-creates-missing-key; Version explicit-recipe fold / recipe-epochs-wins / missing-hyperparameters-key; CLI non-object --hyperparameters and --train-recipe assert structured error with no adapter call, --train-recipe + --epochs asserts merged body). Full suite 916 passing; ruff + mypy clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…explicit recipes Addresses two Jarbas review warnings on #510 plus trivia: Non-object JSON flag values (e.g. --hyperparameters '5', --train-recipe '[1]') previously passed the syntax-only parse and either crashed with a raw TypeError in build_train_recipe (after a wasted recipe fetch) or shipped a malformed trainRecipe to the server. _parse_json_flag now requires a JSON object and exits with a structured error before any network call. epochs was still silently ignored when combined with an explicit train_recipe: the server dense-fills recipe hyperparameters, so recipe epochs always beats body epochs, and a template-derived recipe + --epochs 5 smoke run would train the full default epochs at GPU cost. Explicit recipes are now run through the same build_train_recipe fold (setdefault semantics: an epochs set inside the recipe still wins; the fold creates the hyperparameters key when a hand-written recipe omits it). Docstrings and --train-recipe help updated from "recipe epochs takes precedence" to the fold semantics. Trivia: sweep docstring example now imports time (it calls time.sleep); CHANGELOG.md gains the Unreleased entry for the v2 trainings SDK methods, CLI command, and flags (#510). Tests: +7 (util fold-creates-missing-key; Version explicit-recipe fold / recipe-epochs-wins / missing-hyperparameters-key; CLI non-object --hyperparameters and --train-recipe assert structured error with no adapter call, --train-recipe + --epochs asserts merged body). Full suite 916 passing; ruff + mypy clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
b4dea8a to
f625a3b
Compare
Per review direction on #510: no need for both hyperparameters and train_recipe — one way to do it. The template-fetch-and-merge path is removed; recipe training is now describe -> edit -> submit: template = version.describe_train_recipe("rfdetr-medium")["template"] recipe = copy.deepcopy(template) recipe["hyperparameters"] = {"lr": lr} version.start_training(model_type="rfdetr-medium", train_recipe=recipe) Version.start_training drops the hyperparameters and online_augmentation kwargs, the describe-and-merge branch, and both now-dead ValueError guards (both-given; knobs-without-model_type). The CLI drops --hyperparameters from `train start` and the bare `train` callback; _start_v2 no longer fetches the template. Kept: the `roboflow train recipe` describe command and the get_train_recipe adapter (describe stays a first-class capability); --train-recipe with its non-object JSON validation; and the epochs fold into an explicit recipe (correctness guard for the pre-existing epochs param — the server resolves dense-filled recipe epochs ahead of the body value). build_train_recipe shrinks accordingly and is renamed fold_epochs_into_recipe (deepcopy + setdefault + creates a missing hyperparameters key + never mutates input). Docs: CLI-COMMANDS.md shows the describe -> edit -> --train-recipe flow; CHANGELOG Unreleased entry updated; start_training docstring sweep example rewritten around the template (import copy + import time included for copy-paste). Tests: merge-path, knob-validation, and --hyperparameters cases removed; epochs-fold and non-object --train-recipe validation kept. Full suite 902 passing; ruff + mypy clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
4660e81 to
540ebcf
Compare
…explicit recipes Addresses two Jarbas review warnings on #510 plus trivia: Non-object JSON flag values (e.g. --hyperparameters '5', --train-recipe '[1]') previously passed the syntax-only parse and either crashed with a raw TypeError in build_train_recipe (after a wasted recipe fetch) or shipped a malformed trainRecipe to the server. _parse_json_flag now requires a JSON object and exits with a structured error before any network call. epochs was still silently ignored when combined with an explicit train_recipe: the server dense-fills recipe hyperparameters, so recipe epochs always beats body epochs, and a template-derived recipe + --epochs 5 smoke run would train the full default epochs at GPU cost. Explicit recipes are now run through the same build_train_recipe fold (setdefault semantics: an epochs set inside the recipe still wins; the fold creates the hyperparameters key when a hand-written recipe omits it). Docstrings and --train-recipe help updated from "recipe epochs takes precedence" to the fold semantics. Trivia: sweep docstring example now imports time (it calls time.sleep); CHANGELOG.md gains the Unreleased entry for the v2 trainings SDK methods, CLI command, and flags (#510). Tests: +7 (util fold-creates-missing-key; Version explicit-recipe fold / recipe-epochs-wins / missing-hyperparameters-key; CLI non-object --hyperparameters and --train-recipe assert structured error with no adapter call, --train-recipe + --epochs asserts merged body). Full suite 916 passing; ruff + mypy clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per review direction on #510: no need for both hyperparameters and train_recipe — one way to do it. The template-fetch-and-merge path is removed; recipe training is now describe -> edit -> submit: template = version.describe_train_recipe("rfdetr-medium")["template"] recipe = copy.deepcopy(template) recipe["hyperparameters"] = {"lr": lr} version.start_training(model_type="rfdetr-medium", train_recipe=recipe) Version.start_training drops the hyperparameters and online_augmentation kwargs, the describe-and-merge branch, and both now-dead ValueError guards (both-given; knobs-without-model_type). The CLI drops --hyperparameters from `train start` and the bare `train` callback; _start_v2 no longer fetches the template. Kept: the `roboflow train recipe` describe command and the get_train_recipe adapter (describe stays a first-class capability); --train-recipe with its non-object JSON validation; and the epochs fold into an explicit recipe (correctness guard for the pre-existing epochs param — the server resolves dense-filled recipe epochs ahead of the body value). build_train_recipe shrinks accordingly and is renamed fold_epochs_into_recipe (deepcopy + setdefault + creates a missing hyperparameters key + never mutates input). Docs: CLI-COMMANDS.md shows the describe -> edit -> --train-recipe flow; CHANGELOG Unreleased entry updated; start_training docstring sweep example rewritten around the template (import copy + import time included for copy-paste). Tests: merge-path, knob-validation, and --hyperparameters cases removed; epochs-fold and non-object --train-recipe validation kept. Full suite 902 passing; ruff + mypy clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
7af393d to
8d15b56
Compare
|
/jarbas review, considering https://github.com/roboflow/roboflow/pull/13699 and https://github.com/roboflow/roboflow-mcp/pull/120 as well. Ensure you read the PR descriptions as they contain the bulk of the test evidence. |
Rebased #510 onto the merged MMPV surface (#497), deduplicating against its canonical v2 trainings plumbing instead of carrying parallel implementations: - rfapi.get_train_recipe — GET .../v2/trainings/recipe (schema + editable template) for a model type. New; no #497 equivalent. - rfapi.create_training_v2 gains train_recipe (camelCase trainRecipe body key) and business_context; replaces this branch's former parallel rfapi.create_training. rfapi.list_version_trainings/get_version_training are dropped in favor of #497's list_trainings_for_version/get_training. - Version.describe_train_recipe(model_type) — SDK wrapper for the describe endpoint. - Version.create_training gains train_recipe/business_context; replaces the former Version.start_training. Keeps the pre-network guard (train_recipe requires model_type — recipes are minted per model type) and folds a top-level epochs into the recipe's hyperparameters via util.train_recipe.fold_epochs_into_recipe (recipe epochs wins; missing hyperparameters key is created), because the server resolves dense-filled recipe epochs ahead of the body value. Version.list_trainings/get_training are dropped: #497's trainings()/Training.refresh() cover them. - Tests re-homed onto the canonical functions; #497's exact-kwargs create assertion extended with the two new pass-through kwargs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- roboflow train recipe -p <project> -v <N> -m <model_type> — print the recipe schema and ready-to-edit template as JSON (both output modes). - --train-recipe on train start (and the bare train callback): inline JSON or a curl-style @file reference. Routes through the canonical rfapi.create_training_v2 (from #497) and prints the new trainingId instead of blocking. Structured errors, all before any network call: unreadable @file, invalid JSON, non-object JSON, and a missing model type (recipes are minted per model type). - --epochs is folded into the recipe's hyperparameters unless the recipe already sets epochs (the server resolves recipe epochs ahead of the body value). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CHANGELOG entry reconciled with #497's 1.4.0 MMPV section: #510 now describes only what it adds on top of the canonical surface — the recipe describe endpoint, train_recipe/business_context on Version.create_training, the epochs fold, and the CLI recipe surface. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
0d308a6 to
b047ebe
Compare
|
Status: ❌ Request Changes — Gist: https://gist.github.com/jarbasrf/9f37f34dfe806a002ab789e6885d414a. Resumo: An explicitly empty |
An empty supplied value (classic unset shell variable expansion) passed the truthiness check and fell through to the legacy training endpoint, silently starting a different, billable training. Presence is now 'is not None', so the empty string reaches the JSON validator and fails with a structured error before any network call. Addresses external review feedback on PR #510. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Status: ✅ Approve — Gist: https://gist.github.com/jarbasrf/c90908d109b3044acdda67b4e193ffce. Resumo: The prior empty |
Unrelated to recipes: it is a pre-existing platform field the MCP tool already exposes, and it slipped into this PR as payload parity rather than a considered scope decision. If SDK parity is wanted it can ride its own follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1.4.0 already shipped; this PR's additions ride the next release. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mkaic
left a comment
There was a problem hiding this comment.
LGTM!
I think your new _start_v2 function has some duplicated logic from the new MMPV Version.create_training, but after chatting with Fable I think the right thing to do would be to make that method use your code, not the other way around, I think the way you implemented things is potentially more idiomatic. Anyhow, not really in-scope for this PR, so approving, and I can always open a consolidation/cleanup PR myself later.
What does this PR do?
Adds SDK + CLI support for training with a custom
trainRecipe(hyperparameter/augmentation sweeps) against the v2 trainings API. The surface is deliberately consolidated ontrain_recipeas the single recipe input: describe → edit the server template → submit.roboflow/adapters/rfapi.py):get_train_recipe(GET.../v2/trainings/recipe?modelType=) is new; the canonical v2 create adapter from Feature: Multiple Models Per Version #497,create_training_v2, gainstrain_recipe(camelCasetrainRecipebody key). Listing/fetching runs uses Feature: Multiple Models Per Version #497'slist_trainings_for_version/get_trainingunchanged.roboflow/core/version.py):Version.describe_train_recipe(model_type)(schema + ready-to-submit template + availability) is new; Feature: Multiple Models Per Version #497's canonicalVersion.create_training(...)gainstrain_recipe(returns aTrainingobject — poll via.refresh(), list runs viaVersion.trainings()). The model_type guard and epochs fold run before any network call. The blockingVersion.train()is untouched.roboflow/util/train_recipe.py):fold_epochs_into_recipe— the server dense-fills recipe hyperparameters and resolves recipe epochs ahead of bodyepochs, so a providedepochs=is folded into the submitted recipe unless the recipe sets its own (explicit recipe value wins).roboflow train recipe -p <project> -v <N> -m <model_type>(prints schema+template JSON) and--train-recipe '<json>'onroboflow train start(v2 path, prints the new trainingId; non-object JSON fails fast with a structured error before any network call).Related Issue(s): Companion to roboflow/roboflow#13699 (the backend describe endpoint —
describe_train_recipe/train reciperequire it; deploy order: platform first, then release this). MCP companion (same consolidated shape): roboflow/roboflow-mcp#120. Rebased on #497 (the canonical MMPV SDK surface): this PR now adds only the recipe capabilities on top — no duplicated trainings plumbing. Note one adopted behavior change: training creation returns aTrainingobject rather than a raw dict.Type of Change
Testing
Test details:
Staging evidence:
roboflow train recipe -p lee-sandbox/chess-pieces-fmhpz -v 110 -m rfdetr-mediumCLI outputtrain_recipe.jsonChecklist
CLI-COMMANDS.md+CHANGELOG.mdupdated; docs.roboflow.com reference is a tracked follow-upAdditional Context
Sweep usage the PR enables:
Recipe-based training requires the workspace's online-augmentation training entitlement (403 otherwise); the describe response's
availabilityfield reports both gates up front.🤖 Generated with Claude Code