Skip to content

fix(loadtesting): scrub 'protocol' from plu descriptions so the agent shows 'API' #451

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sourabhd-cbu merged 1 commit into from
Sep 28, 2026
Merged

fix(loadtesting): scrub 'protocol' from plu descriptions so the agent shows 'API' #451

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions capability/loadtesting.capability-index.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -823,8 +823,8 @@
"fitsInQuota / remainingAfterEstimate compare the estimate against current quota.",
"For the same check at start time, call startLoadTestRun with dryRun:true.",
"Always use this for a cost or VU-hour estimate of a hypothetical run — it models ramp-up. Do not compute VU-hours by hand from VUs × duration.",
"A protocol (plu) run is billed at a full load-generator pod's capacity (e.g. 1000 VUs for k6), not the VUs it actually uses, and carries a minimum billing floor (5 minutes) — so a small or short plu run can estimate far higher than VUs × duration would suggest. That is expected; report the returned estimatedVuHours as-is rather than second-guessing it as an error.",
"The estimate is for the test's OWN type: a browser (blu) run costs about 10x a protocol (plu) run at the same VUs and duration (browser VUs carry a 10x weight). So the number depends heavily on whether the test is plu, blu or hybrid — always state which type the estimate is for. When the user asks about a hypothetical run without fixing the type, do not silently inherit the type of whatever test you priced against: say the type explicitly, and if it is genuinely open, give both the plu and blu figures (they differ ~10x) or ask which they mean."
"An API (plu) run is billed at a full load-generator pod's capacity (e.g. 1000 VUs for k6), not the VUs it actually uses, and carries a minimum billing floor (5 minutes) — so a small or short plu run can estimate far higher than VUs × duration would suggest. That is expected; report the returned estimatedVuHours as-is rather than second-guessing it as an error.",
"The estimate is for the test's OWN type: a browser (blu) run costs about 10x an API (plu) run at the same VUs and duration (browser VUs carry a 10x weight). So the number depends heavily on whether the test is plu, blu or hybrid — always state which type the estimate is for. When the user asks about a hypothetical run without fixing the type, do not silently inherit the type of whatever test you priced against: say the type explicitly, and if it is genuinely open, give both the plu and blu figures (they differ ~10x) or ask which they mean."
],
"returns": [
"estimatedVuHours",
Expand Down Expand Up @@ -947,7 +947,7 @@
"name": "testType",
"type": "string",
"required": true,
"description": "plu = protocol/API load; blu = real-browser load; hybrid = BOTH in one test (one PLU leg + one BLU leg, supplied via children). These tokens are INTERNAL wire values — send them here but never show them to the user; the user-facing labels are 'API' (plu), 'Browser' (blu) and 'Hybrid' (hybrid). All three are valid, user-selectable options — hybrid is not a rare/advanced variant, offer it alongside the other two.",
"description": "plu = API load (server-side, non-browser); blu = real-browser load; hybrid = BOTH in one test (one PLU leg + one BLU leg, supplied via children). These tokens are INTERNAL wire values — send them here but never show them to the user; the user-facing labels are 'API' (plu), 'Browser' (blu) and 'Hybrid' (hybrid). All three are valid, user-selectable options — hybrid is not a rare/advanced variant, offer it alongside the other two.",
"values": [
"plu",
"blu",
Expand All @@ -958,7 +958,7 @@
"name": "framework",
"type": "string",
"required": true,
"description": "PLU (protocol) frameworks: k6, jmeter, gatling, locust. BLU (browser) frameworks: playwright, selenium, webdriverio, nightwatch, lcncnightwatch. Must match testType — a PLU framework for testType plu, a BLU framework for testType blu (the backend rejects a mismatched pair).",
"description": "PLU (API) frameworks: k6, jmeter, gatling, locust. BLU (browser) frameworks: playwright, selenium, webdriverio, nightwatch, lcncnightwatch. Must match testType — a PLU framework for testType plu, a BLU framework for testType blu (the backend rejects a mismatched pair).",
"values": [
"k6",
"jmeter",
Expand Down Expand Up @@ -989,7 +989,7 @@
{
"name": "children",
"type": "array",
"description": "Child scenarios for a hybrid test. A hybrid is exactly one PLU (protocol) leg + one BLU (browser) leg — each leg carries its own framework + scriptRef (a sample or an uploaded script). Send the BLU leg as the top-level testType='blu'/framework/config and put the PLU leg in children[] (one entry, testType='plu' with its framework + config.scriptRef). The backend requires exactly one PLU and one BLU leg and rejects any other composition."
"description": "Child scenarios for a hybrid test. A hybrid is exactly one PLU (API) leg + one BLU (browser) leg — each leg carries its own framework + scriptRef (a sample or an uploaded script). Send the BLU leg as the top-level testType='blu'/framework/config and put the PLU leg in children[] (one entry, testType='plu' with its framework + config.scriptRef). The backend requires exactly one PLU and one BLU leg and rejects any other composition."
},
{
"name": "idempotencyKey",
Expand Down Expand Up @@ -1026,7 +1026,7 @@
"Script upload is two-phase: send pendingScriptUpload to get a presigned uploadUrl + s3Key, PUT the file, then call create again with the scriptRef source and s3Key returned in the response's nextStep.",
"To spin up a test from a built-in sample with NO file handling (the dashboard's 'use sample script'), pass scriptRef:{source:'sample'} — the platform's bundled sample for the given framework is used. For Selenium, also pass scriptRef.subType (testng|junit|vanillajava|python|pytest|jest|serenity-cucumber) to pick the binding. testType and framework are still needed; the load profile defaults to constant 1 VU / 30s (see below).",
"Pass idempotencyKey so a retried create does not duplicate the test.",
"Only THREE things must be known before creating, and each is taken from the user's request when they've stated it — ask ONLY for the ones still missing, and ask about nothing else: (1) testType — plu (protocol / API load), blu (real-browser load) or hybrid (both); (2) framework — k6 / jmeter / gatling / locust for plu, playwright / selenium / webdriverio / nightwatch for blu; (3) the script — a built-in sample (scriptRef:{source:'sample'}; for Selenium also pass scriptRef.subType) OR the user's own script via the pendingScriptUpload flow. Do not guess a framework/testType, but do not re-ask for one the user already gave.",
"Only THREE things must be known before creating, and each is taken from the user's request when they've stated it — ask ONLY for the ones still missing, and ask about nothing else: (1) testType — plu (API load), blu (real-browser load) or hybrid (both); (2) framework — k6 / jmeter / gatling / locust for plu, playwright / selenium / webdriverio / nightwatch for blu; (3) the script — a built-in sample (scriptRef:{source:'sample'}; for Selenium also pass scriptRef.subType) OR the user's own script via the pendingScriptUpload flow. Do not guess a framework/testType, but do not re-ask for one the user already gave.",
"There is no clone capability. To duplicate a test, getLoadTest the source and copy its full config into this create — vuRamp/vus, durationSec, loadGeneratorLocations and slaThresholds included; nothing is inherited from the source, so anything you omit is dropped.",
"DEFAULTS apply ONLY after the three mandatory fields (testType, framework, script) are known — they never substitute for a missing one of those. Everything BEYOND those three — do not ask about it; apply the default unless the user explicitly specifies otherwise: loadProfile='constant', config.vus=1, config.durationSec=30, region us-east-1 (omit loadGeneratorLocations so the backend defaults it), and NO advanced features (do not enable log capture, response capture, artifacts, thresholds, etc.). Send config.vus=1 + config.durationSec=30 with no vuRamp for the default constant profile.",
"Only when the user explicitly asks for a different load shape, set config.loadProfile: 'ramping' (config.vuRamp: array of {vus, durationSec} stages; ramp up/down is inferred from adjacent VU deltas), 'iterations' (PLU only — config.iterations = fixed run count, plus config.vus), or 'throughput' (PLU and only JMeter/Gatling/locust/k6 — config.targetRps + config.maxVus + config.durationSec). A stage's own 'type' is NOT honored — every stage persists as a hold.",
Expand Down