Skip to content

docs(cloud): set the rate limit high enough for the selected template set - #242

Open
patrick-projectdiscovery wants to merge 6 commits into
mainfrom
docs/rate-limit-coverage-impact
Open

docs(cloud): set the rate limit high enough for the selected template set#242
patrick-projectdiscovery wants to merge 6 commits into
mainfrom
docs/rate-limit-coverage-impact

Conversation

@patrick-projectdiscovery

@patrick-projectdiscovery patrick-projectdiscovery commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Why

cloud/scanning/parameters.mdx is the only page documenting the per-host rate limit, and it presented a slower scan as the only cost of a low value. Scans run under a per-host time budget, so a rate limit set too low for the number of templates selected means not every template runs. The page gave no indication of that, and its guidance pointed the other way:

Location Before Problem
Best Practices "Start with lower values (10-30 requests/second)" Recommends a value that can drop templates on a full-set scan
Corporate example Rate limit per host: 10 Worked example at the same low value
Rate Limiting note "reduce scan speed but are more respectful" Presents slowness as the only cost
Scope recommendation "Use global application for rate limiting" Applies one host's low value to every scan

What changed

cloud/scanning/parameters.mdx

  • A Warning noting the per-host time budget, and that a rate limit too low for the selected template count drops the exploitation templates that find Critical and High issues.
  • Guidance to use 100 requests/second or higher for full template-set scans, and to narrow the template selection to match where a host cannot take that. Points at the Recommended profile, which excludes informational templates.
  • Inverted the 10-30 requests/second Best Practices guideline, raised the 10 worked example, and noted that a globally applied rate limit reaches every scan.

cloud/scanning/external-scan.mdx

  • Linked the Rate Limiting card to the guidance, so a reader configuring a rate from this page reaches it.
  • Fixed two links that 404 today, both reported by mint broken-links: /cloud/scanning/nuclei-scan to /cloud/scanning/internal-scan (page renamed; the link text already says "internal scanning"), and /cloud/editor/introduction to /cloud/editor/overview.
  • Fixed "ProjectDiscovery offers Enterprise users gain additional features", which has two verbs.

Checks

mint broken-links clean on both pages, including the new #rate-limiting anchor and the Recommended profile link. The external scanning page went from 2 broken links to 0. JSX components balanced; both pages present in mint.json.

🤖 Generated with Claude Code

Patrick Gleason and others added 3 commits September 9, 2026 12:13
…speed

The rate limit control was documented in isolation from the per-host time
budget that makes a low value dangerous. The page framed the only cost of
throttling as a slower scan, recommended 10-30 req/sec as a starting point,
and carried a worked example at 10 req/sec.

A customer scanning the full template set at 10 req/sec silently lost every
vulnerability finding: the scan hit the per-host budget after the passive
templates and reported finished at 100% rather than erroring.

- State the coverage consequence and the finished-at-100% symptom
- Recommend 100+ req/sec for full template-set scans, and scope low values
  to small targeted template selections
- Raise the 10 req/sec worked example and note why
- Caveat applying a rate limit globally
- Link the external scanning rate-limit card to the guidance

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both 404 today. `mint broken-links` reports them.

- /cloud/scanning/nuclei-scan -> /cloud/scanning/internal-scan (page renamed;
  the link text already says "learn more about internal scanning")
- /cloud/editor/introduction -> /cloud/editor/overview (only introduction_old
  remains; overview is the live page other docs link to)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Per Davis, who diagnosed the customer case: under a small rate limit only the
informational templates fire. Naming that is more useful than "passive checks",
because the customer's observable symptom is a completed scan reporting `info`
findings only, with the Critical and High findings gone.

Also points at the Recommended profile as the lever for a target that genuinely
cannot take 100 req/sec, since it already excludes informational templates.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mintlify

mintlify Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
projectdiscovery 🟢 Ready View Preview Sep 9, 2026, 5:53 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

"ProjectDiscovery offers Enterprise users gain additional features" has two
verbs. Same line as the link fix in the previous commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Davis's review: too transparent, and it read AI.

Removed the description of a scan reporting finished at 100% after stopping at
the budget. That is a UX flaw being fixed, so it does not belong in customer
docs; the guidance stands on its own without it.

Cut the Tip and the symptom paragraph into the Warning, dropped the invented
label prefixes ("Symptom to look for", "Sizing the rate limit"), the "not just
X" construction and the trailing justifications on each bullet. The Rate
Limiting section is 763 characters against about 1,900.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@patrick-projectdiscovery patrick-projectdiscovery changed the title docs(cloud): a low rate limit costs vulnerability coverage, not just scan speed docs(cloud): set the rate limit high enough for the selected template set Sep 9, 2026
Davis: the templates may find Critical and High issues, and can be dropped.
The previous wording asserted both as certainties, which is not true of any
given target. Reworked the leading clause to an if/then so his phrasing does
not sit next to a second statement of the same idea.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@DDFranklin DDFranklin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants