[GitMehdi-sys]

…ent Storm topology settings #2131

Thank you for contributing to Apache StormCrawler.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes

• Is there a issue associated with this PR? Is it referenced in the commit message?

• Does your PR title start with #XXXX where XXXX is the issue number you are trying to resolve?

• Has your PR been rebased against the latest commit within the target branch (typically main)?

• Is your initial contribution a single, squashed commit?

• Is the code properly formatted with mvn git-code-format:format-code -Dgcf.globPattern="**/*" -Dskip.format.code=false?

For code changes

• Have you ensured that the full suite of tests is executed via mvn clean verify?
• Have you written or updated unit tests to verify your changes?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE file, including the main LICENSE file?
• If applicable, have you updated the NOTICE file, including the main NOTICE file?

[rzo1]

Thanks for picking this up, the documentation of the Storm settings is a good addition on its own.

Before we change the defaults in the archetypes, though, I'd like to see some numbers. Neither the issue nor the PR measures the effect so far, and the archetype config is what most new users start from. Could one of you (@GitMehdi-sys or @GGraziadei) run the archetype topology with batch size 1 and with 32 + dynamic: true, and compare:

• throughput (pages fetched per second, acks per second)
• complete latency of the spout tuples, and whether anything times out
• CPU per worker

ideally once under light load (a few hosts, politeness-bound) and once under sustained load (many hosts). That would also tell us whether 32 is a good value or whether something smaller works as well. jnioche's StatusUpdaterBenchmark (#2213) might help for the status updater part.

Small things: the THIRD-PARTY.txt change looks unrelated and can be dropped, and it would be nice to fill in the PR description with a short summary and the issue number.