fix(runner): ensure atomic resume.cfg persistence and in-flight target drain on interrupt - #2561
Open
gcoinstash-cmd wants to merge 38 commits into
Open
Conversation
…projectdiscovery#2530) Bumps the modules group with 4 updates in the / directory: [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck), [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer), [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) and [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo). Updates `github.com/projectdiscovery/cdncheck` from 1.2.42 to 1.2.44 - [Release notes](https://github.com/projectdiscovery/cdncheck/releases) - [Commits](projectdiscovery/cdncheck@v1.2.42...v1.2.44) Updates `github.com/projectdiscovery/fastdialer` from 0.5.11 to 0.5.13 - [Release notes](https://github.com/projectdiscovery/fastdialer/releases) - [Commits](projectdiscovery/fastdialer@v0.5.11...v0.5.13) Updates `github.com/projectdiscovery/networkpolicy` from 0.1.41 to 0.1.42 - [Release notes](https://github.com/projectdiscovery/networkpolicy/releases) - [Commits](projectdiscovery/networkpolicy@v0.1.41...v0.1.42) Updates `github.com/projectdiscovery/retryablehttp-go` from 1.3.16 to 1.3.18 - [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases) - [Commits](projectdiscovery/retryablehttp-go@v1.3.16...v1.3.18) Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.87 to 0.2.89 - [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases) - [Commits](projectdiscovery/wappalyzergo@v0.2.87...v0.2.89) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/cdncheck dependency-version: 1.2.43 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/fastdialer dependency-version: 0.5.12 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/networkpolicy dependency-version: 0.1.42 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/retryablehttp-go dependency-version: 1.3.17 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/wappalyzergo dependency-version: 0.2.88 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
….29 (projectdiscovery#2531) Bumps [github.com/happyhackingspace/dit](https://github.com/happyhackingspace/dit) from 0.0.28 to 0.0.29. - [Release notes](https://github.com/happyhackingspace/dit/releases) - [Commits](HappyHackingSpace/dit@v0.0.28...v0.0.29) --- updated-dependencies: - dependency-name: github.com/happyhackingspace/dit dependency-version: 0.0.29 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…covery#2534) Bumps [golang.org/x/text](https://github.com/golang/text) from 0.38.0 to 0.40.0. - [Release notes](https://github.com/golang/text/releases) - [Commits](golang/text@v0.38.0...v0.40.0) --- updated-dependencies: - dependency-name: golang.org/x/text dependency-version: 0.40.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…overy#2535) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.56.0 to 0.57.0. - [Commits](golang/net@v0.56.0...v0.57.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-version: 0.57.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
…ry#2527) * docs: add Common Recipes section for well-known files Per maintainer feedback on PR projectdiscovery#2472, add a 'Common Recipes' section demonstrating how to detect security.txt, robots.txt, sitemap.xml, and other well-known URIs using httpx's composable primitives. Includes: - security.txt detection with RFC 9116 compliance checks - robots.txt and sitemap.xml probes - IANA well-known URIs registry reference - One-liner examples using -path, -mc, -mct, -mr flags Reference: projectdiscovery#2468 (comment) * extend recipes * fix errcheck * review fixes * pure tests --------- Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com> Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
…rojectdiscovery#2538) * fix cpe * fix suffixes
…e-fqdn-extraction parser for fqdn
…personate-strategy Add chrome and JA3 strategies for -tls-impersonate
Bumps the modules group with 3 updates: [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck), [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) and [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo). Updates `github.com/projectdiscovery/cdncheck` from 1.2.44 to 1.2.45 - [Release notes](https://github.com/projectdiscovery/cdncheck/releases) - [Commits](projectdiscovery/cdncheck@v1.2.44...v1.2.45) Updates `github.com/projectdiscovery/retryablehttp-go` from 1.3.18 to 1.3.19 - [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases) - [Commits](projectdiscovery/retryablehttp-go@v1.3.18...v1.3.19) Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.89 to 0.2.90 - [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases) - [Commits](projectdiscovery/wappalyzergo@v0.2.89...v0.2.90) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/cdncheck dependency-version: 1.2.45 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/retryablehttp-go dependency-version: 1.3.19 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/wappalyzergo dependency-version: 0.2.90 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [github.com/go-faker/faker/v4](https://github.com/go-faker/faker) from 4.9.0 to 4.10.0. - [Release notes](https://github.com/go-faker/faker/releases) - [Commits](go-faker/faker@v4.9.0...v4.10.0) --- updated-dependencies: - dependency-name: github.com/go-faker/faker/v4 dependency-version: 4.10.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [github.com/happyhackingspace/dit](https://github.com/happyhackingspace/dit) from 0.0.29 to 0.0.31. - [Release notes](https://github.com/happyhackingspace/dit/releases) - [Commits](HappyHackingSpace/dit@v0.0.29...v0.0.31) --- updated-dependencies: - dependency-name: github.com/happyhackingspace/dit dependency-version: 0.0.31 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
…abot-group-external group external dependabot updates
…abot/go_modules/dev/modules-04b6a3fb12 chore(deps): bump the modules group with 3 updates
…abot/go_modules/dev/github.com/go-faker/faker/v4-4.10.0 chore(deps): bump github.com/go-faker/faker/v4 from 4.9.0 to 4.10.0
…abot/go_modules/dev/github.com/happyhackingspace/dit-0.0.31 chore(deps): bump github.com/happyhackingspace/dit from 0.0.29 to 0.0.31
Bumps the projectdiscovery group with 6 updates: | Package | From | To | | --- | --- | --- | | [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) | `1.2.45` | `1.2.46` | | [github.com/projectdiscovery/goflags](https://github.com/projectdiscovery/goflags) | `0.1.74` | `0.1.75` | | [github.com/projectdiscovery/networkpolicy](https://github.com/projectdiscovery/networkpolicy) | `0.1.42` | `0.1.43` | | [github.com/projectdiscovery/rawhttp](https://github.com/projectdiscovery/rawhttp) | `0.1.90` | `0.1.91` | | [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) | `1.3.19` | `1.3.20` | | [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) | `0.2.90` | `0.2.91` | Updates `github.com/projectdiscovery/cdncheck` from 1.2.45 to 1.2.46 - [Release notes](https://github.com/projectdiscovery/cdncheck/releases) - [Commits](projectdiscovery/cdncheck@v1.2.45...v1.2.46) Updates `github.com/projectdiscovery/goflags` from 0.1.74 to 0.1.75 - [Release notes](https://github.com/projectdiscovery/goflags/releases) - [Commits](projectdiscovery/goflags@v0.1.74...v0.1.75) Updates `github.com/projectdiscovery/networkpolicy` from 0.1.42 to 0.1.43 - [Release notes](https://github.com/projectdiscovery/networkpolicy/releases) - [Commits](projectdiscovery/networkpolicy@v0.1.42...v0.1.43) Updates `github.com/projectdiscovery/rawhttp` from 0.1.90 to 0.1.91 - [Release notes](https://github.com/projectdiscovery/rawhttp/releases) - [Commits](projectdiscovery/rawhttp@v0.1.90...v0.1.91) Updates `github.com/projectdiscovery/retryablehttp-go` from 1.3.19 to 1.3.20 - [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases) - [Commits](projectdiscovery/retryablehttp-go@v1.3.19...v1.3.20) Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.90 to 0.2.91 - [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases) - [Commits](projectdiscovery/wappalyzergo@v0.2.90...v0.2.91) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/cdncheck dependency-version: 1.2.46 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: projectdiscovery - dependency-name: github.com/projectdiscovery/goflags dependency-version: 0.1.75 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: projectdiscovery - dependency-name: github.com/projectdiscovery/networkpolicy dependency-version: 0.1.43 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: projectdiscovery - dependency-name: github.com/projectdiscovery/rawhttp dependency-version: 0.1.91 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: projectdiscovery - dependency-name: github.com/projectdiscovery/retryablehttp-go dependency-version: 1.3.20 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: projectdiscovery - dependency-name: github.com/projectdiscovery/wappalyzergo dependency-version: 0.2.91 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: projectdiscovery ... Signed-off-by: dependabot[bot] <support@github.com>
…abot/go_modules/dev/projectdiscovery-ce7893f9dd chore(deps): bump the projectdiscovery group with 6 updates
…nary-body-test fix: make binary body test use a local server
…te resume.cfg atomically
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Addresses intermittent
resume.cfgcorruption and incomplete target state persistence during SIGINT/SIGTERM interrupts.Fixes #2345
/claim #2345
Key Changes
runner/runner.go): Synchronizes worker termination with target flushing on interrupt so in-flight requests finish before channel teardown.runner/resume.go): Introduces sequential index completion mapping to ensure out-of-order responses do not skip unprocessed target pointers.runner/resume.go): Implements atomic temp-file write + fsync + rename semantics to prevent partial file writes upon sudden termination.runner/resume_test.go): Added atomic save, out-of-order completion, and multi-threaded interrupt/resume test cases.Verification
go vet ./...: Passed (0 warnings)go test -count=1 ./...: Passed (100% test pass rate)