Repository navigation
fix(batch-scrape): exit 1 when a waited batch ends failed or cancelled - #386
Open
pcbeingused333 wants to merge 1 commit into
Open
pcbeingused333 wants to merge 1 commit into
pcbeingused333 wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
1 issue found across 2 files
Confidence score: 3/5
- With
--wait,src/commands/batch-scrape.tscan treat failed or cancelled job-ID status reads as failures, changing the status-read behavior even thoughexecuteBatchScrapedoes not wait for job IDs. Keep this path unchanged.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/commands/batch-scrape.ts">
<violation number="1" location="src/commands/batch-scrape.ts:250">
P2: This also marks failed or cancelled job-ID status reads as failures when `--wait` is supplied, even though `executeBatchScrape` does not wait for job IDs and the status-read path is supposed to remain unchanged. Exclude `options.jobId` from this final-status check.</violation>
</file>
Shadow auto-approve: would not auto-approve because issues were found.
View guided diff | Re-trigger cubic
| ); | ||
| } | ||
|
|
||
| const finalStatus = !started && result.data ? result.data.status : undefined; |
Contributor
There was a problem hiding this comment.
P2: This also marks failed or cancelled job-ID status reads as failures when --wait is supplied, even though executeBatchScrape does not wait for job IDs and the status-read path is supposed to remain unchanged. Exclude options.jobId from this final-status check.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/commands/batch-scrape.ts, line 250:
<comment>This also marks failed or cancelled job-ID status reads as failures when `--wait` is supplied, even though `executeBatchScrape` does not wait for job IDs and the status-read path is supposed to remain unchanged. Exclude `options.jobId` from this final-status check.</comment>
<file context>
@@ -246,4 +246,14 @@ export async function handleBatchScrapeCommand(
);
}
+
+ const finalStatus = !started && result.data ? result.data.status : undefined;
+ if (
+ options.wait &&
</file context>
Suggested change
| const finalStatus = !started && result.data ? result.data.status : undefined; | |
| const finalStatus = | |
| !started && !options.jobId && result.data ? result.data.status : undefined; |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Exit with status 1 when
batch-scrape --waitends infailedorcancelled, ascrawl --waitalready does.Problem.
handleBatchScrapeCommandnever set an exit code for the final job status. With--wait(with or without--progress), a batch that finishedfailedorcancelledprinted its JSON and exited 0, so scripts and CI pipelines treated it as a success.crawl --waitalready exits 1 in the same situation.Reproduction. Run
firecrawl batch-scrape <urls> --waitagainst a job whose final status isfailed: the process exits 0. With a mocked client returningstatus: 'failed'frombatchScrape,process.exitCodestaysundefined.Fix. After writing the output, if
--waitwas used and the final status isfailedorcancelled, printError: Batch scrape <status>.to stderr and setprocess.exitCode = 1, mirroringcrawl.ts. The JSON output is unchanged. Starting a job without--wait, reading a status by job ID, andcompletedare unaffected.Test. New
handleBatchScrapeCommand exit statustests inbatch-scrape.test.tscoverfailedandcancelled(exit 1, JSON still printed, stderr message) and confirm exit status is left alone forcompletedand for reading a failed job without--wait. The two failing cases fail on main and pass with the change (28/28 in the file);tsc --noEmitandprettier --checkare clean.This pull request was prepared with the assistance of an AI coding agent (Claude Code) and submitted by Alex Castillo, who is responsible for its content.
Summary by cubic
Makes
batch-scrape --waitexit with status 1 when the batch endsfailedorcancelled, matchingcrawl --wait. Previously it printed the final JSON and exited 0, so scripts and CI pipelines treated failed batches as successful.Error: Batch scrape <status>.to stderr and setsprocess.exitCode = 1.completed, no--wait,--cancel --wait) are unchanged.Written for commit 52e2816. Summary will update on new commits.