Skip to content

fix(batch-scrape): exit 1 when a waited batch ends failed or cancelled - #386

Open
pcbeingused333 wants to merge 1 commit into
firecrawl:mainfrom
pcbeingused333:fix/batch-scrape-wait-exit-code
Open

pcbeingused333 wants to merge 1 commit into
firecrawl:mainfrom
pcbeingused333:fix/batch-scrape-wait-exit-code

Conversation

@pcbeingused333

@pcbeingused333 pcbeingused333 commented Nov 15, 2026 •

Copy link
Copy Markdown

Exit with status 1 when batch-scrape --wait ends in failed or cancelled, as crawl --wait already does.

Problem. handleBatchScrapeCommand never set an exit code for the final job status. With --wait (with or without --progress), a batch that finished failed or cancelled printed its JSON and exited 0, so scripts and CI pipelines treated it as a success. crawl --wait already exits 1 in the same situation.

Reproduction. Run firecrawl batch-scrape <urls> --wait against a job whose final status is failed: the process exits 0. With a mocked client returning status: 'failed' from batchScrape, process.exitCode stays undefined.

Fix. After writing the output, if --wait was used and the final status is failed or cancelled, print Error: Batch scrape <status>. to stderr and set process.exitCode = 1, mirroring crawl.ts. The JSON output is unchanged. Starting a job without --wait, reading a status by job ID, and completed are unaffected.

Test. New handleBatchScrapeCommand exit status tests in batch-scrape.test.ts cover failed and cancelled (exit 1, JSON still printed, stderr message) and confirm exit status is left alone for completed and for reading a failed job without --wait. The two failing cases fail on main and pass with the change (28/28 in the file); tsc --noEmit and prettier --check are clean.

This pull request was prepared with the assistance of an AI coding agent (Claude Code) and submitted by Alex Castillo, who is responsible for its content.


Summary by cubic

Makes batch-scrape --wait exit with status 1 when the batch ends failed or cancelled, matching crawl --wait. Previously it printed the final JSON and exited 0, so scripts and CI pipelines treated failed batches as successful.

  • Prints Error: Batch scrape <status>. to stderr and sets process.exitCode = 1.
  • JSON output and all other code paths (completed, no --wait, --cancel --wait) are unchanged.

Written for commit 52e2816. Summary will update on new commits.

View guided diff

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 2 files

Confidence score: 3/5

  • With --wait, src/commands/batch-scrape.ts can treat failed or cancelled job-ID status reads as failures, changing the status-read behavior even though executeBatchScrape does not wait for job IDs. Keep this path unchanged.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/commands/batch-scrape.ts">

<violation number="1" location="src/commands/batch-scrape.ts:250">
P2: This also marks failed or cancelled job-ID status reads as failures when `--wait` is supplied, even though `executeBatchScrape` does not wait for job IDs and the status-read path is supposed to remain unchanged. Exclude `options.jobId` from this final-status check.</violation>
</file>

Shadow auto-approve: would not auto-approve because issues were found.

View guided diff | Re-trigger cubic

);
}

const finalStatus = !started && result.data ? result.data.status : undefined;

@cubic-dev-ai cubic-dev-ai Bot Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This also marks failed or cancelled job-ID status reads as failures when --wait is supplied, even though executeBatchScrape does not wait for job IDs and the status-read path is supposed to remain unchanged. Exclude options.jobId from this final-status check.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/commands/batch-scrape.ts, line 250:

<comment>This also marks failed or cancelled job-ID status reads as failures when `--wait` is supplied, even though `executeBatchScrape` does not wait for job IDs and the status-read path is supposed to remain unchanged. Exclude `options.jobId` from this final-status check.</comment>

<file context>
@@ -246,4 +246,14 @@ export async function handleBatchScrapeCommand(
     );
   }
+
+  const finalStatus = !started && result.data ? result.data.status : undefined;
+  if (
+    options.wait &&
</file context>
Suggested change
const finalStatus = !started && result.data ? result.data.status : undefined;
const finalStatus =
!started && !options.jobId && result.data ? result.data.status : undefined;
Fix with cubic

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant