Skip to content

chore: add Cargo http options to handle download errors#23314

Open
comphead wants to merge 1 commit into
apache:mainfrom
comphead:chore2
Open

chore: add Cargo http options to handle download errors#23314
comphead wants to merge 1 commit into
apache:mainfrom
comphead:chore2

Conversation

@comphead

@comphead comphead commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

  • Closes #.

Rationale for this change

CI more and more often fails with HTTP errors, during downloading from Cargo. This leads the entire CI needs to be rerun in the merge queue. Adding retry for Cargo HTTP and network settings.

Example

https://github.com/apache/datafusion/actions/runs/28677739346/job/85055401549?pr=23313

error: failed to get `windows` as a dependency of package `sysinfo v0.39.5`
    ... which satisfies dependency `sysinfo = "^0.39.3"` (locked to 0.39.5) of package `datafusion v54.0.0 (/__w/datafusion/datafusion/datafusion/core)`

Caused by:
  failed to load source for dependency `windows`

Caused by:
  unable to update registry `crates-io`

Caused by:
  download of wi/nd/windows failed

Caused by:
  curl failed

Caused by:
  [55] Failed sending data to the peer (OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0)
Error: Process completed with exit code 101.

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions Bot added the development-process Related to development process of DataFusion label Jul 3, 2026

@2010YOUY01 2010YOUY01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you.

I'll manually rerun CI several times on this PR, to ensure it solves the issue.

# Work around intermittent "[16] Error in the HTTP2 framing layer"
# failures from curl when cargo fetches crates from crates.io.
# Disabling HTTP/2 multiplexing forces cargo to serialize requests,
# and raising retries makes transient network hiccups self-heal.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# and raising retries makes transient network hiccups self-heal.
# and raising retries makes transient network hiccups self-heal.
#
# Reference:
# https://doc.rust-lang.org/cargo/reference/config.html?#httpmultiplexing
# https://doc.rust-lang.org/cargo/reference/config.html?#netretry

Let's add some reference links to the options.

However, I can't find the doc for CARGO_HTTP_RETRY, was that valid?

@2010YOUY01 2010YOUY01 closed this Jul 4, 2026
@2010YOUY01 2010YOUY01 reopened this Jul 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Related to development process of DataFusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants