Skip to content

Go SDK: delete the XComs of earlier tries before a task runs - #74178

Open
FrankYang0529 wants to merge 1 commit into
apache:mainfrom
FrankYang0529:airflow-go-sdk-clear-earlier-try-xcoms
Open

FrankYang0529 wants to merge 1 commit into
apache:mainfrom
FrankYang0529:airflow-go-sdk-clear-earlier-try-xcoms

Conversation

@FrankYang0529

Copy link
Copy Markdown
Member

Follow-up for #74075 (comment).

When a task instance starts a new try, Airflow keeps the XComs it already has and lists their keys in ti_context.xcom_keys_to_clear for the runtime to delete. The Python task runner deletes them before it renders templates. The Go runtime never read the field, so a retried Go task still saw the XComs of its previous try. With this change the Go runtime deletes them too, and the retried task no longer sees them.

  • RunTask sends a DeleteXCom for each key after it finds the task and before it binds arguments or runs the task.
  • If a delete fails, the task does not run. It ends with a RetryTask when should_retry is set and with a FAILED TaskState otherwise.
  • dag.If no longer writes {"skipped": []} to skipmixin_key before the condition runs, since the delete now removes the list from an earlier try.

Was generative AI tooling used to co-author this PR?
  • Yes - Claude Code

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Signed-off-by: PoAn Yang <payang@apache.org>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant