Skip to content

Improve Post-Task logs to show exception in failure#66735

Open
jscheffl wants to merge 1 commit into
apache:mainfrom
jscheffl:feature/improve-post-task-logs
Open

Improve Post-Task logs to show exception in failure#66735
jscheffl wants to merge 1 commit into
apache:mainfrom
jscheffl:feature/improve-post-task-logs

Conversation

@jscheffl
Copy link
Copy Markdown
Contributor

@jscheffl jscheffl commented May 11, 2026

Following-up on the PR by @ashb in #66037 I realized the same shotcoming with the change as some (my local) users complained about in Airflow 2.x as well: If the task fails the Exception details are swallowed/hidden in the post execution block. Experienced users complianed they always need to click to un-fold to see details and in-experienced users did not see any error and wondered why task was failing.

Before (so after @ashb added folding for Pre/Post task (THANKS!!!):
Successful task:
image

Failed task (mis-leading as last log line has no error and is INFO!):
image

Need to un-fold to see the error actually:
image

With this PR the exception in case of task failure is moved above the Post-Execution log group:
image

(So no un-folding of group needed, only "boring" logs below exception):
image

I admit the code is not as elegant as before with a single group but now scattered across multiple different exception catching blocks. Ideas for more elegant placing welcome. But UX should be good first (over code beautiness)

Note: Just adjusted primary code, will fix pytests tomorrow but leave PR here for catching comments on change.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@jscheffl jscheffl added this to the Airflow 3.2.2 milestone May 11, 2026
@vatsrahul1001 vatsrahul1001 added the backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch label May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:task-sdk backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants