Skip to content

Add ShortCircuitOperator#548

Merged
mistercrunch merged 2 commits intoapache:masterfrom
jlowin:shortcircuit
Oct 24, 2015
Merged

Add ShortCircuitOperator#548
mistercrunch merged 2 commits intoapache:masterfrom
jlowin:shortcircuit

Conversation

@jlowin
Copy link
Copy Markdown
Member

@jlowin jlowin commented Oct 23, 2015

A common idiom here is to use a PythonBranchOperator to check if a condition has been met and short circuit the DAG's execution if it hasn't. We started using this ShortCircuitOperator instead because it's much simpler to implement and (critically) to understand: if the condition returns False, DAG execution stops.

This makes it simple to gently introduce non-Task dependencies (file not available, table not updated, don't run on weekends, don't run on holidays, don't run if some metric isn't at a critical level).

@mistercrunch
Copy link
Copy Markdown
Member

Nice. Very useful indeed!

mistercrunch added a commit that referenced this pull request Oct 24, 2015
@mistercrunch mistercrunch merged commit f1e5db1 into apache:master Oct 24, 2015
@jlowin jlowin deleted the shortcircuit branch October 24, 2015 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants