Skip to content

example

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

example

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
src
src
 
 
README.md
README.md
 
 
build.gradle
build.gradle
 
 
gradle.properties
gradle.properties
 
 
settings.gradle
settings.gradle
 
 

README.md

Java SDK examples

Example Dags for the Apache Airflow Java SDK. See the Java SDK guide for the full documentation.

Build

Publish the SDK to the local Maven repository, then build the example bundle into build/bundle/:

cd java-sdk
./gradlew publishToMavenLocal -PskipSigning=true

cd example
../gradlew bundle

Configure Airflow

Register build/bundle/ as a Dag bundle, and route the java queue to the Java coordinator:

export AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST='[
  {"name": "dags-folder", "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle", "kwargs": {}},
  {
    "name": "java-task-handlers",
    "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle",
    "kwargs": {"path": "/absolute/path/to/airflow/java-sdk/example/build/bundle"}
  }
]'
export AIRFLOW__SDK__COORDINATORS='{
  "java": {
    "classpath": "airflow.sdk.coordinators.java.JavaCoordinator",
    "kwargs": {"task_handler_bundle_name": "java-task-handlers"}
  }
}'
export AIRFLOW__SDK__QUEUE_TO_COORDINATOR='{"java": "java"}'

The workers and the Dag processor need Java 11 or later on PATH.

Dag definition

These Dags are declared entirely in Java, in src/java/org/apache/airflow/example/nativedag/:

Dag Source Shows
java_native_annotation_example AnnotationExample.java The annotation style: task inputs, a task group, @Airflow.If, @Airflow.Switch, and TriggerDagRun
java_native_interface_example InterfaceExample.java A similar Dag in the interface style, with DagDef
java_native_target_example TargetExample.java The Dag the other two trigger 
airflow dags trigger java_native_annotation_example
airflow dags trigger java_native_interface_example

Python Dag with stub TaskHandler

These Dags are declared in Python, in src/resources/dags/java_examples.py, and their stub tasks are implemented in Java:

Dag Source Shows
java_task_handler_annotation_example AnnotationExample.java @Airflow.TaskHandler methods, with arguments by position and by name
java_task_handler_interface_example InterfaceExampleBuilder.java Task and InputTask classes registered by Dag and task id
java_task_handler_xcom_casting_example XComCastingExample.java How values passed between tasks convert to the declared Java types

Copy src/resources/dags/java_examples.py into your Airflow Dags folder, and create the Connection and Variable the examples read:

export AIRFLOW_CONN_TEST_HTTP='{
    "conn_type": "http",
    "login": "user",
    "password": "pass",
    "host": "example.com",
    "port": 1234,
    "extra": {"param1": "val1", "param2": "val2"}
}'
export AIRFLOW_VAR_MY_VARIABLE=123
airflow dags trigger java_task_handler_annotation_example
airflow dags trigger java_task_handler_interface_example
airflow dags trigger java_task_handler_xcom_casting_example