trigger = TriggerDagRunOperator( task_id='trigger_other', trigger_dag_id='consumer_dag', conf="xcom_value": " ti.xcom_pull(task_ids='producer_task') " )
xcom_backend = airflow.providers.redis.xcom.RedisXCom airflow xcom exclusive
. This allows you to store the actual data "exclusively" in external object storage while only keeping a reference in the Airflow DB. Apache Airflow Object Storage Backend : You can configure Airflow to use Google Cloud Storage Azure Blob Storage Implementation : To build a custom one, you must subclass and override the serialize_value deserialize_value Thresholding : You can set a size threshold (e.g., xcom_objectstorage_threshold trigger = TriggerDagRunOperator( task_id='trigger_other'
In a multi-tenant environment, you might want to ensure that Task B can pull data from Task A, but Task C (perhaps a notification task) cannot. While Airflow doesn't have native "per-key" permissions, developers implement exclusivity through: xcom_objectstorage_threshold In a multi-tenant environment
XComs are a mechanism for between tasks.