You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Anonymous (Jira)" <ji...@apache.org> on 2022/02/15 13:24:00 UTC
[jira] [Assigned] (AIRFLOW-3379) Support for AWS triggers (S3, SNS, SQS, CloudWatch, etc.)
[ https://issues.apache.org/jira/browse/AIRFLOW-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anonymous reassigned AIRFLOW-3379:
----------------------------------
Assignee: Rohan Bhanderi
> Support for AWS triggers (S3, SNS, SQS, CloudWatch, etc.)
> ---------------------------------------------------------
>
> Key: AIRFLOW-3379
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3379
> Project: Apache Airflow
> Issue Type: Improvement
> Components: aws
> Reporter: Brylie Christopher Oxley
> Assignee: Rohan Bhanderi
> Priority: Major
>
> We would like to build a 'reactive', or event-driven, data pipeline. From what I can gather, the primary Airflow DAG trigger is a timer (cron). However, it would be useful to trigger dags on external events, namely AWS events like S3 file, SQS, SNS, and/or CloudWatch.
> I note there is an experimental API, which could be triggered from an AWS Lambda, but would add boilerplate and brittleness to the data pipeline.
> What are our options for triggering Airflow DAGS from external AWS events?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)