[jira] [Updated] (AIRFLOW-1575) Add AWS Kinesis Firehose hook for inserting batch records

["Siddharth (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-1575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Siddharth updated AIRFLOW-1575:
-------------------------------
    Description: One of the key components of ETL is data ingestion into multiple sources. Airflow provides a great platform for multiple data sources to integrate with each other and transfer data (hive to druid or hive to S3 etc). In AWS ecosystem Kinesis Firehose is an important component which transfers data to other systems within AWS. Data can directly read in Airflow from any system (druid, hive, s3, mysql or csv) and can be pushed to Firehose. This PR creates a firehose hook for inserting batch items. Next - we can build an airflow operator to transfer data from hive to firehose.

> Add AWS Kinesis Firehose hook for inserting batch records
> ---------------------------------------------------------
>
>                 Key: AIRFLOW-1575
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1575
>             Project: Apache Airflow
>          Issue Type: New Feature
>            Reporter: Siddharth
>            Assignee: Siddharth
>
> One of the key components of ETL is data ingestion into multiple sources. Airflow provides a great platform for multiple data sources to integrate with each other and transfer data (hive to druid or hive to S3 etc). In AWS ecosystem Kinesis Firehose is an important component which transfers data to other systems within AWS. Data can directly read in Airflow from any system (druid, hive, s3, mysql or csv) and can be pushed to Firehose. This PR creates a firehose hook for inserting batch items. Next - we can build an airflow operator to transfer data from hive to firehose.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)