[jira] [Resolved] (AIRFLOW-2620) New property called "RetainSameConnection"

["Aizhamal Nurmamat kyzy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aizhamal Nurmamat kyzy resolved AIRFLOW-2620.
---------------------------------------------
    Resolution: Fixed

Resolving the issue after adding 'aws' component, removing 'redshift' component and adding "redshift' label for component refactor.

> New property called "RetainSameConnection"
> ------------------------------------------
>
>                 Key: AIRFLOW-2620
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2620
>             Project: Apache Airflow
>          Issue Type: Wish
>          Components: aws, configuration, operators
>    Affects Versions: 2.0.0
>            Reporter: fclesio
>            Priority: Critical
>              Labels: connection, redshift
>
> Some ETL stuff relies on the -high- reusability of the connections in the middle of the data flow, and part of this is achieved using temporary tables.
> I performed a test [1] in Airflow and the behaviour is the for each command that we use the "PostgresOperator" hits the database again, and if the DAG had a lot of connections it can be a nightmare. 
> As I said in SO, for several cases where you have cases of processing some tables in stages the reuse a temporary table across multiple tasks can be very useful and plus: can remove a lot of pressure in the RDBMS cause you won't need to hit several times.
> That's why a purpose a feature of the connection called "RetainSameConnection" that will keep the same connection during the execution of the DAG.
>  
> Reference: [1] - [My question in Stack Overflow|https://stackoverflow.com/questions/50858770/airflow-retain-the-same-database-connection/] about it



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)