[jira] [Resolved] (AIRFLOW-1770) Add option to file and hiveconfs in HiveOperator

["Bolke de Bruin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bolke de Bruin resolved AIRFLOW-1770.
-------------------------------------
       Resolution: Fixed
    Fix Version/s: 2.0.0
                   1.9.1
                   1.10.0

Issue resolved by pull request #2752
[https://github.com/apache/incubator-airflow/pull/2752]

> Add option to file and hiveconfs in HiveOperator
> ------------------------------------------------
>
>                 Key: AIRFLOW-1770
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1770
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Ace Haidrey
>            Assignee: Alan Ma
>              Labels: operators
>             Fix For: 1.10.0, 1.9.1, 2.0.0
>
>
> The HiveOperator as it currently stands is not flexible enough to accept a hive file and operate on that. You need to read in the contents and pass it and if you do that you need to change the way hiveconfs are in your file to jinja templating.
> Many teams already have their existing sql/hql files and don't want to convert them to make them as portable and decoupled as possible.
> To accomplish this all we need to do is add the option to pass a hql_file and hiveconfs to the HiveOperator. We change the code in the execute to throw an error if both a hql_file and an hql statement are passed. If just hql_file the simplest way without changing the code of the hive hook is to just read the content of the hql_file and set it to be the self.hql. The hiveconfs get passed directly to the run_cli method and we can combine them with the already passed in hiveconfs.
> If we want to make it optional to pass in the context as hiveconfs we can add that too as related to AIRFLOW-788.
> I've included some simple tests to show it all works how we expect.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)