You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Luca Falsina (JIRA)" <ji...@apache.org> on 2017/06/27 14:01:00 UTC

[jira] [Created] (AIRFLOW-1350) Add "query_uri" parameter for Google DataProc operator for both SparkSQL and Hive jobs

Luca Falsina created AIRFLOW-1350:
-------------------------------------

             Summary: Add "query_uri" parameter for Google DataProc operator for both SparkSQL and Hive jobs
                 Key: AIRFLOW-1350
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1350
             Project: Apache Airflow
          Issue Type: Improvement
          Components: gcp, operators
            Reporter: Luca Falsina
            Assignee: Luca Falsina


Google DataProc APIs allow to provide as an input for some types of job (Pig, SparkSQL, Hive) a "query_file" parameter pointing to a file stored in Google Cloud Storage.

The current implementation of the DataProc operator allows this functionality only for Pig jobs. The code modification should be trivial as I plan to just reuse the same implementation in SparkSQL and Hive operator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)