[jira] [Updated] (AIRFLOW-3734) function `load_file` will not run if param `partition` is None

["Aizhamal Nurmamat kyzy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aizhamal Nurmamat kyzy updated AIRFLOW-3734:
--------------------------------------------
         Labels: easyfix hive hive-hooks patch  (was: easyfix patch)
    Component/s: hooks

adding 'hooks' component and tagging with 'hive' label

> function `load_file` will not run if param `partition` is None
> --------------------------------------------------------------
>
>                 Key: AIRFLOW-3734
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3734
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: hive_hooks, hooks
>    Affects Versions: 1.10.1
>            Reporter: zhongjiajie
>            Assignee: zhongjiajie
>            Priority: Major
>              Labels: easyfix, hive, hive-hooks, patch
>             Fix For: 1.10.3
>
>
> the code in master branch in hive_hooks.py in function load_file like below
> {code:java}
> hql = "LOAD DATA LOCAL INPATH '{filepath}' "
> if overwrite:
>     hql += "OVERWRITE "
> hql += "INTO TABLE {table} "
> if partition:
>     pvals = ", ".join(
>         ["{0}='{1}'".format(k, v) for k, v in partition.items()])
>     hql += "PARTITION ({pvals});"
> # As a workaround for HIVE-10541, add a newline character
> # at the end of hql (AIRFLOW-2412).
> hql += '\n'{code}
> when param partition is None, the hql will like this 
> {code:java}
> LOAD DATA LOCAL INPATH <FILEPATH> OVERWRITE INTO TABLE <TABLE>\n{code}
> without `;` syntax the hql will not run, so I think we should move `;` from `hql += "PARTITION (\{pvals});"` to `hql += ';\n'`
>  
> so the code will like 
> {code:java}
> hql = "LOAD DATA LOCAL INPATH '{filepath}' "
> if overwrite:
>     hql += "OVERWRITE "
> hql += "INTO TABLE {table} "
> if partition:
>     pvals = ", ".join(
>         ["{0}='{1}'".format(k, v) for k, v in partition.items()])
>     hql += "PARTITION ({pvals})"
> # As a workaround for HIVE-10541, add a newline character
> # at the end of hql (AIRFLOW-2412).
> hql += ';\n'
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)