[jira] [Created] (AIRFLOW-3734) function `load_file` will not run if param `partition` is None

["zhongjiajie (JIRA)" <ji...@apache.org>]

zhongjiajie created AIRFLOW-3734:
------------------------------------

             Summary: function `load_file` will not run if param `partition` is None
                 Key: AIRFLOW-3734
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3734
             Project: Apache Airflow
          Issue Type: Bug
          Components: hive_hooks
    Affects Versions: 1.10.1
            Reporter: zhongjiajie
            Assignee: zhongjiajie


the code in master branch in hive_hooks.py in function load_file like below
{code:java}
hql = "LOAD DATA LOCAL INPATH '{filepath}' "
if overwrite:
    hql += "OVERWRITE "
hql += "INTO TABLE {table} "
if partition:
    pvals = ", ".join(
        ["{0}='{1}'".format(k, v) for k, v in partition.items()])
    hql += "PARTITION ({pvals});"

# As a workaround for HIVE-10541, add a newline character
# at the end of hql (AIRFLOW-2412).
hql += '\n'{code}
when param partition is None, the hql will like this 
{code:java}
LOAD DATA LOCAL INPATH <FILEPATH> OVERWRITE INTO TABLE <TABLE>\n{code}
without `;` syntax the hql will not run, so I think we should move `;` from `hql += "PARTITION (\{pvals});"` to `hql += ';\n'`

 

so the code will like 
{code:java}
hql = "LOAD DATA LOCAL INPATH '{filepath}' "
if overwrite:
    hql += "OVERWRITE "
hql += "INTO TABLE {table} "
if partition:
    pvals = ", ".join(
        ["{0}='{1}'".format(k, v) for k, v in partition.items()])
    hql += "PARTITION ({pvals})"

# As a workaround for HIVE-10541, add a newline character
# at the end of hql (AIRFLOW-2412).
hql += ';\n'
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)