[jira] [Created] (HIVE-9389) INSERT OVERWRITE DIRECTORY fails to delete old data files

["Andy Skelton (JIRA)" <ji...@apache.org>]

Andy Skelton created HIVE-9389:
----------------------------------

             Summary: INSERT OVERWRITE DIRECTORY fails to delete old data files
                 Key: HIVE-9389
                 URL: https://issues.apache.org/jira/browse/HIVE-9389
             Project: Hive
          Issue Type: Bug
    Affects Versions: 0.13.1
         Environment: CDH 5.3.0, non-secure hdfs, perm checking off
            Reporter: Andy Skelton


{code:sql}
FROM myview INSERT OVERWRITE DIRECTORY 'hdfs://nameservice/path/' SELECT COUNT(DISTINCT mycol);
{code}

This always produces one row. Sometimes the output is two files, {{000000_0}} and {{000000_1}}, one of which is empty. Sometimes we have seen new results in {{000000_0}} while old results remain in {{000000_1}}.

We were alerted to this because Sqoop was exporting the output files in order by filename, writing first the new value and then overwriting with the old value, triggering an alert because the value stopped increasing in our database.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)