You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Sergey Shelukhin (JIRA)" <ji...@apache.org> on 2018/04/23 21:24:00 UTC
[jira] [Updated] (HIVE-19279) remove magic directory skipping from
CopyTask
[ https://issues.apache.org/jira/browse/HIVE-19279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sergey Shelukhin updated HIVE-19279:
------------------------------------
Description:
Follow up from HIVE-17657.
Code exists in copytask that copies files (fancy that); however, when listing the files, if a single directory exists at the source with no other files, it will skip the directory and copy the files inside instead.
This directory in various tests is either the "data" directory from export, or some random partition directory ("foo=bar") that if not skipped makes it into the real partition directory at the destination.
The directory is not skipped if it's not by itself, i.e. any other files or directories are present.
This seems brittle. Caller of the CopyTask should specify exactly what it wants copied instead of relying on this behavior.
was:
Follow up from HIVE-17657.
Code exists in copytask that copies files (fancy that); however, when listing the files, if a single directory exists at the source with no other files, it will skip the directory and copy the files inside instead.
This directory in various tests is either the "data" directory from export, or some random partition directory ("foo=bar") that if not skipped makes it into the real partition directory at the destination.
It won't do that if any other files or directories are present.
This seems brittle. Caller of the CopyTask should specify exactly what it wants copied instead of relying on this behavior.
> remove magic directory skipping from CopyTask
> ---------------------------------------------
>
> Key: HIVE-19279
> URL: https://issues.apache.org/jira/browse/HIVE-19279
> Project: Hive
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Priority: Major
>
> Follow up from HIVE-17657.
> Code exists in copytask that copies files (fancy that); however, when listing the files, if a single directory exists at the source with no other files, it will skip the directory and copy the files inside instead.
> This directory in various tests is either the "data" directory from export, or some random partition directory ("foo=bar") that if not skipped makes it into the real partition directory at the destination.
> The directory is not skipped if it's not by itself, i.e. any other files or directories are present.
> This seems brittle. Caller of the CopyTask should specify exactly what it wants copied instead of relying on this behavior.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)