You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ashutosh Chauhan (JIRA)" <ji...@apache.org> on 2013/12/12 18:42:06 UTC
[jira] [Comment Edited] (HIVE-6024) Load data local inpath
unnecessarily creates a copy task
[ https://issues.apache.org/jira/browse/HIVE-6024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846485#comment-13846485 ]
Ashutosh Chauhan edited comment on HIVE-6024 at 12/12/13 5:41 PM:
------------------------------------------------------------------
This results in inconsistent semantic that while loading from local source files are *not* moved but copied, but while loading from DFS sources files are moved and thus are deleted at source location after operation. Ideally same semantic of load (delete at source) should be provided while loading either from DFS or local. What exactly should be semantic can be debated, however scope for this jira is limited to not a create an additional copy task while loading from local, but rather do the copy (instead of move) in MoveTask itself and thus saving on unnecessary task execution and FS operations.
was (Author: ashutoshc):
This results in inconsistent semantic that while loading from local source files are *not* moved but copied, but while loading from DFS sources files are moved and thus are deleted at source location after operation. Ideally same semantic of move (delete at source) should be provided while loading either from DFS or local. What exactly should be semantic can be debated, however scope for this jira is limited to not a create an additional copy task while loading from local, but rather do the copy (instead of move) in MoveTask itself and thus saving on unnecessary task execution and FS operations.
> Load data local inpath unnecessarily creates a copy task
> --------------------------------------------------------
>
> Key: HIVE-6024
> URL: https://issues.apache.org/jira/browse/HIVE-6024
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Ashutosh Chauhan
>
> Load data command creates an additional copy task only when its loading from {{local}} It doesn't create this additional copy task while loading from DFS though.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)