You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Eva Tse <et...@netflix.com> on 2009/08/01 03:01:29 UTC
Load data inpath w/o overwrite does not move the files
We discovered a problem where loading into a new partition in hive w/o
specifying overwrite¹ doesn¹t work.
If the specified partition does not exist yet, running the following
statement would not move the data files, but only create the directory on
HDFS.
hive -e "load data inpath
'/user/output/nccp/P20090731T233339/sessionsFacts_L20090718T03-r-00000' into
table nccp_session_facts partition (dateint = 20090718, hour = 4)
If we run the same statement one more time, it will move the file.
However, if we run the same statement with overwrite¹, it will create the
directory and move the file just fine.
We don¹t think it is expected behavior. Hence, would like to know if this is
a bug.
Hive version: r795559 with 487.4 patch.
Hadoop: 0.20
Thanks a lot!
Eva.
Re: Load data inpath w/o overwrite does not move the files
Posted by Zheng Shao <zs...@gmail.com>.
I reproduced the bug and opened HIVE-718.
Zheng
On Fri, Jul 31, 2009 at 6:01 PM, Eva Tse<et...@netflix.com> wrote:
>
>
> We discovered a problem where loading into a new partition in hive w/o
> specifying ‘overwrite’ doesn’t work.
>
> If the specified partition does not exist yet, running the following
> statement would not move the data files, but only create the directory on
> HDFS.
> hive -e "load data inpath
> '/user/output/nccp/P20090731T233339/sessionsFacts_L20090718T03-r-00000' into
> table nccp_session_facts partition (dateint = 20090718, hour = 4)
>
> If we run the same statement one more time, it will move the file.
>
> However, if we run the same statement with ‘overwrite’, it will create the
> directory and move the file just fine.
>
> We don’t think it is expected behavior. Hence, would like to know if this is
> a bug.
>
>
> Hive version: r795559 with 487.4 patch.
> Hadoop: 0.20
>
>
> Thanks a lot!
> Eva.
>
--
Yours,
Zheng