You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Qian Xu <qi...@intel.com> on 2015/04/29 20:00:49 UTC
Review Request 33688: SQOOP-2339: Move sub-directory might fail in
append mode
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33688/
-----------------------------------------------------------
Review request for Sqoop.
Bugs: SQOOP-2339
https://issues.apache.org/jira/browse/SQOOP-2339
Repository: sqoop-trunk
Description
-------
In append mode, Sqoop will first create dataset in a temporary directory and then move it target directory. I noticed that if the temporary directory contains sub-directory and the target directory has a same directory, the import might fail with an IOException. This will happen in old hadoop distribution such as hadoop-2.0.4-alpha.
There is no problem with hadoop-2.5.0. With hadoop-2.5.0, duplicated directory will be renamed by adding a suffix "-0000" (the digit might be increased).
Diffs
-----
src/java/org/apache/sqoop/util/AppendUtils.java c53a6e2
src/test/com/cloudera/sqoop/TestParquetImport.java ae5427c
src/test/com/cloudera/sqoop/testutil/ImportJobTestCase.java dbd9061
Diff: https://reviews.apache.org/r/33688/diff/
Testing
-------
New test cases added
Thanks,
Qian Xu
Re: Review Request 33688: SQOOP-2339: Move sub-directory might fail in
append mode
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33688/#review82012
-----------------------------------------------------------
Ship it!
Ship It!
- Jarek Cecho
On April 29, 2015, 6 p.m., Qian Xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33688/
> -----------------------------------------------------------
>
> (Updated April 29, 2015, 6 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2339
> https://issues.apache.org/jira/browse/SQOOP-2339
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> In append mode, Sqoop will first create dataset in a temporary directory and then move it target directory. I noticed that if the temporary directory contains sub-directory and the target directory has a same directory, the import might fail with an IOException. This will happen in old hadoop distribution such as hadoop-2.0.4-alpha.
> There is no problem with hadoop-2.5.0. With hadoop-2.5.0, duplicated directory will be renamed by adding a suffix "-0000" (the digit might be increased).
>
>
> Diffs
> -----
>
> src/java/org/apache/sqoop/util/AppendUtils.java c53a6e2
> src/test/com/cloudera/sqoop/TestParquetImport.java ae5427c
> src/test/com/cloudera/sqoop/testutil/ImportJobTestCase.java dbd9061
>
> Diff: https://reviews.apache.org/r/33688/diff/
>
>
> Testing
> -------
>
> New test cases added
>
>
> Thanks,
>
> Qian Xu
>
>