You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Clément MATHIEU <cl...@unportant.info> on 2015/10/03 17:00:33 UTC

Re: Review Request 38372: Support of glob paths during export

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38372/
-----------------------------------------------------------

(Updated oct. 3, 2015, 3 après-midi)


Review request for Sqoop.


Changes
-------

Add an unit test


Bugs: SQOOP-1281
    https://issues.apache.org/jira/browse/SQOOP-1281


Repository: sqoop-trunk


Description
-------

The only thing required to add support of glob paths is to update ExportJobBase.getFileType.

Current code fails with an exception when globbing is used. The patch add support of glob paths while preserving the original logic:

 - Returns the type of the first file
 - Fails if no file can be found


Diffs (updated)
-----

  src/java/org/apache/sqoop/mapreduce/ExportJobBase.java f9fa7f3 
  src/test/com/cloudera/sqoop/TestAvroExport.java 663828c 

Diff: https://reviews.apache.org/r/38372/diff/


Testing
-------

It has been tested & used with Sqoop 1.4.5 on top of a CDH 4.7, 5.3 & 5.4 (we have jobs taking glob paths as input parameter)

I quickly crawled the test suite to see how I could unit test this change but failed to find a trivial implementation. I believe that I have to fiddle with getTablePath / TestExport but I'm not sure where it will take me. Any advice is welcome.


Thanks,

Clément MATHIEU

Re: Review Request 38372: Support of glob paths during export

Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38372/#review101649
-----------------------------------------------------------

Ship it!


Ship It!

- Jarek Cecho


On Oct. 3, 2015, 3 p.m., Clément MATHIEU wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38372/
> -----------------------------------------------------------
> 
> (Updated Oct. 3, 2015, 3 p.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-1281
>     https://issues.apache.org/jira/browse/SQOOP-1281
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> The only thing required to add support of glob paths is to update ExportJobBase.getFileType.
> 
> Current code fails with an exception when globbing is used. The patch add support of glob paths while preserving the original logic:
> 
>  - Returns the type of the first file
>  - Fails if no file can be found
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/mapreduce/ExportJobBase.java f9fa7f3 
>   src/test/com/cloudera/sqoop/TestAvroExport.java 663828c 
> 
> Diff: https://reviews.apache.org/r/38372/diff/
> 
> 
> Testing
> -------
> 
> It has been tested & used with Sqoop 1.4.5 on top of a CDH 4.7, 5.3 & 5.4 (we have jobs taking glob paths as input parameter)
> 
> I quickly crawled the test suite to see how I could unit test this change but failed to find a trivial implementation. I believe that I have to fiddle with getTablePath / TestExport but I'm not sure where it will take me. Any advice is welcome.
> 
> 
> Thanks,
> 
> Clément MATHIEU
> 
>