You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Szabolcs Vasas <va...@gmail.com> on 2018/08/09 15:10:51 UTC
Re: Review Request 68024: SQOOP-3348: Add hadoop-aws dependency and S3
credential generator logic for tests
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68024/#review207036
-----------------------------------------------------------
COMPILING.txt
Lines 311 (patched)
<https://reviews.apache.org/r/68024/#comment290196>
Based on org.apache.sqoop.testutil.DefaultS3CredentialGenerator#generateS3Credentials I think it only works with " ".
COMPILING.txt
Lines 316 (patched)
<https://reviews.apache.org/r/68024/#comment290197>
Can you please add the test command for gradle as well?
src/test/org/apache/sqoop/s3/TestS3AvroImport.java
Lines 54 (patched)
<https://reviews.apache.org/r/68024/#comment290198>
When the generator command is not specified this method throws an exception.
This does not cause an issue in case of Ant, the tests will be green but it the Gradle build will fail. Can you please fix it?
src/test/org/apache/sqoop/testutil/SequenceFileTestUtils.java
Lines 59 (patched)
<https://reviews.apache.org/r/68024/#comment290200>
The constructor
org.apache.hadoop.io.SequenceFile.Reader#Reader(org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration) is deprecated so please use org.apache.hadoop.io.SequenceFile.Reader#Reader(org.apache.hadoop.conf.Configuration, org.apache.hadoop.io.SequenceFile.Reader.Option...) instead.
- Szabolcs Vasas
On July 23, 2018, 4:07 p.m., Boglarka Egyed wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68024/
> -----------------------------------------------------------
>
> (Updated July 23, 2018, 4:07 p.m.)
>
>
> Review request for Sqoop, Fero Szabo and Szabolcs Vasas.
>
>
> Bugs: SQOOP-3348
> https://issues.apache.org/jira/browse/SQOOP-3348
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> * Added hadoop-aws dependency
> * Added an Amazon S3 credential generator logic to enable automated testing
> * Added s3.bucket.url and s3.generator.command properties for testing purposes
> * Added first S3 import test cases for text, Avro and Sequence file formats
> * Extended COMPILING.txt with Amazon S3 test running guide
>
>
> Diffs
> -----
>
> COMPILING.txt 9abf0b211b24de65b53351ece170b09e4851938b
> build.xml 0ae729bce035f6f1afef78702a3d888bb6c5a9ff
> ivy.xml 796ef70ae38f254169587ce7d3401646f4a80a3a
> src/test/org/apache/sqoop/s3/TestS3AvroImport.java PRE-CREATION
> src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java PRE-CREATION
> src/test/org/apache/sqoop/s3/TestS3TextImport.java PRE-CREATION
> src/test/org/apache/sqoop/testutil/DefaultS3CredentialGenerator.java PRE-CREATION
> src/test/org/apache/sqoop/testutil/S3CredentialGenerator.java PRE-CREATION
> src/test/org/apache/sqoop/testutil/S3TestUtils.java PRE-CREATION
> src/test/org/apache/sqoop/testutil/SequenceFileTestUtils.java PRE-CREATION
> src/test/org/apache/sqoop/testutil/TextFileTestUtils.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/68024/diff/1/
>
>
> Testing
> -------
>
> Added new S3 import test cases with text, Avro and Sequence file formats.
>
> ant clean test -Ds3.bucket.url=<bucket-url> -Ds3.generator.command=<credential-generator-command>
>
>
> Thanks,
>
> Boglarka Egyed
>
>