You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Szabolcs Vasas <va...@gmail.com> on 2018/11/15 11:27:58 UTC

Review Request 69346: Categorize all tests in the project

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69346/
-----------------------------------------------------------

Review request for Sqoop.


Bugs: SQOOP-3404
    https://issues.apache.org/jira/browse/SQOOP-3404


Repository: sqoop-trunk


Description
-------

- All tests are categorized now
- Introduced S3Test category as a subcategory of ThirdPartyTest
- Reorganized test tasks: we have unitTest, integrationTest, kerberizedTest, test, s3Test, allTest tasks now
- jacocoTestReport task is fixed to contain the coverage information of the kerberizedTest task too. This is needed because the kerberizedTest needs the forkEvery parameter to be set to 1 and because of that it has to be a separate task which generates separate coverage information too. However it is automatically triggered after the test task so the invocation is more convenient for the tester.


Diffs
-----

  build.gradle cb9eeca74bbf278c3e5fd15de608d8c37c917ddb 
  src/test/org/apache/sqoop/importjob/SplitByImportTest.java c6fe4f2e8a80c96ad667f4fe4a26510af96562dc 
  src/test/org/apache/sqoop/manager/TestMainframeManager.java c84f05f660c396a06a5031e00abdae77ffbcf2aa 
  src/test/org/apache/sqoop/manager/oracle/TestOraOopDBInputSplitGetDebugDetails.java 6f33ad3b650436b7f268b4ef5bfd451bd5e6958e 
  src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java 5e558717c0d43301ecbf81a37d5ee3fd35756d65 
  src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java 1a6943786834d27f27523f484d76cf678f18cf48 
  src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetBinaryRecord.java b4cba28c3611400b5c4227a5166b6c91e9152dc4 
  src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileGdgEntryParser.java 521a04266e8806321fe7aa6a89c064f369174523 
  src/test/org/apache/sqoop/s3/TestS3AvroImport.java 7f5f5d62c5cab10f932aa22c3a713b13fefc2b58 
  src/test/org/apache/sqoop/s3/TestS3ExternalHiveTableImport.java 0c3161e5a783446e35f4754124f86715d103ec0b 
  src/test/org/apache/sqoop/s3/TestS3ImportWithHadoopCredProvider.java 3a0d6365dc20f8eef5bdd67a4a2dc9c68ff74d7f 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java 5faf59ea80c48fe025294cabd100e7d176032138 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java a4f986423ea299716a29f9d02f7c8453a7f2ba02 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java d271588c5af060bbc3d301a845f45c46d0f6a2ba 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java 52d89c775b5f1219471df44d222fd92a59ed408c 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java 39238c5fab56b54a85dde5aed0d4bb2c77382fa6 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java 597e3def2cc33adebeeb3bc1ee35ad8a7f4b990d 
  src/test/org/apache/sqoop/s3/TestS3ParquetImport.java c9785d816d4a7a5870d74c51a9faa229f6d3818e 
  src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java bba8b74ebe639df26e977abf377f4904144dcfaa 
  src/test/org/apache/sqoop/s3/TestS3TextImport.java 114f97cbb8857a7633cae5d030769ac4a90e36aa 
  src/test/org/apache/sqoop/testcategories/thirdpartytest/S3Test.java PRE-CREATION 
  src/test/org/apache/sqoop/tool/TestS3IncrementalImportOptionValidations.java 7745f1b07e6d6c457b0164deeace12587ec058d0 


Diff: https://reviews.apache.org/r/69346/diff/1/


Testing
-------

./gradlew unitTest
./gradlew integrationTest
./gradlew kerberizedTest
./gradlew ... s3Test
./gradlew test
./gradlew ... thirdPartyTest
./gradlew allTest


Thanks,

Szabolcs Vasas


Re: Review Request 69346: Categorize all tests in the project

Posted by Szabolcs Vasas <va...@gmail.com>.

> On Nov. 15, 2018, 5:12 p.m., Boglarka Egyed wrote:
> > COMPILING.txt
> > Lines 374-430 (patched)
> > <https://reviews.apache.org/r/69346/diff/2/?file=2108121#file2108121line378>
> >
> >     Shouldn't we include S3 related properties here too as those re part of the third party test suite as well?

Yes, this makes sense.


- Szabolcs


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69346/#review210577
-----------------------------------------------------------


On Nov. 15, 2018, 5:45 p.m., Szabolcs Vasas wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69346/
> -----------------------------------------------------------
> 
> (Updated Nov. 15, 2018, 5:45 p.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-3404
>     https://issues.apache.org/jira/browse/SQOOP-3404
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> - All tests are categorized now
> - Introduced S3Test category as a subcategory of ThirdPartyTest
> - Reorganized test tasks: we have unitTest, integrationTest, kerberizedTest, test, s3Test, allTest tasks now
> - jacocoTestReport task is fixed to contain the coverage information of the kerberizedTest task too. This is needed because the kerberizedTest needs the forkEvery parameter to be set to 1 and because of that it has to be a separate task which generates separate coverage information too. However it is automatically triggered after the test task so the invocation is more convenient for the tester.
> 
> 
> Diffs
> -----
> 
>   COMPILING.txt 835ba33b1e89158bed0e05698b188ab3323eb881 
>   build.gradle cb9eeca74bbf278c3e5fd15de608d8c37c917ddb 
>   src/test/org/apache/sqoop/importjob/SplitByImportTest.java c6fe4f2e8a80c96ad667f4fe4a26510af96562dc 
>   src/test/org/apache/sqoop/manager/TestMainframeManager.java c84f05f660c396a06a5031e00abdae77ffbcf2aa 
>   src/test/org/apache/sqoop/manager/oracle/TestOraOopDBInputSplitGetDebugDetails.java 6f33ad3b650436b7f268b4ef5bfd451bd5e6958e 
>   src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java 5e558717c0d43301ecbf81a37d5ee3fd35756d65 
>   src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java 1a6943786834d27f27523f484d76cf678f18cf48 
>   src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetBinaryRecord.java b4cba28c3611400b5c4227a5166b6c91e9152dc4 
>   src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileGdgEntryParser.java 521a04266e8806321fe7aa6a89c064f369174523 
>   src/test/org/apache/sqoop/s3/TestS3AvroImport.java 7f5f5d62c5cab10f932aa22c3a713b13fefc2b58 
>   src/test/org/apache/sqoop/s3/TestS3ExternalHiveTableImport.java 0c3161e5a783446e35f4754124f86715d103ec0b 
>   src/test/org/apache/sqoop/s3/TestS3ImportWithHadoopCredProvider.java 3a0d6365dc20f8eef5bdd67a4a2dc9c68ff74d7f 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java 5faf59ea80c48fe025294cabd100e7d176032138 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java a4f986423ea299716a29f9d02f7c8453a7f2ba02 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java d271588c5af060bbc3d301a845f45c46d0f6a2ba 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java 52d89c775b5f1219471df44d222fd92a59ed408c 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java 39238c5fab56b54a85dde5aed0d4bb2c77382fa6 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java 597e3def2cc33adebeeb3bc1ee35ad8a7f4b990d 
>   src/test/org/apache/sqoop/s3/TestS3ParquetImport.java c9785d816d4a7a5870d74c51a9faa229f6d3818e 
>   src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java bba8b74ebe639df26e977abf377f4904144dcfaa 
>   src/test/org/apache/sqoop/s3/TestS3TextImport.java 114f97cbb8857a7633cae5d030769ac4a90e36aa 
>   src/test/org/apache/sqoop/testcategories/thirdpartytest/S3Test.java PRE-CREATION 
>   src/test/org/apache/sqoop/tool/TestS3IncrementalImportOptionValidations.java 7745f1b07e6d6c457b0164deeace12587ec058d0 
> 
> 
> Diff: https://reviews.apache.org/r/69346/diff/3/
> 
> 
> Testing
> -------
> 
> ./gradlew unitTest
> ./gradlew integrationTest
> ./gradlew kerberizedTest
> ./gradlew ... s3Test
> ./gradlew test
> ./gradlew ... thirdPartyTest
> ./gradlew allTest
> 
> 
> Thanks,
> 
> Szabolcs Vasas
> 
>


Re: Review Request 69346: Categorize all tests in the project

Posted by Boglarka Egyed <bo...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69346/#review210577
-----------------------------------------------------------


Fix it, then Ship it!




Hi Szabolcs,

Thanks for this improvement, it makes Sqoop tests pretty neat!

I have checked all the categories and they seem to be OK for me. I have two minor findings, please find them below.

Thank you,
Bogi


COMPILING.txt
Lines 374-430 (patched)
<https://reviews.apache.org/r/69346/#comment295282>

    Shouldn't we include S3 related properties here too as those re part of the third party test suite as well?



build.gradle
Line 188 (original), 204 (patched)
<https://reviews.apache.org/r/69346/#comment295284>

    Typo: This the same


- Boglarka Egyed


On Nov. 15, 2018, 1:40 p.m., Szabolcs Vasas wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69346/
> -----------------------------------------------------------
> 
> (Updated Nov. 15, 2018, 1:40 p.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-3404
>     https://issues.apache.org/jira/browse/SQOOP-3404
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> - All tests are categorized now
> - Introduced S3Test category as a subcategory of ThirdPartyTest
> - Reorganized test tasks: we have unitTest, integrationTest, kerberizedTest, test, s3Test, allTest tasks now
> - jacocoTestReport task is fixed to contain the coverage information of the kerberizedTest task too. This is needed because the kerberizedTest needs the forkEvery parameter to be set to 1 and because of that it has to be a separate task which generates separate coverage information too. However it is automatically triggered after the test task so the invocation is more convenient for the tester.
> 
> 
> Diffs
> -----
> 
>   COMPILING.txt 835ba33b1e89158bed0e05698b188ab3323eb881 
>   build.gradle cb9eeca74bbf278c3e5fd15de608d8c37c917ddb 
>   src/test/org/apache/sqoop/importjob/SplitByImportTest.java c6fe4f2e8a80c96ad667f4fe4a26510af96562dc 
>   src/test/org/apache/sqoop/manager/TestMainframeManager.java c84f05f660c396a06a5031e00abdae77ffbcf2aa 
>   src/test/org/apache/sqoop/manager/oracle/TestOraOopDBInputSplitGetDebugDetails.java 6f33ad3b650436b7f268b4ef5bfd451bd5e6958e 
>   src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java 5e558717c0d43301ecbf81a37d5ee3fd35756d65 
>   src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java 1a6943786834d27f27523f484d76cf678f18cf48 
>   src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetBinaryRecord.java b4cba28c3611400b5c4227a5166b6c91e9152dc4 
>   src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileGdgEntryParser.java 521a04266e8806321fe7aa6a89c064f369174523 
>   src/test/org/apache/sqoop/s3/TestS3AvroImport.java 7f5f5d62c5cab10f932aa22c3a713b13fefc2b58 
>   src/test/org/apache/sqoop/s3/TestS3ExternalHiveTableImport.java 0c3161e5a783446e35f4754124f86715d103ec0b 
>   src/test/org/apache/sqoop/s3/TestS3ImportWithHadoopCredProvider.java 3a0d6365dc20f8eef5bdd67a4a2dc9c68ff74d7f 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java 5faf59ea80c48fe025294cabd100e7d176032138 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java a4f986423ea299716a29f9d02f7c8453a7f2ba02 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java d271588c5af060bbc3d301a845f45c46d0f6a2ba 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java 52d89c775b5f1219471df44d222fd92a59ed408c 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java 39238c5fab56b54a85dde5aed0d4bb2c77382fa6 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java 597e3def2cc33adebeeb3bc1ee35ad8a7f4b990d 
>   src/test/org/apache/sqoop/s3/TestS3ParquetImport.java c9785d816d4a7a5870d74c51a9faa229f6d3818e 
>   src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java bba8b74ebe639df26e977abf377f4904144dcfaa 
>   src/test/org/apache/sqoop/s3/TestS3TextImport.java 114f97cbb8857a7633cae5d030769ac4a90e36aa 
>   src/test/org/apache/sqoop/testcategories/thirdpartytest/S3Test.java PRE-CREATION 
>   src/test/org/apache/sqoop/tool/TestS3IncrementalImportOptionValidations.java 7745f1b07e6d6c457b0164deeace12587ec058d0 
> 
> 
> Diff: https://reviews.apache.org/r/69346/diff/2/
> 
> 
> Testing
> -------
> 
> ./gradlew unitTest
> ./gradlew integrationTest
> ./gradlew kerberizedTest
> ./gradlew ... s3Test
> ./gradlew test
> ./gradlew ... thirdPartyTest
> ./gradlew allTest
> 
> 
> Thanks,
> 
> Szabolcs Vasas
> 
>


Re: Review Request 69346: Categorize all tests in the project

Posted by Szabolcs Vasas <va...@gmail.com>.

> On Nov. 16, 2018, 9:43 p.m., Fero Szabo wrote:
> > COMPILING.txt
> > Line 465 (original), 633 (patched)
> > <https://reviews.apache.org/r/69346/diff/2/?file=2108121#file2108121line639>
> >
> >     I believe you require a fully qualified classname here, right? A concrete example here might come in handy. What do you think?

You don't have to specify a full class name here, and you can use wildcards too. I have updated the example and removed -Dtest.single because it is deprecated now.


- Szabolcs


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69346/#review210576
-----------------------------------------------------------


On Nov. 19, 2018, 9:10 a.m., Szabolcs Vasas wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69346/
> -----------------------------------------------------------
> 
> (Updated Nov. 19, 2018, 9:10 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-3404
>     https://issues.apache.org/jira/browse/SQOOP-3404
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> - All tests are categorized now
> - Introduced S3Test category as a subcategory of ThirdPartyTest
> - Reorganized test tasks: we have unitTest, integrationTest, kerberizedTest, test, s3Test, allTest tasks now
> - jacocoTestReport task is fixed to contain the coverage information of the kerberizedTest task too. This is needed because the kerberizedTest needs the forkEvery parameter to be set to 1 and because of that it has to be a separate task which generates separate coverage information too. However it is automatically triggered after the test task so the invocation is more convenient for the tester.
> 
> 
> Diffs
> -----
> 
>   COMPILING.txt 835ba33b1e89158bed0e05698b188ab3323eb881 
>   build.gradle cb9eeca74bbf278c3e5fd15de608d8c37c917ddb 
>   src/test/org/apache/sqoop/importjob/SplitByImportTest.java c6fe4f2e8a80c96ad667f4fe4a26510af96562dc 
>   src/test/org/apache/sqoop/manager/TestMainframeManager.java c84f05f660c396a06a5031e00abdae77ffbcf2aa 
>   src/test/org/apache/sqoop/manager/oracle/TestOraOopDBInputSplitGetDebugDetails.java 6f33ad3b650436b7f268b4ef5bfd451bd5e6958e 
>   src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java 5e558717c0d43301ecbf81a37d5ee3fd35756d65 
>   src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java 1a6943786834d27f27523f484d76cf678f18cf48 
>   src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetBinaryRecord.java b4cba28c3611400b5c4227a5166b6c91e9152dc4 
>   src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileGdgEntryParser.java 521a04266e8806321fe7aa6a89c064f369174523 
>   src/test/org/apache/sqoop/s3/TestS3AvroImport.java 7f5f5d62c5cab10f932aa22c3a713b13fefc2b58 
>   src/test/org/apache/sqoop/s3/TestS3ExternalHiveTableImport.java 0c3161e5a783446e35f4754124f86715d103ec0b 
>   src/test/org/apache/sqoop/s3/TestS3ImportWithHadoopCredProvider.java 3a0d6365dc20f8eef5bdd67a4a2dc9c68ff74d7f 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java 5faf59ea80c48fe025294cabd100e7d176032138 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java a4f986423ea299716a29f9d02f7c8453a7f2ba02 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java d271588c5af060bbc3d301a845f45c46d0f6a2ba 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java 52d89c775b5f1219471df44d222fd92a59ed408c 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java 39238c5fab56b54a85dde5aed0d4bb2c77382fa6 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java 597e3def2cc33adebeeb3bc1ee35ad8a7f4b990d 
>   src/test/org/apache/sqoop/s3/TestS3ParquetImport.java c9785d816d4a7a5870d74c51a9faa229f6d3818e 
>   src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java bba8b74ebe639df26e977abf377f4904144dcfaa 
>   src/test/org/apache/sqoop/s3/TestS3TextImport.java 114f97cbb8857a7633cae5d030769ac4a90e36aa 
>   src/test/org/apache/sqoop/testcategories/thirdpartytest/S3Test.java PRE-CREATION 
>   src/test/org/apache/sqoop/tool/TestS3IncrementalImportOptionValidations.java 7745f1b07e6d6c457b0164deeace12587ec058d0 
> 
> 
> Diff: https://reviews.apache.org/r/69346/diff/4/
> 
> 
> Testing
> -------
> 
> ./gradlew unitTest
> ./gradlew integrationTest
> ./gradlew kerberizedTest
> ./gradlew ... s3Test
> ./gradlew test
> ./gradlew ... thirdPartyTest
> ./gradlew allTest
> 
> 
> Thanks,
> 
> Szabolcs Vasas
> 
>


Re: Review Request 69346: Categorize all tests in the project

Posted by Fero Szabo via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69346/#review210576
-----------------------------------------------------------


Ship it!




Hi Szabi,

lgtm, just a minor suggestion.


COMPILING.txt
Line 465 (original), 633 (patched)
<https://reviews.apache.org/r/69346/#comment295281>

    I believe you require a fully qualified classname here, right? A concrete example here might come in handy. What do you think?


- Fero Szabo


On Nov. 15, 2018, 5:45 p.m., Szabolcs Vasas wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69346/
> -----------------------------------------------------------
> 
> (Updated Nov. 15, 2018, 5:45 p.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-3404
>     https://issues.apache.org/jira/browse/SQOOP-3404
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> - All tests are categorized now
> - Introduced S3Test category as a subcategory of ThirdPartyTest
> - Reorganized test tasks: we have unitTest, integrationTest, kerberizedTest, test, s3Test, allTest tasks now
> - jacocoTestReport task is fixed to contain the coverage information of the kerberizedTest task too. This is needed because the kerberizedTest needs the forkEvery parameter to be set to 1 and because of that it has to be a separate task which generates separate coverage information too. However it is automatically triggered after the test task so the invocation is more convenient for the tester.
> 
> 
> Diffs
> -----
> 
>   COMPILING.txt 835ba33b1e89158bed0e05698b188ab3323eb881 
>   build.gradle cb9eeca74bbf278c3e5fd15de608d8c37c917ddb 
>   src/test/org/apache/sqoop/importjob/SplitByImportTest.java c6fe4f2e8a80c96ad667f4fe4a26510af96562dc 
>   src/test/org/apache/sqoop/manager/TestMainframeManager.java c84f05f660c396a06a5031e00abdae77ffbcf2aa 
>   src/test/org/apache/sqoop/manager/oracle/TestOraOopDBInputSplitGetDebugDetails.java 6f33ad3b650436b7f268b4ef5bfd451bd5e6958e 
>   src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java 5e558717c0d43301ecbf81a37d5ee3fd35756d65 
>   src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java 1a6943786834d27f27523f484d76cf678f18cf48 
>   src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetBinaryRecord.java b4cba28c3611400b5c4227a5166b6c91e9152dc4 
>   src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileGdgEntryParser.java 521a04266e8806321fe7aa6a89c064f369174523 
>   src/test/org/apache/sqoop/s3/TestS3AvroImport.java 7f5f5d62c5cab10f932aa22c3a713b13fefc2b58 
>   src/test/org/apache/sqoop/s3/TestS3ExternalHiveTableImport.java 0c3161e5a783446e35f4754124f86715d103ec0b 
>   src/test/org/apache/sqoop/s3/TestS3ImportWithHadoopCredProvider.java 3a0d6365dc20f8eef5bdd67a4a2dc9c68ff74d7f 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java 5faf59ea80c48fe025294cabd100e7d176032138 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java a4f986423ea299716a29f9d02f7c8453a7f2ba02 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java d271588c5af060bbc3d301a845f45c46d0f6a2ba 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java 52d89c775b5f1219471df44d222fd92a59ed408c 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java 39238c5fab56b54a85dde5aed0d4bb2c77382fa6 
>   src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java 597e3def2cc33adebeeb3bc1ee35ad8a7f4b990d 
>   src/test/org/apache/sqoop/s3/TestS3ParquetImport.java c9785d816d4a7a5870d74c51a9faa229f6d3818e 
>   src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java bba8b74ebe639df26e977abf377f4904144dcfaa 
>   src/test/org/apache/sqoop/s3/TestS3TextImport.java 114f97cbb8857a7633cae5d030769ac4a90e36aa 
>   src/test/org/apache/sqoop/testcategories/thirdpartytest/S3Test.java PRE-CREATION 
>   src/test/org/apache/sqoop/tool/TestS3IncrementalImportOptionValidations.java 7745f1b07e6d6c457b0164deeace12587ec058d0 
> 
> 
> Diff: https://reviews.apache.org/r/69346/diff/3/
> 
> 
> Testing
> -------
> 
> ./gradlew unitTest
> ./gradlew integrationTest
> ./gradlew kerberizedTest
> ./gradlew ... s3Test
> ./gradlew test
> ./gradlew ... thirdPartyTest
> ./gradlew allTest
> 
> 
> Thanks,
> 
> Szabolcs Vasas
> 
>


Re: Review Request 69346: Categorize all tests in the project

Posted by Szabolcs Vasas <va...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69346/
-----------------------------------------------------------

(Updated Nov. 19, 2018, 9:10 a.m.)


Review request for Sqoop.


Bugs: SQOOP-3404
    https://issues.apache.org/jira/browse/SQOOP-3404


Repository: sqoop-trunk


Description
-------

- All tests are categorized now
- Introduced S3Test category as a subcategory of ThirdPartyTest
- Reorganized test tasks: we have unitTest, integrationTest, kerberizedTest, test, s3Test, allTest tasks now
- jacocoTestReport task is fixed to contain the coverage information of the kerberizedTest task too. This is needed because the kerberizedTest needs the forkEvery parameter to be set to 1 and because of that it has to be a separate task which generates separate coverage information too. However it is automatically triggered after the test task so the invocation is more convenient for the tester.


Diffs (updated)
-----

  COMPILING.txt 835ba33b1e89158bed0e05698b188ab3323eb881 
  build.gradle cb9eeca74bbf278c3e5fd15de608d8c37c917ddb 
  src/test/org/apache/sqoop/importjob/SplitByImportTest.java c6fe4f2e8a80c96ad667f4fe4a26510af96562dc 
  src/test/org/apache/sqoop/manager/TestMainframeManager.java c84f05f660c396a06a5031e00abdae77ffbcf2aa 
  src/test/org/apache/sqoop/manager/oracle/TestOraOopDBInputSplitGetDebugDetails.java 6f33ad3b650436b7f268b4ef5bfd451bd5e6958e 
  src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java 5e558717c0d43301ecbf81a37d5ee3fd35756d65 
  src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java 1a6943786834d27f27523f484d76cf678f18cf48 
  src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetBinaryRecord.java b4cba28c3611400b5c4227a5166b6c91e9152dc4 
  src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileGdgEntryParser.java 521a04266e8806321fe7aa6a89c064f369174523 
  src/test/org/apache/sqoop/s3/TestS3AvroImport.java 7f5f5d62c5cab10f932aa22c3a713b13fefc2b58 
  src/test/org/apache/sqoop/s3/TestS3ExternalHiveTableImport.java 0c3161e5a783446e35f4754124f86715d103ec0b 
  src/test/org/apache/sqoop/s3/TestS3ImportWithHadoopCredProvider.java 3a0d6365dc20f8eef5bdd67a4a2dc9c68ff74d7f 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java 5faf59ea80c48fe025294cabd100e7d176032138 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java a4f986423ea299716a29f9d02f7c8453a7f2ba02 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java d271588c5af060bbc3d301a845f45c46d0f6a2ba 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java 52d89c775b5f1219471df44d222fd92a59ed408c 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java 39238c5fab56b54a85dde5aed0d4bb2c77382fa6 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java 597e3def2cc33adebeeb3bc1ee35ad8a7f4b990d 
  src/test/org/apache/sqoop/s3/TestS3ParquetImport.java c9785d816d4a7a5870d74c51a9faa229f6d3818e 
  src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java bba8b74ebe639df26e977abf377f4904144dcfaa 
  src/test/org/apache/sqoop/s3/TestS3TextImport.java 114f97cbb8857a7633cae5d030769ac4a90e36aa 
  src/test/org/apache/sqoop/testcategories/thirdpartytest/S3Test.java PRE-CREATION 
  src/test/org/apache/sqoop/tool/TestS3IncrementalImportOptionValidations.java 7745f1b07e6d6c457b0164deeace12587ec058d0 


Diff: https://reviews.apache.org/r/69346/diff/4/

Changes: https://reviews.apache.org/r/69346/diff/3-4/


Testing
-------

./gradlew unitTest
./gradlew integrationTest
./gradlew kerberizedTest
./gradlew ... s3Test
./gradlew test
./gradlew ... thirdPartyTest
./gradlew allTest


Thanks,

Szabolcs Vasas


Re: Review Request 69346: Categorize all tests in the project

Posted by Szabolcs Vasas <va...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69346/
-----------------------------------------------------------

(Updated Nov. 15, 2018, 5:45 p.m.)


Review request for Sqoop.


Changes
-------

Fixed review findings.


Bugs: SQOOP-3404
    https://issues.apache.org/jira/browse/SQOOP-3404


Repository: sqoop-trunk


Description
-------

- All tests are categorized now
- Introduced S3Test category as a subcategory of ThirdPartyTest
- Reorganized test tasks: we have unitTest, integrationTest, kerberizedTest, test, s3Test, allTest tasks now
- jacocoTestReport task is fixed to contain the coverage information of the kerberizedTest task too. This is needed because the kerberizedTest needs the forkEvery parameter to be set to 1 and because of that it has to be a separate task which generates separate coverage information too. However it is automatically triggered after the test task so the invocation is more convenient for the tester.


Diffs (updated)
-----

  COMPILING.txt 835ba33b1e89158bed0e05698b188ab3323eb881 
  build.gradle cb9eeca74bbf278c3e5fd15de608d8c37c917ddb 
  src/test/org/apache/sqoop/importjob/SplitByImportTest.java c6fe4f2e8a80c96ad667f4fe4a26510af96562dc 
  src/test/org/apache/sqoop/manager/TestMainframeManager.java c84f05f660c396a06a5031e00abdae77ffbcf2aa 
  src/test/org/apache/sqoop/manager/oracle/TestOraOopDBInputSplitGetDebugDetails.java 6f33ad3b650436b7f268b4ef5bfd451bd5e6958e 
  src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java 5e558717c0d43301ecbf81a37d5ee3fd35756d65 
  src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java 1a6943786834d27f27523f484d76cf678f18cf48 
  src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetBinaryRecord.java b4cba28c3611400b5c4227a5166b6c91e9152dc4 
  src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileGdgEntryParser.java 521a04266e8806321fe7aa6a89c064f369174523 
  src/test/org/apache/sqoop/s3/TestS3AvroImport.java 7f5f5d62c5cab10f932aa22c3a713b13fefc2b58 
  src/test/org/apache/sqoop/s3/TestS3ExternalHiveTableImport.java 0c3161e5a783446e35f4754124f86715d103ec0b 
  src/test/org/apache/sqoop/s3/TestS3ImportWithHadoopCredProvider.java 3a0d6365dc20f8eef5bdd67a4a2dc9c68ff74d7f 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java 5faf59ea80c48fe025294cabd100e7d176032138 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java a4f986423ea299716a29f9d02f7c8453a7f2ba02 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java d271588c5af060bbc3d301a845f45c46d0f6a2ba 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java 52d89c775b5f1219471df44d222fd92a59ed408c 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java 39238c5fab56b54a85dde5aed0d4bb2c77382fa6 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java 597e3def2cc33adebeeb3bc1ee35ad8a7f4b990d 
  src/test/org/apache/sqoop/s3/TestS3ParquetImport.java c9785d816d4a7a5870d74c51a9faa229f6d3818e 
  src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java bba8b74ebe639df26e977abf377f4904144dcfaa 
  src/test/org/apache/sqoop/s3/TestS3TextImport.java 114f97cbb8857a7633cae5d030769ac4a90e36aa 
  src/test/org/apache/sqoop/testcategories/thirdpartytest/S3Test.java PRE-CREATION 
  src/test/org/apache/sqoop/tool/TestS3IncrementalImportOptionValidations.java 7745f1b07e6d6c457b0164deeace12587ec058d0 


Diff: https://reviews.apache.org/r/69346/diff/3/

Changes: https://reviews.apache.org/r/69346/diff/2-3/


Testing
-------

./gradlew unitTest
./gradlew integrationTest
./gradlew kerberizedTest
./gradlew ... s3Test
./gradlew test
./gradlew ... thirdPartyTest
./gradlew allTest


Thanks,

Szabolcs Vasas


Re: Review Request 69346: Categorize all tests in the project

Posted by Szabolcs Vasas <va...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69346/
-----------------------------------------------------------

(Updated Nov. 15, 2018, 1:40 p.m.)


Review request for Sqoop.


Changes
-------

Updated COMPILING.txt to document the test tasks we have now with Gradle.
It was outdated and not properly formatted in a few places so I updated those parts as well.
Please check the formatting by running asciidoc COMPILING.txt and opening the output HTML.


Bugs: SQOOP-3404
    https://issues.apache.org/jira/browse/SQOOP-3404


Repository: sqoop-trunk


Description
-------

- All tests are categorized now
- Introduced S3Test category as a subcategory of ThirdPartyTest
- Reorganized test tasks: we have unitTest, integrationTest, kerberizedTest, test, s3Test, allTest tasks now
- jacocoTestReport task is fixed to contain the coverage information of the kerberizedTest task too. This is needed because the kerberizedTest needs the forkEvery parameter to be set to 1 and because of that it has to be a separate task which generates separate coverage information too. However it is automatically triggered after the test task so the invocation is more convenient for the tester.


Diffs (updated)
-----

  COMPILING.txt 835ba33b1e89158bed0e05698b188ab3323eb881 
  build.gradle cb9eeca74bbf278c3e5fd15de608d8c37c917ddb 
  src/test/org/apache/sqoop/importjob/SplitByImportTest.java c6fe4f2e8a80c96ad667f4fe4a26510af96562dc 
  src/test/org/apache/sqoop/manager/TestMainframeManager.java c84f05f660c396a06a5031e00abdae77ffbcf2aa 
  src/test/org/apache/sqoop/manager/oracle/TestOraOopDBInputSplitGetDebugDetails.java 6f33ad3b650436b7f268b4ef5bfd451bd5e6958e 
  src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java 5e558717c0d43301ecbf81a37d5ee3fd35756d65 
  src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java 1a6943786834d27f27523f484d76cf678f18cf48 
  src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetBinaryRecord.java b4cba28c3611400b5c4227a5166b6c91e9152dc4 
  src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileGdgEntryParser.java 521a04266e8806321fe7aa6a89c064f369174523 
  src/test/org/apache/sqoop/s3/TestS3AvroImport.java 7f5f5d62c5cab10f932aa22c3a713b13fefc2b58 
  src/test/org/apache/sqoop/s3/TestS3ExternalHiveTableImport.java 0c3161e5a783446e35f4754124f86715d103ec0b 
  src/test/org/apache/sqoop/s3/TestS3ImportWithHadoopCredProvider.java 3a0d6365dc20f8eef5bdd67a4a2dc9c68ff74d7f 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java 5faf59ea80c48fe025294cabd100e7d176032138 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java a4f986423ea299716a29f9d02f7c8453a7f2ba02 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java d271588c5af060bbc3d301a845f45c46d0f6a2ba 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java 52d89c775b5f1219471df44d222fd92a59ed408c 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java 39238c5fab56b54a85dde5aed0d4bb2c77382fa6 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java 597e3def2cc33adebeeb3bc1ee35ad8a7f4b990d 
  src/test/org/apache/sqoop/s3/TestS3ParquetImport.java c9785d816d4a7a5870d74c51a9faa229f6d3818e 
  src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java bba8b74ebe639df26e977abf377f4904144dcfaa 
  src/test/org/apache/sqoop/s3/TestS3TextImport.java 114f97cbb8857a7633cae5d030769ac4a90e36aa 
  src/test/org/apache/sqoop/testcategories/thirdpartytest/S3Test.java PRE-CREATION 
  src/test/org/apache/sqoop/tool/TestS3IncrementalImportOptionValidations.java 7745f1b07e6d6c457b0164deeace12587ec058d0 


Diff: https://reviews.apache.org/r/69346/diff/2/

Changes: https://reviews.apache.org/r/69346/diff/1-2/


Testing
-------

./gradlew unitTest
./gradlew integrationTest
./gradlew kerberizedTest
./gradlew ... s3Test
./gradlew test
./gradlew ... thirdPartyTest
./gradlew allTest


Thanks,

Szabolcs Vasas