You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Jarek Cecho <ja...@apache.org> on 2013/06/30 20:25:12 UTC
Review Request 12201: SQOOP-1114 Sqoop2: Integration: Add tests for various
Generic JDBC Connector splitters
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12201/
-----------------------------------------------------------
Review request for Sqoop.
Bugs: SQOOP-1114
https://issues.apache.org/jira/browse/SQOOP-1114
Repository: sqoop-sqoop2
Description
-------
I've added new test case that is testing most of splitters available in Generic JDBC Connector. For those tests I've prepared new testing data set that is based on Ubuntu releases and updated junit to version 4.11 for better Parameterized runner.
Diffs
-----
pom.xml 8785e01000cc6e7d5a74ffeb96b83d236a657df8
test/src/main/java/org/apache/sqoop/test/data/UbuntuReleases.java PRE-CREATION
test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 6aeadd4e54e47e5644270b15813b2d9c4cedc059
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/SplitterTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/12201/diff/
Testing
-------
2 out of 4 new tests are currently failing, both appear to be code (and not test) issues, so I've filled SQOOP-1112 and SQOOP-1113 to deal with them separately.
Thanks,
Jarek Cecho
Re: Review Request 12201: SQOOP-1114 Sqoop2: Integration: Add tests for
various Generic JDBC Connector splitters
Posted by Hari Shreedharan <hs...@cloudera.com>.
> On July 30, 2013, 8:53 p.m., Hari Shreedharan wrote:
> > This looks good. I will wait for SQOOP-1112 and SQOOP-1113 to be fixed and committed before committing this one - so all the tests actually pass.
Looks like those have been committed, but I still see test failures:
Failed tests: testSplitter[code_name-13-3](org.apache.sqoop.integration.connector.jdbc.generic.imports.PartitionerTest): Output do not match expectations.
testSplitter[code_name-13-5](org.apache.sqoop.integration.connector.jdbc.generic.imports.PartitionerTest): Output do not match expectations.
testSplitter[code_name-13-10](org.apache.sqoop.integration.connector.jdbc.generic.imports.PartitionerTest): Output do not match expectations.
testSplitter[code_name-13-13](org.apache.sqoop.integration.connector.jdbc.generic.imports.PartitionerTest): Output do not match expectations.
- Hari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12201/#review24281
-----------------------------------------------------------
On July 30, 2013, 4:10 a.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12201/
> -----------------------------------------------------------
>
> (Updated July 30, 2013, 4:10 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1114
> https://issues.apache.org/jira/browse/SQOOP-1114
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> I've added new test case that is testing most of splitters available in Generic JDBC Connector. For those tests I've prepared new testing data set that is based on Ubuntu releases and updated junit to version 4.11 for better Parameterized runner.
>
>
> Diffs
> -----
>
> pom.xml 520c107662ff1215820369d873daa25289ce492f
> test/src/main/java/org/apache/sqoop/test/asserts/HdfsAsserts.java 056e6124b918ce5821c389e388a0cdfa68fd7fc0
> test/src/main/java/org/apache/sqoop/test/data/UbuntuReleases.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java bf1c91f21784cbb57a4d5f07711ad57deaaf3f7d
> test/src/main/java/org/apache/sqoop/test/testcases/TomcatTestCase.java ca77e64486d80b38306b5b30e185e6278ad7eaf1
> test/src/main/java/org/apache/sqoop/test/utils/ParametrizedUtils.java PRE-CREATION
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/PartitionerTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/12201/diff/
>
>
> Testing
> -------
>
> 2 out of 4 new tests are currently failing, both appear to be code (and not test) issues, so I've filled SQOOP-1112 and SQOOP-1113 to deal with them separately.
>
>
> Thanks,
>
> Jarek Cecho
>
>
Re: Review Request 12201: SQOOP-1114 Sqoop2: Integration: Add tests for
various Generic JDBC Connector splitters
Posted by Jarek Cecho <ja...@apache.org>.
> On July 30, 2013, 8:53 p.m., Hari Shreedharan wrote:
> > This looks good. I will wait for SQOOP-1112 and SQOOP-1113 to be fixed and committed before committing this one - so all the tests actually pass.
>
> Hari Shreedharan wrote:
> Looks like those have been committed, but I still see test failures:
> Failed tests: testSplitter[code_name-13-3](org.apache.sqoop.integration.connector.jdbc.generic.imports.PartitionerTest): Output do not match expectations.
> testSplitter[code_name-13-5](org.apache.sqoop.integration.connector.jdbc.generic.imports.PartitionerTest): Output do not match expectations.
> testSplitter[code_name-13-10](org.apache.sqoop.integration.connector.jdbc.generic.imports.PartitionerTest): Output do not match expectations.
> testSplitter[code_name-13-13](org.apache.sqoop.integration.connector.jdbc.generic.imports.PartitionerTest): Output do not match expectations.
Hi Hari,
thank you much for your review! The build failures are expected and are being tracked by SQOOP-1154. The JIRA itself is already in "Patch available" state, so feel free to review it as well!
Jarcec
- Jarek
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12201/#review24281
-----------------------------------------------------------
On July 30, 2013, 4:10 a.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12201/
> -----------------------------------------------------------
>
> (Updated July 30, 2013, 4:10 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1114
> https://issues.apache.org/jira/browse/SQOOP-1114
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> I've added new test case that is testing most of splitters available in Generic JDBC Connector. For those tests I've prepared new testing data set that is based on Ubuntu releases and updated junit to version 4.11 for better Parameterized runner.
>
>
> Diffs
> -----
>
> pom.xml 520c107662ff1215820369d873daa25289ce492f
> test/src/main/java/org/apache/sqoop/test/asserts/HdfsAsserts.java 056e6124b918ce5821c389e388a0cdfa68fd7fc0
> test/src/main/java/org/apache/sqoop/test/data/UbuntuReleases.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java bf1c91f21784cbb57a4d5f07711ad57deaaf3f7d
> test/src/main/java/org/apache/sqoop/test/testcases/TomcatTestCase.java ca77e64486d80b38306b5b30e185e6278ad7eaf1
> test/src/main/java/org/apache/sqoop/test/utils/ParametrizedUtils.java PRE-CREATION
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/PartitionerTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/12201/diff/
>
>
> Testing
> -------
>
> 2 out of 4 new tests are currently failing, both appear to be code (and not test) issues, so I've filled SQOOP-1112 and SQOOP-1113 to deal with them separately.
>
>
> Thanks,
>
> Jarek Cecho
>
>
Re: Review Request 12201: SQOOP-1114 Sqoop2: Integration: Add tests for
various Generic JDBC Connector splitters
Posted by Hari Shreedharan <hs...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12201/#review24281
-----------------------------------------------------------
This looks good. I will wait for SQOOP-1112 and SQOOP-1113 to be fixed and committed before committing this one - so all the tests actually pass.
- Hari Shreedharan
On July 30, 2013, 4:10 a.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12201/
> -----------------------------------------------------------
>
> (Updated July 30, 2013, 4:10 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1114
> https://issues.apache.org/jira/browse/SQOOP-1114
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> I've added new test case that is testing most of splitters available in Generic JDBC Connector. For those tests I've prepared new testing data set that is based on Ubuntu releases and updated junit to version 4.11 for better Parameterized runner.
>
>
> Diffs
> -----
>
> pom.xml 520c107662ff1215820369d873daa25289ce492f
> test/src/main/java/org/apache/sqoop/test/asserts/HdfsAsserts.java 056e6124b918ce5821c389e388a0cdfa68fd7fc0
> test/src/main/java/org/apache/sqoop/test/data/UbuntuReleases.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java bf1c91f21784cbb57a4d5f07711ad57deaaf3f7d
> test/src/main/java/org/apache/sqoop/test/testcases/TomcatTestCase.java ca77e64486d80b38306b5b30e185e6278ad7eaf1
> test/src/main/java/org/apache/sqoop/test/utils/ParametrizedUtils.java PRE-CREATION
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/PartitionerTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/12201/diff/
>
>
> Testing
> -------
>
> 2 out of 4 new tests are currently failing, both appear to be code (and not test) issues, so I've filled SQOOP-1112 and SQOOP-1113 to deal with them separately.
>
>
> Thanks,
>
> Jarek Cecho
>
>
Re: Review Request 12201: SQOOP-1114 Sqoop2: Integration: Add tests for
various Generic JDBC Connector splitters
Posted by Hari Shreedharan <hs...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12201/#review24328
-----------------------------------------------------------
Ship it!
Ship It!
- Hari Shreedharan
On July 30, 2013, 4:10 a.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12201/
> -----------------------------------------------------------
>
> (Updated July 30, 2013, 4:10 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1114
> https://issues.apache.org/jira/browse/SQOOP-1114
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> I've added new test case that is testing most of splitters available in Generic JDBC Connector. For those tests I've prepared new testing data set that is based on Ubuntu releases and updated junit to version 4.11 for better Parameterized runner.
>
>
> Diffs
> -----
>
> pom.xml 520c107662ff1215820369d873daa25289ce492f
> test/src/main/java/org/apache/sqoop/test/asserts/HdfsAsserts.java 056e6124b918ce5821c389e388a0cdfa68fd7fc0
> test/src/main/java/org/apache/sqoop/test/data/UbuntuReleases.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java bf1c91f21784cbb57a4d5f07711ad57deaaf3f7d
> test/src/main/java/org/apache/sqoop/test/testcases/TomcatTestCase.java ca77e64486d80b38306b5b30e185e6278ad7eaf1
> test/src/main/java/org/apache/sqoop/test/utils/ParametrizedUtils.java PRE-CREATION
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/PartitionerTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/12201/diff/
>
>
> Testing
> -------
>
> 2 out of 4 new tests are currently failing, both appear to be code (and not test) issues, so I've filled SQOOP-1112 and SQOOP-1113 to deal with them separately.
>
>
> Thanks,
>
> Jarek Cecho
>
>
Re: Review Request 12201: SQOOP-1114 Sqoop2: Integration: Add tests for
various Generic JDBC Connector splitters
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12201/
-----------------------------------------------------------
(Updated July 30, 2013, 4:10 a.m.)
Review request for Sqoop.
Changes
-------
Rebasing on current head. Full build will still fail because of SQOOP-1154 (there was typo in my previous comment).
Bugs: SQOOP-1114
https://issues.apache.org/jira/browse/SQOOP-1114
Repository: sqoop-sqoop2
Description
-------
I've added new test case that is testing most of splitters available in Generic JDBC Connector. For those tests I've prepared new testing data set that is based on Ubuntu releases and updated junit to version 4.11 for better Parameterized runner.
Diffs (updated)
-----
pom.xml 520c107662ff1215820369d873daa25289ce492f
test/src/main/java/org/apache/sqoop/test/asserts/HdfsAsserts.java 056e6124b918ce5821c389e388a0cdfa68fd7fc0
test/src/main/java/org/apache/sqoop/test/data/UbuntuReleases.java PRE-CREATION
test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java bf1c91f21784cbb57a4d5f07711ad57deaaf3f7d
test/src/main/java/org/apache/sqoop/test/testcases/TomcatTestCase.java ca77e64486d80b38306b5b30e185e6278ad7eaf1
test/src/main/java/org/apache/sqoop/test/utils/ParametrizedUtils.java PRE-CREATION
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/PartitionerTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/12201/diff/
Testing
-------
2 out of 4 new tests are currently failing, both appear to be code (and not test) issues, so I've filled SQOOP-1112 and SQOOP-1113 to deal with them separately.
Thanks,
Jarek Cecho
Re: Review Request 12201: SQOOP-1114 Sqoop2: Integration: Add tests for
various Generic JDBC Connector splitters
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12201/
-----------------------------------------------------------
(Updated July 30, 2013, 3:21 a.m.)
Review request for Sqoop.
Changes
-------
Changes:
* Rebased on current head of Sqoop2 branch.
* Incorporated Hari's feedback by counting number of output files
* Improved the test case by calling several different splitters
* Currently the integration test is failing on SQOOP-1113
Bugs: SQOOP-1114
https://issues.apache.org/jira/browse/SQOOP-1114
Repository: sqoop-sqoop2
Description
-------
I've added new test case that is testing most of splitters available in Generic JDBC Connector. For those tests I've prepared new testing data set that is based on Ubuntu releases and updated junit to version 4.11 for better Parameterized runner.
Diffs (updated)
-----
pom.xml 520c107662ff1215820369d873daa25289ce492f
test/src/main/java/org/apache/sqoop/test/asserts/HdfsAsserts.java 056e6124b918ce5821c389e388a0cdfa68fd7fc0
test/src/main/java/org/apache/sqoop/test/data/UbuntuReleases.java PRE-CREATION
test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java bf1c91f21784cbb57a4d5f07711ad57deaaf3f7d
test/src/main/java/org/apache/sqoop/test/testcases/TomcatTestCase.java ca77e64486d80b38306b5b30e185e6278ad7eaf1
test/src/main/java/org/apache/sqoop/test/utils/ParametrizedUtils.java PRE-CREATION
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/PartitionerTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/12201/diff/
Testing
-------
2 out of 4 new tests are currently failing, both appear to be code (and not test) issues, so I've filled SQOOP-1112 and SQOOP-1113 to deal with them separately.
Thanks,
Jarek Cecho
Re: Review Request 12201: SQOOP-1114 Sqoop2: Integration: Add tests for
various Generic JDBC Connector splitters
Posted by Jarek Cecho <ja...@apache.org>.
> On July 12, 2013, 12:32 a.m., Hari Shreedharan wrote:
> > Jarcec,
> >
> > This test really does not test if the input data was really split. It only tests that the input data was received on the other side. We need to ensure that the data was indeed split and each split was really processed separately.
That is very good point Hari, thank you for rising this concern. I'll fix that shortly.
- Jarek
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12201/#review23042
-----------------------------------------------------------
On June 30, 2013, 6:25 p.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12201/
> -----------------------------------------------------------
>
> (Updated June 30, 2013, 6:25 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1114
> https://issues.apache.org/jira/browse/SQOOP-1114
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> I've added new test case that is testing most of splitters available in Generic JDBC Connector. For those tests I've prepared new testing data set that is based on Ubuntu releases and updated junit to version 4.11 for better Parameterized runner.
>
>
> Diffs
> -----
>
> pom.xml 8785e01000cc6e7d5a74ffeb96b83d236a657df8
> test/src/main/java/org/apache/sqoop/test/data/UbuntuReleases.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 6aeadd4e54e47e5644270b15813b2d9c4cedc059
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/SplitterTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/12201/diff/
>
>
> Testing
> -------
>
> 2 out of 4 new tests are currently failing, both appear to be code (and not test) issues, so I've filled SQOOP-1112 and SQOOP-1113 to deal with them separately.
>
>
> Thanks,
>
> Jarek Cecho
>
>
Re: Review Request 12201: SQOOP-1114 Sqoop2: Integration: Add tests for
various Generic JDBC Connector splitters
Posted by Hari Shreedharan <hs...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12201/#review23042
-----------------------------------------------------------
Jarcec,
This test really does not test if the input data was really split. It only tests that the input data was received on the other side. We need to ensure that the data was indeed split and each split was really processed separately.
- Hari Shreedharan
On June 30, 2013, 6:25 p.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12201/
> -----------------------------------------------------------
>
> (Updated June 30, 2013, 6:25 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1114
> https://issues.apache.org/jira/browse/SQOOP-1114
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> I've added new test case that is testing most of splitters available in Generic JDBC Connector. For those tests I've prepared new testing data set that is based on Ubuntu releases and updated junit to version 4.11 for better Parameterized runner.
>
>
> Diffs
> -----
>
> pom.xml 8785e01000cc6e7d5a74ffeb96b83d236a657df8
> test/src/main/java/org/apache/sqoop/test/data/UbuntuReleases.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 6aeadd4e54e47e5644270b15813b2d9c4cedc059
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/SplitterTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/12201/diff/
>
>
> Testing
> -------
>
> 2 out of 4 new tests are currently failing, both appear to be code (and not test) issues, so I've filled SQOOP-1112 and SQOOP-1113 to deal with them separately.
>
>
> Thanks,
>
> Jarek Cecho
>
>