You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Sean Mackrory <ma...@gmail.com> on 2012/12/19 19:15:34 UTC
Review Request: SQOOP-796: Unable to use new Hadoop environment variables
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8685/
-----------------------------------------------------------
Review request for Sqoop.
Description
-------
See JIRA for full description of the problem and solution.
This addresses bug SQOOP-796.
https://issues.apache.org/jira/browse/SQOOP-796
Diffs
-----
bin/configure-sqoop ee1dd7b
bin/sqoop bc69083
conf/sqoop-env-template.sh 0294f17
src/docs/man/common-args.txt 129f5d1
src/docs/man/sqoop.txt 44b4f4e
src/docs/user/common-args.txt eff9939
src/docs/user/help.txt 11b4f0c
src/docs/user/tools.txt ce83a24
src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java a5f72f7
src/java/org/apache/sqoop/SqoopOptions.java 2b5db29
src/java/org/apache/sqoop/orm/CompilationManager.java 26adcbf
src/java/org/apache/sqoop/tool/BaseSqoopTool.java 1ab3e5f
src/scripts/run-perftest.sh 2a4ba21
src/test/com/cloudera/sqoop/manager/PostgresqlImportTest.java 512b1d5
Diff: https://reviews.apache.org/r/8685/diff/
Testing
-------
Performed an import from MySQL on a pre-YARN MapReduce cluster (with HADOOP_HOME set) and a YARN cluster (with HADOOP_COMMON_HOME and HADOOP_MAPRED_HOME set). Although the YARN cluster failed, it appears to be after Sqoop has located all of it's dependencies and that the problem is with my YARN setup.
Thanks,
Sean Mackrory
Re: Review Request: SQOOP-796: Unable to use new Hadoop environment variables
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8685/#review14885
-----------------------------------------------------------
Hi Sean,
thank you very much for your patch. I just have one high level point:
1) Would you mind running "ant checkstyle" and fixing introduced style violations?
src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java
<https://reviews.apache.org/r/8685/#comment31811>
Please do not touch com.cloudera packages. They are deprecated and should not be used any more.
Thanks for your time!
Jarcec
- Jarek Cecho
On Dec. 19, 2012, 6:15 p.m., Sean Mackrory wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8685/
> -----------------------------------------------------------
>
> (Updated Dec. 19, 2012, 6:15 p.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> See JIRA for full description of the problem and solution.
>
>
> This addresses bug SQOOP-796.
> https://issues.apache.org/jira/browse/SQOOP-796
>
>
> Diffs
> -----
>
> bin/configure-sqoop ee1dd7b
> bin/sqoop bc69083
> conf/sqoop-env-template.sh 0294f17
> src/docs/man/common-args.txt 129f5d1
> src/docs/man/sqoop.txt 44b4f4e
> src/docs/user/common-args.txt eff9939
> src/docs/user/help.txt 11b4f0c
> src/docs/user/tools.txt ce83a24
> src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java a5f72f7
> src/java/org/apache/sqoop/SqoopOptions.java 2b5db29
> src/java/org/apache/sqoop/orm/CompilationManager.java 26adcbf
> src/java/org/apache/sqoop/tool/BaseSqoopTool.java 1ab3e5f
> src/scripts/run-perftest.sh 2a4ba21
> src/test/com/cloudera/sqoop/manager/PostgresqlImportTest.java 512b1d5
>
> Diff: https://reviews.apache.org/r/8685/diff/
>
>
> Testing
> -------
>
> Performed an import from MySQL on a pre-YARN MapReduce cluster (with HADOOP_HOME set) and a YARN cluster (with HADOOP_COMMON_HOME and HADOOP_MAPRED_HOME set). Although the YARN cluster failed, it appears to be after Sqoop has located all of it's dependencies and that the problem is with my YARN setup.
>
>
> Thanks,
>
> Sean Mackrory
>
>
Re: Review Request: SQOOP-796: Unable to use new Hadoop environment variables
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8685/#review15230
-----------------------------------------------------------
Ship it!
Hi Sean,
I do have just few small notes to your patch. Please feel free to fix them and upload your patch directly to JIRA (without another round on review board).
src/docs/man/common-args.txt
<https://reviews.apache.org/r/8685/#comment32846>
Can we keep in the parameter list also the original --hadoop-home and just say that it's deprecated?
src/docs/user/help.txt
<https://reviews.apache.org/r/8685/#comment32845>
Can we keep in the parameter list also the original --hadoop-home and just say that it's deprecated?
src/docs/user/tools.txt
<https://reviews.apache.org/r/8685/#comment32843>
Would you mind changing reference from Cloudera to BigTop?
src/docs/user/tools.txt
<https://reviews.apache.org/r/8685/#comment32844>
Can we keep in the parameter list also the original --hadoop-home and just say that it's deprecated?
Jarcec
- Jarek Cecho
On Jan. 9, 2013, 5:39 p.m., Sean Mackrory wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8685/
> -----------------------------------------------------------
>
> (Updated Jan. 9, 2013, 5:39 p.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> See JIRA for full description of the problem and solution.
>
>
> This addresses bug SQOOP-796.
> https://issues.apache.org/jira/browse/SQOOP-796
>
>
> Diffs
> -----
>
> bin/configure-sqoop ee1dd7b
> bin/sqoop bc69083
> conf/sqoop-env-template.sh 0294f17
> src/docs/man/common-args.txt 129f5d1
> src/docs/man/sqoop.txt 44b4f4e
> src/docs/user/common-args.txt 7c69886
> src/docs/user/help.txt 11b4f0c
> src/docs/user/tools.txt ce83a24
> src/java/org/apache/sqoop/SqoopOptions.java 3e0ec3e
> src/java/org/apache/sqoop/orm/CompilationManager.java 26adcbf
> src/java/org/apache/sqoop/tool/BaseSqoopTool.java c0221c9
> src/scripts/run-perftest.sh 2a4ba21
>
> Diff: https://reviews.apache.org/r/8685/diff/
>
>
> Testing
> -------
>
> Performed an import from MySQL on a pre-YARN MapReduce cluster (with HADOOP_HOME set) and a YARN cluster (with HADOOP_COMMON_HOME and HADOOP_MAPRED_HOME set). Although the YARN cluster failed, it appears to be after Sqoop has located all of it's dependencies and that the problem is with my YARN setup.
>
>
> Thanks,
>
> Sean Mackrory
>
>
Re: Review Request: SQOOP-796: Unable to use new Hadoop environment variables
Posted by Sean Mackrory <ma...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8685/
-----------------------------------------------------------
(Updated Jan. 9, 2013, 5:39 p.m.)
Review request for Sqoop.
Changes
-------
Removed changes to Cloudera namespace (my apologies - the patch was originally written during a global search - I just replaced every occurence of HADOOP_HOME) and fixed 1 style violation (a line was longer than 80 characters). I have since done more testing and verified that this also works in YARN clusters with the proper environment.
Description
-------
See JIRA for full description of the problem and solution.
This addresses bug SQOOP-796.
https://issues.apache.org/jira/browse/SQOOP-796
Diffs (updated)
-----
bin/configure-sqoop ee1dd7b
bin/sqoop bc69083
conf/sqoop-env-template.sh 0294f17
src/docs/man/common-args.txt 129f5d1
src/docs/man/sqoop.txt 44b4f4e
src/docs/user/common-args.txt 7c69886
src/docs/user/help.txt 11b4f0c
src/docs/user/tools.txt ce83a24
src/java/org/apache/sqoop/SqoopOptions.java 3e0ec3e
src/java/org/apache/sqoop/orm/CompilationManager.java 26adcbf
src/java/org/apache/sqoop/tool/BaseSqoopTool.java c0221c9
src/scripts/run-perftest.sh 2a4ba21
Diff: https://reviews.apache.org/r/8685/diff/
Testing
-------
Performed an import from MySQL on a pre-YARN MapReduce cluster (with HADOOP_HOME set) and a YARN cluster (with HADOOP_COMMON_HOME and HADOOP_MAPRED_HOME set). Although the YARN cluster failed, it appears to be after Sqoop has located all of it's dependencies and that the problem is with my YARN setup.
Thanks,
Sean Mackrory