You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Benjamin Hindman <be...@berkeley.edu> on 2013/08/02 09:09:55 UTC
Review Request 13225: Refactored Hadoop on Mesos from contrib to JAR.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13225/
-----------------------------------------------------------
Review request for mesos, Ben Mahler, Brenden Matthews, and Vinod Kone.
Repository: mesos-git
Description
-------
First part of cleaning up Hadoop on Mesos. Ultimatly we'd like to get this out of the repository all together. Note that the only code change here was directly invoking 'bin/hadoop org.apache.hadoop.mapred.MesosExecutor' rather than 'mesos-executor' (which we deleted). Given that these source files were moved via 'git mv' I believe that existing patches can still be rebased appropriately and thus shouldn't block this review.
Diffs
-----
hadoop/HadoopPipes.cc.patch e0ca33a0888aeb90f1a26fb94cac32b1a7e15f4a
hadoop/Makefile.am 2702924dd27fb6b32f15a021233b2a3224787627
hadoop/NOTES 1c9948c70367e006d0148a43c3b3f1ee0c545d3c
hadoop/TUTORIAL.sh f4285d5b50833cfad1b3893de38c463617638d64
hadoop/hadoop-0.20.2-cdh3u3_hadoop-env.sh.patch 88c0754ca607bb3acf377b10f144f7173a7750a9
hadoop/hadoop-0.20.2-cdh3u3_mesos.patch 9284788a84abded26ca67d6e0768e1cc694a0e53
hadoop/hadoop-0.20.2-cdh3u5_hadoop-env.sh.patch 38eca6c15134ff1449506583bbe2948e68f5a7b2
hadoop/hadoop-0.20.2-cdh3u5_mesos.patch 9284788a84abded26ca67d6e0768e1cc694a0e53
hadoop/hadoop-0.20.205.0_hadoop-env.sh.patch 066f71538be7ac6f51bbe115663cbe654f1e1d3d
hadoop/hadoop-0.20.205.0_mesos.patch 35fe3ba2bbaa28b206da91a551bf6bcced19bb7d
hadoop/hadoop-2.0.0-mr1-cdh4.1.2_hadoop-env.sh.patch 9dfe4a3ab70507f2db454c85cd258230a19b2c4d
hadoop/hadoop-2.0.0-mr1-cdh4.1.2_mesos.patch 2fc254f3ed194dfb4e68d60e5c643f49014973fc
hadoop/hadoop-2.0.0-mr1-cdh4.2.1_hadoop-env.sh.patch d57c1e7cdf623b2aeb6df0531554415e04c46817
hadoop/hadoop-2.0.0-mr1-cdh4.2.1_mesos.patch 8a394440d1c02e862b2732ecb74886b06446a301
hadoop/hadoop-7698-1.patch 0ab424b0bd16cda79f16f5c23771b1837eb718c5
hadoop/hadoop-gridmix.patch 92b55e0afd76f93d5eac9d5be37ab1693b77f82d
hadoop/mapred-site.xml.patch
hadoop/mesos-executor 7902e8084a16f4348df55abbaf41cbce14fe2189
hadoop/mesos/build.xml 9edf9e2c413cc5be5683051c8e3f2b3037b8900b
hadoop/mesos/ivy.xml fc4fb16f8914a40b31858a2972de0c82cf493505
hadoop/mesos/ivy/libraries.properties d740528dfb3b72270e1acb22cb2cd31a7cccfe55
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 027389d16dd852f1da065d359ff2189e614774b2
hadoop/pom.xml PRE-CREATION
Diff: https://reviews.apache.org/r/13225/diff/
Testing
-------
Thanks,
Benjamin Hindman
Re: Review Request 13225: Refactored Hadoop on Mesos from contrib to JAR.
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13225/
-----------------------------------------------------------
(Updated Aug. 6, 2013, 7:01 p.m.)
Review request for mesos, Ben Mahler, Brenden Matthews, and Vinod Kone.
Bugs: MESOS-618
https://issues.apache.org/jira/browse/MESOS-618
Repository: mesos-git
Description
-------
First part of cleaning up Hadoop on Mesos. Ultimatly we'd like to get this out of the repository all together. Note that the only code change here was directly invoking 'bin/hadoop org.apache.hadoop.mapred.MesosExecutor' rather than 'mesos-executor' (which we deleted). Given that these source files were moved via 'git mv' I believe that existing patches can still be rebased appropriately and thus shouldn't block this review.
Diffs
-----
hadoop/HadoopPipes.cc.patch e0ca33a0888aeb90f1a26fb94cac32b1a7e15f4a
hadoop/Makefile.am 2702924dd27fb6b32f15a021233b2a3224787627
hadoop/NOTES 1c9948c70367e006d0148a43c3b3f1ee0c545d3c
hadoop/TUTORIAL.sh f4285d5b50833cfad1b3893de38c463617638d64
hadoop/hadoop-0.20.2-cdh3u3_hadoop-env.sh.patch 88c0754ca607bb3acf377b10f144f7173a7750a9
hadoop/hadoop-0.20.2-cdh3u3_mesos.patch 9284788a84abded26ca67d6e0768e1cc694a0e53
hadoop/hadoop-0.20.2-cdh3u5_hadoop-env.sh.patch 38eca6c15134ff1449506583bbe2948e68f5a7b2
hadoop/hadoop-0.20.2-cdh3u5_mesos.patch 9284788a84abded26ca67d6e0768e1cc694a0e53
hadoop/hadoop-0.20.205.0_hadoop-env.sh.patch 066f71538be7ac6f51bbe115663cbe654f1e1d3d
hadoop/hadoop-0.20.205.0_mesos.patch 35fe3ba2bbaa28b206da91a551bf6bcced19bb7d
hadoop/hadoop-2.0.0-mr1-cdh4.1.2_hadoop-env.sh.patch 9dfe4a3ab70507f2db454c85cd258230a19b2c4d
hadoop/hadoop-2.0.0-mr1-cdh4.1.2_mesos.patch 2fc254f3ed194dfb4e68d60e5c643f49014973fc
hadoop/hadoop-2.0.0-mr1-cdh4.2.1_hadoop-env.sh.patch d57c1e7cdf623b2aeb6df0531554415e04c46817
hadoop/hadoop-2.0.0-mr1-cdh4.2.1_mesos.patch 8a394440d1c02e862b2732ecb74886b06446a301
hadoop/hadoop-7698-1.patch 0ab424b0bd16cda79f16f5c23771b1837eb718c5
hadoop/hadoop-gridmix.patch 92b55e0afd76f93d5eac9d5be37ab1693b77f82d
hadoop/mapred-site.xml.patch
hadoop/mesos-executor 7902e8084a16f4348df55abbaf41cbce14fe2189
hadoop/mesos/build.xml 9edf9e2c413cc5be5683051c8e3f2b3037b8900b
hadoop/mesos/ivy.xml fc4fb16f8914a40b31858a2972de0c82cf493505
hadoop/mesos/ivy/libraries.properties d740528dfb3b72270e1acb22cb2cd31a7cccfe55
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 027389d16dd852f1da065d359ff2189e614774b2
hadoop/pom.xml PRE-CREATION
Diff: https://reviews.apache.org/r/13225/diff/
Testing
-------
Thanks,
Benjamin Hindman
Re: Review Request 13225: Refactored Hadoop on Mesos from contrib to JAR.
Posted by Benjamin Hindman <be...@berkeley.edu>.
> On Aug. 2, 2013, 5:08 p.m., Vinod Kone wrote:
> > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java, line 641
> > <https://reviews.apache.org/r/13225/diff/2/?file=334398#file334398line641>
> >
> > did this path work out? iirc, we had to do the canonical path thingy because "bin" was not in the cwd of the task tracker.
Yup, I preserved the 'local' semantics by doing:
new File("bin/hadoop").getCanonicalPath() + " org.apache.hadoop.mapred.MesosExecutor"
Which still doesn't require 'mesos-executor'!
> On Aug. 2, 2013, 5:08 p.m., Vinod Kone wrote:
> > hadoop/pom.xml, lines 26-32
> > <https://reviews.apache.org/r/13225/diff/2/?file=334399#file334399line26>
> >
> > Can you add a comment here that this is for short term only? Once we publish 0.12.1 jar we should just point to apache repo.
Done.
> On Aug. 2, 2013, 5:08 p.m., Vinod Kone wrote:
> > hadoop/pom.xml, lines 47-49
> > <https://reviews.apache.org/r/13225/diff/2/?file=334399#file334399line47>
> >
> > ditto
Done.
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13225/#review24539
-----------------------------------------------------------
On Aug. 2, 2013, 7:38 a.m., Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13225/
> -----------------------------------------------------------
>
> (Updated Aug. 2, 2013, 7:38 a.m.)
>
>
> Review request for mesos, Ben Mahler, Brenden Matthews, and Vinod Kone.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> First part of cleaning up Hadoop on Mesos. Ultimatly we'd like to get this out of the repository all together. Note that the only code change here was directly invoking 'bin/hadoop org.apache.hadoop.mapred.MesosExecutor' rather than 'mesos-executor' (which we deleted). Given that these source files were moved via 'git mv' I believe that existing patches can still be rebased appropriately and thus shouldn't block this review.
>
>
> Diffs
> -----
>
> hadoop/HadoopPipes.cc.patch e0ca33a0888aeb90f1a26fb94cac32b1a7e15f4a
> hadoop/Makefile.am 2702924dd27fb6b32f15a021233b2a3224787627
> hadoop/NOTES 1c9948c70367e006d0148a43c3b3f1ee0c545d3c
> hadoop/TUTORIAL.sh f4285d5b50833cfad1b3893de38c463617638d64
> hadoop/hadoop-0.20.2-cdh3u3_hadoop-env.sh.patch 88c0754ca607bb3acf377b10f144f7173a7750a9
> hadoop/hadoop-0.20.2-cdh3u3_mesos.patch 9284788a84abded26ca67d6e0768e1cc694a0e53
> hadoop/hadoop-0.20.2-cdh3u5_hadoop-env.sh.patch 38eca6c15134ff1449506583bbe2948e68f5a7b2
> hadoop/hadoop-0.20.2-cdh3u5_mesos.patch 9284788a84abded26ca67d6e0768e1cc694a0e53
> hadoop/hadoop-0.20.205.0_hadoop-env.sh.patch 066f71538be7ac6f51bbe115663cbe654f1e1d3d
> hadoop/hadoop-0.20.205.0_mesos.patch 35fe3ba2bbaa28b206da91a551bf6bcced19bb7d
> hadoop/hadoop-2.0.0-mr1-cdh4.1.2_hadoop-env.sh.patch 9dfe4a3ab70507f2db454c85cd258230a19b2c4d
> hadoop/hadoop-2.0.0-mr1-cdh4.1.2_mesos.patch 2fc254f3ed194dfb4e68d60e5c643f49014973fc
> hadoop/hadoop-2.0.0-mr1-cdh4.2.1_hadoop-env.sh.patch d57c1e7cdf623b2aeb6df0531554415e04c46817
> hadoop/hadoop-2.0.0-mr1-cdh4.2.1_mesos.patch 8a394440d1c02e862b2732ecb74886b06446a301
> hadoop/hadoop-7698-1.patch 0ab424b0bd16cda79f16f5c23771b1837eb718c5
> hadoop/hadoop-gridmix.patch 92b55e0afd76f93d5eac9d5be37ab1693b77f82d
> hadoop/mapred-site.xml.patch
> hadoop/mesos-executor 7902e8084a16f4348df55abbaf41cbce14fe2189
> hadoop/mesos/build.xml 9edf9e2c413cc5be5683051c8e3f2b3037b8900b
> hadoop/mesos/ivy.xml fc4fb16f8914a40b31858a2972de0c82cf493505
> hadoop/mesos/ivy/libraries.properties d740528dfb3b72270e1acb22cb2cd31a7cccfe55
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 027389d16dd852f1da065d359ff2189e614774b2
> hadoop/pom.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13225/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Benjamin Hindman
>
>
Re: Review Request 13225: Refactored Hadoop on Mesos from contrib to JAR.
Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13225/#review24539
-----------------------------------------------------------
Ship it!
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java
<https://reviews.apache.org/r/13225/#comment48567>
did this path work out? iirc, we had to do the canonical path thingy because "bin" was not in the cwd of the task tracker.
hadoop/pom.xml
<https://reviews.apache.org/r/13225/#comment48552>
Can you add a comment here that this is for short term only? Once we publish 0.12.1 jar we should just point to apache repo.
hadoop/pom.xml
<https://reviews.apache.org/r/13225/#comment48554>
ditto
- Vinod Kone
On Aug. 2, 2013, 7:38 a.m., Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13225/
> -----------------------------------------------------------
>
> (Updated Aug. 2, 2013, 7:38 a.m.)
>
>
> Review request for mesos, Ben Mahler, Brenden Matthews, and Vinod Kone.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> First part of cleaning up Hadoop on Mesos. Ultimatly we'd like to get this out of the repository all together. Note that the only code change here was directly invoking 'bin/hadoop org.apache.hadoop.mapred.MesosExecutor' rather than 'mesos-executor' (which we deleted). Given that these source files were moved via 'git mv' I believe that existing patches can still be rebased appropriately and thus shouldn't block this review.
>
>
> Diffs
> -----
>
> hadoop/HadoopPipes.cc.patch e0ca33a0888aeb90f1a26fb94cac32b1a7e15f4a
> hadoop/Makefile.am 2702924dd27fb6b32f15a021233b2a3224787627
> hadoop/NOTES 1c9948c70367e006d0148a43c3b3f1ee0c545d3c
> hadoop/TUTORIAL.sh f4285d5b50833cfad1b3893de38c463617638d64
> hadoop/hadoop-0.20.2-cdh3u3_hadoop-env.sh.patch 88c0754ca607bb3acf377b10f144f7173a7750a9
> hadoop/hadoop-0.20.2-cdh3u3_mesos.patch 9284788a84abded26ca67d6e0768e1cc694a0e53
> hadoop/hadoop-0.20.2-cdh3u5_hadoop-env.sh.patch 38eca6c15134ff1449506583bbe2948e68f5a7b2
> hadoop/hadoop-0.20.2-cdh3u5_mesos.patch 9284788a84abded26ca67d6e0768e1cc694a0e53
> hadoop/hadoop-0.20.205.0_hadoop-env.sh.patch 066f71538be7ac6f51bbe115663cbe654f1e1d3d
> hadoop/hadoop-0.20.205.0_mesos.patch 35fe3ba2bbaa28b206da91a551bf6bcced19bb7d
> hadoop/hadoop-2.0.0-mr1-cdh4.1.2_hadoop-env.sh.patch 9dfe4a3ab70507f2db454c85cd258230a19b2c4d
> hadoop/hadoop-2.0.0-mr1-cdh4.1.2_mesos.patch 2fc254f3ed194dfb4e68d60e5c643f49014973fc
> hadoop/hadoop-2.0.0-mr1-cdh4.2.1_hadoop-env.sh.patch d57c1e7cdf623b2aeb6df0531554415e04c46817
> hadoop/hadoop-2.0.0-mr1-cdh4.2.1_mesos.patch 8a394440d1c02e862b2732ecb74886b06446a301
> hadoop/hadoop-7698-1.patch 0ab424b0bd16cda79f16f5c23771b1837eb718c5
> hadoop/hadoop-gridmix.patch 92b55e0afd76f93d5eac9d5be37ab1693b77f82d
> hadoop/mapred-site.xml.patch
> hadoop/mesos-executor 7902e8084a16f4348df55abbaf41cbce14fe2189
> hadoop/mesos/build.xml 9edf9e2c413cc5be5683051c8e3f2b3037b8900b
> hadoop/mesos/ivy.xml fc4fb16f8914a40b31858a2972de0c82cf493505
> hadoop/mesos/ivy/libraries.properties d740528dfb3b72270e1acb22cb2cd31a7cccfe55
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 027389d16dd852f1da065d359ff2189e614774b2
> hadoop/pom.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13225/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Benjamin Hindman
>
>
Re: Review Request 13225: Refactored Hadoop on Mesos from contrib to JAR.
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13225/
-----------------------------------------------------------
(Updated Aug. 2, 2013, 7:38 a.m.)
Review request for mesos, Ben Mahler, Brenden Matthews, and Vinod Kone.
Repository: mesos-git
Description
-------
First part of cleaning up Hadoop on Mesos. Ultimatly we'd like to get this out of the repository all together. Note that the only code change here was directly invoking 'bin/hadoop org.apache.hadoop.mapred.MesosExecutor' rather than 'mesos-executor' (which we deleted). Given that these source files were moved via 'git mv' I believe that existing patches can still be rebased appropriately and thus shouldn't block this review.
Diffs (updated)
-----
hadoop/HadoopPipes.cc.patch e0ca33a0888aeb90f1a26fb94cac32b1a7e15f4a
hadoop/Makefile.am 2702924dd27fb6b32f15a021233b2a3224787627
hadoop/NOTES 1c9948c70367e006d0148a43c3b3f1ee0c545d3c
hadoop/TUTORIAL.sh f4285d5b50833cfad1b3893de38c463617638d64
hadoop/hadoop-0.20.2-cdh3u3_hadoop-env.sh.patch 88c0754ca607bb3acf377b10f144f7173a7750a9
hadoop/hadoop-0.20.2-cdh3u3_mesos.patch 9284788a84abded26ca67d6e0768e1cc694a0e53
hadoop/hadoop-0.20.2-cdh3u5_hadoop-env.sh.patch 38eca6c15134ff1449506583bbe2948e68f5a7b2
hadoop/hadoop-0.20.2-cdh3u5_mesos.patch 9284788a84abded26ca67d6e0768e1cc694a0e53
hadoop/hadoop-0.20.205.0_hadoop-env.sh.patch 066f71538be7ac6f51bbe115663cbe654f1e1d3d
hadoop/hadoop-0.20.205.0_mesos.patch 35fe3ba2bbaa28b206da91a551bf6bcced19bb7d
hadoop/hadoop-2.0.0-mr1-cdh4.1.2_hadoop-env.sh.patch 9dfe4a3ab70507f2db454c85cd258230a19b2c4d
hadoop/hadoop-2.0.0-mr1-cdh4.1.2_mesos.patch 2fc254f3ed194dfb4e68d60e5c643f49014973fc
hadoop/hadoop-2.0.0-mr1-cdh4.2.1_hadoop-env.sh.patch d57c1e7cdf623b2aeb6df0531554415e04c46817
hadoop/hadoop-2.0.0-mr1-cdh4.2.1_mesos.patch 8a394440d1c02e862b2732ecb74886b06446a301
hadoop/hadoop-7698-1.patch 0ab424b0bd16cda79f16f5c23771b1837eb718c5
hadoop/hadoop-gridmix.patch 92b55e0afd76f93d5eac9d5be37ab1693b77f82d
hadoop/mapred-site.xml.patch
hadoop/mesos-executor 7902e8084a16f4348df55abbaf41cbce14fe2189
hadoop/mesos/build.xml 9edf9e2c413cc5be5683051c8e3f2b3037b8900b
hadoop/mesos/ivy.xml fc4fb16f8914a40b31858a2972de0c82cf493505
hadoop/mesos/ivy/libraries.properties d740528dfb3b72270e1acb22cb2cd31a7cccfe55
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosExecutor.java
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java 027389d16dd852f1da065d359ff2189e614774b2
hadoop/pom.xml PRE-CREATION
Diff: https://reviews.apache.org/r/13225/diff/
Testing
-------
Thanks,
Benjamin Hindman