You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by "Hiral Patel (Created) (JIRA)" <ji...@apache.org> on 2012/04/09 07:13:24 UTC
[jira] [Created] (MESOS-182) Patch for CDH3u3 for mesos 0.9.0
Patch for CDH3u3 for mesos 0.9.0
--------------------------------
Key: MESOS-182
URL: https://issues.apache.org/jira/browse/MESOS-182
Project: Mesos
Issue Type: Improvement
Components: build
Affects Versions: 0.9.0
Reporter: Hiral Patel
Priority: Trivial
I've attached a patch for cdh3u3 which uses the patch for 0.20.205 as a bases plus adds protobuff jar and a mesos-0.9.0.jar compiled from RC3. I tested it on my local machine with wordcount to validate the tasks run on mesos.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MESOS-182) Patch for CDH3u3 for mesos 0.9.0
Posted by "Hiral Patel (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hiral Patel updated MESOS-182:
------------------------------
Attachment: 0001-Manually-patched-using-0.20.250-patch-for-mesos-plus.patch
git patch from original cdh3u3 tarball
> Patch for CDH3u3 for mesos 0.9.0
> --------------------------------
>
> Key: MESOS-182
> URL: https://issues.apache.org/jira/browse/MESOS-182
> Project: Mesos
> Issue Type: Improvement
> Components: build
> Affects Versions: 0.9.0
> Reporter: Hiral Patel
> Priority: Trivial
> Labels: patch
> Attachments: 0001-Manually-patched-using-0.20.250-patch-for-mesos-plus.patch
>
>
> I've attached a patch for cdh3u3 which uses the patch for 0.20.205 as a bases plus adds protobuff jar and a mesos-0.9.0.jar compiled from RC3. I tested it on my local machine with wordcount to validate the tasks run on mesos.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-182) Patch for CDH3u3 for mesos 0.9.0
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264185#comment-13264185 ]
jiraposter@reviews.apache.org commented on MESOS-182:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4922/
-----------------------------------------------------------
Review request for mesos and Matei Zaharia.
Summary
-------
I wanted to break down the changes to Hadoop so that someone could see exactly how minimal they really are. In addition, I wanted the patches to the Hadoop source code itself to be clearly identifiable. So, I broke the mega patch up. Now, we apply the small Hadoop patch (which has been submitted to Hadoop trunk), COPY the Mesos contrib code over, and then apply two other small patches for hadoop-env.sh and mapred-site.xml.
In addition, we also copy the native library into lib as well as the mesos.jar and protobuf.jar so that we don't need to point to any build directory. Likewise, we don't even need to set MESOS_NATIVE_LIBRARY at all! The resulting patch for hadoop-env.sh is very small indeed.
I also deleted the hadoop.tar.gz that we included because I figured it would be just as easy to download it via the tutorial (once it's downloaded once, the tutorial just keeps using that, only deleting it if you do a 'make clean').
The final directory you get after running 'make hadoop-0.20.205.0' can be zipped, copied, and used right away in a Mesos cluster.
I also used the patch from https://issues.apache.org/jira/browse/MESOS-182 to do the same thing for cdh3u3. You get this via 'make hadoop-0.20.2-cdh3u3'.
This addresses bug MESOS-182.
https://issues.apache.org/jira/browse/MESOS-182
Diffs
-----
hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkScheduler.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkExecutor.java PRE-CREATION
hadoop/mesos/ivy/libraries.properties PRE-CREATION
hadoop/mapred-site.xml.patch PRE-CREATION
hadoop/mesos-executor PRE-CREATION
hadoop/mesos/build.xml PRE-CREATION
hadoop/mesos/ivy.xml PRE-CREATION
hadoop/hadoop-0.20.205.0_hadoop-env.sh.patch PRE-CREATION
hadoop/hadoop-0.20.205.0_mesos.patch PRE-CREATION
hadoop/hadoop-0.20.205.0_conf_mapred-site.xml.patch 4764496
hadoop/hadoop-0.20.205.0.tar.gz bc605a3
hadoop/hadoop-0.20.205.0_conf_hadoop-env.sh.patch 6733ef5
hadoop/hadoop-0.20.2-cdh3u3_hadoop-env.sh.patch PRE-CREATION
hadoop/hadoop-0.20.2-cdh3u3_mesos.patch PRE-CREATION
hadoop/hadoop-0.20.205.0.patch c113001
hadoop/hadoop-0.20.2-cdh3u3.patch PRE-CREATION
hadoop/TUTORIAL.sh c122387
hadoop/TUTORIAL 36f5fb0
hadoop/Makefile.am PRE-CREATION
README 0d58255
configure.ac f1e2c3a
ec2/Makefile.am PRE-CREATION
Makefile.am 99a21d7
hadoop/mesos/src/java/org/apache/hadoop/mapred/HadoopFrameworkMessage.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosTaskTrackerInstrumentation.java PRE-CREATION
Diff: https://reviews.apache.org/r/4922/diff
Testing
-------
$ pwd
/path/to/mesos/build
$ make hadoop-0.20.205.0
$ make hadoop-0.20.2-cdh3u3
Thanks,
Benjamin
> Patch for CDH3u3 for mesos 0.9.0
> --------------------------------
>
> Key: MESOS-182
> URL: https://issues.apache.org/jira/browse/MESOS-182
> Project: Mesos
> Issue Type: Improvement
> Components: build
> Affects Versions: 0.9.0
> Reporter: Hiral Patel
> Priority: Trivial
> Labels: patch
> Attachments: 0001-Manually-patched-using-0.20.250-patch-for-mesos-plus.patch
>
>
> I've attached a patch for cdh3u3 which uses the patch for 0.20.205 as a bases plus adds protobuff jar and a mesos-0.9.0.jar compiled from RC3. I tested it on my local machine with wordcount to validate the tasks run on mesos.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-182) Patch for CDH3u3 for mesos 0.9.0
Posted by "Benjamin Hindman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265505#comment-13265505 ]
Benjamin Hindman commented on MESOS-182:
----------------------------------------
Thanks so much for the patch Hiral!
> Patch for CDH3u3 for mesos 0.9.0
> --------------------------------
>
> Key: MESOS-182
> URL: https://issues.apache.org/jira/browse/MESOS-182
> Project: Mesos
> Issue Type: Improvement
> Components: build
> Affects Versions: 0.9.0
> Reporter: Hiral Patel
> Priority: Trivial
> Labels: patch
> Attachments: 0001-Manually-patched-using-0.20.250-patch-for-mesos-plus.patch
>
>
> I've attached a patch for cdh3u3 which uses the patch for 0.20.205 as a bases plus adds protobuff jar and a mesos-0.9.0.jar compiled from RC3. I tested it on my local machine with wordcount to validate the tasks run on mesos.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-182) Patch for CDH3u3 for mesos 0.9.0
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265504#comment-13265504 ]
jiraposter@reviews.apache.org commented on MESOS-182:
-----------------------------------------------------
bq. On 2012-04-28 21:21:44, Charles Reiss wrote:
bq. > Looks okay to me. If you want to avoid the library finding, you should be able to do ${MESOS_BUILD_DIR}/libtool --mode=install cp ${MESOS_BUILD_DIR}/src/libmesos.la lib/native/${PLATFORM}, but I'm not convinced this is better.
Yeah, I'll leave it for now.
bq. On 2012-04-28 21:21:44, Charles Reiss wrote:
bq. > hadoop/Makefile.am, line 47
bq. > <https://reviews.apache.org/r/4922/diff/2/?file=105208#file105208line47>
bq. >
bq. > Why the dependency on 'all'? (and elsewhere)
This is an artifact of being copied from the top-level Makefile.am. Removed, good catch.
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4922/#review7341
-----------------------------------------------------------
On 2012-04-28 01:28:09, Benjamin Hindman wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4922/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-28 01:28:09)
bq.
bq.
bq. Review request for mesos, Charles Reiss and Matei Zaharia.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. I wanted to break down the changes to Hadoop so that someone could see exactly how minimal they really are. In addition, I wanted the patches to the Hadoop source code itself to be clearly identifiable. So, I broke the mega patch up. Now, we apply the small Hadoop patch (which has been submitted to Hadoop trunk), COPY the Mesos contrib code over, and then apply two other small patches for hadoop-env.sh and mapred-site.xml.
bq.
bq. In addition, we also copy the native library into lib as well as the mesos.jar and protobuf.jar so that we don't need to point to any build directory. Likewise, we don't even need to set MESOS_NATIVE_LIBRARY at all! The resulting patch for hadoop-env.sh is very small indeed.
bq.
bq. I also deleted the hadoop.tar.gz that we included because I figured it would be just as easy to download it via the tutorial (once it's downloaded once, the tutorial just keeps using that, only deleting it if you do a 'make clean').
bq.
bq. The final directory you get after running 'make hadoop-0.20.205.0' can be zipped, copied, and used right away in a Mesos cluster.
bq.
bq. I also used the patch from https://issues.apache.org/jira/browse/MESOS-182 to do the same thing for cdh3u3. You get this via 'make hadoop-0.20.2-cdh3u3'.
bq.
bq.
bq. This addresses bug MESOS-182.
bq. https://issues.apache.org/jira/browse/MESOS-182
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. Makefile.am 99a21d7
bq. README 0d58255
bq. configure.ac f1e2c3a
bq. ec2/Makefile.am PRE-CREATION
bq. hadoop/Makefile.am PRE-CREATION
bq. hadoop/TUTORIAL 36f5fb0
bq. hadoop/TUTORIAL.sh c122387
bq. hadoop/hadoop-0.20.2-cdh3u3.patch PRE-CREATION
bq. hadoop/hadoop-0.20.2-cdh3u3_hadoop-env.sh.patch PRE-CREATION
bq. hadoop/hadoop-0.20.2-cdh3u3_mesos.patch PRE-CREATION
bq. hadoop/hadoop-0.20.205.0.patch c113001
bq. hadoop/hadoop-0.20.205.0.tar.gz bc605a3
bq. hadoop/hadoop-0.20.205.0_conf_hadoop-env.sh.patch 6733ef5
bq. hadoop/hadoop-0.20.205.0_conf_mapred-site.xml.patch 4764496
bq. hadoop/hadoop-0.20.205.0_hadoop-env.sh.patch PRE-CREATION
bq. hadoop/hadoop-0.20.205.0_mesos.patch PRE-CREATION
bq. hadoop/mapred-site.xml.patch PRE-CREATION
bq. hadoop/mesos-executor PRE-CREATION
bq. hadoop/mesos/build.xml PRE-CREATION
bq. hadoop/mesos/ivy.xml PRE-CREATION
bq. hadoop/mesos/ivy/libraries.properties PRE-CREATION
bq. hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkExecutor.java PRE-CREATION
bq. hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkScheduler.java PRE-CREATION
bq. hadoop/mesos/src/java/org/apache/hadoop/mapred/HadoopFrameworkMessage.java PRE-CREATION
bq. hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java PRE-CREATION
bq. hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosTaskTrackerInstrumentation.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/4922/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. $ pwd
bq. /path/to/mesos/build/hadoop
bq. $ make hadoop-0.20.205.0
bq. $ make hadoop-0.20.2-cdh3u3
bq.
bq.
bq. Thanks,
bq.
bq. Benjamin
bq.
bq.
> Patch for CDH3u3 for mesos 0.9.0
> --------------------------------
>
> Key: MESOS-182
> URL: https://issues.apache.org/jira/browse/MESOS-182
> Project: Mesos
> Issue Type: Improvement
> Components: build
> Affects Versions: 0.9.0
> Reporter: Hiral Patel
> Priority: Trivial
> Labels: patch
> Attachments: 0001-Manually-patched-using-0.20.250-patch-for-mesos-plus.patch
>
>
> I've attached a patch for cdh3u3 which uses the patch for 0.20.205 as a bases plus adds protobuff jar and a mesos-0.9.0.jar compiled from RC3. I tested it on my local machine with wordcount to validate the tasks run on mesos.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-182) Patch for CDH3u3 for mesos 0.9.0
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264189#comment-13264189 ]
jiraposter@reviews.apache.org commented on MESOS-182:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4922/
-----------------------------------------------------------
(Updated 2012-04-28 01:27:41.349376)
Review request for mesos, Charles Reiss and Matei Zaharia.
Changes
-------
Added Charles as a reviewer since he had reviewed this stuff for me before.
Summary
-------
I wanted to break down the changes to Hadoop so that someone could see exactly how minimal they really are. In addition, I wanted the patches to the Hadoop source code itself to be clearly identifiable. So, I broke the mega patch up. Now, we apply the small Hadoop patch (which has been submitted to Hadoop trunk), COPY the Mesos contrib code over, and then apply two other small patches for hadoop-env.sh and mapred-site.xml.
In addition, we also copy the native library into lib as well as the mesos.jar and protobuf.jar so that we don't need to point to any build directory. Likewise, we don't even need to set MESOS_NATIVE_LIBRARY at all! The resulting patch for hadoop-env.sh is very small indeed.
I also deleted the hadoop.tar.gz that we included because I figured it would be just as easy to download it via the tutorial (once it's downloaded once, the tutorial just keeps using that, only deleting it if you do a 'make clean').
The final directory you get after running 'make hadoop-0.20.205.0' can be zipped, copied, and used right away in a Mesos cluster.
I also used the patch from https://issues.apache.org/jira/browse/MESOS-182 to do the same thing for cdh3u3. You get this via 'make hadoop-0.20.2-cdh3u3'.
This addresses bug MESOS-182.
https://issues.apache.org/jira/browse/MESOS-182
Diffs
-----
Makefile.am 99a21d7
README 0d58255
configure.ac f1e2c3a
ec2/Makefile.am PRE-CREATION
hadoop/Makefile.am PRE-CREATION
hadoop/TUTORIAL 36f5fb0
hadoop/TUTORIAL.sh c122387
hadoop/hadoop-0.20.2-cdh3u3.patch PRE-CREATION
hadoop/hadoop-0.20.2-cdh3u3_hadoop-env.sh.patch PRE-CREATION
hadoop/hadoop-0.20.2-cdh3u3_mesos.patch PRE-CREATION
hadoop/hadoop-0.20.205.0.patch c113001
hadoop/hadoop-0.20.205.0.tar.gz bc605a3
hadoop/hadoop-0.20.205.0_conf_hadoop-env.sh.patch 6733ef5
hadoop/hadoop-0.20.205.0_conf_mapred-site.xml.patch 4764496
hadoop/hadoop-0.20.205.0_hadoop-env.sh.patch PRE-CREATION
hadoop/hadoop-0.20.205.0_mesos.patch PRE-CREATION
hadoop/mapred-site.xml.patch PRE-CREATION
hadoop/mesos-executor PRE-CREATION
hadoop/mesos/build.xml PRE-CREATION
hadoop/mesos/ivy.xml PRE-CREATION
hadoop/mesos/ivy/libraries.properties PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkExecutor.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkScheduler.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/HadoopFrameworkMessage.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosTaskTrackerInstrumentation.java PRE-CREATION
Diff: https://reviews.apache.org/r/4922/diff
Testing
-------
$ pwd
/path/to/mesos/build
$ make hadoop-0.20.205.0
$ make hadoop-0.20.2-cdh3u3
Thanks,
Benjamin
> Patch for CDH3u3 for mesos 0.9.0
> --------------------------------
>
> Key: MESOS-182
> URL: https://issues.apache.org/jira/browse/MESOS-182
> Project: Mesos
> Issue Type: Improvement
> Components: build
> Affects Versions: 0.9.0
> Reporter: Hiral Patel
> Priority: Trivial
> Labels: patch
> Attachments: 0001-Manually-patched-using-0.20.250-patch-for-mesos-plus.patch
>
>
> I've attached a patch for cdh3u3 which uses the patch for 0.20.205 as a bases plus adds protobuff jar and a mesos-0.9.0.jar compiled from RC3. I tested it on my local machine with wordcount to validate the tasks run on mesos.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-182) Patch for CDH3u3 for mesos 0.9.0
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264190#comment-13264190 ]
jiraposter@reviews.apache.org commented on MESOS-182:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4922/
-----------------------------------------------------------
(Updated 2012-04-28 01:28:09.177037)
Review request for mesos, Charles Reiss and Matei Zaharia.
Changes
-------
Fixed the 'Testing Done:' section.
Summary
-------
I wanted to break down the changes to Hadoop so that someone could see exactly how minimal they really are. In addition, I wanted the patches to the Hadoop source code itself to be clearly identifiable. So, I broke the mega patch up. Now, we apply the small Hadoop patch (which has been submitted to Hadoop trunk), COPY the Mesos contrib code over, and then apply two other small patches for hadoop-env.sh and mapred-site.xml.
In addition, we also copy the native library into lib as well as the mesos.jar and protobuf.jar so that we don't need to point to any build directory. Likewise, we don't even need to set MESOS_NATIVE_LIBRARY at all! The resulting patch for hadoop-env.sh is very small indeed.
I also deleted the hadoop.tar.gz that we included because I figured it would be just as easy to download it via the tutorial (once it's downloaded once, the tutorial just keeps using that, only deleting it if you do a 'make clean').
The final directory you get after running 'make hadoop-0.20.205.0' can be zipped, copied, and used right away in a Mesos cluster.
I also used the patch from https://issues.apache.org/jira/browse/MESOS-182 to do the same thing for cdh3u3. You get this via 'make hadoop-0.20.2-cdh3u3'.
This addresses bug MESOS-182.
https://issues.apache.org/jira/browse/MESOS-182
Diffs
-----
Makefile.am 99a21d7
README 0d58255
configure.ac f1e2c3a
ec2/Makefile.am PRE-CREATION
hadoop/Makefile.am PRE-CREATION
hadoop/TUTORIAL 36f5fb0
hadoop/TUTORIAL.sh c122387
hadoop/hadoop-0.20.2-cdh3u3.patch PRE-CREATION
hadoop/hadoop-0.20.2-cdh3u3_hadoop-env.sh.patch PRE-CREATION
hadoop/hadoop-0.20.2-cdh3u3_mesos.patch PRE-CREATION
hadoop/hadoop-0.20.205.0.patch c113001
hadoop/hadoop-0.20.205.0.tar.gz bc605a3
hadoop/hadoop-0.20.205.0_conf_hadoop-env.sh.patch 6733ef5
hadoop/hadoop-0.20.205.0_conf_mapred-site.xml.patch 4764496
hadoop/hadoop-0.20.205.0_hadoop-env.sh.patch PRE-CREATION
hadoop/hadoop-0.20.205.0_mesos.patch PRE-CREATION
hadoop/mapred-site.xml.patch PRE-CREATION
hadoop/mesos-executor PRE-CREATION
hadoop/mesos/build.xml PRE-CREATION
hadoop/mesos/ivy.xml PRE-CREATION
hadoop/mesos/ivy/libraries.properties PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkExecutor.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkScheduler.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/HadoopFrameworkMessage.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosTaskTrackerInstrumentation.java PRE-CREATION
Diff: https://reviews.apache.org/r/4922/diff
Testing (updated)
-------
$ pwd
/path/to/mesos/build/hadoop
$ make hadoop-0.20.205.0
$ make hadoop-0.20.2-cdh3u3
Thanks,
Benjamin
> Patch for CDH3u3 for mesos 0.9.0
> --------------------------------
>
> Key: MESOS-182
> URL: https://issues.apache.org/jira/browse/MESOS-182
> Project: Mesos
> Issue Type: Improvement
> Components: build
> Affects Versions: 0.9.0
> Reporter: Hiral Patel
> Priority: Trivial
> Labels: patch
> Attachments: 0001-Manually-patched-using-0.20.250-patch-for-mesos-plus.patch
>
>
> I've attached a patch for cdh3u3 which uses the patch for 0.20.205 as a bases plus adds protobuff jar and a mesos-0.9.0.jar compiled from RC3. I tested it on my local machine with wordcount to validate the tasks run on mesos.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-182) Patch for CDH3u3 for mesos 0.9.0
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264188#comment-13264188 ]
jiraposter@reviews.apache.org commented on MESOS-182:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4922/
-----------------------------------------------------------
(Updated 2012-04-28 01:27:14.227261)
Review request for mesos and Matei Zaharia.
Changes
-------
Fixed 'make dist'.
Summary
-------
I wanted to break down the changes to Hadoop so that someone could see exactly how minimal they really are. In addition, I wanted the patches to the Hadoop source code itself to be clearly identifiable. So, I broke the mega patch up. Now, we apply the small Hadoop patch (which has been submitted to Hadoop trunk), COPY the Mesos contrib code over, and then apply two other small patches for hadoop-env.sh and mapred-site.xml.
In addition, we also copy the native library into lib as well as the mesos.jar and protobuf.jar so that we don't need to point to any build directory. Likewise, we don't even need to set MESOS_NATIVE_LIBRARY at all! The resulting patch for hadoop-env.sh is very small indeed.
I also deleted the hadoop.tar.gz that we included because I figured it would be just as easy to download it via the tutorial (once it's downloaded once, the tutorial just keeps using that, only deleting it if you do a 'make clean').
The final directory you get after running 'make hadoop-0.20.205.0' can be zipped, copied, and used right away in a Mesos cluster.
I also used the patch from https://issues.apache.org/jira/browse/MESOS-182 to do the same thing for cdh3u3. You get this via 'make hadoop-0.20.2-cdh3u3'.
This addresses bug MESOS-182.
https://issues.apache.org/jira/browse/MESOS-182
Diffs (updated)
-----
Makefile.am 99a21d7
README 0d58255
configure.ac f1e2c3a
ec2/Makefile.am PRE-CREATION
hadoop/Makefile.am PRE-CREATION
hadoop/TUTORIAL 36f5fb0
hadoop/TUTORIAL.sh c122387
hadoop/hadoop-0.20.2-cdh3u3.patch PRE-CREATION
hadoop/hadoop-0.20.2-cdh3u3_hadoop-env.sh.patch PRE-CREATION
hadoop/hadoop-0.20.2-cdh3u3_mesos.patch PRE-CREATION
hadoop/hadoop-0.20.205.0.patch c113001
hadoop/hadoop-0.20.205.0.tar.gz bc605a3
hadoop/hadoop-0.20.205.0_conf_hadoop-env.sh.patch 6733ef5
hadoop/hadoop-0.20.205.0_conf_mapred-site.xml.patch 4764496
hadoop/hadoop-0.20.205.0_hadoop-env.sh.patch PRE-CREATION
hadoop/hadoop-0.20.205.0_mesos.patch PRE-CREATION
hadoop/mapred-site.xml.patch PRE-CREATION
hadoop/mesos-executor PRE-CREATION
hadoop/mesos/build.xml PRE-CREATION
hadoop/mesos/ivy.xml PRE-CREATION
hadoop/mesos/ivy/libraries.properties PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkExecutor.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkScheduler.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/HadoopFrameworkMessage.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java PRE-CREATION
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosTaskTrackerInstrumentation.java PRE-CREATION
Diff: https://reviews.apache.org/r/4922/diff
Testing
-------
$ pwd
/path/to/mesos/build
$ make hadoop-0.20.205.0
$ make hadoop-0.20.2-cdh3u3
Thanks,
Benjamin
> Patch for CDH3u3 for mesos 0.9.0
> --------------------------------
>
> Key: MESOS-182
> URL: https://issues.apache.org/jira/browse/MESOS-182
> Project: Mesos
> Issue Type: Improvement
> Components: build
> Affects Versions: 0.9.0
> Reporter: Hiral Patel
> Priority: Trivial
> Labels: patch
> Attachments: 0001-Manually-patched-using-0.20.250-patch-for-mesos-plus.patch
>
>
> I've attached a patch for cdh3u3 which uses the patch for 0.20.205 as a bases plus adds protobuff jar and a mesos-0.9.0.jar compiled from RC3. I tested it on my local machine with wordcount to validate the tasks run on mesos.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (MESOS-182) Patch for CDH3u3 for mesos 0.9.0
Posted by "Benjamin Hindman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Hindman closed MESOS-182.
----------------------------------
> Patch for CDH3u3 for mesos 0.9.0
> --------------------------------
>
> Key: MESOS-182
> URL: https://issues.apache.org/jira/browse/MESOS-182
> Project: Mesos
> Issue Type: Improvement
> Components: build
> Affects Versions: 0.9.0
> Reporter: Hiral Patel
> Assignee: Benjamin Hindman
> Priority: Trivial
> Labels: patch
> Attachments: 0001-Manually-patched-using-0.20.250-patch-for-mesos-plus.patch
>
>
> I've attached a patch for cdh3u3 which uses the patch for 0.20.205 as a bases plus adds protobuff jar and a mesos-0.9.0.jar compiled from RC3. I tested it on my local machine with wordcount to validate the tasks run on mesos.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-182) Patch for CDH3u3 for mesos 0.9.0
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264403#comment-13264403 ]
jiraposter@reviews.apache.org commented on MESOS-182:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4922/#review7341
-----------------------------------------------------------
Ship it!
Looks okay to me. If you want to avoid the library finding, you should be able to do ${MESOS_BUILD_DIR}/libtool --mode=install cp ${MESOS_BUILD_DIR}/src/libmesos.la lib/native/${PLATFORM}, but I'm not convinced this is better.
hadoop/Makefile.am
<https://reviews.apache.org/r/4922/#comment16220>
Why the dependency on 'all'? (and elsewhere)
- Charles
On 2012-04-28 01:28:09, Benjamin Hindman wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4922/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-28 01:28:09)
bq.
bq.
bq. Review request for mesos, Charles Reiss and Matei Zaharia.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. I wanted to break down the changes to Hadoop so that someone could see exactly how minimal they really are. In addition, I wanted the patches to the Hadoop source code itself to be clearly identifiable. So, I broke the mega patch up. Now, we apply the small Hadoop patch (which has been submitted to Hadoop trunk), COPY the Mesos contrib code over, and then apply two other small patches for hadoop-env.sh and mapred-site.xml.
bq.
bq. In addition, we also copy the native library into lib as well as the mesos.jar and protobuf.jar so that we don't need to point to any build directory. Likewise, we don't even need to set MESOS_NATIVE_LIBRARY at all! The resulting patch for hadoop-env.sh is very small indeed.
bq.
bq. I also deleted the hadoop.tar.gz that we included because I figured it would be just as easy to download it via the tutorial (once it's downloaded once, the tutorial just keeps using that, only deleting it if you do a 'make clean').
bq.
bq. The final directory you get after running 'make hadoop-0.20.205.0' can be zipped, copied, and used right away in a Mesos cluster.
bq.
bq. I also used the patch from https://issues.apache.org/jira/browse/MESOS-182 to do the same thing for cdh3u3. You get this via 'make hadoop-0.20.2-cdh3u3'.
bq.
bq.
bq. This addresses bug MESOS-182.
bq. https://issues.apache.org/jira/browse/MESOS-182
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. Makefile.am 99a21d7
bq. README 0d58255
bq. configure.ac f1e2c3a
bq. ec2/Makefile.am PRE-CREATION
bq. hadoop/Makefile.am PRE-CREATION
bq. hadoop/TUTORIAL 36f5fb0
bq. hadoop/TUTORIAL.sh c122387
bq. hadoop/hadoop-0.20.2-cdh3u3.patch PRE-CREATION
bq. hadoop/hadoop-0.20.2-cdh3u3_hadoop-env.sh.patch PRE-CREATION
bq. hadoop/hadoop-0.20.2-cdh3u3_mesos.patch PRE-CREATION
bq. hadoop/hadoop-0.20.205.0.patch c113001
bq. hadoop/hadoop-0.20.205.0.tar.gz bc605a3
bq. hadoop/hadoop-0.20.205.0_conf_hadoop-env.sh.patch 6733ef5
bq. hadoop/hadoop-0.20.205.0_conf_mapred-site.xml.patch 4764496
bq. hadoop/hadoop-0.20.205.0_hadoop-env.sh.patch PRE-CREATION
bq. hadoop/hadoop-0.20.205.0_mesos.patch PRE-CREATION
bq. hadoop/mapred-site.xml.patch PRE-CREATION
bq. hadoop/mesos-executor PRE-CREATION
bq. hadoop/mesos/build.xml PRE-CREATION
bq. hadoop/mesos/ivy.xml PRE-CREATION
bq. hadoop/mesos/ivy/libraries.properties PRE-CREATION
bq. hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkExecutor.java PRE-CREATION
bq. hadoop/mesos/src/java/org/apache/hadoop/mapred/FrameworkScheduler.java PRE-CREATION
bq. hadoop/mesos/src/java/org/apache/hadoop/mapred/HadoopFrameworkMessage.java PRE-CREATION
bq. hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java PRE-CREATION
bq. hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosTaskTrackerInstrumentation.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/4922/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. $ pwd
bq. /path/to/mesos/build/hadoop
bq. $ make hadoop-0.20.205.0
bq. $ make hadoop-0.20.2-cdh3u3
bq.
bq.
bq. Thanks,
bq.
bq. Benjamin
bq.
bq.
> Patch for CDH3u3 for mesos 0.9.0
> --------------------------------
>
> Key: MESOS-182
> URL: https://issues.apache.org/jira/browse/MESOS-182
> Project: Mesos
> Issue Type: Improvement
> Components: build
> Affects Versions: 0.9.0
> Reporter: Hiral Patel
> Priority: Trivial
> Labels: patch
> Attachments: 0001-Manually-patched-using-0.20.250-patch-for-mesos-plus.patch
>
>
> I've attached a patch for cdh3u3 which uses the patch for 0.20.205 as a bases plus adds protobuff jar and a mesos-0.9.0.jar compiled from RC3. I tested it on my local machine with wordcount to validate the tasks run on mesos.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MESOS-182) Patch for CDH3u3 for mesos 0.9.0
Posted by "Benjamin Hindman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Hindman resolved MESOS-182.
------------------------------------
Resolution: Fixed
Assignee: Benjamin Hindman
> Patch for CDH3u3 for mesos 0.9.0
> --------------------------------
>
> Key: MESOS-182
> URL: https://issues.apache.org/jira/browse/MESOS-182
> Project: Mesos
> Issue Type: Improvement
> Components: build
> Affects Versions: 0.9.0
> Reporter: Hiral Patel
> Assignee: Benjamin Hindman
> Priority: Trivial
> Labels: patch
> Attachments: 0001-Manually-patched-using-0.20.250-patch-for-mesos-plus.patch
>
>
> I've attached a patch for cdh3u3 which uses the patch for 0.20.205 as a bases plus adds protobuff jar and a mesos-0.9.0.jar compiled from RC3. I tested it on my local machine with wordcount to validate the tasks run on mesos.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira