You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2012/05/01 01:55:50 UTC
[jira] [Created] (BIGTOP-570) Build package for Apache Giraph
Andrew Purtell created BIGTOP-570:
-------------------------------------
Summary: Build package for Apache Giraph
Key: BIGTOP-570
URL: https://issues.apache.org/jira/browse/BIGTOP-570
Project: Bigtop
Issue Type: New Feature
Reporter: Andrew Purtell
Priority: Minor
Attachments: 0001-Giraph-package-WIP.patch.gz
We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated BIGTOP-570:
----------------------------------
Attachment: 0001-Giraph-package-WIP.patch.gz
Got it.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz, 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280464#comment-13280464 ]
Andrew Purtell commented on BIGTOP-570:
---------------------------------------
Making progress.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz, 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288103#comment-13288103 ]
Andrew Purtell commented on BIGTOP-570:
---------------------------------------
GIRAPH-37 may make GIRAPH-198 moot. Suggest updating the SHA of the source tarball pull for latest Giraph trunk.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: BIGTOP-570-debian.patch.txt, BIGTOP-570.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287534#comment-13287534 ]
Roman Shaposhnik commented on BIGTOP-570:
-----------------------------------------
Andrew, the latest patches seem to be ok, but we're now blocked on the following 2 Giraph issues: GIRAPH-198 and GIRAPH-199
I can personally provide patches for GIRAPH-199 but I would appreciate if you could help with rounding up folks who can take a look at GIRAPH-198
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: BIGTOP-570-debian.patch.txt, BIGTOP-570.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272640#comment-13272640 ]
Andrew Purtell commented on BIGTOP-570:
---------------------------------------
@Roman, ok I'll commit to getting that to you early next week.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272629#comment-13272629 ]
Roman Shaposhnik commented on BIGTOP-570:
-----------------------------------------
Andrew, I have a first cut at Debian packaging. If you can help us with creating a smoke test for Giraph -- it'll be all we need.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (BIGTOP-570) Build package for Apache Giraph
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik resolved BIGTOP-570.
-------------------------------------
Resolution: Fixed
Fix Version/s: 0.4.0
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Fix For: 0.4.0
>
> Attachments: 0001-Giraph-package-WIP.patch.gz, BIGTOP-570-debian.patch.txt, BIGTOP-570.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Peter Linnell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265662#comment-13265662 ]
Peter Linnell commented on BIGTOP-570:
--------------------------------------
I had a look at the rpm side and it looks ready to commit. Thanks for the patch!
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated BIGTOP-570:
----------------------------------
Attachment: 0001-Giraph-package-WIP.patch.gz
Attached is an updated patch that includes a Giraph smoke test, cribbed from a unit test. I wish I could report more progress. It took too much time just to figure out Maven was pulling in dependencies to some defunct java.net repo from ZooKeeper. Now I can't seem to get Maven to select the correct Hadoop artifact ("java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapre\
duce.JobContext, but class was expected
"). Perhaps this is a familiar problem?
Unfortunately just executing 'hadoop jar ...' using iTest Shell is not an option because Giraph requires execution of more than one task in parallel, something not supported by LocalTaskRunner.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz, 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated BIGTOP-570:
----------------------------------
Attachment: 0001-Giraph-package-WIP.patch.gz
Thanks Roman, attached is the latest that incorporates your patch plus changes the smoke test to a very simple Groovy script that assumes a live test cluster, much smaller/simpler, thanks for the guidance there.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz, 0001-Giraph-package-WIP.patch.gz, 0001-Giraph-package-WIP.patch.gz, BIGTOP-570-debian.patch.txt
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13293170#comment-13293170 ]
Roman Shaposhnik commented on BIGTOP-570:
-----------------------------------------
+1 and committed.
Once again -- thanks a million for the patch!
One last request -- I really would like to have Giraph 0.2.0-incubating in Bigtop 0.4.0 distribution. In order to do that we have to have it be released. Any chance you can help
influencing Giraph community to have that release within next couple of weeks? Obviously it shouldn't be rushed, but if the schedule allows it can come very handy
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Fix For: 0.4.0
>
> Attachments: 0001-Giraph-package-WIP.patch.gz, BIGTOP-570-debian.patch.txt, BIGTOP-570.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated BIGTOP-570:
----------------------------------
Attachment: 0001-Giraph-package-WIP.patch.gz
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269076#comment-13269076 ]
Patrick Hunt commented on BIGTOP-570:
-------------------------------------
Giraph is about to graduate to TLP status, would be great to see this available in the next release!
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291845#comment-13291845 ]
Andrew Purtell commented on BIGTOP-570:
---------------------------------------
I see on GIRAPH-198 we are missing a property definition to use the new Netty RPC. Will update the patch momentarily.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: BIGTOP-570-debian.patch.txt, BIGTOP-570.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280496#comment-13280496 ]
Andrew Purtell commented on BIGTOP-570:
---------------------------------------
I had to touch a bunch of POMs, if there might be a better way, please advise.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz, 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288678#comment-13288678 ]
Roman Shaposhnik commented on BIGTOP-570:
-----------------------------------------
Andrew, we're already at the tip of trunk with your patch https://github.com/apache/giraph/commit/79962a3c8c5b9a29aa346e66959ad6b5ddd79022 and according to CHANGELOG it does include GIRAPH-37
Any other suggestions?
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: BIGTOP-570-debian.patch.txt, BIGTOP-570.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (BIGTOP-570) Build package for Apache Giraph
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik reassigned BIGTOP-570:
---------------------------------------
Assignee: Roman Shaposhnik
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265657#comment-13265657 ]
Roman Shaposhnik commented on BIGTOP-570:
-----------------------------------------
Excellent news! Thanks a million for the patch, Andrew! Giraph is a keeper. Now, given that I'm a Debian geek, I'll take a look at what it takes to add deb part to your patch. I'm sure James Page would also be interested ;-)
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (BIGTOP-570) Build package for Apache
Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280441#comment-13280441 ]
Andrew Purtell edited comment on BIGTOP-570 at 5/21/12 8:20 PM:
----------------------------------------------------------------
Attached is an updated patch that includes a Giraph smoke test, cribbed from a unit test. I wish I could report more progress. It took too much time just to figure out Maven was pulling in dependencies to some defunct java.net repo from ZooKeeper. Now I can't seem to get Maven to select the correct Hadoop artifact ("java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected"). Perhaps this is a familiar problem?
Unfortunately just executing 'hadoop jar ...' using iTest Shell is not an option because Giraph requires execution of more than one task in parallel, something not supported by LocalTaskRunner.
Edit: Fixed formatting.
was (Author: apurtell):
Attached is an updated patch that includes a Giraph smoke test, cribbed from a unit test. I wish I could report more progress. It took too much time just to figure out Maven was pulling in dependencies to some defunct java.net repo from ZooKeeper. Now I can't seem to get Maven to select the correct Hadoop artifact ("java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapre\
duce.JobContext, but class was expected
"). Perhaps this is a familiar problem?
Unfortunately just executing 'hadoop jar ...' using iTest Shell is not an option because Giraph requires execution of more than one task in parallel, something not supported by LocalTaskRunner.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz, 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated BIGTOP-570:
----------------------------------
Attachment: 0001-Giraph-package-WIP.patch.gz
Attached updated package WIP.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz, BIGTOP-570-debian.patch.txt, BIGTOP-570.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated BIGTOP-570:
----------------------------------
Attachment: (was: 0001-Giraph-package-WIP.patch.gz)
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: BIGTOP-570-debian.patch.txt
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated BIGTOP-570:
----------------------------------
Attachment: BIGTOP-570.patch.gz
Updated patch.
- Giraph do-component-build should -Dhadoop.version=2.0.0-alpha
- Fix file not found issue with config dir in RPM specfile.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: BIGTOP-570-debian.patch.txt, BIGTOP-570.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-570) Build package for Apache Giraph
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik updated BIGTOP-570:
------------------------------------
Attachment: BIGTOP-570-debian.patch.txt
Andrew, I'm attaching a debian side of things plus some trivial modifications to your RPM code.
Please update your test code to run against a real cluster and I think I can then submit this to our testing machinery to see how it fares.
Once again -- thanks a million for helping us add this useful project to Bigtop!
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz, 0001-Giraph-package-WIP.patch.gz, BIGTOP-570-debian.patch.txt
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-570) Build package for Apache Giraph
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated BIGTOP-570:
----------------------------------
Attachment: (was: 0001-Giraph-package-WIP.patch.gz)
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-570) Build package for Apache Giraph
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281841#comment-13281841 ]
Roman Shaposhnik commented on BIGTOP-570:
-----------------------------------------
Looking at it. Apologies for the complexity of POM logic -- we're actually working on a Groovy solution to simplify it.
> Build package for Apache Giraph
> -------------------------------
>
> Key: BIGTOP-570
> URL: https://issues.apache.org/jira/browse/BIGTOP-570
> Project: Bigtop
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Roman Shaposhnik
> Priority: Minor
> Attachments: 0001-Giraph-package-WIP.patch.gz, 0001-Giraph-package-WIP.patch.gz
>
>
> We are basing our next internal Hadoop distribution on Bigtop and want to make Apache Giraph available to our users as part of the platform. Apache Giraph is a promising platform for running large-scale graph algorithms on top of existing Hadoop infrastructure, currently in incubation. For more information on the project, please see: http://incubator.apache.org/giraph/ If this is of interest to anyone, I've ported the RPM specfile and common build scripts to both Bigtop and Giraph trunks. Unfortunately I don't know how to package for Debian.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira