You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Lars George (JIRA)" <ji...@apache.org> on 2011/01/04 09:29:45 UTC
[jira] Created: (WHIRR-191) [CDH] Start other services based on
CDH, not just HDFS and MR
[CDH] Start other services based on CDH, not just HDFS and MR
-------------------------------------------------------------
Key: WHIRR-191
URL: https://issues.apache.org/jira/browse/WHIRR-191
Project: Whirr
Issue Type: New Feature
Components: service/hadoop
Affects Versions: 0.3.0
Reporter: Lars George
Fix For: 0.3.0
When using the CDH service it currently only starts HDFS and MR, similar to the Apache Hadoop service (the one that uses the tarballs). We do have for example also an Apache ZooKeeper or Apache HBase service, but for CDH we do not. Either add (and change the current service) a CDH Hadoop, CDH HBase and so on or come up with another means of starting up other processes apart from HDFS and MR based on the CDH release.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Updated] (WHIRR-191) [CDH] Start other services based on
CDH, not just HDFS and MR
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-191:
----------------------------
Fix Version/s: 0.5.0
> [CDH] Start other services based on CDH, not just HDFS and MR
> -------------------------------------------------------------
>
> Key: WHIRR-191
> URL: https://issues.apache.org/jira/browse/WHIRR-191
> Project: Whirr
> Issue Type: New Feature
> Components: service/hadoop
> Affects Versions: 0.3.0
> Reporter: Lars George
> Fix For: 0.5.0
>
> Attachments: WHIRR-191.patch
>
>
> When using the CDH service it currently only starts HDFS and MR, similar to the Apache Hadoop service (the one that uses the tarballs). We do have for example also an Apache ZooKeeper or Apache HBase service, but for CDH we do not. Either add (and change the current service) a CDH Hadoop, CDH HBase and so on or come up with another means of starting up other processes apart from HDFS and MR based on the CDH release.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-191) [CDH] Start other services based on
CDH, not just HDFS and MR
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13029610#comment-13029610 ]
Lars George commented on WHIRR-191:
-----------------------------------
I looked through the patch, looks good to me. If the tests pass then +1 from me.
> [CDH] Start other services based on CDH, not just HDFS and MR
> -------------------------------------------------------------
>
> Key: WHIRR-191
> URL: https://issues.apache.org/jira/browse/WHIRR-191
> Project: Whirr
> Issue Type: New Feature
> Components: service/hadoop
> Affects Versions: 0.3.0
> Reporter: Lars George
> Fix For: 0.5.0
>
> Attachments: WHIRR-191.patch, WHIRR-191.patch, WHIRR-191.patch
>
>
> When using the CDH service it currently only starts HDFS and MR, similar to the Apache Hadoop service (the one that uses the tarballs). We do have for example also an Apache ZooKeeper or Apache HBase service, but for CDH we do not. Either add (and change the current service) a CDH Hadoop, CDH HBase and so on or come up with another means of starting up other processes apart from HDFS and MR based on the CDH release.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-191) [CDH] Start other services based on
CDH, not just HDFS and MR
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-191:
----------------------------
Attachment: WHIRR-191.patch
This is now working (the problem was with passing nulls to get the scanner). I've tested all the Hadoop, HBase, ZooKeeper services and they pass on EC2 so this should be ready to go.
> [CDH] Start other services based on CDH, not just HDFS and MR
> -------------------------------------------------------------
>
> Key: WHIRR-191
> URL: https://issues.apache.org/jira/browse/WHIRR-191
> Project: Whirr
> Issue Type: New Feature
> Components: service/hadoop
> Affects Versions: 0.3.0
> Reporter: Lars George
> Fix For: 0.5.0
>
> Attachments: WHIRR-191.patch, WHIRR-191.patch, WHIRR-191.patch
>
>
> When using the CDH service it currently only starts HDFS and MR, similar to the Apache Hadoop service (the one that uses the tarballs). We do have for example also an Apache ZooKeeper or Apache HBase service, but for CDH we do not. Either add (and change the current service) a CDH Hadoop, CDH HBase and so on or come up with another means of starting up other processes apart from HDFS and MR based on the CDH release.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (WHIRR-191) [CDH] Start other services based on
CDH, not just HDFS and MR
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977384#action_12977384 ]
Tom White commented on WHIRR-191:
---------------------------------
Can we do what we do for HDFS and MR? That is, have separate install and configure scripts, but share the ClusterActionHandlers?
> [CDH] Start other services based on CDH, not just HDFS and MR
> -------------------------------------------------------------
>
> Key: WHIRR-191
> URL: https://issues.apache.org/jira/browse/WHIRR-191
> Project: Whirr
> Issue Type: New Feature
> Components: service/hadoop
> Affects Versions: 0.3.0
> Reporter: Lars George
> Fix For: 0.3.0
>
>
> When using the CDH service it currently only starts HDFS and MR, similar to the Apache Hadoop service (the one that uses the tarballs). We do have for example also an Apache ZooKeeper or Apache HBase service, but for CDH we do not. Either add (and change the current service) a CDH Hadoop, CDH HBase and so on or come up with another means of starting up other processes apart from HDFS and MR based on the CDH release.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Updated] (WHIRR-191) [CDH] Start other services based on
CDH, not just HDFS and MR
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-191:
----------------------------
Attachment: WHIRR-191.patch
Work in progress patch.
> [CDH] Start other services based on CDH, not just HDFS and MR
> -------------------------------------------------------------
>
> Key: WHIRR-191
> URL: https://issues.apache.org/jira/browse/WHIRR-191
> Project: Whirr
> Issue Type: New Feature
> Components: service/hadoop
> Affects Versions: 0.3.0
> Reporter: Lars George
> Attachments: WHIRR-191.patch
>
>
> When using the CDH service it currently only starts HDFS and MR, similar to the Apache Hadoop service (the one that uses the tarballs). We do have for example also an Apache ZooKeeper or Apache HBase service, but for CDH we do not. Either add (and change the current service) a CDH Hadoop, CDH HBase and so on or come up with another means of starting up other processes apart from HDFS and MR based on the CDH release.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-191) [CDH] Start other services based on
CDH, not just HDFS and MR
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-191:
----------------------------
Attachment: WHIRR-191.patch
With help from Lars I updated the test to use the Thrift server, but I'm having problems getting a scanner (as a test that HBase is running). The exception I get is:
{noformat}
org.apache.thrift.TApplicationException: Internal error processing scannerOpen
at org.apache.thrift.TApplicationException.read(TApplicationException.java:107)
at org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_scannerOpen(Hbase.java:1473)
at org.apache.hadoop.hbase.thrift.generated.Hbase$Client.scannerOpen(Hbase.java:1454)
at org.apache.whirr.service.hbase.integration.HBaseServiceController.getScanner(HBaseServiceController.java:154)
at org.apache.whirr.service.hbase.integration.HBaseServiceController.waitForMaster(HBaseServiceController.java:137)
at org.apache.whirr.service.hbase.integration.HBaseServiceController.startup(HBaseServiceController.java:102)
at org.apache.whirr.service.hbase.integration.HBaseServiceController.ensureClusterRunning(HBaseServiceController.java:78)
at org.apache.whirr.service.hbase.integration.HBaseServiceTest.setUp(HBaseServiceTest.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:115)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:102)
at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
{noformat}
Any ideas about how to make progress from here?
> [CDH] Start other services based on CDH, not just HDFS and MR
> -------------------------------------------------------------
>
> Key: WHIRR-191
> URL: https://issues.apache.org/jira/browse/WHIRR-191
> Project: Whirr
> Issue Type: New Feature
> Components: service/hadoop
> Affects Versions: 0.3.0
> Reporter: Lars George
> Fix For: 0.5.0
>
> Attachments: WHIRR-191.patch, WHIRR-191.patch
>
>
> When using the CDH service it currently only starts HDFS and MR, similar to the Apache Hadoop service (the one that uses the tarballs). We do have for example also an Apache ZooKeeper or Apache HBase service, but for CDH we do not. Either add (and change the current service) a CDH Hadoop, CDH HBase and so on or come up with another means of starting up other processes apart from HDFS and MR based on the CDH release.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (WHIRR-191) [CDH] Start other services based on
CDH, not just HDFS and MR
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977421#action_12977421 ]
Lars George commented on WHIRR-191:
-----------------------------------
I would think so, since it is only up on the servers which is different, i.e. they are all CDH based. We need to add a parameter for repo as well.
> [CDH] Start other services based on CDH, not just HDFS and MR
> -------------------------------------------------------------
>
> Key: WHIRR-191
> URL: https://issues.apache.org/jira/browse/WHIRR-191
> Project: Whirr
> Issue Type: New Feature
> Components: service/hadoop
> Affects Versions: 0.3.0
> Reporter: Lars George
> Fix For: 0.3.0
>
>
> When using the CDH service it currently only starts HDFS and MR, similar to the Apache Hadoop service (the one that uses the tarballs). We do have for example also an Apache ZooKeeper or Apache HBase service, but for CDH we do not. Either add (and change the current service) a CDH Hadoop, CDH HBase and so on or come up with another means of starting up other processes apart from HDFS and MR based on the CDH release.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Resolved] (WHIRR-191) [CDH] Start other services based on
CDH, not just HDFS and MR
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White resolved WHIRR-191.
-----------------------------
Resolution: Fixed
Assignee: Tom White
I've just committed this.
> [CDH] Start other services based on CDH, not just HDFS and MR
> -------------------------------------------------------------
>
> Key: WHIRR-191
> URL: https://issues.apache.org/jira/browse/WHIRR-191
> Project: Whirr
> Issue Type: New Feature
> Components: service/hadoop
> Affects Versions: 0.3.0
> Reporter: Lars George
> Assignee: Tom White
> Fix For: 0.5.0
>
> Attachments: WHIRR-191.patch, WHIRR-191.patch, WHIRR-191.patch
>
>
> When using the CDH service it currently only starts HDFS and MR, similar to the Apache Hadoop service (the one that uses the tarballs). We do have for example also an Apache ZooKeeper or Apache HBase service, but for CDH we do not. Either add (and change the current service) a CDH Hadoop, CDH HBase and so on or come up with another means of starting up other processes apart from HDFS and MR based on the CDH release.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira