You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Bruno Dumon (JIRA)" <ji...@apache.org> on 2011/07/15 15:08:02 UTC
[jira] [Created] (WHIRR-338) byon cluster with hostnames defined in
/etc/hosts
byon cluster with hostnames defined in /etc/hosts
-------------------------------------------------
Key: WHIRR-338
URL: https://issues.apache.org/jira/browse/WHIRR-338
Project: Whirr
Issue Type: Improvement
Components: core
Reporter: Bruno Dumon
Attachments: WHIRR-338.patch
Basically the subject combined with the single-line patch should make it clear, what follows is the long story.
I am unable to get HBase to work when using BYON nodes due to the following error:
{noformat}
2011-07-14 14:46:07,837 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
java.lang.IllegalArgumentException: Wrong FS: hdfs://192.168.56.11:8020/hbase, expected: hdfs://ubuntutest2
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
at org.apache.hadoop.hdfs.DistributedFileSystem.checkPath(DistributedFileSystem.java:106)
{noformat}
The situation is I'm using two VirtualBox instances whose hostnames I defined manually through /etc/hosts
In the byon yaml file, I specify the host by IP address:
{noformat}
- id: ubuntutest2
hostname: 192.168.56.11
{noformat}
When specifying the hostname instead, I get an error when starting Whirr:
invalid IP address: ubuntutest2
This message is produced in the constructor of Cluster.Instance, earlier Whirr got the address by calling NodeMetadata.getPublicAddresses(), so jclouds returns there a hostname rather than an address (which might be considered a bug in jclouds, but even if it returned an ip address, the problem described next stays true).
Back to the case where I specify the IP address in the byon yaml:
The address in the hdfs URL is determined in HBaseMasterClusterActionHandler and comes down to:
instance.getPublicAddress().getHostName()
which is determined by xbill/dnsjava (see DnsUtil), which does not make use of /etc/hosts.
As fix (see attached patch), I propose to fall back to Java's getCanonicalHostName() in case dnsjava did not return any results.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (WHIRR-338) byon cluster with hostnames defined
in /etc/hosts
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White resolved WHIRR-338.
-----------------------------
Resolution: Fixed
Fix Version/s: 0.6.0
Assignee: Bruno Dumon
I've just committed this. Thanks, Bruno!
> byon cluster with hostnames defined in /etc/hosts
> -------------------------------------------------
>
> Key: WHIRR-338
> URL: https://issues.apache.org/jira/browse/WHIRR-338
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Reporter: Bruno Dumon
> Assignee: Bruno Dumon
> Fix For: 0.6.0
>
> Attachments: WHIRR-338.patch
>
>
> Basically the subject combined with the single-line patch should make it clear, what follows is the long story.
> I am unable to get HBase to work when using BYON nodes due to the following error:
> {noformat}
> 2011-07-14 14:46:07,837 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
> java.lang.IllegalArgumentException: Wrong FS: hdfs://192.168.56.11:8020/hbase, expected: hdfs://ubuntutest2
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> at org.apache.hadoop.hdfs.DistributedFileSystem.checkPath(DistributedFileSystem.java:106)
> {noformat}
> The situation is I'm using two VirtualBox instances whose hostnames I defined manually through /etc/hosts
> In the byon yaml file, I specify the host by IP address:
> {noformat}
> - id: ubuntutest2
> hostname: 192.168.56.11
> {noformat}
> When specifying the hostname instead, I get an error when starting Whirr:
> invalid IP address: ubuntutest2
> This message is produced in the constructor of Cluster.Instance, earlier Whirr got the address by calling NodeMetadata.getPublicAddresses(), so jclouds returns there a hostname rather than an address (which might be considered a bug in jclouds, but even if it returned an ip address, the problem described next stays true).
> Back to the case where I specify the IP address in the byon yaml:
> The address in the hdfs URL is determined in HBaseMasterClusterActionHandler and comes down to:
> instance.getPublicAddress().getHostName()
> which is determined by xbill/dnsjava (see DnsUtil), which does not make use of /etc/hosts.
> As fix (see attached patch), I propose to fall back to Java's getCanonicalHostName() in case dnsjava did not return any results.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-338) byon cluster with hostnames defined in
/etc/hosts
Posted by "Bruno Dumon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bruno Dumon updated WHIRR-338:
------------------------------
Attachment: WHIRR-338.patch
Added patch, also available as https://github.com/bdumon/whirr/commit/8d44cf874619c0b5bafae1e7da834ef047fc224e
> byon cluster with hostnames defined in /etc/hosts
> -------------------------------------------------
>
> Key: WHIRR-338
> URL: https://issues.apache.org/jira/browse/WHIRR-338
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Reporter: Bruno Dumon
> Attachments: WHIRR-338.patch
>
>
> Basically the subject combined with the single-line patch should make it clear, what follows is the long story.
> I am unable to get HBase to work when using BYON nodes due to the following error:
> {noformat}
> 2011-07-14 14:46:07,837 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
> java.lang.IllegalArgumentException: Wrong FS: hdfs://192.168.56.11:8020/hbase, expected: hdfs://ubuntutest2
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> at org.apache.hadoop.hdfs.DistributedFileSystem.checkPath(DistributedFileSystem.java:106)
> {noformat}
> The situation is I'm using two VirtualBox instances whose hostnames I defined manually through /etc/hosts
> In the byon yaml file, I specify the host by IP address:
> {noformat}
> - id: ubuntutest2
> hostname: 192.168.56.11
> {noformat}
> When specifying the hostname instead, I get an error when starting Whirr:
> invalid IP address: ubuntutest2
> This message is produced in the constructor of Cluster.Instance, earlier Whirr got the address by calling NodeMetadata.getPublicAddresses(), so jclouds returns there a hostname rather than an address (which might be considered a bug in jclouds, but even if it returned an ip address, the problem described next stays true).
> Back to the case where I specify the IP address in the byon yaml:
> The address in the hdfs URL is determined in HBaseMasterClusterActionHandler and comes down to:
> instance.getPublicAddress().getHostName()
> which is determined by xbill/dnsjava (see DnsUtil), which does not make use of /etc/hosts.
> As fix (see attached patch), I propose to fall back to Java's getCanonicalHostName() in case dnsjava did not return any results.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira