You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Tim Robertson (Created) (JIRA)" <ji...@apache.org> on 2012/01/30 15:52:10 UTC
[jira] [Created] (WHIRR-492) Hadoop TaskTrackers on EC2 mix of
internal and external host names
Hadoop TaskTrackers on EC2 mix of internal and external host names
------------------------------------------------------------------
Key: WHIRR-492
URL: https://issues.apache.org/jira/browse/WHIRR-492
Project: Whirr
Issue Type: Bug
Components: service/hadoop
Affects Versions: 0.7.0
Environment: EC2
Reporter: Tim Robertson
Running a 1 master, 3 node cluster of Hadoop + HBase, I see on the Hadoop task trackers
the following hosts:
1) domU-12-31-39-0B-C1-81.compute-1.internal
2) ip-10-66-197-215.ec2.internal
3) domU-12-31-39-0E-F2-41.compute-1.internal
This is consistent with both the trunk (today) and 0.7.0.2, and all
tasks on node 2) above then fail.
My config is the following:
whirr.cluster-name=hbase
whirr.instance-templates=1
zookeeper+hadoop-namenode+hadoop-jobtracker+hbase-master,3
hadoop-datanode+hadoop-tasktracker+hbase-regionserver
whirr.provider=aws-ec2
whirr.identity=${env:AWS_ACCESS_KEY_ID}
whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
whirr.hardware-id=c1.xlarge
whirr.image-id=us-east-1/ami-da0cf8b3
whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-492) Hadoop TaskTrackers on EC2 mix of
internal and external host names
Posted by "Tom White (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198405#comment-13198405 ]
Tom White commented on WHIRR-492:
---------------------------------
I think these are all internal addresses, just different DNS names. I wonder if it would help to constrain everything to one zone rather than a whole region: http://whirr.apache.org/faq.html#How_can_I_start_a_cluster_in_a_particular_location
> Hadoop TaskTrackers on EC2 mix of internal and external host names
> ------------------------------------------------------------------
>
> Key: WHIRR-492
> URL: https://issues.apache.org/jira/browse/WHIRR-492
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.7.0
> Environment: EC2
> Reporter: Tim Robertson
>
> Running a 1 master, 3 node cluster of Hadoop + HBase, I see on the Hadoop task trackers
> the following hosts:
> 1) domU-12-31-39-0B-C1-81.compute-1.internal
> 2) ip-10-66-197-215.ec2.internal
> 3) domU-12-31-39-0E-F2-41.compute-1.internal
> This is consistent with both the trunk (today) and 0.7.0.2, and all
> tasks on node 2) above then fail.
> My config is the following:
> whirr.cluster-name=hbase
> whirr.instance-templates=1
> zookeeper+hadoop-namenode+hadoop-jobtracker+hbase-master,3
> hadoop-datanode+hadoop-tasktracker+hbase-regionserver
> whirr.provider=aws-ec2
> whirr.identity=${env:AWS_ACCESS_KEY_ID}
> whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-492) Hadoop TaskTrackers on EC2 mix of
internal and external host names
Posted by "Tim Robertson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199743#comment-13199743 ]
Tim Robertson commented on WHIRR-492:
-------------------------------------
Confirmed it appears to work consistently when using us-east-1b
Thanks Tom, Andrei
> Hadoop TaskTrackers on EC2 mix of internal and external host names
> ------------------------------------------------------------------
>
> Key: WHIRR-492
> URL: https://issues.apache.org/jira/browse/WHIRR-492
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.7.0
> Environment: EC2
> Reporter: Tim Robertson
>
> Running a 1 master, 3 node cluster of Hadoop + HBase, I see on the Hadoop task trackers
> the following hosts:
> 1) domU-12-31-39-0B-C1-81.compute-1.internal
> 2) ip-10-66-197-215.ec2.internal
> 3) domU-12-31-39-0E-F2-41.compute-1.internal
> This is consistent with both the trunk (today) and 0.7.0.2, and all
> tasks on node 2) above then fail.
> My config is the following:
> whirr.cluster-name=hbase
> whirr.instance-templates=1
> zookeeper+hadoop-namenode+hadoop-jobtracker+hbase-master,3
> hadoop-datanode+hadoop-tasktracker+hbase-regionserver
> whirr.provider=aws-ec2
> whirr.identity=${env:AWS_ACCESS_KEY_ID}
> whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-492) Hadoop TaskTrackers on EC2 mix of
internal and external host names
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198825#comment-13198825 ]
Andrei Savu commented on WHIRR-492:
-----------------------------------
Good point! Tim can you give it a try using us-east-1a (or similar) as the location-id?
> Hadoop TaskTrackers on EC2 mix of internal and external host names
> ------------------------------------------------------------------
>
> Key: WHIRR-492
> URL: https://issues.apache.org/jira/browse/WHIRR-492
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.7.0
> Environment: EC2
> Reporter: Tim Robertson
>
> Running a 1 master, 3 node cluster of Hadoop + HBase, I see on the Hadoop task trackers
> the following hosts:
> 1) domU-12-31-39-0B-C1-81.compute-1.internal
> 2) ip-10-66-197-215.ec2.internal
> 3) domU-12-31-39-0E-F2-41.compute-1.internal
> This is consistent with both the trunk (today) and 0.7.0.2, and all
> tasks on node 2) above then fail.
> My config is the following:
> whirr.cluster-name=hbase
> whirr.instance-templates=1
> zookeeper+hadoop-namenode+hadoop-jobtracker+hbase-master,3
> hadoop-datanode+hadoop-tasktracker+hbase-regionserver
> whirr.provider=aws-ec2
> whirr.identity=${env:AWS_ACCESS_KEY_ID}
> whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-492) Hadoop TaskTrackers on EC2 mix of
internal and external host names
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200131#comment-13200131 ]
Andrei Savu commented on WHIRR-492:
-----------------------------------
Any ideas on how we should incorporate this? Only as documentation?
> Hadoop TaskTrackers on EC2 mix of internal and external host names
> ------------------------------------------------------------------
>
> Key: WHIRR-492
> URL: https://issues.apache.org/jira/browse/WHIRR-492
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.7.0
> Environment: EC2
> Reporter: Tim Robertson
>
> Running a 1 master, 3 node cluster of Hadoop + HBase, I see on the Hadoop task trackers
> the following hosts:
> 1) domU-12-31-39-0B-C1-81.compute-1.internal
> 2) ip-10-66-197-215.ec2.internal
> 3) domU-12-31-39-0E-F2-41.compute-1.internal
> This is consistent with both the trunk (today) and 0.7.0.2, and all
> tasks on node 2) above then fail.
> My config is the following:
> whirr.cluster-name=hbase
> whirr.instance-templates=1
> zookeeper+hadoop-namenode+hadoop-jobtracker+hbase-master,3
> hadoop-datanode+hadoop-tasktracker+hbase-regionserver
> whirr.provider=aws-ec2
> whirr.identity=${env:AWS_ACCESS_KEY_ID}
> whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira