You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Chris Embree <ce...@gmail.com> on 2013/02/20 21:17:11 UTC
Topology script frequency
I've checked all of the documentation, books and google searches I can
think of....
I have a working topology script. I have dynamic IP's. I have an
automated process to update the rack data when a datanode changes IP.
What I don't have is any clue as to when the NN reads this script. If I
execute it right now with 10.10.10.10 it'll return /dc1/rack3 When I run
hadoop balancer I see /default/rack/10.10.10.10:50010
I really don't want to have to restart the NN for IP changes to be noticed.
Any brilliant insights?
Thanks
Chris
答复: Topology script frequency
Posted by 谢良 <xi...@xiaomi.com>.
I'd submitted a patch(HADOOP-8928.txt) to reload topology config dynamically like "hdfs dfsadmin -refreshTopology" style several months before, see HADOOP-8928 for more details.
Regards,
Liang
________________________________
发件人: Chris Embree [cembree@gmail.com]
发送时间: 2013年2月21日 4:17
收件人: user@hadoop.apache.org
主题: Topology script frequency
I've checked all of the documentation, books and google searches I can think of....
I have a working topology script. I have dynamic IP's. I have an automated process to update the rack data when a datanode changes IP.
What I don't have is any clue as to when the NN reads this script. If I execute it right now with 10.10.10.10 it'll return /dc1/rack3 When I run hadoop balancer I see /default/rack/10.10.10.10:50010<http://10.10.10.10:50010>
I really don't want to have to restart the NN for IP changes to be noticed.
Any brilliant insights?
Thanks
Chris
Re: Topology script frequency
Posted by Harsh J <ha...@cloudera.com>.
NN refers to the rack topology script/class when a new node joins
(i.e. it doesn't have the node's IP already in cache), when it starts
up, and (I think) also when you issue -refreshNodes.
The ideal way to "add" a node to the rack right now is to first update
the rack config at the NN, then boot the DN up.
On Thu, Feb 21, 2013 at 1:47 AM, Chris Embree <ce...@gmail.com> wrote:
> I've checked all of the documentation, books and google searches I can think
> of....
>
> I have a working topology script. I have dynamic IP's. I have an automated
> process to update the rack data when a datanode changes IP.
>
> What I don't have is any clue as to when the NN reads this script. If I
> execute it right now with 10.10.10.10 it'll return /dc1/rack3 When I run
> hadoop balancer I see /default/rack/10.10.10.10:50010
> I really don't want to have to restart the NN for IP changes to be noticed.
>
> Any brilliant insights?
>
> Thanks
> Chris
--
Harsh J
答复: Topology script frequency
Posted by 谢良 <xi...@xiaomi.com>.
I'd submitted a patch(HADOOP-8928.txt) to reload topology config dynamically like "hdfs dfsadmin -refreshTopology" style several months before, see HADOOP-8928 for more details.
Regards,
Liang
________________________________
发件人: Chris Embree [cembree@gmail.com]
发送时间: 2013年2月21日 4:17
收件人: user@hadoop.apache.org
主题: Topology script frequency
I've checked all of the documentation, books and google searches I can think of....
I have a working topology script. I have dynamic IP's. I have an automated process to update the rack data when a datanode changes IP.
What I don't have is any clue as to when the NN reads this script. If I execute it right now with 10.10.10.10 it'll return /dc1/rack3 When I run hadoop balancer I see /default/rack/10.10.10.10:50010<http://10.10.10.10:50010>
I really don't want to have to restart the NN for IP changes to be noticed.
Any brilliant insights?
Thanks
Chris
Re: Topology script frequency
Posted by Harsh J <ha...@cloudera.com>.
NN refers to the rack topology script/class when a new node joins
(i.e. it doesn't have the node's IP already in cache), when it starts
up, and (I think) also when you issue -refreshNodes.
The ideal way to "add" a node to the rack right now is to first update
the rack config at the NN, then boot the DN up.
On Thu, Feb 21, 2013 at 1:47 AM, Chris Embree <ce...@gmail.com> wrote:
> I've checked all of the documentation, books and google searches I can think
> of....
>
> I have a working topology script. I have dynamic IP's. I have an automated
> process to update the rack data when a datanode changes IP.
>
> What I don't have is any clue as to when the NN reads this script. If I
> execute it right now with 10.10.10.10 it'll return /dc1/rack3 When I run
> hadoop balancer I see /default/rack/10.10.10.10:50010
> I really don't want to have to restart the NN for IP changes to be noticed.
>
> Any brilliant insights?
>
> Thanks
> Chris
--
Harsh J
答复: Topology script frequency
Posted by 谢良 <xi...@xiaomi.com>.
I'd submitted a patch(HADOOP-8928.txt) to reload topology config dynamically like "hdfs dfsadmin -refreshTopology" style several months before, see HADOOP-8928 for more details.
Regards,
Liang
________________________________
发件人: Chris Embree [cembree@gmail.com]
发送时间: 2013年2月21日 4:17
收件人: user@hadoop.apache.org
主题: Topology script frequency
I've checked all of the documentation, books and google searches I can think of....
I have a working topology script. I have dynamic IP's. I have an automated process to update the rack data when a datanode changes IP.
What I don't have is any clue as to when the NN reads this script. If I execute it right now with 10.10.10.10 it'll return /dc1/rack3 When I run hadoop balancer I see /default/rack/10.10.10.10:50010<http://10.10.10.10:50010>
I really don't want to have to restart the NN for IP changes to be noticed.
Any brilliant insights?
Thanks
Chris
Re: Topology script frequency
Posted by Harsh J <ha...@cloudera.com>.
NN refers to the rack topology script/class when a new node joins
(i.e. it doesn't have the node's IP already in cache), when it starts
up, and (I think) also when you issue -refreshNodes.
The ideal way to "add" a node to the rack right now is to first update
the rack config at the NN, then boot the DN up.
On Thu, Feb 21, 2013 at 1:47 AM, Chris Embree <ce...@gmail.com> wrote:
> I've checked all of the documentation, books and google searches I can think
> of....
>
> I have a working topology script. I have dynamic IP's. I have an automated
> process to update the rack data when a datanode changes IP.
>
> What I don't have is any clue as to when the NN reads this script. If I
> execute it right now with 10.10.10.10 it'll return /dc1/rack3 When I run
> hadoop balancer I see /default/rack/10.10.10.10:50010
> I really don't want to have to restart the NN for IP changes to be noticed.
>
> Any brilliant insights?
>
> Thanks
> Chris
--
Harsh J
答复: Topology script frequency
Posted by 谢良 <xi...@xiaomi.com>.
I'd submitted a patch(HADOOP-8928.txt) to reload topology config dynamically like "hdfs dfsadmin -refreshTopology" style several months before, see HADOOP-8928 for more details.
Regards,
Liang
________________________________
发件人: Chris Embree [cembree@gmail.com]
发送时间: 2013年2月21日 4:17
收件人: user@hadoop.apache.org
主题: Topology script frequency
I've checked all of the documentation, books and google searches I can think of....
I have a working topology script. I have dynamic IP's. I have an automated process to update the rack data when a datanode changes IP.
What I don't have is any clue as to when the NN reads this script. If I execute it right now with 10.10.10.10 it'll return /dc1/rack3 When I run hadoop balancer I see /default/rack/10.10.10.10:50010<http://10.10.10.10:50010>
I really don't want to have to restart the NN for IP changes to be noticed.
Any brilliant insights?
Thanks
Chris
Re: Topology script frequency
Posted by Harsh J <ha...@cloudera.com>.
NN refers to the rack topology script/class when a new node joins
(i.e. it doesn't have the node's IP already in cache), when it starts
up, and (I think) also when you issue -refreshNodes.
The ideal way to "add" a node to the rack right now is to first update
the rack config at the NN, then boot the DN up.
On Thu, Feb 21, 2013 at 1:47 AM, Chris Embree <ce...@gmail.com> wrote:
> I've checked all of the documentation, books and google searches I can think
> of....
>
> I have a working topology script. I have dynamic IP's. I have an automated
> process to update the rack data when a datanode changes IP.
>
> What I don't have is any clue as to when the NN reads this script. If I
> execute it right now with 10.10.10.10 it'll return /dc1/rack3 When I run
> hadoop balancer I see /default/rack/10.10.10.10:50010
> I really don't want to have to restart the NN for IP changes to be noticed.
>
> Any brilliant insights?
>
> Thanks
> Chris
--
Harsh J