You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by lei liu <li...@gmail.com> on 2013/10/28 09:59:25 UTC
Decommission DataNode
In CDH3u5, when the DataNode is Decommissioned, the DataNode progress will
be shutdown by NameNode.
But In CDH4.3.1, when the DataNode is Decommissioned, the DataNode progress
will be not shutdown by NameNode.
When the datanode is Decommissioned, why the datanode is not automatically
shutdown by NameNode in CDH4.3.1?
Thanks,
LiuLei
Re: Decommission DataNode
Posted by Suresh Srinivas <su...@hortonworks.com>.
No this is not a bug. This is the new behavior. Please see for details -
https://issues.apache.org/jira/browse/HDFS-1547
On Tue, Oct 29, 2013 at 1:48 AM, lei liu <li...@gmail.com> wrote:
> Should the datanode be shutdown when it is Decommissioned?
>
> I think if this is bug, I can fix it.
>
>
> 2013/10/29 lei liu <li...@gmail.com>
>
> > HI Steve, thanks for your reply.
> >
> >
> > There is the question in hadoop-2.2.0.
> >
> >
> > In hadoop-2.2.0, there are below code in DatanodeManager.handleHeartbeat
> > method :
> > // Check if this datanode should actually be shutdown instead.
> > if (nodeinfo != null &&* nodeinfo.isDisallowed()*) {
> > setDatanodeDead(nodeinfo);
> > throw new DisallowedDatanodeException(nodeinfo);
> > }
> >
> >
> > In hadoop-1.0.1, there are below code in FSNamesystem.handleHeartbeat
> > method:
> > // Check if this datanode should actually be shutdown instead.
> > if (nodeinfo != null && shouldNodeShutdown(nodeinfo)) {
> > setDatanodeDead(nodeinfo);
> > throw new DisallowedDatanodeException(nodeinfo);
> > }
> >
> > private boolean shouldNodeShutdown(DatanodeDescriptor node) {
> > return (node.isDecommissioned());
> > }
> >
> >
> > So in hadoop-1.0.1, when the datanode is Decommissioned, the datanode
> will
> > be kill by NameNode, but in hadoop-2.2.0, when the datanode is
> > Decommissioned, the datanode still is lived.
> >
> >
> >
> > 2013/10/28 Steve Loughran <st...@hortonworks.com>
> >
> >> sounds like a question for the cloudera support forums
> >>
> >>
> >> On 28 October 2013 08:59, lei liu <li...@gmail.com> wrote:
> >>
> >> > In CDH3u5, when the DataNode is Decommissioned, the DataNode progress
> >> will
> >> > be shutdown by NameNode.
> >> >
> >> > But In CDH4.3.1, when the DataNode is Decommissioned, the DataNode
> >> progress
> >> > will be not shutdown by NameNode.
> >> >
> >> >
> >> > When the datanode is Decommissioned, why the datanode is not
> >> automatically
> >> > shutdown by NameNode in CDH4.3.1?
> >> >
> >> >
> >> > Thanks,
> >> >
> >> > LiuLei
> >> >
> >>
> >> --
> >> CONFIDENTIALITY NOTICE
> >> NOTICE: This message is intended for the use of the individual or entity
> >> to
> >> which it is addressed and may contain information that is confidential,
> >> privileged and exempt from disclosure under applicable law. If the
> reader
> >> of this message is not the intended recipient, you are hereby notified
> >> that
> >> any printing, copying, dissemination, distribution, disclosure or
> >> forwarding of this communication is strictly prohibited. If you have
> >> received this communication in error, please contact the sender
> >> immediately
> >> and delete it from your system. Thank You.
> >>
> >
> >
>
--
http://hortonworks.com/download/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Decommission DataNode
Posted by lei liu <li...@gmail.com>.
Should the datanode be shutdown when it is Decommissioned?
I think if this is bug, I can fix it.
2013/10/29 lei liu <li...@gmail.com>
> HI Steve, thanks for your reply.
>
>
> There is the question in hadoop-2.2.0.
>
>
> In hadoop-2.2.0, there are below code in DatanodeManager.handleHeartbeat
> method :
> // Check if this datanode should actually be shutdown instead.
> if (nodeinfo != null &&* nodeinfo.isDisallowed()*) {
> setDatanodeDead(nodeinfo);
> throw new DisallowedDatanodeException(nodeinfo);
> }
>
>
> In hadoop-1.0.1, there are below code in FSNamesystem.handleHeartbeat
> method:
> // Check if this datanode should actually be shutdown instead.
> if (nodeinfo != null && shouldNodeShutdown(nodeinfo)) {
> setDatanodeDead(nodeinfo);
> throw new DisallowedDatanodeException(nodeinfo);
> }
>
> private boolean shouldNodeShutdown(DatanodeDescriptor node) {
> return (node.isDecommissioned());
> }
>
>
> So in hadoop-1.0.1, when the datanode is Decommissioned, the datanode will
> be kill by NameNode, but in hadoop-2.2.0, when the datanode is
> Decommissioned, the datanode still is lived.
>
>
>
> 2013/10/28 Steve Loughran <st...@hortonworks.com>
>
>> sounds like a question for the cloudera support forums
>>
>>
>> On 28 October 2013 08:59, lei liu <li...@gmail.com> wrote:
>>
>> > In CDH3u5, when the DataNode is Decommissioned, the DataNode progress
>> will
>> > be shutdown by NameNode.
>> >
>> > But In CDH4.3.1, when the DataNode is Decommissioned, the DataNode
>> progress
>> > will be not shutdown by NameNode.
>> >
>> >
>> > When the datanode is Decommissioned, why the datanode is not
>> automatically
>> > shutdown by NameNode in CDH4.3.1?
>> >
>> >
>> > Thanks,
>> >
>> > LiuLei
>> >
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>
Re: Decommission DataNode
Posted by lei liu <li...@gmail.com>.
HI Steve, thanks for your reply.
There is the question in hadoop-2.2.0.
In hadoop-2.2.0, there are below code in
DatanodeManager<eclipse-javadoc:%E2%98%82=hadoop-hdfs/src%5C/main%5C/java%3Corg.apache.hadoop.hdfs.server.blockmanagement%7BDatanodeManager.java%E2%98%83DatanodeManager>.handleHeartbeat
method :
// Check if this datanode should actually be shutdown instead.
if (nodeinfo != null &&* nodeinfo.isDisallowed()*) {
setDatanodeDead(nodeinfo);
throw new DisallowedDatanodeException(nodeinfo);
}
In hadoop-1.0.1, there are below code in FSNamesystem.handleHeartbeat
method:
// Check if this datanode should actually be shutdown instead.
if (nodeinfo != null && shouldNodeShutdown(nodeinfo)) {
setDatanodeDead(nodeinfo);
throw new DisallowedDatanodeException(nodeinfo);
}
private boolean shouldNodeShutdown(DatanodeDescriptor node) {
return (node.isDecommissioned());
}
So in hadoop-1.0.1, when the datanode is Decommissioned, the datanode will
be kill by NameNode, but in hadoop-2.2.0, when the datanode is
Decommissioned, the datanode still is lived.
2013/10/28 Steve Loughran <st...@hortonworks.com>
> sounds like a question for the cloudera support forums
>
>
> On 28 October 2013 08:59, lei liu <li...@gmail.com> wrote:
>
> > In CDH3u5, when the DataNode is Decommissioned, the DataNode progress
> will
> > be shutdown by NameNode.
> >
> > But In CDH4.3.1, when the DataNode is Decommissioned, the DataNode
> progress
> > will be not shutdown by NameNode.
> >
> >
> > When the datanode is Decommissioned, why the datanode is not
> automatically
> > shutdown by NameNode in CDH4.3.1?
> >
> >
> > Thanks,
> >
> > LiuLei
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: Decommission DataNode
Posted by Steve Loughran <st...@hortonworks.com>.
sounds like a question for the cloudera support forums
On 28 October 2013 08:59, lei liu <li...@gmail.com> wrote:
> In CDH3u5, when the DataNode is Decommissioned, the DataNode progress will
> be shutdown by NameNode.
>
> But In CDH4.3.1, when the DataNode is Decommissioned, the DataNode progress
> will be not shutdown by NameNode.
>
>
> When the datanode is Decommissioned, why the datanode is not automatically
> shutdown by NameNode in CDH4.3.1?
>
>
> Thanks,
>
> LiuLei
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.