You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Stas Oskin <st...@gmail.com> on 2009/04/27 13:11:27 UTC
Blocks replication in downtime even
Hi.
I have a question:
If I have N of DataNodes, and one or several of the nodes have become
unavailable, would HDFS re-synchronize the blocks automatically, according
to replication level set?
And if yes, when? As soon as the offline node was detected, or only on file
access?
Regards.
Re: Blocks replication in downtime even
Posted by Stas Oskin <st...@gmail.com>.
Hi.
I think one needs to run balancer in order to clean out the redundant
blocks.
Can anyone confirm this?
Regards.
2009/4/28 Piotr <pi...@gmail.com>
> Hi
>
> What happens when the node rejoins then ?
>
> - The replication level of several blocks increases ?
> - The old replicas are removed in favor of new replicas ? (Or the new
> ones)
>
> regards
> Piotr
>
> 2009/4/27 Stas Oskin <st...@gmail.com>
>
> > Thanks.
> >
> > 2009/4/27 Koji Noguchi <kn...@yahoo-inc.com>
> >
> > >
> http://hadoop.apache.org/core/docs/current/hdfs_design.html#Data+Disk+Fa
> > > ilure%2C+Heartbeats+and+Re-Replication<
> >
> http://hadoop.apache.org/core/docs/current/hdfs_design.html#Data+Disk+Fa%0Ailure%2C+Heartbeats+and+Re-Replication
> > >
> > >
> > > hope this helps.
> > >
> > > Koji
> > >
> > > -----Original Message-----
> > > From: Stas Oskin [mailto:stas.oskin@gmail.com]
> > > Sent: Monday, April 27, 2009 4:11 AM
> > > To: core-user@hadoop.apache.org
> > > Subject: Blocks replication in downtime even
> > >
> > > Hi.
> > >
> > > I have a question:
> > >
> > > If I have N of DataNodes, and one or several of the nodes have become
> > > unavailable, would HDFS re-synchronize the blocks automatically,
> > > according
> > > to replication level set?
> > > And if yes, when? As soon as the offline node was detected, or only on
> > > file
> > > access?
> > >
> > > Regards.
> > >
> >
>
Re: Blocks replication in downtime even
Posted by Piotr <pi...@gmail.com>.
Hi
What happens when the node rejoins then ?
- The replication level of several blocks increases ?
- The old replicas are removed in favor of new replicas ? (Or the new
ones)
regards
Piotr
2009/4/27 Stas Oskin <st...@gmail.com>
> Thanks.
>
> 2009/4/27 Koji Noguchi <kn...@yahoo-inc.com>
>
> > http://hadoop.apache.org/core/docs/current/hdfs_design.html#Data+Disk+Fa
> > ilure%2C+Heartbeats+and+Re-Replication<
> http://hadoop.apache.org/core/docs/current/hdfs_design.html#Data+Disk+Fa%0Ailure%2C+Heartbeats+and+Re-Replication
> >
> >
> > hope this helps.
> >
> > Koji
> >
> > -----Original Message-----
> > From: Stas Oskin [mailto:stas.oskin@gmail.com]
> > Sent: Monday, April 27, 2009 4:11 AM
> > To: core-user@hadoop.apache.org
> > Subject: Blocks replication in downtime even
> >
> > Hi.
> >
> > I have a question:
> >
> > If I have N of DataNodes, and one or several of the nodes have become
> > unavailable, would HDFS re-synchronize the blocks automatically,
> > according
> > to replication level set?
> > And if yes, when? As soon as the offline node was detected, or only on
> > file
> > access?
> >
> > Regards.
> >
>
Re: Blocks replication in downtime even
Posted by Stas Oskin <st...@gmail.com>.
Thanks.
2009/4/27 Koji Noguchi <kn...@yahoo-inc.com>
> http://hadoop.apache.org/core/docs/current/hdfs_design.html#Data+Disk+Fa
> ilure%2C+Heartbeats+and+Re-Replication<http://hadoop.apache.org/core/docs/current/hdfs_design.html#Data+Disk+Fa%0Ailure%2C+Heartbeats+and+Re-Replication>
>
> hope this helps.
>
> Koji
>
> -----Original Message-----
> From: Stas Oskin [mailto:stas.oskin@gmail.com]
> Sent: Monday, April 27, 2009 4:11 AM
> To: core-user@hadoop.apache.org
> Subject: Blocks replication in downtime even
>
> Hi.
>
> I have a question:
>
> If I have N of DataNodes, and one or several of the nodes have become
> unavailable, would HDFS re-synchronize the blocks automatically,
> according
> to replication level set?
> And if yes, when? As soon as the offline node was detected, or only on
> file
> access?
>
> Regards.
>
RE: Blocks replication in downtime even
Posted by Koji Noguchi <kn...@yahoo-inc.com>.
http://hadoop.apache.org/core/docs/current/hdfs_design.html#Data+Disk+Fa
ilure%2C+Heartbeats+and+Re-Replication
hope this helps.
Koji
-----Original Message-----
From: Stas Oskin [mailto:stas.oskin@gmail.com]
Sent: Monday, April 27, 2009 4:11 AM
To: core-user@hadoop.apache.org
Subject: Blocks replication in downtime even
Hi.
I have a question:
If I have N of DataNodes, and one or several of the nodes have become
unavailable, would HDFS re-synchronize the blocks automatically,
according
to replication level set?
And if yes, when? As soon as the offline node was detected, or only on
file
access?
Regards.