You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Tsz Wo (Nicholas), Sze" <s2...@yahoo.com> on 2009/10/19 23:34:01 UTC
[VOTE] port HDFS-127 (DFSClient block read failures cause open DFSInputStream to become unusable) to hadoop 0.20/0.21
DFSClient has a retry mechanism on block acquiring for read. If the number of retries attends to a certain limit (defined by dfs.client.max.block.acquire.failures), DFSClient will throw a BlockMissingException back to the user application. In the current implementation, DFSClient counts the failures across multiple block acquiring operations but the block acquiring operations are supposed to be independent. HDFS-127 fixes this problem by counting the failures within a single operation.
I propose to commit HDFS-127 to 0.20 and above since this fix is safe and very useful.
Nicholas Sze
Re: [VOTE] port HDFS-127 (DFSClient block read failures cause open DFSInputStream to become unusable) to hadoop 0.20/0.21
Posted by "Tsz Wo (Nicholas), Sze" <s2...@yahoo.com>.
With five +1 (Suresh, Chris, Dhruba, Stack and myself) and no -1, the vote passed. I am going to commitHDFS-127 to 0.20 and above.
Nicholas
----- Original Message ----
> From: "Tsz Wo (Nicholas), Sze" <s2...@yahoo.com>
> To: hdfs-dev@hadoop.apache.org
> Sent: Mon, October 19, 2009 2:34:01 PM
> Subject: [VOTE] port HDFS-127 (DFSClient block read failures cause open DFSInputStream to become unusable) to hadoop 0.20/0.21
>
> DFSClient has a retry mechanism on block acquiring for read. If the number of
> retries attends to a certain limit (defined by
> dfs.client.max.block.acquire.failures), DFSClient will throw a
> BlockMissingException back to the user application. In the current
> implementation, DFSClient counts the failures across multiple block acquiring
> operations but the block acquiring operations are supposed to be independent.
> HDFS-127 fixes this problem by counting the failures within a single operation.
>
> I propose to commit HDFS-127 to 0.20 and above since this fix is safe and very
> useful.
>
> Nicholas Sze
Re: [VOTE] port HDFS-127 (DFSClient block read failures cause open
DFSInputStream to become unusable) to hadoop 0.20/0.21
Posted by Dhruba Borthakur <dh...@gmail.com>.
+1.
On Mon, Oct 19, 2009 at 3:55 PM, Chris Douglas <ch...@gmail.com>wrote:
> +1 This fix has been in limbo for a long time; thanks for finishing
> it, Nicholas. -C
>
> On Mon, Oct 19, 2009 at 2:34 PM, Tsz Wo (Nicholas), Sze
> <s2...@yahoo.com> wrote:
> > DFSClient has a retry mechanism on block acquiring for read. If the
> number of retries attends to a certain limit (defined by
> dfs.client.max.block.acquire.failures), DFSClient will throw a
> BlockMissingException back to the user application. In the current
> implementation, DFSClient counts the failures across multiple block
> acquiring operations but the block acquiring operations are supposed to be
> independent. HDFS-127 fixes this problem by counting the failures within a
> single operation.
> >
> > I propose to commit HDFS-127 to 0.20 and above since this fix is safe and
> very useful.
> >
> > Nicholas Sze
> >
> >
>
--
Connect to me at http://www.facebook.com/dhruba
Re: [VOTE] port HDFS-127 (DFSClient block read failures cause open
DFSInputStream to become unusable) to hadoop 0.20/0.21
Posted by Chris Douglas <ch...@gmail.com>.
+1 This fix has been in limbo for a long time; thanks for finishing
it, Nicholas. -C
On Mon, Oct 19, 2009 at 2:34 PM, Tsz Wo (Nicholas), Sze
<s2...@yahoo.com> wrote:
> DFSClient has a retry mechanism on block acquiring for read. If the number of retries attends to a certain limit (defined by dfs.client.max.block.acquire.failures), DFSClient will throw a BlockMissingException back to the user application. In the current implementation, DFSClient counts the failures across multiple block acquiring operations but the block acquiring operations are supposed to be independent. HDFS-127 fixes this problem by counting the failures within a single operation.
>
> I propose to commit HDFS-127 to 0.20 and above since this fix is safe and very useful.
>
> Nicholas Sze
>
>
Re: [VOTE] port HDFS-127 (DFSClient block read failures cause open
DFSInputStream to become unusable) to hadoop 0.20/0.21
Posted by stack <st...@duboce.net>.
+1
On Mon, Oct 19, 2009 at 2:34 PM, Tsz Wo (Nicholas), Sze <
s29752-hadoopdev@yahoo.com> wrote:
> DFSClient has a retry mechanism on block acquiring for read. If the number
> of retries attends to a certain limit (defined by
> dfs.client.max.block.acquire.failures), DFSClient will throw a
> BlockMissingException back to the user application. In the current
> implementation, DFSClient counts the failures across multiple block
> acquiring operations but the block acquiring operations are supposed to be
> independent. HDFS-127 fixes this problem by counting the failures within a
> single operation.
>
> I propose to commit HDFS-127 to 0.20 and above since this fix is safe and
> very useful.
>
> Nicholas Sze
>
>
Re: [VOTE] port HDFS-127 (DFSClient block read failures cause open
DFSInputStream to become unusable) to hadoop 0.20/0.21
Posted by Suresh Srinivas <su...@yahoo-inc.com>.
+1
On 10/19/09 2:34 PM, "Tsz Wo (Nicholas), Sze" <s2...@yahoo.com> wrote:
DFSClient has a retry mechanism on block acquiring for read. If the number of retries attends to a certain limit (defined by dfs.client.max.block.acquire.failures), DFSClient will throw a BlockMissingException back to the user application. In the current implementation, DFSClient counts the failures across multiple block acquiring operations but the block acquiring operations are supposed to be independent. HDFS-127 fixes this problem by counting the failures within a single operation.
I propose to commit HDFS-127 to 0.20 and above since this fix is safe and very useful.
Nicholas Sze