You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jonathan Hsieh <jo...@cloudera.com> on 2012/03/07 00:25:55 UTC
Re: Why ScannerTimeoutException is not handled in TableInputFormat?
Hi Jan,
What version were you on before and what version are you on currently?
The HBASE-4269 tried to restore the semantics changed in HBASE-4196 from
0.90.3 to 0.90.5. If we caught the exception we may skip records if the
timeout exceptions are skipped which is undesirable.
It might make sense to change Scanner Timeout to be derived from a
different exception but this will take a bit of digging and testing to
verify (DNRIOEx is used in about 40 places)
Jon.
On Fri, Feb 24, 2012 at 5:08 AM, Jan Lukavský
<ja...@firma.seznam.cz>wrote:
> Hi all,
>
> patch to HBASE-4269 removed handling of ScannerTimeoutException from
> TableRecordReader. Is there a reason for this? We are now seeing a lot of
> ScannerTimeoutExceptions in our jobs, which were previously handled. The
> problem is caused by ScannerTimeoutException being derived from
> DoNotRetryIOException (why's that?). Would it be safe to just derive this
> exception from IOException? And if not, should the TableRecordReader
> explictly handle this exception? The same problem may arise with
> LeaseException, which is also derived from DoNotRetryIOException. Both of
> these exception may have the same cause and in my understanding can be
> handled in the RecordReader without any harm. Or am I wrong?
>
> Thanks for answer,
> Jan
>
>
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com
Re: Why ScannerTimeoutException is not handled in TableInputFormat?
Posted by Jan Lukavský <ja...@firma.seznam.cz>.
Hi Vrushali,
the issue is https://issues.apache.org/jira/browse/HBASE-5757. There are
two patches, one is applicable against trunk, the other simple simple
version should be applicable against hbase-0.90.5. It should fix your
problem.
Jan
On 20.5.2012 23:26, Vrushali C wrote:
> Hi Jan
>
> I am facing the same issue as you did. If I change from hbase-0.90.3 to using
> hbase-0.90.5, I get a lot of Scanner timeout exceptions. Did you file the JIRA?
>
> Also, in my case, the setup method in my mapper takes a long time (like 10 mins)
> and by the time the map function is invoked, it throws a scanner timeout
> exception. My map is not executed even once.
> thanks
> Vrushali
>
>
>
>
> ________________________________
> From: Jan Lukavský<ja...@firma.seznam.cz>
> To: "user@hbase.apache.org"<us...@hbase.apache.org>
> Cc: Jonathan Hsieh<jo...@cloudera.com>; "hbase-user@hadoop.apache.org"<hb...@hadoop.apache.org>
> Sent: Monday, March 12, 2012 3:03 AM
> Subject: Re: Why ScannerTimeoutException is not handled in TableInputFormat?
>
> Hi Jon,
>
> sorry I forgot to include the details of our environment. We are using
> cdh3u3, which includes the patch (I think cdh3u2 did not). I went a bit
> through the patches for the two issues (HBASE-4196) and (HBASE-4269) and
> I think the problem is that the patch to HBASE-4196 did not change the
> sematics for *mapreduce* package. The change was related to *mapred
> *(the older API). We are using the new one and hence for us the
> semantics was in fact changed by HBASE-4269. IMO, the correct way of
> restoring the sematics would be to catch DNRIOEx only in the mapred
> implementation of the record reader.
>
> Shall I file another JIRA?
> Thanks,
> Jan
>
> On 7.3.2012 00:25, Jonathan Hsieh wrote:
>> Hi Jan,
>>
>> What version were you on before and what version are you on currently?
>>
>> The HBASE-4269 tried to restore the semantics changed in HBASE-4196 from
>> 0.90.3 to 0.90.5. If we caught the exception we may skip records if the
>> timeout exceptions are skipped which is undesirable.
>>
>> It might make sense to change Scanner Timeout to be derived from a
>> different exception but this will take a bit of digging and testing to
>> verify (DNRIOEx is used in about 40 places)
>>
>> Jon.
>>
>> On Fri, Feb 24, 2012 at 5:08 AM, Jan Lukavský
>> <ja...@firma.seznam.cz>wrote:
>>
>>> Hi all,
>>>
>>> patch to HBASE-4269 removed handling of ScannerTimeoutException from
>>> TableRecordReader. Is there a reason for this? We are now seeing a lot of
>>> ScannerTimeoutExceptions in our jobs, which were previously handled. The
>>> problem is caused by ScannerTimeoutException being derived from
>>> DoNotRetryIOException (why's that?). Would it be safe to just derive this
>>> exception from IOException? And if not, should the TableRecordReader
>>> explictly handle this exception? The same problem may arise with
>>> LeaseException, which is also derived from DoNotRetryIOException. Both of
>>> these exception may have the same cause and in my understanding can be
>>> handled in the RecordReader without any harm. Or am I wrong?
>>>
>>> Thanks for answer,
>>> Jan
>>>
>>>
Re: Why ScannerTimeoutException is not handled in TableInputFormat?
Posted by Vrushali C <vr...@ymail.com>.
Hi Jan
I am facing the same issue as you did. If I change from hbase-0.90.3 to using
hbase-0.90.5, I get a lot of Scanner timeout exceptions. Did you file the JIRA?
Also, in my case, the setup method in my mapper takes a long time (like 10 mins)
and by the time the map function is invoked, it throws a scanner timeout
exception. My map is not executed even once.
thanks
Vrushali
________________________________
From: Jan Lukavský <ja...@firma.seznam.cz>
To: "user@hbase.apache.org" <us...@hbase.apache.org>
Cc: Jonathan Hsieh <jo...@cloudera.com>; "hbase-user@hadoop.apache.org" <hb...@hadoop.apache.org>
Sent: Monday, March 12, 2012 3:03 AM
Subject: Re: Why ScannerTimeoutException is not handled in TableInputFormat?
Hi Jon,
sorry I forgot to include the details of our environment. We are using
cdh3u3, which includes the patch (I think cdh3u2 did not). I went a bit
through the patches for the two issues (HBASE-4196) and (HBASE-4269) and
I think the problem is that the patch to HBASE-4196 did not change the
sematics for *mapreduce* package. The change was related to *mapred
*(the older API). We are using the new one and hence for us the
semantics was in fact changed by HBASE-4269. IMO, the correct way of
restoring the sematics would be to catch DNRIOEx only in the mapred
implementation of the record reader.
Shall I file another JIRA?
Thanks,
Jan
On 7.3.2012 00:25, Jonathan Hsieh wrote:
> Hi Jan,
>
> What version were you on before and what version are you on currently?
>
> The HBASE-4269 tried to restore the semantics changed in HBASE-4196 from
> 0.90.3 to 0.90.5. If we caught the exception we may skip records if the
> timeout exceptions are skipped which is undesirable.
>
> It might make sense to change Scanner Timeout to be derived from a
> different exception but this will take a bit of digging and testing to
> verify (DNRIOEx is used in about 40 places)
>
> Jon.
>
> On Fri, Feb 24, 2012 at 5:08 AM, Jan Lukavský
> <ja...@firma.seznam.cz>wrote:
>
>> Hi all,
>>
>> patch to HBASE-4269 removed handling of ScannerTimeoutException from
>> TableRecordReader. Is there a reason for this? We are now seeing a lot of
>> ScannerTimeoutExceptions in our jobs, which were previously handled. The
>> problem is caused by ScannerTimeoutException being derived from
>> DoNotRetryIOException (why's that?). Would it be safe to just derive this
>> exception from IOException? And if not, should the TableRecordReader
>> explictly handle this exception? The same problem may arise with
>> LeaseException, which is also derived from DoNotRetryIOException. Both of
>> these exception may have the same cause and in my understanding can be
>> handled in the RecordReader without any harm. Or am I wrong?
>>
>> Thanks for answer,
>> Jan
>>
>>
>
Re: Why ScannerTimeoutException is not handled in TableInputFormat?
Posted by Jan Lukavský <ja...@firma.seznam.cz>.
Hi Jon,
sorry I forgot to include the details of our environment. We are using
cdh3u3, which includes the patch (I think cdh3u2 did not). I went a bit
through the patches for the two issues (HBASE-4196) and (HBASE-4269) and
I think the problem is that the patch to HBASE-4196 did not change the
sematics for *mapreduce* package. The change was related to *mapred
*(the older API). We are using the new one and hence for us the
semantics was in fact changed by HBASE-4269. IMO, the correct way of
restoring the sematics would be to catch DNRIOEx only in the mapred
implementation of the record reader.
Shall I file another JIRA?
Thanks,
Jan
On 7.3.2012 00:25, Jonathan Hsieh wrote:
> Hi Jan,
>
> What version were you on before and what version are you on currently?
>
> The HBASE-4269 tried to restore the semantics changed in HBASE-4196 from
> 0.90.3 to 0.90.5. If we caught the exception we may skip records if the
> timeout exceptions are skipped which is undesirable.
>
> It might make sense to change Scanner Timeout to be derived from a
> different exception but this will take a bit of digging and testing to
> verify (DNRIOEx is used in about 40 places)
>
> Jon.
>
> On Fri, Feb 24, 2012 at 5:08 AM, Jan Lukavský
> <ja...@firma.seznam.cz>wrote:
>
>> Hi all,
>>
>> patch to HBASE-4269 removed handling of ScannerTimeoutException from
>> TableRecordReader. Is there a reason for this? We are now seeing a lot of
>> ScannerTimeoutExceptions in our jobs, which were previously handled. The
>> problem is caused by ScannerTimeoutException being derived from
>> DoNotRetryIOException (why's that?). Would it be safe to just derive this
>> exception from IOException? And if not, should the TableRecordReader
>> explictly handle this exception? The same problem may arise with
>> LeaseException, which is also derived from DoNotRetryIOException. Both of
>> these exception may have the same cause and in my understanding can be
>> handled in the RecordReader without any harm. Or am I wrong?
>>
>> Thanks for answer,
>> Jan
>>
>>
>