You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Doğacan Güney (JIRA)" <ji...@apache.org> on 2009/07/08 13:45:15 UTC
[jira] Created: (HBASE-1627) TableInputFormatBase#nextKeyValue
catches the wrong exception
TableInputFormatBase#nextKeyValue catches the wrong exception
-------------------------------------------------------------
Key: HBASE-1627
URL: https://issues.apache.org/jira/browse/HBASE-1627
Project: Hadoop HBase
Issue Type: Bug
Affects Versions: 0.20.0
Reporter: Doğacan Güney
Priority: Minor
Attachments: tableinput.patch
TableInputFormatBase#nextKeyValue only catches UnknownScannerException from Scanner.next. However, scanner may throw other exceptions:
{code}
/* from HTable.ClientScanner#next */
try {
values = getConnection().getRegionServerWithRetries(callable);
} catch (IOException e) {
if (e instanceof UnknownScannerException &&
lastNext + scannerTimeout < System.currentTimeMillis()) {
ScannerTimeoutException ex = new ScannerTimeoutException();
ex.initCause(e);
throw ex;
}
throw e;
}
{code}
Is there any reason why TIFB does not catch ScannerTimeoutException?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1627) TableInputFormatBase#nextKeyValue
catches the wrong exception
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-1627.
--------------------------
Resolution: Fixed
Fix Version/s: 0.20.0
Assignee: Doğacan Güney
Hadoop Flags: [Reviewed]
Committed. Thanks for the patch Doğacan.
> TableInputFormatBase#nextKeyValue catches the wrong exception
> -------------------------------------------------------------
>
> Key: HBASE-1627
> URL: https://issues.apache.org/jira/browse/HBASE-1627
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Doğacan Güney
> Assignee: Doğacan Güney
> Priority: Minor
> Fix For: 0.20.0
>
> Attachments: tableinput.patch
>
>
> TableInputFormatBase#nextKeyValue only catches UnknownScannerException from Scanner.next. However, scanner may throw other exceptions:
> {code}
> /* from HTable.ClientScanner#next */
> try {
> values = getConnection().getRegionServerWithRetries(callable);
> } catch (IOException e) {
> if (e instanceof UnknownScannerException &&
> lastNext + scannerTimeout < System.currentTimeMillis()) {
> ScannerTimeoutException ex = new ScannerTimeoutException();
> ex.initCause(e);
> throw ex;
> }
> throw e;
> }
> {code}
> Is there any reason why TIFB does not catch ScannerTimeoutException?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1627) TableInputFormatBase#nextKeyValue
catches the wrong exception
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729774#action_12729774 ]
stack commented on HBASE-1627:
------------------------------
Thanks for persisting. Its I who is think. Yes, will retry once only since we redo the next inside the catch block. Let me commit. Good stuff.
> TableInputFormatBase#nextKeyValue catches the wrong exception
> -------------------------------------------------------------
>
> Key: HBASE-1627
> URL: https://issues.apache.org/jira/browse/HBASE-1627
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Doğacan Güney
> Priority: Minor
> Attachments: tableinput.patch
>
>
> TableInputFormatBase#nextKeyValue only catches UnknownScannerException from Scanner.next. However, scanner may throw other exceptions:
> {code}
> /* from HTable.ClientScanner#next */
> try {
> values = getConnection().getRegionServerWithRetries(callable);
> } catch (IOException e) {
> if (e instanceof UnknownScannerException &&
> lastNext + scannerTimeout < System.currentTimeMillis()) {
> ScannerTimeoutException ex = new ScannerTimeoutException();
> ex.initCause(e);
> throw ex;
> }
> throw e;
> }
> {code}
> Is there any reason why TIFB does not catch ScannerTimeoutException?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1627) TableInputFormatBase#nextKeyValue
catches the wrong exception
Posted by "Doğacan Güney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729766#action_12729766 ]
Doğacan Güney commented on HBASE-1627:
--------------------------------------
Sorry for being thick but I still don't understand. AFAICS we only retry once. So if the IOE that we capture is fatal then when we try again we will get the fatal exception again and the job will fail. So IIRC we ONLY try once.
> TableInputFormatBase#nextKeyValue catches the wrong exception
> -------------------------------------------------------------
>
> Key: HBASE-1627
> URL: https://issues.apache.org/jira/browse/HBASE-1627
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Doğacan Güney
> Priority: Minor
> Attachments: tableinput.patch
>
>
> TableInputFormatBase#nextKeyValue only catches UnknownScannerException from Scanner.next. However, scanner may throw other exceptions:
> {code}
> /* from HTable.ClientScanner#next */
> try {
> values = getConnection().getRegionServerWithRetries(callable);
> } catch (IOException e) {
> if (e instanceof UnknownScannerException &&
> lastNext + scannerTimeout < System.currentTimeMillis()) {
> ScannerTimeoutException ex = new ScannerTimeoutException();
> ex.initCause(e);
> throw ex;
> }
> throw e;
> }
> {code}
> Is there any reason why TIFB does not catch ScannerTimeoutException?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1627) TableInputFormatBase#nextKeyValue
catches the wrong exception
Posted by "Doğacan Güney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728920#action_12728920 ]
Doğacan Güney commented on HBASE-1627:
--------------------------------------
Yeah sure I can change it to STE but why is IOE being broad a problem? I mean, we catch the IOE and recreate the scanner, try next()ing again and fail if there is an exception again. Is there any scenario where this is harmful?
> TableInputFormatBase#nextKeyValue catches the wrong exception
> -------------------------------------------------------------
>
> Key: HBASE-1627
> URL: https://issues.apache.org/jira/browse/HBASE-1627
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Doğacan Güney
> Priority: Minor
> Attachments: tableinput.patch
>
>
> TableInputFormatBase#nextKeyValue only catches UnknownScannerException from Scanner.next. However, scanner may throw other exceptions:
> {code}
> /* from HTable.ClientScanner#next */
> try {
> values = getConnection().getRegionServerWithRetries(callable);
> } catch (IOException e) {
> if (e instanceof UnknownScannerException &&
> lastNext + scannerTimeout < System.currentTimeMillis()) {
> ScannerTimeoutException ex = new ScannerTimeoutException();
> ex.initCause(e);
> throw ex;
> }
> throw e;
> }
> {code}
> Is there any reason why TIFB does not catch ScannerTimeoutException?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1627) TableInputFormatBase#nextKeyValue
catches the wrong exception
Posted by "Doğacan Güney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doğacan Güney updated HBASE-1627:
---------------------------------
Attachment: tableinput.patch
A simple patch to change catch clause to catch all IOException-s. Are there any downsides to this? If an IOException is thrown, reset your scanner, try again and if it fails again your job fails. Makes sense to me :D
> TableInputFormatBase#nextKeyValue catches the wrong exception
> -------------------------------------------------------------
>
> Key: HBASE-1627
> URL: https://issues.apache.org/jira/browse/HBASE-1627
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Doğacan Güney
> Priority: Minor
> Attachments: tableinput.patch
>
>
> TableInputFormatBase#nextKeyValue only catches UnknownScannerException from Scanner.next. However, scanner may throw other exceptions:
> {code}
> /* from HTable.ClientScanner#next */
> try {
> values = getConnection().getRegionServerWithRetries(callable);
> } catch (IOException e) {
> if (e instanceof UnknownScannerException &&
> lastNext + scannerTimeout < System.currentTimeMillis()) {
> ScannerTimeoutException ex = new ScannerTimeoutException();
> ex.initCause(e);
> throw ex;
> }
> throw e;
> }
> {code}
> Is there any reason why TIFB does not catch ScannerTimeoutException?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1627) TableInputFormatBase#nextKeyValue
catches the wrong exception
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728855#action_12728855 ]
stack commented on HBASE-1627:
------------------------------
IOE seems overly broad (almost everything is IOE at its base).
Above looks like a bug in TIFB. It should have been changed to handle STE instead of USE. If you are good with making this change, I'll commit your patch changing your overly general IOE to STE?
> TableInputFormatBase#nextKeyValue catches the wrong exception
> -------------------------------------------------------------
>
> Key: HBASE-1627
> URL: https://issues.apache.org/jira/browse/HBASE-1627
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Doğacan Güney
> Priority: Minor
> Attachments: tableinput.patch
>
>
> TableInputFormatBase#nextKeyValue only catches UnknownScannerException from Scanner.next. However, scanner may throw other exceptions:
> {code}
> /* from HTable.ClientScanner#next */
> try {
> values = getConnection().getRegionServerWithRetries(callable);
> } catch (IOException e) {
> if (e instanceof UnknownScannerException &&
> lastNext + scannerTimeout < System.currentTimeMillis()) {
> ScannerTimeoutException ex = new ScannerTimeoutException();
> ex.initCause(e);
> throw ex;
> }
> throw e;
> }
> {code}
> Is there any reason why TIFB does not catch ScannerTimeoutException?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1627) TableInputFormatBase#nextKeyValue
catches the wrong exception
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729748#action_12729748 ]
stack commented on HBASE-1627:
------------------------------
Because near everything is an IOE at its base in hbase, its hard to distingush the retryable from the IOE that is fatal. My fear is that if we make the catch overly broad, it will retry for ever rather than fail. If OK with you, I'll commit STE instead of IOE? Thats a fix for a breakage. We can keep an eye on this and if other IOE subclasses that can bear retry, lets add them?
> TableInputFormatBase#nextKeyValue catches the wrong exception
> -------------------------------------------------------------
>
> Key: HBASE-1627
> URL: https://issues.apache.org/jira/browse/HBASE-1627
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Doğacan Güney
> Priority: Minor
> Attachments: tableinput.patch
>
>
> TableInputFormatBase#nextKeyValue only catches UnknownScannerException from Scanner.next. However, scanner may throw other exceptions:
> {code}
> /* from HTable.ClientScanner#next */
> try {
> values = getConnection().getRegionServerWithRetries(callable);
> } catch (IOException e) {
> if (e instanceof UnknownScannerException &&
> lastNext + scannerTimeout < System.currentTimeMillis()) {
> ScannerTimeoutException ex = new ScannerTimeoutException();
> ex.initCause(e);
> throw ex;
> }
> throw e;
> }
> {code}
> Is there any reason why TIFB does not catch ScannerTimeoutException?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.