You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Mikhail Bautin (JIRA)" <ji...@apache.org> on 2012/10/04 10:27:07 UTC
[jira] [Created] (HBASE-6943) [89-fbDo not catch certain exceptions
trying
Mikhail Bautin created HBASE-6943:
-------------------------------------
Summary: [89-fbDo not catch certain exceptions trying
Key: HBASE-6943
URL: https://issues.apache.org/jira/browse/HBASE-6943
Project: HBase
Issue Type: Bug
Reporter: Mikhail Bautin
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6943) [89-fb] Do not catch certain
exceptions trying to get an RS connection
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469799#comment-13469799 ]
Phabricator commented on HBASE-6943:
------------------------------------
mbautin has closed the revision "[jira] [HBASE-6943] [89-fb] Do not catch certain exceptions trying to get an RS connection".
CHANGED PRIOR TO COMMIT
https://reviews.facebook.net/D5877?vs=19413&id=19425#differential-review-toc
REVISION DETAIL
https://reviews.facebook.net/D5877
COMMIT
https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH1394307
To: Kannan, Liyin, Karthik, JIRA, mbautin
Cc: avf, adela, pritamdamania, aaiyer, nspiegelberg, amirshim, mycnyc
> [89-fb] Do not catch certain exceptions trying to get an RS connection
> ----------------------------------------------------------------------
>
> Key: HBASE-6943
> URL: https://issues.apache.org/jira/browse/HBASE-6943
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Attachments: D5877.1.patch, D5877.2.patch, D5877.3.patch, D5877.4.patch
>
>
> When getting a regionserver connection in 0.89-fb in HBaseClient, we catch all types of Throwable. I have observed a real case when the client looked stuck. On debugging it turned out that a NoSuchMethodError was thrown and caught, leaving the connection in an inconsistent state (initialized socket but null streams). All following attempts resulted in NPEs that were also caught, and no errors were logged. From the user's perspective the client was just stuck. The root cause was the absence of a required jar (hence the NoSuchMethodError) but it was not reported properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6943) [89-fb] Do not catch certain
exceptions trying to get an RS connection
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-6943:
-------------------------------
Attachment: D5877.2.patch
mbautin updated the revision "[jira] [HBASE-6943] [89-fb] Do not catch certain exceptions trying to get an RS connection".
Reviewers: Kannan, Liyin, Karthik, JIRA
Addressing Amit's feedback
REVISION DETAIL
https://reviews.facebook.net/D5877
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
To: Kannan, Liyin, Karthik, JIRA, mbautin
Cc: avf, adela, pritamdamania, aaiyer, nspiegelberg, amirshim, mycnyc
> [89-fb] Do not catch certain exceptions trying to get an RS connection
> ----------------------------------------------------------------------
>
> Key: HBASE-6943
> URL: https://issues.apache.org/jira/browse/HBASE-6943
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Attachments: D5877.1.patch, D5877.2.patch
>
>
> When getting a regionserver connection in 0.89-fb in HBaseClient, we catch all types of Throwable. I have observed a real case when the client looked stuck. On debugging it turned out that a NoSuchMethodError was thrown and caught, leaving the connection in an inconsistent state (initialized socket but null streams). All following attempts resulted in NPEs that were also caught, and no errors were logged. From the user's perspective the client was just stuck. The root cause was the absence of a required jar (hence the NoSuchMethodError) but it was not reported properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6943) [89-fb] Do not catch certain
exceptions trying to get an RS connection
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469607#comment-13469607 ]
Phabricator commented on HBASE-6943:
------------------------------------
aaiyer has commented on the revision "[jira] [HBASE-6943] [89-fb] Do not catch certain exceptions trying to get an RS connection".
INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java:1383 Do we need to do this in other places as well? -- getRegionServer with/without retries?
Perhaps a good place to do this is to do it in translateException. If we see that the throwable is one of these bad ones. we can just throw ie again.
REVISION DETAIL
https://reviews.facebook.net/D5877
BRANCH
stuck_client_v4
To: Kannan, Liyin, Karthik, JIRA, mbautin
Cc: avf, adela, pritamdamania, aaiyer, nspiegelberg, amirshim, mycnyc
> [89-fb] Do not catch certain exceptions trying to get an RS connection
> ----------------------------------------------------------------------
>
> Key: HBASE-6943
> URL: https://issues.apache.org/jira/browse/HBASE-6943
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Attachments: D5877.1.patch
>
>
> When getting a regionserver connection in 0.89-fb in HBaseClient, we catch all types of Throwable. I have observed a real case when the client looked stuck. On debugging it turned out that a NoSuchMethodError was thrown and caught, leaving the connection in an inconsistent state (initialized socket but null streams). All following attempts resulted in NPEs that were also caught, and no errors were logged. From the user's perspective the client was just stuck. The root cause was the absence of a required jar (hence the NoSuchMethodError) but it was not reported properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6943) [89-fb] Do not catch certain
exceptions trying to get an RS connection
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-6943:
-------------------------------
Attachment: D5877.1.patch
mbautin requested code review of "[jira] [HBASE-6943] [89-fb] Do not catch certain exceptions trying to get an RS connection".
Reviewers: Kannan, Liyin, Karthik, JIRA
When getting a regionserver connection in 0.89-fb in HBaseClient, we catch all types of Throwable. I have observed a real case when the client looked stuck. On debugging it turned out that a NoSuchMethodError was thrown and caught, leaving the connection in an inconsistent state (initialized socket but null streams). All following attempts resulted in NPEs that were also caught, and no errors were logged. From the user's perspective the client was just stuck. The root cause was the absence of a required jar (hence the NoSuchMethodError) but it was not reported properly.
TEST PLAN
Run a client with the same configuration as before and verify it does not get stuck.
REVISION DETAIL
https://reviews.facebook.net/D5877
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/13929/
To: Kannan, Liyin, Karthik, JIRA, mbautin
> [89-fb] Do not catch certain exceptions trying to get an RS connection
> ----------------------------------------------------------------------
>
> Key: HBASE-6943
> URL: https://issues.apache.org/jira/browse/HBASE-6943
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Attachments: D5877.1.patch
>
>
> When getting a regionserver connection in 0.89-fb in HBaseClient, we catch all types of Throwable. I have observed a real case when the client looked stuck. On debugging it turned out that a NoSuchMethodError was thrown and caught, leaving the connection in an inconsistent state (initialized socket but null streams). All following attempts resulted in NPEs that were also caught, and no errors were logged. From the user's perspective the client was just stuck. The root cause was the absence of a required jar (hence the NoSuchMethodError) but it was not reported properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6943) [89-fb] Do not catch certain
exceptions trying to get an RS connection
Posted by "Mikhail Bautin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mikhail Bautin updated HBASE-6943:
----------------------------------
Description: When getting a regionserver connection in 0.89-fb in HBaseClient, we catch all types of Throwable. I have observed a real case when the client looked stuck. On debugging it turned out that a NoSuchMethodError was thrown and caught, leaving the connection in an inconsistent state (initialized socket but null streams). All following attempts resulted in NPEs that were also caught, and no errors were logged. From the user's perspective the client was just stuck. The root cause was the absence of a required jar (hence the NoSuchMethodError) but it was not reported properly.
Summary: [89-fb] Do not catch certain exceptions trying to get an RS connection (was: [89-fbDo not catch certain exceptions trying )
> [89-fb] Do not catch certain exceptions trying to get an RS connection
> ----------------------------------------------------------------------
>
> Key: HBASE-6943
> URL: https://issues.apache.org/jira/browse/HBASE-6943
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
>
> When getting a regionserver connection in 0.89-fb in HBaseClient, we catch all types of Throwable. I have observed a real case when the client looked stuck. On debugging it turned out that a NoSuchMethodError was thrown and caught, leaving the connection in an inconsistent state (initialized socket but null streams). All following attempts resulted in NPEs that were also caught, and no errors were logged. From the user's perspective the client was just stuck. The root cause was the absence of a required jar (hence the NoSuchMethodError) but it was not reported properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6943) [89-fb] Do not catch certain
exceptions trying to get an RS connection
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469535#comment-13469535 ]
Phabricator commented on HBASE-6943:
------------------------------------
Kannan has added CCs to the revision "[jira] [HBASE-6943] [89-fb] Do not catch certain exceptions trying to get an RS connection".
Added CCs: avf, adela, pritamdamania, aaiyer, nspiegelberg, amirshim, mycnyc
Let's try to cc individually until we can figure out how to more easily email the group automatically.
REVISION DETAIL
https://reviews.facebook.net/D5877
To: Kannan, Liyin, Karthik, JIRA, mbautin
Cc: avf, adela, pritamdamania, aaiyer, nspiegelberg, amirshim, mycnyc
> [89-fb] Do not catch certain exceptions trying to get an RS connection
> ----------------------------------------------------------------------
>
> Key: HBASE-6943
> URL: https://issues.apache.org/jira/browse/HBASE-6943
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Attachments: D5877.1.patch
>
>
> When getting a regionserver connection in 0.89-fb in HBaseClient, we catch all types of Throwable. I have observed a real case when the client looked stuck. On debugging it turned out that a NoSuchMethodError was thrown and caught, leaving the connection in an inconsistent state (initialized socket but null streams). All following attempts resulted in NPEs that were also caught, and no errors were logged. From the user's perspective the client was just stuck. The root cause was the absence of a required jar (hence the NoSuchMethodError) but it was not reported properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6943) [89-fb] Do not catch certain
exceptions trying to get an RS connection
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469537#comment-13469537 ]
Phabricator commented on HBASE-6943:
------------------------------------
Kannan has accepted the revision "[jira] [HBASE-6943] [89-fb] Do not catch certain exceptions trying to get an RS connection".
lgtm! good catch...
REVISION DETAIL
https://reviews.facebook.net/D5877
BRANCH
stuck_client_v4
To: Kannan, Liyin, Karthik, JIRA, mbautin
Cc: avf, adela, pritamdamania, aaiyer, nspiegelberg, amirshim, mycnyc
> [89-fb] Do not catch certain exceptions trying to get an RS connection
> ----------------------------------------------------------------------
>
> Key: HBASE-6943
> URL: https://issues.apache.org/jira/browse/HBASE-6943
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Attachments: D5877.1.patch
>
>
> When getting a regionserver connection in 0.89-fb in HBaseClient, we catch all types of Throwable. I have observed a real case when the client looked stuck. On debugging it turned out that a NoSuchMethodError was thrown and caught, leaving the connection in an inconsistent state (initialized socket but null streams). All following attempts resulted in NPEs that were also caught, and no errors were logged. From the user's perspective the client was just stuck. The root cause was the absence of a required jar (hence the NoSuchMethodError) but it was not reported properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6943) [89-fb] Do not catch certain
exceptions trying to get an RS connection
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-6943:
-------------------------------
Attachment: D5877.4.patch
mbautin updated the revision "[jira] [HBASE-6943] [89-fb] Do not catch certain exceptions trying to get an RS connection".
Reviewers: Kannan, Liyin, Karthik, JIRA
Removing some unnecessary changes.
REVISION DETAIL
https://reviews.facebook.net/D5877
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
To: Kannan, Liyin, Karthik, JIRA, mbautin
Cc: avf, adela, pritamdamania, aaiyer, nspiegelberg, amirshim, mycnyc
> [89-fb] Do not catch certain exceptions trying to get an RS connection
> ----------------------------------------------------------------------
>
> Key: HBASE-6943
> URL: https://issues.apache.org/jira/browse/HBASE-6943
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Attachments: D5877.1.patch, D5877.2.patch, D5877.3.patch, D5877.4.patch
>
>
> When getting a regionserver connection in 0.89-fb in HBaseClient, we catch all types of Throwable. I have observed a real case when the client looked stuck. On debugging it turned out that a NoSuchMethodError was thrown and caught, leaving the connection in an inconsistent state (initialized socket but null streams). All following attempts resulted in NPEs that were also caught, and no errors were logged. From the user's perspective the client was just stuck. The root cause was the absence of a required jar (hence the NoSuchMethodError) but it was not reported properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6943) [89-fb] Do not catch certain
exceptions trying to get an RS connection
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-6943:
-------------------------------
Attachment: D5877.3.patch
mbautin updated the revision "[jira] [HBASE-6943] [89-fb] Do not catch certain exceptions trying to get an RS connection".
Reviewers: Kannan, Liyin, Karthik, JIRA
Catching an arbitrary throwable and wrapping it with an IOException in setupIOstreams.
REVISION DETAIL
https://reviews.facebook.net/D5877
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
To: Kannan, Liyin, Karthik, JIRA, mbautin
Cc: avf, adela, pritamdamania, aaiyer, nspiegelberg, amirshim, mycnyc
> [89-fb] Do not catch certain exceptions trying to get an RS connection
> ----------------------------------------------------------------------
>
> Key: HBASE-6943
> URL: https://issues.apache.org/jira/browse/HBASE-6943
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Attachments: D5877.1.patch, D5877.2.patch, D5877.3.patch
>
>
> When getting a regionserver connection in 0.89-fb in HBaseClient, we catch all types of Throwable. I have observed a real case when the client looked stuck. On debugging it turned out that a NoSuchMethodError was thrown and caught, leaving the connection in an inconsistent state (initialized socket but null streams). All following attempts resulted in NPEs that were also caught, and no errors were logged. From the user's perspective the client was just stuck. The root cause was the absence of a required jar (hence the NoSuchMethodError) but it was not reported properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira