You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Igor Yurinok (JIRA)" <ji...@apache.org> on 2012/10/05 12:12:03 UTC

[jira] [Created] (HBASE-6956) Do not return back to HTablePool closed connections

Igor Yurinok created HBASE-6956:
-----------------------------------

             Summary: Do not return back to HTablePool closed connections
                 Key: HBASE-6956
                 URL: https://issues.apache.org/jira/browse/HBASE-6956
             Project: HBase
          Issue Type: Bug
          Components: Client
    Affects Versions: 0.90.6
            Reporter: Igor Yurinok


Sometimes we see a lot of Exception about closed connections:
{code}
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
{code}

After investigation we assumed that it occurs because closed connection returns back into HTablePool. 

For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.

But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

Posted by "Micah Whitacre (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507634#comment-13507634 ] 

Micah Whitacre commented on HBASE-6956:
---------------------------------------

Igor, can you explain how you are detecting the connection is closed?  Your isClosed() method only checks to see if the HTable boolean value has been closed which is independent of the HConnection.  

Additionally why do you avoid trying to close the HConnection by informing the HConnectionManager that the connection is stale?

The last question is how does this solve the problem of evicting HTableInterface instances from the pool that are no longer valid?  Altering the HTableInterfaceFactory + HTableInterface implementation only solves the problem of new instances needed to be created.
                
> Do not return back to HTablePool closed connections
> ---------------------------------------------------
>
>                 Key: HBASE-6956
>                 URL: https://issues.apache.org/jira/browse/HBASE-6956
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.90.6
>            Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>  org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> {code}
> After investigation we assumed that it occurs because closed connection returns back into HTablePool. 
> For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

Posted by "Bryan Baugher (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507345#comment-13507345 ] 

Bryan Baugher commented on HBASE-6956:
--------------------------------------

We have also ran into this issue specifically during major failure of the cluster (i.e. hmaster, zookeeper, regionservers are down).

[~amuraru] did you manage to find a good way to flush a bad table from your pool when the connection becomes closed?
                
> Do not return back to HTablePool closed connections
> ---------------------------------------------------
>
>                 Key: HBASE-6956
>                 URL: https://issues.apache.org/jira/browse/HBASE-6956
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.90.6
>            Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>  org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> {code}
> After investigation we assumed that it occurs because closed connection returns back into HTablePool. 
> For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

Posted by "shengjie min (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487037#comment-13487037 ] 

shengjie min commented on HBASE-6956:
-------------------------------------

In my case I am using HTablePool, my understanding is the following:

so when the hbase query comes,

if pool is avaiable:
    I get a PooledHtable instance, once I am done with query, I call HtablePool.PooledHtable.close(), it will return PooledHtable instance to the pool.
if pool is out:
    I get a normal Htable instance, once I am done with query, I call Htable.close(), it will just release Htable instance.

I have no clue why I ended up having closed HConnectionImplementation in the pool.


                
> Do not return back to HTablePool closed connections
> ---------------------------------------------------
>
>                 Key: HBASE-6956
>                 URL: https://issues.apache.org/jira/browse/HBASE-6956
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.90.6
>            Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>  org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> {code}
> After investigation we assumed that it occurs because closed connection returns back into HTablePool. 
> For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

Posted by "Adrian Muraru (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491169#comment-13491169 ] 

Adrian Muraru commented on HBASE-6956:
--------------------------------------

I'm seeing a similar stack trace when using HTablePool:
{code}
java.lang.RuntimeException: java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@5c429be9 closed
        at org.apache.hadoop.hbase.client.HTableFactory.createHTableInterface(HTableFactory.java:38) ~[hbase-0.92.2-26.jar:0.92.2-26]
        at org.apache.hadoop.hbase.client.HTablePool.createHTable(HTablePool.java:268) ~[hbase-0.92.2-26.jar:0.92.2-26]
        at org.apache.hadoop.hbase.client.HTablePool.findOrCreateTable(HTablePool.java:198) ~[hbase-0.92.2-26.jar:0.92.2-26]
        at org.apache.hadoop.hbase.client.HTablePool.getTable(HTablePool.java:173) ~[hbase-0.92.2-26.jar:0.92.2-26]
{code}

The issue is not with a closed HTable instance returned back to the pool.
Instead, following my investigation it is caused by the shared *HConnection* instance being closed. 
Once you get into this state - the HTablePool#getTable() will always fail and never recover.

A possible fix is to drop the HConnection used by HTablePool#getTable() in case of an IOException. 
Not trivial though as it is created based on a private Configuration instance in the pool :)
{code}
this.connection = HConnectionManager.getConnection(conf)
{code}

A more brutal way of escaping this state is to drop the pool altogether a create a new one (not really doable in all cases depending on the usage)

I'll try to come up with a patch to clean up the faulty HConnection and keep the pool up.

                
> Do not return back to HTablePool closed connections
> ---------------------------------------------------
>
>                 Key: HBASE-6956
>                 URL: https://issues.apache.org/jira/browse/HBASE-6956
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.90.6
>            Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>  org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> {code}
> After investigation we assumed that it occurs because closed connection returns back into HTablePool. 
> For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487013#comment-13487013 ] 

Ted Yu commented on HBASE-6956:
-------------------------------

For putTable(), I see this javadoc:
{code}
  /**
   * This method is not needed anymore, clients should call
   * HTableInterface.close() rather than returning the tables to the pool
   * 
   * @param table
   *          the proxy table user got from pool
   * @deprecated
   */
  public void putTable(HTableInterface table) throws IOException {
{code}
@Igor, @Shengjie:
Can you tell us whether you followed the above suggestion ?

Thanks
                
> Do not return back to HTablePool closed connections
> ---------------------------------------------------
>
>                 Key: HBASE-6956
>                 URL: https://issues.apache.org/jira/browse/HBASE-6956
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.90.6
>            Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>  org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> {code}
> After investigation we assumed that it occurs because closed connection returns back into HTablePool. 
> For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

Posted by "Adrian Muraru (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491367#comment-13491367 ] 

Adrian Muraru commented on HBASE-6956:
--------------------------------------

[~shengjie_min] 0.92.2
                
> Do not return back to HTablePool closed connections
> ---------------------------------------------------
>
>                 Key: HBASE-6956
>                 URL: https://issues.apache.org/jira/browse/HBASE-6956
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.90.6
>            Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>  org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> {code}
> After investigation we assumed that it occurs because closed connection returns back into HTablePool. 
> For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

Posted by "shengjie min (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487006#comment-13487006 ] 

shengjie min commented on HBASE-6956:
-------------------------------------

I still have this issue even in 0.92.1. any updates?
                
> Do not return back to HTablePool closed connections
> ---------------------------------------------------
>
>                 Key: HBASE-6956
>                 URL: https://issues.apache.org/jira/browse/HBASE-6956
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.90.6
>            Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>  org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> {code}
> After investigation we assumed that it occurs because closed connection returns back into HTablePool. 
> For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

Posted by "stack (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507413#comment-13507413 ] 

stack commented on HBASE-6956:
------------------------------

This seems like an ugly enough oversight on our part.  Any chance of a patch to address it?  Could we change HConnection Interface to make it easier getting a new one [~amuraru]?  (I like the name of your class [~iyurinok])
                
> Do not return back to HTablePool closed connections
> ---------------------------------------------------
>
>                 Key: HBASE-6956
>                 URL: https://issues.apache.org/jira/browse/HBASE-6956
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.90.6
>            Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>  org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> {code}
> After investigation we assumed that it occurs because closed connection returns back into HTablePool. 
> For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

Posted by "shengjie min (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491365#comment-13491365 ] 

shengjie min commented on HBASE-6956:
-------------------------------------

[~amuraru]what version you are using?
                
> Do not return back to HTablePool closed connections
> ---------------------------------------------------
>
>                 Key: HBASE-6956
>                 URL: https://issues.apache.org/jira/browse/HBASE-6956
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.90.6
>            Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>  org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> {code}
> After investigation we assumed that it occurs because closed connection returns back into HTablePool. 
> For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

Posted by "Igor Yurinok (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507353#comment-13507353 ] 

Igor Yurinok commented on HBASE-6956:
-------------------------------------

We implemented our own HTableFactory and HTable which able to check whether connection closed:
{code}
public class HumbleHTableFactory implements HTableInterfaceFactory {

    @Override
    public HTableInterface createHTableInterface(Configuration config, byte[] tableName) {
        try {
            return new HumbleHTable(config, tableName);
        } catch (IOException ioe) {
            throw new RuntimeException(ioe);
        }
    }

    @Override
    public void releaseHTableInterface(HTableInterface table) {
        try {
            table.close();
        } catch (IOException ioe) {
            throw new RuntimeException(ioe);
        }
    }

}
{code}
{code}
public class HumbleHTable extends HTable {

    private static final Log log = LogFactory.getLog(HumbleHTable.class);

    public HumbleHTable(String tableName) throws IOException {
        super(tableName);
    }

    public HumbleHTable(byte[] tableName) throws IOException {
        super(tableName);
    }

    public HumbleHTable(Configuration conf, String tableName) throws IOException {
        super(conf, tableName);
    }

    public HumbleHTable(Configuration conf, byte[] tableName) throws IOException {
        super(conf, tableName);
    }

    /**
     * Harmless clean-up - HConnection isn't touched. Only the executor pool is shut down
     *
     * @throws IOException
     */
    @Override
    public void close() throws IOException {
        if (isClosed()) {
            return;
        }
        flushCommits();
        ExecutorService pool = getPool();
        if (pool != null) {
            pool.shutdown();
        }
        setClosed();
    }

    private boolean isClosed() {
        try {
            return getSuperField("closed").getBoolean(this);
        } catch (Exception e) {
            log.error(e);
            return false;
        }
    }

    private void setClosed() {
        try {
            getSuperField("closed").setBoolean(this, true);
        } catch (Exception e) {
            log.error(e);
        }
    }

    private ExecutorService getPool() {
        try {
            return (ExecutorService) getSuperField("pool").get(this);
        } catch (Exception e) {
            log.error(e);
            return null;
        }
    }

    /**
     * Loads stuff from the parent class via reflection
     */
    private Field getSuperField(String fieldName) throws NoSuchFieldException {
        Field field = HTable.class.getDeclaredField(fieldName);
        field.setAccessible(true);
        return field;
    }

}
{code}
                
> Do not return back to HTablePool closed connections
> ---------------------------------------------------
>
>                 Key: HBASE-6956
>                 URL: https://issues.apache.org/jira/browse/HBASE-6956
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.90.6
>            Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>  org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> {code}
> After investigation we assumed that it occurs because closed connection returns back into HTablePool. 
> For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

Posted by "Adrian Muraru (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507639#comment-13507639 ] 

Adrian Muraru commented on HBASE-6956:
--------------------------------------

There is an ongoing discussion on HBASE-6580 addressing this issue, a broader scope but would resolve the broken HConnection discussed here.
The agreement there was to deprecate HTablePool and rely on lightweight (read cheap) HTable (HBASE-4805) retrieved from a new HConnection#getTable()

I have a patch on HBASE-6580 that proposes a new HTableInterfaceFactory :https://issues.apache.org/jira/secure/attachment/12554094/HBASE-6580_v1.patch
and can be used as a custom factory when creating the HTablePool 
{noformat}org.apache.hadoop.hbase.client.HTablePool.HTablePool(Configuration config, int maxSize, HTableInterfaceFactory tableFactory){noformat}
                
> Do not return back to HTablePool closed connections
> ---------------------------------------------------
>
>                 Key: HBASE-6956
>                 URL: https://issues.apache.org/jira/browse/HBASE-6956
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.90.6
>            Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>  org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> {code}
> After investigation we assumed that it occurs because closed connection returns back into HTablePool. 
> For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

Posted by "stack (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472779#comment-13472779 ] 

stack commented on HBASE-6956:
------------------------------

You have a patch Igor?
                
> Do not return back to HTablePool closed connections
> ---------------------------------------------------
>
>                 Key: HBASE-6956
>                 URL: https://issues.apache.org/jira/browse/HBASE-6956
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.90.6
>            Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>  org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> org.apache.hadoop.hbase.client.ClosedConnectionException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068 closed
> {code}
> After investigation we assumed that it occurs because closed connection returns back into HTablePool. 
> For our opinion best solution is  check whether the table is closed in method HTablePool.putTable and if true don't add it into the queue and release such HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira