You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "gaojinchao (Created) (JIRA)" <ji...@apache.org> on 2011/12/07 03:47:40 UTC
[jira] [Created] (HBASE-4970) Add a parameter to change
keepAliveTime of Htable thread pool.
Add a parameter to change keepAliveTime of Htable thread pool.
---------------------------------------------------------------
Key: HBASE-4970
URL: https://issues.apache.org/jira/browse/HBASE-4970
Project: HBase
Issue Type: Improvement
Components: client
Reporter: gaojinchao
Assignee: gaojinchao
Priority: Trivial
In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4970) Add a parameter to change
keepAliveTime of Htable thread pool.
Posted by "gaojinchao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
gaojinchao updated HBASE-4970:
------------------------------
Attachment: HBASE-4970_Branch90.patch
> Add a parameter to change keepAliveTime of Htable thread pool.
> ---------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
> Attachments: HBASE-4970_Branch90.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165565#comment-13165565 ]
Lars Hofhansl commented on HBASE-4970:
--------------------------------------
V2 looks good to me. +1
Is this good enough for you gaojinchao? It will put more burden on the application to manage the ExecutorService and HConnection.
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171968#comment-13171968 ]
Lars Hofhansl commented on HBASE-4970:
--------------------------------------
Ok... In that case, let's just add the new parameter everywhere. Sorry for leading you astray.
I'll commit the initial patch and the 0.92 and trunk patches later today, unless somebody objects.
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "gaojinchao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165995#comment-13165995 ]
gaojinchao commented on HBASE-4970:
-----------------------------------
In HBASE-4970_Branch90_V2.patch I added a parameter to change keepAliveTime as well.
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Add a parameter to change
keepAliveTime of Htable thread pool.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164123#comment-13164123 ]
Zhihong Yu commented on HBASE-4970:
-----------------------------------
@Jinchao:
Can you upload a patch ?
Thanks
> Add a parameter to change keepAliveTime of Htable thread pool.
> ---------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "gaojinchao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168310#comment-13168310 ]
gaojinchao commented on HBASE-4970:
-----------------------------------
@Lars
Thanks for your comments
My consideration:
1. For original users, Only change the configure and don't need manage so many resources.
2. For the new users, They can manage the ExecutorService and HConnection by themselves if they need.
I will make a patch for Trunk and 0.92. :)
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "gaojinchao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
gaojinchao updated HBASE-4970:
------------------------------
Attachment: HBASE-4970_Trunk_V2.patch
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "gaojinchao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
gaojinchao updated HBASE-4970:
------------------------------
Summary: Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch) (was: Add a parameter to change keepAliveTime of Htable thread pool.)
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "gaojinchao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165802#comment-13165802 ]
gaojinchao commented on HBASE-4970:
-----------------------------------
I also want to add a parameter to change keepAliveTime of Htable thread pool. so that clients can have more option
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Add a parameter so that
keepAliveTime of Htable thread pool can be changed
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172053#comment-13172053 ]
Zhihong Yu commented on HBASE-4970:
-----------------------------------
I think the likelihood of 0.90.5 RC going through is high.
So we should add 0.90.6 section to CHANGES.txt of 0.92 and TRUNK.
> Add a parameter so that keepAliveTime of Htable thread pool can be changed
> --------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Add a parameter to change
keepAliveTime of Htable thread pool.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164274#comment-13164274 ]
Ted Yu commented on HBASE-4970:
-------------------------------
I think there shouldn't be upper case letters in name of new config.
> Add a parameter to change keepAliveTime of Htable thread pool.
> ---------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
> Attachments: HBASE-4970_Branch90.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Add a parameter to change
keepAliveTime of Htable thread pool.
Posted by "gaojinchao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164364#comment-13164364 ]
gaojinchao commented on HBASE-4970:
-----------------------------------
Fixed Lars's comment.
@Lars
Please review firstly, I will test it in real cluster tomorrow.
> Add a parameter to change keepAliveTime of Htable thread pool.
> ---------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "gaojinchao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
gaojinchao updated HBASE-4970:
------------------------------
Attachment: HBASE-4970_Branch92_V2.patch
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "gaojinchao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171986#comment-13171986 ]
gaojinchao commented on HBASE-4970:
-----------------------------------
No problem, Thanks for your work!
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167670#comment-13167670 ]
stack commented on HBASE-4970:
------------------------------
lgtm
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Add a parameter to change
keepAliveTime of Htable thread pool.
Posted by "gaojinchao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164276#comment-13164276 ]
gaojinchao commented on HBASE-4970:
-----------------------------------
Sorry, I didn't see the Lars's comment. I will try to backport HBASE-4805.
> Add a parameter to change keepAliveTime of Htable thread pool.
> ---------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
> Attachments: HBASE-4970_Branch90.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Add a parameter so that
keepAliveTime of Htable thread pool can be changed
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172269#comment-13172269 ]
Hudson commented on HBASE-4970:
-------------------------------
Integrated in HBase-TRUNK-security #37 (See [https://builds.apache.org/job/HBase-TRUNK-security/37/])
HBASE-4970 Add a parameter so that keepAliveTime of Htable thread pool can be changed (gaojinchao)
larsh :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
> Add a parameter so that keepAliveTime of Htable thread pool can be changed
> --------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.94.0, 0.92.1, 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4970) Add a parameter to change
keepAliveTime of Htable thread pool.
Posted by "gaojinchao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
gaojinchao updated HBASE-4970:
------------------------------
Affects Version/s: 0.90.4
Fix Version/s: 0.90.5
> Add a parameter to change keepAliveTime of Htable thread pool.
> ---------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Add a parameter to change
keepAliveTime of Htable thread pool.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164580#comment-13164580 ]
Zhihong Yu commented on HBASE-4970:
-----------------------------------
Patch v2 is a backport and doesn't change keepAliveTime.
I feel we should address the needs of HTable users.
I am fine with the backport - we may want to modify the title of this JIRA accordingly.
> Add a parameter to change keepAliveTime of Htable thread pool.
> ---------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167308#comment-13167308 ]
ramkrishna.s.vasudevan commented on HBASE-4970:
-----------------------------------------------
+1 on patch..
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Add a parameter to change
keepAliveTime of Htable thread pool.
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164119#comment-13164119 ]
ramkrishna.s.vasudevan commented on HBASE-4970:
-----------------------------------------------
+1 on this change.
> Add a parameter to change keepAliveTime of Htable thread pool.
> ---------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "gaojinchao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171381#comment-13171381 ]
gaojinchao commented on HBASE-4970:
-----------------------------------
@Lars
Thanks for your review.
I tend to only modify the parameters.
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "gaojinchao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
gaojinchao updated HBASE-4970:
------------------------------
Attachment: HBASE-4970_Branch90_V2.patch
Thanks for Ted's and lars's reveiw.
Fixed all comments
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Add a parameter so that
keepAliveTime of Htable thread pool can be changed
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172050#comment-13172050 ]
Lars Hofhansl commented on HBASE-4970:
--------------------------------------
About to commit. What do we do with the CHANGES.txt file now. This is a 0.90 change, so I want to add it to the 0.90 section in trunk and 0.92 as well. They still have 0.90.5 marked as unreleased.
In 0.90 there is a 0.90.6 entry also marked as unreleased (in addition to 0.90.5, which is also marked as unreleased). Should I add a 0.90.6 row in 0.92 in trunk?
> Add a parameter so that keepAliveTime of Htable thread pool can be changed
> --------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167684#comment-13167684 ]
Lars Hofhansl commented on HBASE-4970:
--------------------------------------
Hmm... I am -0 on the extra config option. I really think that should be some slightly configurable default behavior, and if that does not fit the requirements, one should manage the ExecutorService separately rather than adding more and more config options.
I'm not opposed, though.
In either case, if we did the config option part, that part would need to be added to 0.92 and trunk as well.
If we did the config option part, does it still make sense to backport my 0.92/trunk change?
gaojinchao, if you had both options at your disposal (separate ExecutorService and your config option), which one would you use? We should do that one thing that you would use.
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource
consumption in HTable (backport HBASE-4805 to 0.90 branch)
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171162#comment-13171162 ]
Lars Hofhansl commented on HBASE-4970:
--------------------------------------
Thanks for the patch. I'm happy to commit this.
Let's add the new config option to all of 0.90, 0.92.1, and trunk.
Should we still port the slightly bigger change back to 0.90? Is that still needed for you gaojinchao, or would like to both change integrated in 0.90?
> Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4970) Add a parameter to change
keepAliveTime of Htable thread pool.
Posted by "gaojinchao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
gaojinchao updated HBASE-4970:
------------------------------
Attachment: HBASE-4970_Branch90_V1_trial.patch
> Add a parameter to change keepAliveTime of Htable thread pool.
> ---------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Add a parameter to change
keepAliveTime of Htable thread pool.
Posted by "gaojinchao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164127#comment-13164127 ]
gaojinchao commented on HBASE-4970:
-----------------------------------
ok, No problem.
> Add a parameter to change keepAliveTime of Htable thread pool.
> ---------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.5
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Add a parameter so that
keepAliveTime of Htable thread pool can be changed
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172171#comment-13172171 ]
Hudson commented on HBASE-4970:
-------------------------------
Integrated in HBase-0.92-security #44 (See [https://builds.apache.org/job/HBase-0.92-security/44/])
HBASE-4970 Add a parameter so that keepAliveTime of Htable thread pool can be changed (gaojinchao)
larsh :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HTable.java
> Add a parameter so that keepAliveTime of Htable thread pool can be changed
> --------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.94.0, 0.92.1, 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4970) Add a parameter so that keepAliveTime
of Htable thread pool can be changed
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-4970:
------------------------------
Summary: Add a parameter so that keepAliveTime of Htable thread pool can be changed (was: Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch))
Switching subject back.
> Add a parameter so that keepAliveTime of Htable thread pool can be changed
> --------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4970) Add a parameter so that
keepAliveTime of Htable thread pool can be changed
Posted by "Lars Hofhansl (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl resolved HBASE-4970.
----------------------------------
Resolution: Fixed
Fix Version/s: 0.92.1
0.94.0
Hadoop Flags: Reviewed
Committed to 0.90, 0.92, and trunk.
Thanks for the patch and your patience, gaojinchao.
> Add a parameter so that keepAliveTime of Htable thread pool can be changed
> --------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.94.0, 0.92.1, 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Add a parameter so that
keepAliveTime of Htable thread pool can be changed
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172237#comment-13172237 ]
Hudson commented on HBASE-4970:
-------------------------------
Integrated in HBase-TRUNK #2558 (See [https://builds.apache.org/job/HBase-TRUNK/2558/])
HBASE-4970 Add a parameter so that keepAliveTime of Htable thread pool can be changed (gaojinchao)
larsh :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
> Add a parameter so that keepAliveTime of Htable thread pool can be changed
> --------------------------------------------------------------------------
>
> Key: HBASE-4970
> URL: https://issues.apache.org/jira/browse/HBASE-4970
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.90.4
> Reporter: gaojinchao
> Assignee: gaojinchao
> Priority: Trivial
> Fix For: 0.94.0, 0.92.1, 0.90.6
>
> Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch, HBASE-4970_Branch92_V2.patch, HBASE-4970_Trunk_V2.patch
>
>
> In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down.
> Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]?
> You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released.
> So I think we can add a parameter to change keepAliveTime of Htable thread pool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira