You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Benoit Sigoure (Created) (JIRA)" <ji...@apache.org> on 2011/11/06 05:00:51 UTC
[jira] [Created] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Don't create an unnecessary LinkedList when evicting from the BlockCache
------------------------------------------------------------------------
Key: HBASE-4752
URL: https://issues.apache.org/jira/browse/HBASE-4752
Project: HBase
Issue Type: Improvement
Components: performance, regionserver
Affects Versions: 0.90.4
Reporter: Benoit Sigoure
Assignee: Benoit Sigoure
Priority: Minor
When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Jonathan Gray (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145693#comment-13145693 ]
Jonathan Gray commented on HBASE-4752:
--------------------------------------
Sorry I didn't chime in earlier, been traveling.
I'm actually -1 on this change at the moment because of the introduction of a Google class (now the block cache has this external dependency). This class is actually used by other projects outside of HBase, so I'd hate to put in an unnecessary dependency. Is there additional value we get out of using the MinMaxPQ? We save a LinkedList allocation?
As for the change in behavior, I'm not sure I follow. Seems like nothing actually changes? (whether the PQ is cleared or not doesn't really matter, behavior-wise?)
The way I'm reading the code, it seems like we could actually just remove the LL completely and leave in place the regular PQ? CachedBlock takes care of the sort order, no?
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4752:
--------------------------
Fix Version/s: 0.94.0
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145004#comment-13145004 ]
Ted Yu commented on HBASE-4752:
-------------------------------
The gist of CachedBlockQueue now lies in its add() method which does heap size accounting.
I think we should keep this class.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Li Pi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145244#comment-13145244 ]
Li Pi commented on HBASE-4752:
------------------------------
This should work. I agree we should drain until we're below minsize, this seems to align with the way the code has been written.
+1 from me.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145703#comment-13145703 ]
Ted Yu commented on HBASE-4752:
-------------------------------
MinMaxPriorityQueue is already used by load balancer. So the dependency is not new.
If we only use the regular PriorityQueue, in LruBlockCache.free(), how do we free the least recently used CachedBlock's ?
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4752:
--------------------------
Status: Patch Available (was: Open)
Patch testing trunk v2.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145089#comment-13145089 ]
Lars Hofhansl commented on HBASE-4752:
--------------------------------------
+1 still on v2
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4752:
--------------------------
Status: Open (was: Patch Available)
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145039#comment-13145039 ]
Lars Hofhansl commented on HBASE-4752:
--------------------------------------
One part that is still different in the old implementation LruBlockCache.free() always drained the entire queue. Now we only drain until we have evicted enough.
Not sure whether that was intentional or not, but to be functionally equivalent we should do this:
{code}
while(...) {
...
if (freedBytes >= toFree) {
queue.clear();
return freedBytes;
}
}
queue.clear();
...
{code}
Or similar (or clear() in a finally block).
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149394#comment-13149394 ]
Hudson commented on HBASE-4752:
-------------------------------
Integrated in HBase-TRUNK #2434 (See [https://builds.apache.org/job/HBase-TRUNK/2434/])
HBASE-4752 Don't create an unnecessary LinkedList when evicting from the BlockCache
tedyu :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/CachedBlockQueue.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCachedBlockQueue.java
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144907#comment-13144907 ]
Ted Yu commented on HBASE-4752:
-------------------------------
@Benoit:
HadoopQA uses -p0 to apply patches.
Please regenerate patch and submit again.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Benoit Sigoure (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoit Sigoure updated HBASE-4752:
----------------------------------
Attachment: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch
HBase 0.90 doesn't compile for me right now, I get a ton of Avro related errors. I haven't tested this patch, but I think it should work. :)
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144918#comment-13144918 ]
Lars Hofhansl commented on HBASE-4752:
--------------------------------------
Doesn't this change the behavior? And I mean not just the API.
Before the change the list was sorted in reverse (addFirst on the LinkedList), so that the least recently used blocks are removed first (until enough was freed).
With the change the most recently used blocks are removed first defeating the purpose of an LRU cache.
Unless I am missing something, this does not look right.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145034#comment-13145034 ]
Ted Yu commented on HBASE-4752:
-------------------------------
I ran TestMasterObserver#testRegionTransitionOperations and it passed.
Test failures didn't seem to be related to the patch in this JIRA.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145054#comment-13145054 ]
Lars Hofhansl commented on HBASE-4752:
--------------------------------------
Hard to say, but i would agree. Maybe jgray can chime in?
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144940#comment-13144940 ]
Lars Hofhansl commented on HBASE-4752:
--------------------------------------
Looking at MinMaxPriorityQueue... can we just get rid of CacheBlockQueue? It only exists to provide the entries in reverse it seems.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144908#comment-13144908 ]
Ted Yu commented on HBASE-4752:
-------------------------------
+1 on patch.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu reassigned HBASE-4752:
-----------------------------
Assignee: Ted Yu (was: Benoit Sigoure)
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4752:
--------------------------
Attachment: 4752-trunk-v2.txt
Added initial size for MinMaxPriorityQueue.
Modified TestCachedBlockQueue to reflect its original intention. Also simplified the assertions with a loop.
TestCachedBlockQueue passes.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149355#comment-13149355 ]
stack commented on HBASE-4752:
------------------------------
@Ted Agreed; lets just use it since its already in CLASSPATH
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144937#comment-13144937 ]
Lars Hofhansl commented on HBASE-4752:
--------------------------------------
Oh? ... Nope, was still referring to Beniot's patch. I missed your new patch.
I am not familiar with MinMaxPriorityQueue, but if it does what its name suggests the patch looks good to me.
Wanna set the initial size (with a builder)? Or remove the initial size calculation?
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4752:
--------------------------
Status: Patch Available (was: Open)
The following tests passed based on patch for TRUNK:
{code}
src/test/java/org/apache/hadoop/hbase/io/hfile/TestBlockCacheColumnFamilySummary.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFileBlockCacheSummary.java
{code}
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144957#comment-13144957 ]
Hadoop QA commented on HBASE-4752:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502646/4752-trunk.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -164 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 49 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMasterFailover
org.apache.hadoop.hbase.io.hfile.TestCachedBlockQueue
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/194//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/194//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/194//console
This message is automatically generated.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145046#comment-13145046 ]
Ted Yu commented on HBASE-4752:
-------------------------------
bq. Now we only drain until we have evicted enough.
I would say this should be the correct behavior.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149367#comment-13149367 ]
Ted Yu commented on HBASE-4752:
-------------------------------
Integrated patch v2 into TRUNK.
Thanks for the finding, Benoit.
Thanks for the reviews Lars, Li, Benoit, Jonathan and Stack.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4752:
--------------------------
Status: Open (was: Patch Available)
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4752:
--------------------------
Resolution: Fixed
Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change)
Status: Resolved (was: Patch Available)
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145025#comment-13145025 ]
Hadoop QA commented on HBASE-4752:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502664/4752-trunk-v2.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -164 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 48 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.master.TestMasterFailover
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/197//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/197//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/197//console
This message is automatically generated.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146769#comment-13146769 ]
Ted Yu commented on HBASE-4752:
-------------------------------
Here are some statistics about how widely HBase codebase uses class(es) from Guava:
{code}
zhihyu$ find src/main -name '*.java' -exec grep Precondition {} \; -print | head
import com.google.common.base.Preconditions;
Preconditions.checkArgument(n >= 0, "limit be positive %s", n);
Preconditions.checkArgument(filterArguments.size() == 1,
src/main/java/org/apache/hadoop/hbase/filter/ColumnCountGetFilter.java
import com.google.common.base.Preconditions;
Preconditions.checkArgument(limit >= 0, "limit must be positive %s", limit);
Preconditions.checkArgument(offset >= 0, "offset must be positive %s", offset);
Preconditions.checkArgument(filterArguments.size() == 2,
src/main/java/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.java
import com.google.common.base.Preconditions;
zhihyu$ find src/main -name '*.java' -exec grep Precondition {} \; | wc
96 270 5149
{code}
If usage of Preconditions is recommended, we shouldn't worry about using MinMaxPriorityQueue which already appears in our codebase.
I am also open to finding other implementations for double-ended queue.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4752:
--------------------------
Attachment: 4752-trunk.txt
How about changing CachedBlockQueue.queue to double-ended queue ?
In BlockBucket.free(), pollLast() would be called.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144934#comment-13144934 ]
Ted Yu commented on HBASE-4752:
-------------------------------
@Lars:
If you were talking about my patch, I assume you have noticed the following:
{code}
- for(CachedBlock cb: blocks) {
+ while ((cb = queue.pollLast()) != null) {
{code}
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Benoit Sigoure (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoit Sigoure updated HBASE-4752:
----------------------------------
Status: Patch Available (was: Open)
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144929#comment-13144929 ]
Lars Hofhansl commented on HBASE-4752:
--------------------------------------
Hmm... The blocks are still sorted in the PriorityQueue. Could use a TreeSet with a comparator, but the properties will be different.
The least optimization one could do is using an ArrayList and fill it from the back (the size of the queue is known, and since PriorityQueue is not threadsafe and there is no thread synchronization in CachedBlockQueue, I assume there is not multithreaded access here).
Strange that those tests pass, though. You do agree that with the patch the entries would be evicted in the opposite order than before, no?
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Benoit Sigoure (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145127#comment-13145127 ]
Benoit Sigoure commented on HBASE-4752:
---------------------------------------
+1 too. Sorry my initial patch was getting entries in the reverse order. As I said, I couldn't test it because branch 0.90 didn't even compile for me.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Ted Yu
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch, 4752-trunk-v2.txt, 4752-trunk.txt
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4752) Don't create an unnecessary
LinkedList when evicting from the BlockCache
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144903#comment-13144903 ]
Hadoop QA commented on HBASE-4752:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502637/0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/191//console
This message is automatically generated.
> Don't create an unnecessary LinkedList when evicting from the BlockCache
> ------------------------------------------------------------------------
>
> Key: HBASE-4752
> URL: https://issues.apache.org/jira/browse/HBASE-4752
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Affects Versions: 0.90.4
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-HBASE-4752-Don-t-create-an-unnecessary-LinkedList-wh.patch
>
>
> When evicting from the BlockCache, the code creates a LinkedList containing every single block sorted by access time. This list is created from a PriorityQueue. I don't believe it is necessary, as the PriorityQueue can be used directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira