You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Doug Meil (JIRA)" <ji...@apache.org> on 2011/08/13 23:43:27 UTC
[jira] [Created] (HBASE-4199) blockCache summary - backend
blockCache summary - backend
----------------------------
Key: HBASE-4199
URL: https://issues.apache.org/jira/browse/HBASE-4199
Project: HBase
Issue Type: Sub-task
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089104#comment-13089104 ]
Jean-Daniel Cryans commented on HBASE-4199:
-------------------------------------------
Still +1, good work!
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: 4199.v5, java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch, java_HBASE_4199_v4.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Meil updated HBASE-4199:
-----------------------------
Attachment: java_HBASE_4199_v4.patch
Ok, v4 patch with requested changes. The only part I feel like I'm between a rock and a hard place is the line-breaks. One one hand, you guys are saying "don't do long lines" - fair enough. But on the other hand you don't like the line-breaking, and I don't see how the way I'm breaking is much different from the way anybody else is doing it.
All the big stuff is taken care of though. BlockCacheSummaryEntry renamed to BlockCacheColumnFamilySummary. Method renamed to getBlockCacheColumnFamilySummary as well.
Expanded Javadoc to include more verbiage on the cost of method (e.g., will do FS scan, etc.) and other places as mentioned.
Fingers crossed!
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch, java_HBASE_4199_v4.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087302#comment-13087302 ]
stack commented on HBASE-4199:
------------------------------
Here's some feedback.
This is very nice funcionality. Thanks for adding it.
The constructor that takes no args has empty javadoc. Remove it or fill it in (perhaps note this method is for deserializing only)
No need to assign "" to table and columnfamily on declaration? Let a NPE out if this class ill-initialized
You are inconsistent w/ lines separating methods. Looks sloppy (Usually we put a single line between methods)
Whats up with this BlockCacheSummaryEntry? Its a summary or is it an entry? If a summary, its odd that I can set things like blocks and heapSize post construction and even table and columnfamily. Maybe this should be an immmutable object where you pass in all variables on construction?
I suppose its a summary for the passed table and cf? Hmm. Thats what you say in the class javadoc so strike the above rumination on class name (the whether it should be immutable comment stands).
+ this.heapSize = this.heapSize + heapSize; is usually written this.heapSize += heapSize... but no biggie.
In the below
{code}
+ public void readFields(DataInput arg0) throws IOException {
{code}
'arg0' is a bad name for a param (Same on the write).
In hashCode you do:
{code}
+ final int prime = 31;
+ int result = 1;
+ result = prime * result
...
{code}
Why the 'int result = 1' at all? Why not 'int result = prime + ....'
I like your equals implementation.
In your toString, why include name of this class?
Should we throw exception if path does not have four elements in it?
Would suggest you add the stuff you have in comments down in createFromStoreFilePath up into the javadoc; would help clarify the kinda path we are expecting.
You are inconsistent with spacings below:
{code}
+ String table = s[ s.length - 4]; // 4th from the end
+ String cf = s[ s.length - 2]; // 2nd from the end
{code}
Be careful w/ line lengths. Usually < 80.
How often is getBlockCacheSummary called? For each invocation we do inspection of all under hbase.rootdir? This seems like a pretty costly operation.
Looking at getTableStoreFilePathMap, we should make a storefiles filter. Seems like it'd get used more than once (but thats for another JIRA).
This comment looks wrong? " Under each of these, should be one file only."
Why do this:
{code}
+ BlockCacheSummaryEntry e2 = new BlockCacheSummaryEntry();
+ e2.setTable(e.getTable());
+ e2.setColumnFamily(e.getColumnFamily());
+ return e2;
{code}
Why not do return BlockCacheSummaryEntry(e.getTable(), e.getColumnFamily());
The below should be a Set?
{code}
+ Map<BlockCacheSummaryEntry, BlockCacheSummaryEntry> bcs =
+ new HashMap<BlockCacheSummaryEntry, BlockCacheSummaryEntry>();
{code}
Or, strike that... I see how you are using it. Its a little unusual that equality is on only two of the datamembers. Nothing wrong with this but I'd call this out in the class comment for this class, that two instances are compared the same if table+cf agree (though counts differ).
I like doing if (s.length <= 0) continue rather than below.
{code}
+ if (s.length > 0) {
{code}
Advantage of former is that save an indentation. Similar for the if (path != null).. I'd rather do if (path == null) continue;
In javadoc should you should say that getBlockCacheSummary returns a list sorted by table name + cf (or you don't want to have sort in contract for this method?)
Why do this:
{code}
+ BlockCacheSummaryEntry[] ar = new BlockCacheSummaryEntry[list.size()];
+ for (int i = 0; i < list.size(); i++) {
+ ar[i]=list.get(i);
+ }
+ return ar;
{code}
Why not return the List? (In future, doing something like above, you can use http://download.oracle.com/javase/6/docs/api/java/util/List.html#toArray(T[]))
I see now that this is a feature use with low frequency so the fact that it is heavyweight should be fine. You might add this to javadoc though that it includes scan of fs
Why the double javadoc? You have 'Performs block cache summary' but then you also have @Override (we will pick up the javadoc from the interface so the extra stuff in here from HREgionServer is not needed).
Tests look good.
Just remove the below:
{code}
+ public void setUp() {
+ }
{code}
Remove javadoc w/ nothing in it. Looks bad.
Good stuff Doug.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088307#comment-13088307 ]
Ted Yu commented on HBASE-4199:
-------------------------------
Patch version 4 is in a commit-table state.
Minor comments:
In BlockCache:
{code}
+ public List<BlockCacheColumnFamilySummary> getBlockCacheColumnFamilySummary(Configuration conf) throws IOException {
{code}
I think getBlockCacheColumnFamilySummaries might be a better name.
For HRegionInterface:
{code}
+ * Performs a BlockCache summary and returns a List of BlockCacheColumnFamily objects.
{code}
BlockCacheColumnFamilySummary objects are returned. Again, the method name should pluralize Summaries.
Good work, Doug.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch, java_HBASE_4199_v4.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087474#comment-13087474 ]
stack commented on HBASE-4199:
------------------------------
On 1., BlockCacheColumnFamilySummary? CF needs the table to qualify it but scope is column family?
On 2., Normally that is so but looking in the Interfaces, e.g. HRegionInterface.java, I see that some methods take Lists.
This is the class that does the serialization: src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
It seems to do some special stuff to do a List.
Presumes List of Writables.
On 3., Yeah. That was obvious (smile).
On 4., Yes. You thing it will be invoked frequently?
On 5., Hmm... yeah, or just check for null when serializing.
Good stuff.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086405#comment-13086405 ]
Doug Meil commented on HBASE-4199:
----------------------------------
Thanks Ted.
I'll fix the Javadoc and the unit test table constants.
I'm not sure I agree about the overloaded constructor. Doing this...
{code}
bcse = new BlockCacheSummaryEntry( s[ s.length - 4], s[s.length - 2]);
{code]
... seems less clear to me. I think the 'setTable' with the comment reminder on why it's being done makes more sense.
And doing this...
{code}
String table = ...;
String cf = ...;
bcse = new BlockCacheSummaryEntry(table, cf);
{code]
... results in basically the same 3 lines of code that exist now.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Meil updated HBASE-4199:
-----------------------------
Attachment: java_HBASE_4199.patch
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084881#comment-13084881 ]
Ted Yu commented on HBASE-4199:
-------------------------------
I was talking about the default ctor for BlockCacheSummaryEntry. Throughout the patch, after BlockCacheSummaryEntry() is called, table and columnFamily are filled in.
So I think BlockCacheSummaryEntry(String table, String columnFamily) should be used.
Please see Mingjie's comment in HBASE-4197 about using review board.
Thanks
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087244#comment-13087244 ]
Doug Meil commented on HBASE-4199:
----------------------------------
Great! Do this need more +1's, or can it be committed?
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Meil updated HBASE-4199:
-----------------------------
Attachment: java_HBASE_4199_v2.patch
Adding an updated version of this patch
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085207#comment-13085207 ]
Jean-Daniel Cryans commented on HBASE-4199:
-------------------------------------------
I'm sorta confused by TestBlockCacheSummary.testBlockCacheSummary, you have 3 region servers but then you create a BC from StoreFile.getBlockCache that isn't related to any RS (because the get a different conf object in MiniHBaseCluster.init). What is it testing exactly? Do you need 3 RS btw?
You should set BlockCacheSummaryEntry's attributes to some default values because you might end up with NPEs that are hard to debug, just a precaution with Writables.
There's some cleanup to do too, long lines and whatnot.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087948#comment-13087948 ]
stack commented on HBASE-4199:
------------------------------
On 3., you can't use 'auto-gen' as excuse (smile). Fix your templates so 'auto-gen' does right thing or fix the auto-gen'd stuff. Else all the rest of your changes seem good.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch, java_HBASE_4199_v4.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087923#comment-13087923 ]
Doug Meil commented on HBASE-4199:
----------------------------------
Also, RS interface changed to return List instead of array.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch, java_HBASE_4199_v4.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4199) blockCache summary - backend
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4199:
--------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: 4199.v5, java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch, java_HBASE_4199_v4.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084856#comment-13084856 ]
Ted Yu commented on HBASE-4199:
-------------------------------
It would be easier for other developers to make review comments if review board was used.
{code}
+/**
+ * This class provides tests for the {@link HTableUtil} class
+ *
+ */
+public class TestBlockCacheSummary {
{code}
Please update javadoc with brief description of what is tested in the above class.
{code}
+ /**
+ *
+ */
+ public BlockCacheSummaryEntry() {
+
+ }
{code}
The above can be made package private. Please also fill in javadoc.
Earlier in the same class:
{code}
+ private String table;
+ private String columnFamily;
{code}
Would it be more general if the above two fields were declared as byte[] ?
{code}
+ * @return heapSize
+ */
+ public int getBlocks() {
+ return blocks;
{code}
Please correct javadoc.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086317#comment-13086317 ]
Ted Yu commented on HBASE-4199:
-------------------------------
Please use better table names below:
{code}
+ private static final String TEST_TABLE = "testFamily";
+ private static final String TEST_TABLE2 = "testFamily2";
{code}
Javadoc for BlockCacheSummaryEntry should mention entry:
{code}
+/**
+ * Represents a summary of the blockCache by Table and ColumnFamily
+ *
+ */
+public class BlockCacheSummaryEntry implements Writable, Comparable<BlockCacheSummaryEntry> {
{code}
I think the code below:
{code}
+ bcse = new BlockCacheSummaryEntry();
+ bcse.setTable(s[ s.length - 4]); // 4th from the end
+ bcse.setColumnFamily(s[ s.length - 2]); // 2nd from the end
{code}
should be replaced with calling the two parameter ctor. The default ctor should be made package private.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085464#comment-13085464 ]
Doug Meil commented on HBASE-4199:
----------------------------------
The unit test has been updated in this v2 patch and now does the flush with an assertion on 2 tables which also now appear in the block cache.
Looked for wide-lines, added default-values in BlockCacheSummaryEntry, added some additional Javadoc.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089229#comment-13089229 ]
Hudson commented on HBASE-4199:
-------------------------------
Integrated in HBase-TRUNK #2131 (See [https://builds.apache.org/job/HBase-TRUNK/2131/])
HBASE-4199 blockCache summary - backend (Doug Meil)
tedyu :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCacheColumnFamilySummary.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFileBlockCacheSummary.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestBlockCacheColumnFamilySummary.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: 4199.v5, java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch, java_HBASE_4199_v4.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087956#comment-13087956 ]
Doug Meil commented on HBASE-4199:
----------------------------------
Fair enough. In my defense, I did change writable methods to use 'arg' instead of 'arg0' in this patch.
What else is left?
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch, java_HBASE_4199_v4.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089121#comment-13089121 ]
Ted Yu commented on HBASE-4199:
-------------------------------
Integrated to TRUNK.
Thanks for the patch Doug.
Thanks for the review Stack and J-D.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: 4199.v5, java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch, java_HBASE_4199_v4.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4199) blockCache summary - backend
Posted by "Jonathan Hsieh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hsieh updated HBASE-4199:
----------------------------------
Fix Version/s: 0.92.0
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 4199.v5, java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch, java_HBASE_4199_v4.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087122#comment-13087122 ]
Jean-Daniel Cryans commented on HBASE-4199:
-------------------------------------------
+1 commit.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087475#comment-13087475 ]
stack commented on HBASE-4199:
------------------------------
On 1., BlockCacheColumnFamilySummary? CF needs the table to qualify it but scope is column family?
On 2., Normally that is so but looking in the Interfaces, e.g. HRegionInterface.java, I see that some methods take Lists.
This is the class that does the serialization: src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
It seems to do some special stuff to do a List.
Presumes List of Writables.
On 3., Yeah. That was obvious (smile).
On 4., Yes. You thing it will be invoked frequently?
On 5., Hmm... yeah, or just check for null when serializing.
Good stuff.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084864#comment-13084864 ]
Doug Meil commented on HBASE-4199:
----------------------------------
BlockCacheSummaryEntry can't be made package private because it's also the class that is eventually returned by the RegionServer API.
I'll fix the javadoc.
How do you use the review board? I don't see anything on "More Actions" or "Workflow" in Jira.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086421#comment-13086421 ]
Ted Yu commented on HBASE-4199:
-------------------------------
Please use curly braces for the code directive.
The advantage of using two parameter ctor is that table and column family would be set at the same time, reducing the chance of inconsistency between them now that the two fields carry default values.
This is my personal opinion.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085257#comment-13085257 ]
Jean-Daniel Cryans commented on HBASE-4199:
-------------------------------------------
bq. I'm just trying to create some activity and have it wind up in the block cache. The only thing that seems to be in there in the unit test is the ROOT table with the 'info' CF, which I'm asserting against.
It'd be preferable to look at some real data, here it seems you saw it contained ROOT by chance and now check against it. Force flush that table you're writing to, then scan and then check the content of the block cache.
bq. How is it different? It's using the same config as the TEST_UTIL.
Like I wrote, this is what MiniHBaseCluster does:
{code}
Configuration rsConf = HBaseConfiguration.create(conf);
User user = HBaseTestingUtility.getDifferentUser(rsConf,
".hfs."+index++);
hbaseCluster.addRegionServer(rsConf, i, user);
{code}
But I'm mixing stuff here, I was thinking the same way we usually use Configs for connections but that's not the case here. The block cache is shared between all region servers in the same JVM, which is kinda wrong if you ask me, but it makes testing somehow easier. Anyways, disregard.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Meil updated HBASE-4199:
-----------------------------
Status: Patch Available (was: Open)
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085251#comment-13085251 ]
Doug Meil commented on HBASE-4199:
----------------------------------
Thanks!
re: "minicluster."
Doesn't need to be 3. I copied that from some other test that happened to be 3. Changed to the no-arg method which is 1.
re: "defaults"
Good one. Changed
re: "different config"
How is it different? It's using the same config as the TEST_UTIL.
{code}
BlockCache bc = StoreFile.getBlockCache(TEST_UTIL.getConfiguration());
List<BlockCacheSummaryEntry> bcs = bc.getBlockCacheSummary(TEST_UTIL.getConfiguration());
{code}
re: "what"
I'm just trying to create some activity and have it wind up in the block cache. The only thing that seems to be in there in the unit test is the -ROOT- table with the 'info' CF, which I'm asserting against.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Meil updated HBASE-4199:
-----------------------------
Attachment: java_HBASE_4199_v3.patch
Uploading v3 with all requested changes.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087469#comment-13087469 ]
Doug Meil commented on HBASE-4199:
----------------------------------
1. BlockCacheSummaryEntry name
I'm struggling with the name. 'BlockCacheSummary' makes it seem like you should get exactly one object back (I.e., a summary). So what 'BlockCacheSummaryEntry' is intended to be is an entry in the summary.
Is BlockCacheTableCfSummary a better class name?
2. RegionServer returning Lists
Can the interface do this? I thought that everything returned had to implement Writable. If RegionServers can return Lists, then I think that's a better structure.
3. Eclipse-generated methods
Equals, hashcode, and the writable signature (e.g., arg0), and toString were all generated by Eclipse.
4. Cost of method
Yeah, this is not an inexpensive operation, but adding that method to FsUtils was the only way I could find getting the lookup from StoreFile to CF/Table. That could be cached, but there are all sorts of issues with cache invalidation every times a new store-file is written.
5. Default Values
JD told me that BlockCacheSummaryEntry should have those with default values for the purpose of serialization (same for default constructor)! :-)
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4199) blockCache summary - backend
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4199:
--------------------------
Attachment: 4199.v5
Patch version 5 changes parameter arg to in/out for readFields()/write()
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: 4199.v5, java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch, java_HBASE_4199_v4.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
Posted by "Doug Meil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087670#comment-13087670 ]
Doug Meil commented on HBASE-4199:
----------------------------------
Thanks Stack!
1. I'll change the class to BlockCacheColumnFamilySummary, and the RS and BlockCache method to getBlockCacheColumnFamilySummary (instead of getBlockCacheSummary). It's more verbose, but I think it helps the conceptual clarity.
2. List vs. array. Knowing now that Lists are allowed in RS interfaces, I'll change the RS interface to List. The only reason I returned an array was that I didn't think it was supported.
3. Since this is auto-generated code, leave it alone?
4. This would be the backing method to the new web-page (another sub-task) to present this data. I think it's "low frequency access." One option if this becomes an issue is to cache the result in the RS with a timestamp of when it was last calculated. And basically don't allow the RS to do a summary of this less than 30 seconds or so. I'd rather do that than trying to introduce any StoreFile caching in FsUtils.
Good point about adding the potential expense of the method in the Javadoc.
5. If it's ok with everybody, I'd rather leave this one alone too.
> blockCache summary - backend
> ----------------------------
>
> Key: HBASE-4199
> URL: https://issues.apache.org/jira/browse/HBASE-4199
> Project: HBase
> Issue Type: Sub-task
> Reporter: Doug Meil
> Assignee: Doug Meil
> Priority: Minor
> Attachments: java_HBASE_4199.patch, java_HBASE_4199_v2.patch, java_HBASE_4199_v3.patch
>
>
> This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer.
> This will NOT include any of the web UI or anything else like that. That is for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira