You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Phabricator (Commented) (JIRA)" <ji...@apache.org> on 2011/12/01 00:53:41 UTC
[jira] [Commented] (HBASE-4908) HBase cluster test tool (port from
0.89-fb)
[ https://issues.apache.org/jira/browse/HBASE-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160476#comment-13160476 ]
Phabricator commented on HBASE-4908:
------------------------------------
stack has commented on the revision "[jira] [HBASE-4908] HBase cluster test tool (port from 0.89-fb)".
Kinda close to PE but I suppose different enough. This can give constant update on rates of read and write. Could add that to PE. Also, can do load+read at same time. Would have to do that in different threads in PE. Has more variable load too.
INLINE COMMENTS
src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:17 is this the right package M? util?
src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:47 A bit of class doc on what it is might help.
Might also just point them at usage it seems since that looks like it might be pretty good.
src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:52 No biggie but these data member names with a trailing underscore a bit silly (smile)
src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:58 I suppose its fine this is hard-coded for now.
src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:92 No biggie: check cols is at least 3 in size so we can return 'nicer' message if less than 3
src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:103 This hard coding should be explained somewhere? Maybe in class comment? The 20/1. Or in usage.
src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:204 Yeah, I think these 'features' need to bubble it up to the class comment so folks can see whats in here w/o having to read code.
src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:275 Hmm... maybe this is good enough when it prints out for figuring whats in here?
src/test/java/org/apache/hadoop/hbase/manual/RestartMetaTest.java:36 Class comment
src/test/java/org/apache/hadoop/hbase/manual/RestartMetaTest.java:17 ditto on package. Should this be in under src/test or under src/main? Maybe package should be tool rather than manual?
src/test/java/org/apache/hadoop/hbase/manual/RestartMetaTest.java:152 There is a lot of overlap w/ previous test here, the HBaseClusterTest tool?
src/test/java/org/apache/hadoop/hbase/manual/utils/DataGenerator.java:24 comment on what this does?
src/test/java/org/apache/hadoop/hbase/manual/utils/DisplayFormatUtils.java:24 This class not needed? Use http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/util/StringUtils.html#humanReadableInt(long)
src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:38 Class comment. HBaseUtils is a pretty generic name; says nothing about content.
src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:53 Remove
src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:54 Why not just let this out? Should at least say in javadoc that it can return a null?
src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:59 This method does not look like it belongs in a generic hbaseutils class. It looks like it belongs to the testing tool that makes up the bulk of this patch. Doesn't seem like method name describes properly what it does either. Why not let out exceptions rather than catch and log.
src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:92 HSA is deprecated.
src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:96 Use MetaReader? e.g:
public static Pair<HRegionInfo, ServerName> getRegion(
CatalogTracker catalogTracker, byte [] regionName)
src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:99 Odd name for this method and it also looks a bit dodgy returning an HBC but it only has zk location set into it.
src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:111 This looks like it belongs in Keying class under util? Or in HBaseSplitter?
src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:124 ditto
src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:139 This should be in Bytes (is it not already?)
src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:151 ditto
src/test/java/org/apache/hadoop/hbase/manual/utils/HdfsAppender.java:36 Does this belong in an hbase test class?
src/test/java/org/apache/hadoop/hbase/manual/utils/KillProcessesAndVerify.java:33 class comment.
src/test/java/org/apache/hadoop/hbase/manual/utils/KillProcessesAndVerify.java:119 Let these out rather than just print them?
src/test/java/org/apache/hadoop/hbase/manual/utils/KillProcessesAndVerify.java:152 What keys are being passed here?
src/test/java/org/apache/hadoop/hbase/manual/utils/MultiThreadedAction.java:31 Class comment
src/test/java/org/apache/hadoop/hbase/manual/utils/MultiThreadedReader.java:156 Remove
src/test/java/org/apache/hadoop/hbase/manual/utils/ProcessBasedLocalHBaseCluster.java:36 Class comment. Is this doing what minihbasecluster does?
src/test/java/org/apache/hadoop/hbase/manual/utils/ProcessBasedLocalHBaseCluster.java:256 Not sure this a good idea.
REVISION DETAIL
https://reviews.facebook.net/D549
> HBase cluster test tool (port from 0.89-fb)
> -------------------------------------------
>
> Key: HBASE-4908
> URL: https://issues.apache.org/jira/browse/HBASE-4908
> Project: HBase
> Issue Type: Sub-task
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Attachments: D549.1.patch
>
>
> Porting one of our HBase cluster test tools (a single-process multi-threaded load generator and verifier) from 0.89-fb to trunk.
> I cleaned up the code a bit compared to what's in 0.89-fb, and discovered that it has some features that I have not tried yet (some kind of a kill test, and some way to run HBase as multiple processes on one machine).
> The main utility of this piece of code for us has been the HBaseClusterTest command-line tool (called HBaseTest in 0.89-fb), which we usually invoke as a load test in our five-node dev cluster testing, e.g.:
> hbase org.apache.hadoop.hbase.manual.HBaseTest -load 1000000000:50:100:20 -tn load_test -read 1:1000000000:50:20 -zk <zk_quorum> -bloom ROWCOL -compression GZIP
> I will be using this code to load-test the delta encoding patch and making fixes, but I am submitting the patch for early feedback. I will probably try out its other functionality and comment on how it works.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira