You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Alex Newman (JIRA)" <ji...@apache.org> on 2010/10/13 22:25:35 UTC
[jira] Created: (HBASE-3107) Breakup HLogSplitTest unit tests.
Breakup HLogSplitTest unit tests.
---------------------------------
Key: HBASE-3107
URL: https://issues.apache.org/jira/browse/HBASE-3107
Project: HBase
Issue Type: Sub-task
Reporter: Alex Newman
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-3107) Breakup HLogSplitTest unit tests.
Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Newman reassigned HBASE-3107:
----------------------------------
Assignee: Alex Newman
> Breakup HLogSplitTest unit tests.
> ---------------------------------
>
> Key: HBASE-3107
> URL: https://issues.apache.org/jira/browse/HBASE-3107
> Project: HBase
> Issue Type: Sub-task
> Reporter: Alex Newman
> Assignee: Alex Newman
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3107) Breakup HLogSplitTest unit tests.
Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920754#action_12920754 ]
Alex Newman commented on HBASE-3107:
------------------------------------
I want to break up the unit tests into 5 files, A BaseHLogSplit which contains all of the common functions. A TestHLogSplit, a TestHLogSplitSkipErrors, DistributedHlogSplit, DistributedHLogSkipErrors. The Distributed versions will load MiniClusters instead of miniHDFS clusters. The inheritance will look roughly like
BaseHLogSplit => TestHLogSplit => DistributedTestHLogSplit
BaseHLogSplit => TestHLogSplitErrors => DistributedTestHlogSplitSkipErrors.
This could significantly increase the overall time to unit test compared to where it does, as we will need to start/stop atleast one additional MiniCluster. I thought I would bring it up in case someone had another suggestion.
> Breakup HLogSplitTest unit tests.
> ---------------------------------
>
> Key: HBASE-3107
> URL: https://issues.apache.org/jira/browse/HBASE-3107
> Project: HBase
> Issue Type: Sub-task
> Reporter: Alex Newman
> Assignee: Alex Newman
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3107) Breakup HLogSplitTest unit tests.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920778#action_12920778 ]
stack commented on HBASE-3107:
------------------------------
How many tests in each Alex? Sounds fine though? You don't want to put TestHLogSplit + TestHLogSplitSkipErrors together? Ditto for distributed stuff?
> Breakup HLogSplitTest unit tests.
> ---------------------------------
>
> Key: HBASE-3107
> URL: https://issues.apache.org/jira/browse/HBASE-3107
> Project: HBase
> Issue Type: Sub-task
> Reporter: Alex Newman
> Assignee: Alex Newman
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3107) Breakup HLogSplitTest unit tests.
Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920931#action_12920931 ]
Alex Newman commented on HBASE-3107:
------------------------------------
I know right, it seems like a lot. So I am very openminded to a change like that but here is what ran through my head. Distributed logsplitting now just enqueues zk nodes when a new split comes in. It's then up to a thread running in the regionserver to do the actual splitting. The choice to skipErrors or not to skipErrors happens by checking the (hbase.hlog.split.skip.errors) variable whenever the real split log happens. However, once I startup the regionserver in the test (@BeforeClass) with skip errors or not skip errors, I worry we are stuck with it, for the length of the test. As an alternative:
- We could embed into the zk data whether or not it's a skip errors log I suppose it could be done, but that seems like a weird place to do it *shrug*
- Alternatively we could change a property on the running regionserver? I don't know of any code which does something like that?
Anyway, reducing unit test run time is a goal of mine as well, as I am lazy , any suggestions are welcome.
> Breakup HLogSplitTest unit tests.
> ---------------------------------
>
> Key: HBASE-3107
> URL: https://issues.apache.org/jira/browse/HBASE-3107
> Project: HBase
> Issue Type: Sub-task
> Reporter: Alex Newman
> Assignee: Alex Newman
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3107) Breakup HLogSplitTest unit tests.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12921692#action_12921692 ]
stack commented on HBASE-3107:
------------------------------
bq. We could embed into the zk data whether or not it's a skip errors log I suppose it could be done, but that seems like a weird place to do it shrug
Well, the new fashion in these circles is making config live and being able to change it while cluster is up so having the config up in zk is actually the right direction to go in I'd say.
For now I'd say, to make progress, you could hack up something into zk. Later we can make it part of new whatever the new 'live config' doo-hickey will look like.
Good on you Alex.
> Breakup HLogSplitTest unit tests.
> ---------------------------------
>
> Key: HBASE-3107
> URL: https://issues.apache.org/jira/browse/HBASE-3107
> Project: HBase
> Issue Type: Sub-task
> Reporter: Alex Newman
> Assignee: Alex Newman
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.