You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Prakash Khemani (JIRA)" <ji...@apache.org> on 2011/06/08 18:09:59 UTC
[jira] [Created] (HBASE-3963) Schedule all log-spliiting at startup
all at once
Schedule all log-spliiting at startup all at once
-------------------------------------------------
Key: HBASE-3963
URL: https://issues.apache.org/jira/browse/HBASE-3963
Project: HBase
Issue Type: Improvement
Reporter: Prakash Khemani
Assignee: Prakash Khemani
When distributed log splitting is enabled then it is better to call splitLog() for all region servers simultaneously. A large number of splitlog tasks will get scheduled - one for each log file. But a splitlog-worker (region server) executes only one task at a time and there shouldn't be a danger of DFS overload. Scheduling all the tasks at once ensures maximum parallelism.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3963) Schedule all log-spliiting at
startup all at once
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047502#comment-13047502 ]
stack commented on HBASE-3963:
------------------------------
I tried to apply your patch but it failed. Can you recut it to apply to trunk (and fix the tabs while you are at it). Thank you Mingjan).
> Schedule all log-spliiting at startup all at once
> -------------------------------------------------
>
> Key: HBASE-3963
> URL: https://issues.apache.org/jira/browse/HBASE-3963
> Project: HBase
> Issue Type: Improvement
> Reporter: Prakash Khemani
> Assignee: Prakash Khemani
> Attachments: schedule-all-splitlog.patch
>
>
> When distributed log splitting is enabled then it is better to call splitLog() for all region servers simultaneously. A large number of splitlog tasks will get scheduled - one for each log file. But a splitlog-worker (region server) executes only one task at a time and there shouldn't be a danger of DFS overload. Scheduling all the tasks at once ensures maximum parallelism.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3963) Schedule all log-spliiting at
startup all at once
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050986#comment-13050986 ]
Hudson commented on HBASE-3963:
-------------------------------
Integrated in HBase-TRUNK #1976 (See [https://builds.apache.org/job/HBase-TRUNK/1976/])
> Schedule all log-spliiting at startup all at once
> -------------------------------------------------
>
> Key: HBASE-3963
> URL: https://issues.apache.org/jira/browse/HBASE-3963
> Project: HBase
> Issue Type: Improvement
> Reporter: Prakash Khemani
> Assignee: Prakash Khemani
> Fix For: 0.92.0
>
> Attachments: schedule-all-splitlog-trunk.patch, schedule-all-splitlog.patch
>
>
> When distributed log splitting is enabled then it is better to call splitLog() for all region servers simultaneously. A large number of splitlog tasks will get scheduled - one for each log file. But a splitlog-worker (region server) executes only one task at a time and there shouldn't be a danger of DFS overload. Scheduling all the tasks at once ensures maximum parallelism.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-3963) Schedule all log-spliiting at
startup all at once
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-3963.
--------------------------
Resolution: Fixed
Fix Version/s: 0.92.0
Hadoop Flags: [Reviewed]
Committed to TRUNK. Thank you for the patch MingJian.
> Schedule all log-spliiting at startup all at once
> -------------------------------------------------
>
> Key: HBASE-3963
> URL: https://issues.apache.org/jira/browse/HBASE-3963
> Project: HBase
> Issue Type: Improvement
> Reporter: Prakash Khemani
> Assignee: Prakash Khemani
> Fix For: 0.92.0
>
> Attachments: schedule-all-splitlog-trunk.patch, schedule-all-splitlog.patch
>
>
> When distributed log splitting is enabled then it is better to call splitLog() for all region servers simultaneously. A large number of splitlog tasks will get scheduled - one for each log file. But a splitlog-worker (region server) executes only one task at a time and there shouldn't be a danger of DFS overload. Scheduling all the tasks at once ensures maximum parallelism.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3963) Schedule all log-spliiting at startup
all at once
Posted by "mingjian (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
mingjian updated HBASE-3963:
----------------------------
Attachment: schedule-all-splitlog-trunk.patch
diff for trunk
> Schedule all log-spliiting at startup all at once
> -------------------------------------------------
>
> Key: HBASE-3963
> URL: https://issues.apache.org/jira/browse/HBASE-3963
> Project: HBase
> Issue Type: Improvement
> Reporter: Prakash Khemani
> Assignee: Prakash Khemani
> Attachments: schedule-all-splitlog-trunk.patch, schedule-all-splitlog.patch
>
>
> When distributed log splitting is enabled then it is better to call splitLog() for all region servers simultaneously. A large number of splitlog tasks will get scheduled - one for each log file. But a splitlog-worker (region server) executes only one task at a time and there shouldn't be a danger of DFS overload. Scheduling all the tasks at once ensures maximum parallelism.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3963) Schedule all log-spliiting at
startup all at once
Posted by "mingjian (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047045#comment-13047045 ]
mingjian commented on HBASE-3963:
---------------------------------
I passed all the tests.Is there any test failed for you?
I used 4 spaces to replace one tab. I will change it.
> Schedule all log-spliiting at startup all at once
> -------------------------------------------------
>
> Key: HBASE-3963
> URL: https://issues.apache.org/jira/browse/HBASE-3963
> Project: HBase
> Issue Type: Improvement
> Reporter: Prakash Khemani
> Assignee: Prakash Khemani
> Attachments: schedule-all-splitlog.patch
>
>
> When distributed log splitting is enabled then it is better to call splitLog() for all region servers simultaneously. A large number of splitlog tasks will get scheduled - one for each log file. But a splitlog-worker (region server) executes only one task at a time and there shouldn't be a danger of DFS overload. Scheduling all the tasks at once ensures maximum parallelism.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3963) Schedule all log-spliiting at
startup all at once
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047003#comment-13047003 ]
stack commented on HBASE-3963:
------------------------------
@Mingjian The patch looks good to me (Did you test it?). You do not have the same tab spacing as the surrounding code (two spaces for a tab not the four or tab character you have). A bunch of your patch is just white space changes. In future try and keep the formatting of surrounding code.
> Schedule all log-spliiting at startup all at once
> -------------------------------------------------
>
> Key: HBASE-3963
> URL: https://issues.apache.org/jira/browse/HBASE-3963
> Project: HBase
> Issue Type: Improvement
> Reporter: Prakash Khemani
> Assignee: Prakash Khemani
> Attachments: schedule-all-splitlog.patch
>
>
> When distributed log splitting is enabled then it is better to call splitLog() for all region servers simultaneously. A large number of splitlog tasks will get scheduled - one for each log file. But a splitlog-worker (region server) executes only one task at a time and there shouldn't be a danger of DFS overload. Scheduling all the tasks at once ensures maximum parallelism.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3963) Schedule all log-spliiting at
startup all at once
Posted by "Prakash Khemani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048248#comment-13048248 ]
Prakash Khemani commented on HBASE-3963:
----------------------------------------
Patch looks good to me. Thanks.
> Schedule all log-spliiting at startup all at once
> -------------------------------------------------
>
> Key: HBASE-3963
> URL: https://issues.apache.org/jira/browse/HBASE-3963
> Project: HBase
> Issue Type: Improvement
> Reporter: Prakash Khemani
> Assignee: Prakash Khemani
> Attachments: schedule-all-splitlog.patch
>
>
> When distributed log splitting is enabled then it is better to call splitLog() for all region servers simultaneously. A large number of splitlog tasks will get scheduled - one for each log file. But a splitlog-worker (region server) executes only one task at a time and there shouldn't be a danger of DFS overload. Scheduling all the tasks at once ensures maximum parallelism.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3963) Schedule all log-spliiting at startup
all at once
Posted by "mingjian (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
mingjian updated HBASE-3963:
----------------------------
Attachment: schedule-all-splitlog.patch
modified the function "splitLog(final String serverName)" to "splitLog(final List<String> serverName)".
> Schedule all log-spliiting at startup all at once
> -------------------------------------------------
>
> Key: HBASE-3963
> URL: https://issues.apache.org/jira/browse/HBASE-3963
> Project: HBase
> Issue Type: Improvement
> Reporter: Prakash Khemani
> Assignee: Prakash Khemani
> Attachments: schedule-all-splitlog.patch
>
>
> When distributed log splitting is enabled then it is better to call splitLog() for all region servers simultaneously. A large number of splitlog tasks will get scheduled - one for each log file. But a splitlog-worker (region server) executes only one task at a time and there shouldn't be a danger of DFS overload. Scheduling all the tasks at once ensures maximum parallelism.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3963) Schedule all log-spliiting at
startup all at once
Posted by "mingjian (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048552#comment-13048552 ]
mingjian commented on HBASE-3963:
---------------------------------
@stack: try this patch?Is there still any problem with spaces?
> Schedule all log-spliiting at startup all at once
> -------------------------------------------------
>
> Key: HBASE-3963
> URL: https://issues.apache.org/jira/browse/HBASE-3963
> Project: HBase
> Issue Type: Improvement
> Reporter: Prakash Khemani
> Assignee: Prakash Khemani
> Attachments: schedule-all-splitlog-trunk.patch, schedule-all-splitlog.patch
>
>
> When distributed log splitting is enabled then it is better to call splitLog() for all region servers simultaneously. A large number of splitlog tasks will get scheduled - one for each log file. But a splitlog-worker (region server) executes only one task at a time and there shouldn't be a danger of DFS overload. Scheduling all the tasks at once ensures maximum parallelism.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira