You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Zhihong Yu (Created) (JIRA)" <ji...@apache.org> on 2012/01/06 16:05:39 UTC
[jira] [Created] (HBASE-5136) Redundant MonitoredTask instances in
case of distributed log splitting retry
Redundant MonitoredTask instances in case of distributed log splitting retry
----------------------------------------------------------------------------
Key: HBASE-5136
URL: https://issues.apache.org/jira/browse/HBASE-5136
Project: HBase
Issue Type: Task
Reporter: Zhihong Yu
In case of log splitting retry, the following code would be executed multiple times:
{code}
public long splitLogDistributed(final List<Path> logDirs) throws IOException {
MonitoredTask status = TaskMonitor.get().createStatus(
"Doing distributed log split in " + logDirs);
{code}
leading to multiple MonitoredTask instances.
User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Sergey Shelukhin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484496#comment-13484496 ]
Sergey Shelukhin commented on HBASE-5136:
-----------------------------------------
Hi. Should this be ok to commit?
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 5136-trunk.patch, 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484518#comment-13484518 ]
Ted Yu commented on HBASE-5136:
-------------------------------
For Jimmy's patch:
{code}
+ Integer logDirsHash = logDirs.hashCode();
{code}
What if there is hash collision between two log directories ?
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 5136-trunk.patch, 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181618#comment-13181618 ]
Zhihong Yu commented on HBASE-5136:
-----------------------------------
Can SplitLogManager maintain mapping from hash of logDirs to MonitoredTask ?
If hash of logDirs isn't found in this mapping, we create MonitoredTask.
@Prakash:
What do you think ?
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484537#comment-13484537 ]
Jimmy Xiang commented on HBASE-5136:
------------------------------------
@Ted, were you referring to 5136.txt? That's your patch. My patch is 5136-trunk.patch, which is already committed in HBASE-6357.
I think we can close this one as a duplicate.
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 5136-trunk.patch, 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5136) Redundant MonitoredTask instances in
case of distributed log splitting retry
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5136:
------------------------------
Attachment: 5136.txt
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5136) Redundant MonitoredTask instances in
case of distributed log splitting retry
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5136:
------------------------------
Attachment: 5136.txt
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5136) Redundant MonitoredTask instances in
case of distributed log splitting retry
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5136:
-------------------------------
Attachment: 5136-trunk.patch
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Ted Yu
> Assignee: Zhihong Ted Yu
> Attachments: 5136-trunk.patch, 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Prakash Khemani (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181777#comment-13181777 ]
Prakash Khemani commented on HBASE-5136:
----------------------------------------
it will be lot simpler to do status.cleanup() in the finally block in splitLogDistributed()
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5136) Redundant MonitoredTask instances in
case of distributed log splitting retry
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5136:
------------------------------
Attachment: (was: 5136.txt)
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181686#comment-13181686 ]
Zhihong Yu commented on HBASE-5136:
-----------------------------------
Patch v1 is based on the assumption that the list of dead region servers for distributed log splitting stays the same across retries.
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5136) Redundant MonitoredTask instances in
case of distributed log splitting retry
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu resolved HBASE-5136.
---------------------------
Resolution: Duplicate
Manage Attachments dialog reverses the order of the two patches, compared to the order on JIRA page :-)
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 5136-trunk.patch, 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5136) Redundant MonitoredTask instances in
case of distributed log splitting retry
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5136:
------------------------------
Status: Patch Available (was: Open)
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5136) Redundant MonitoredTask instances in
case of distributed log splitting retry
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5136:
-------------------------------
Status: Open (was: Patch Available)
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Ted Yu
> Assignee: Zhihong Ted Yu
> Attachments: 5136-trunk.patch, 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183090#comment-13183090 ]
Zhihong Yu commented on HBASE-5136:
-----------------------------------
@Prakash:
Can you kindly provide your opinion ?
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Jimmy Xiang (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13185688#comment-13185688 ]
Jimmy Xiang commented on HBASE-5136:
------------------------------------
Instead of reuse the same status object, can we abort the original one?
{code}
waitForSplittingCompletion(batch, status);
if (batch.done != batch.installed) {
batch.isDead = true;
tot_mgr_log_split_batch_err.incrementAndGet();
LOG.warn("error while splitting logs in " + logDirs +
" installed = " + batch.installed + " but only " + batch.done + " done");
=> update the status message and abort it here
throw new IOException("error or interrupt while splitting logs in "
+ logDirs + " Task = " + batch);
}
{code}
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5136) Redundant MonitoredTask instances in
case of distributed log splitting retry
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5136:
-------------------------------
Status: Open (was: Patch Available)
Let me create a different related issue.
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Ted Yu
> Assignee: Zhihong Ted Yu
> Attachments: 5136-trunk.patch, 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Jean-Daniel Cryans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13184477#comment-13184477 ]
Jean-Daniel Cryans commented on HBASE-5136:
-------------------------------------------
bq. The suggested approach would produce partially confusing result because there might be multiple MonitoredTask instances for the same logDirs within 60 second period.
But it would say that it's aborted right? I actually prefer that.
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5136) Redundant MonitoredTask instances in
case of distributed log splitting retry
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5136:
-------------------------------
Status: Patch Available (was: Open)
Posted a patch to abort the task status if log splitting failed.
Let HBASE-5174 to handle the duplicated status entry.
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Ted Yu
> Assignee: Zhihong Ted Yu
> Attachments: 5136-trunk.patch, 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181721#comment-13181721 ]
Hadoop QA commented on HBASE-5136:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12509744/5136.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javadoc. The javadoc tool appears to have generated -151 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/692//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/692//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/692//console
This message is automatically generated.
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181779#comment-13181779 ]
Zhihong Yu commented on HBASE-5136:
-----------------------------------
Thanks for the review, Prakash.
So the suggestion is to add finally block to splitLogDistributed().
Here is the implementation for cleanup() where stateTime would be set to current time:
{code}
public void cleanup() {
if (state == State.RUNNING) {
setState(State.ABORTED);
}
}
{code}
TaskMonitor hardcodes expiration timeout as 60 seconds:
{code}
private static final long EXPIRATION_TIME = 60*1000;
{code}
The suggested approach would produce partially confusing result because there might be multiple MonitoredTask instances for the same logDirs within 60 second period.
Please correct me if my interpretation of the task monitoring is wrong.
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13185693#comment-13185693 ]
Zhihong Yu commented on HBASE-5136:
-----------------------------------
This is similar to Prakash's suggestion.
In some situation, several (aborted) tasks for the same server would be shown on master UI at the same time since it takes 1 minute to clean up task display.
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13184377#comment-13184377 ]
Zhihong Yu commented on HBASE-5136:
-----------------------------------
Can someone review the patch ?
Thanks
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5136) Redundant MonitoredTask instances
in case of distributed log splitting retry
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13184507#comment-13184507 ]
Zhihong Yu commented on HBASE-5136:
-----------------------------------
Then this JIRA depends on HBASE-5174.
Please comment on my proposal there.
The patch in this JIRA is just specialized version of my proposal for HBASE-5174.
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
> Attachments: 5136.txt
>
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5136) Redundant MonitoredTask instances in
case of distributed log splitting retry
Posted by "Zhihong Yu (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu reassigned HBASE-5136:
---------------------------------
Assignee: Zhihong Yu
> Redundant MonitoredTask instances in case of distributed log splitting retry
> ----------------------------------------------------------------------------
>
> Key: HBASE-5136
> URL: https://issues.apache.org/jira/browse/HBASE-5136
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Assignee: Zhihong Yu
>
> In case of log splitting retry, the following code would be executed multiple times:
> {code}
> public long splitLogDistributed(final List<Path> logDirs) throws IOException {
> MonitoredTask status = TaskMonitor.get().createStatus(
> "Doing distributed log split in " + logDirs);
> {code}
> leading to multiple MonitoredTask instances.
> User may get confused by multiple distributed log splitting entries for the same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira