You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Devaraj Das (JIRA)" <ji...@apache.org> on 2012/11/23 20:58:58 UTC
[jira] [Created] (HBASE-7213) Have HLog files for .META. edits only
Devaraj Das created HBASE-7213:
----------------------------------
Summary: Have HLog files for .META. edits only
Key: HBASE-7213
URL: https://issues.apache.org/jira/browse/HBASE-7213
Project: HBase
Issue Type: Improvement
Components: master, regionserver
Reporter: Devaraj Das
Assignee: Devaraj Das
Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505256#comment-13505256 ]
ramkrishna.s.vasudevan commented on HBASE-7213:
-----------------------------------------------
@N
Yes balancer is also one thing which is doing it. just verified the code.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504404#comment-13504404 ]
ramkrishna.s.vasudevan commented on HBASE-7213:
-----------------------------------------------
@Deva
Thanks for the update. I remeber there is a hlog created inside a hregion also which is never getting used? Can we make use of it?
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505785#comment-13505785 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
[~ram_krish], thanks for looking. Yes, this patch is not complete yet and the next iteration will have the missing functionalities. I'll also answer your questions in the next iteration.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7213) Have HLog files for .META. edits only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HBASE-7213:
-------------------------------
Attachment: 7213-in-progress.patch
Here is a rough, currently work in progress, patch. At a high level:
1. There is a new MetaServices class that currently provides the services for creating/getting meta hlog.
2. OpenMetaHandler invokes the method for creating the meta hlog, and also makes sure that the opened region gets the right hlog instance.
3. The meta hlog file is created in the same root hlog directory, but is suffixed with .META.
I still need to plumb through the log splitting code, etc. But I thought I'll post what I have to see if there is any immediate feedback.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510057#comment-13510057 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
bq. When we lose the .META. region, does the RS clean up its .META. log?
I don't think I need to do anything specific here. The existing code should handle this already.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7213) Have HLog files for .META. edits only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HBASE-7213:
-------------------------------
Attachment: 7213-in-progress.2.2.patch
Addresses most of the comments..
One thing I didn't change is
bq. The check for ROOT region above is for assigning ROOT ..
In HBASE-3171, there is work underway to remove -ROOT- .. So not much gain here to address things to do with how the root region is handled. I have left it as such (with a known and a very corner case that if the server holding the root's region crashes before the region's edits are persisted, the assignment of the region might be an issue).
Manual tests seemed to indicate things are working well. I'll start the full set of unit tests now.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7213) Have HLog files for .META. edits only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HBASE-7213:
-------------------------------
Attachment: 7213-in-progress.2.patch
A more complete version. Testing in progress. This patch adds APIs to do with META in some classes (as opposed to a flag, as suggested by [~stack] earlier). The logs for META region is written in a special file ending with ".META". The SSH's process method treats META hlogs with higher priority and splits them first, and as soon as the split is done, tries to assign them, and then goes on to split other regions / assignments. I have not made changes to how the -ROOT- region is handled, and the reason is that it is very very likely the case that the -ROOT-'s region's edits are already persisted on disk (and in case folks feel that -ROOT- should be handled as well, it probably can be done in a follow up..).
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504032#comment-13504032 ]
stack commented on HBASE-7213:
------------------------------
Yes
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505501#comment-13505501 ]
ramkrishna.s.vasudevan commented on HBASE-7213:
-----------------------------------------------
@Deva
Patch looks good.. We may need to check on log rolling, log splitting etc...
One thing is if the META has just moved from one RS to another RS and the first RS crashes. We will be having a the HLog for the META also with it(because it was holding it).. So in this i think we need not split that META hlog right? Correct me if am wrong here.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506953#comment-13506953 ]
Todd Lipcon commented on HBASE-7213:
------------------------------------
Yea, I don't think you need to strictly sequence this after the multi-WAL work. But it would be nice to have the "end goal" in mind while doing this work. Sorry, haven't had time to look at the in-progress patch, but if there's a simple solution that works OK now, no sense blocking it for the perfect end-game solution later.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7213) Have HLog files for .META. edits only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HBASE-7213:
-------------------------------
Attachment: 7213-2.6.patch
bq. One more spin and I'd say this patch is ready to go in. Good on you DD.
Sweet... Here is an updated patch with the filename extn change, and with passing PathFilter in splitLogDistributed as per your last comments.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-2.4.patch, 7213-2.6.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7213) Have HLog files for .META. edits only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HBASE-7213:
-------------------------------
Attachment: 7213-2.8.patch
bq. I can remove this on commit:
Oops.. apologies for missing the removal. This patch removes the interface method.
bq. This looks new (and important):
Yeah (smile). I noticed it recently..
bq. Lets wait some time on hbase-3171. If it doesn't show up soon, we'll commit this.
Okay. That makes sense. Thanks folks for reviewing the patch and providing good feedback.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Priority: Critical
> Fix For: 0.96.0
>
> Attachments: 7213-2.4.patch, 7213-2.6.patch, 7213-2.8.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505791#comment-13505791 ]
stack commented on HBASE-7213:
------------------------------
[~ram_krish] On if a .META. moved from one RS to another, and first crashes, then when we go to recover, we will see it wasn't carrying .META. so we won't try and recover it (as is the case w/ any other region).
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505803#comment-13505803 ]
stack commented on HBASE-7213:
------------------------------
On the approach in the sketch patch:
Passing a flag to instantiateHLog when its meta is a little awkward. Ripples all the ways down. Would instantiateMetaHLog and createMetaHLog in the factory be more plain about what is going on? Runs into a wall at FSHLog though though here, is there anything in the log name that would denote it a .META. WAL?
Patch doesn't look bad so far. Not too intrusive for big payback.
I was thinking this facility would come in as part of the multiwal work... if multiwals, make provision for an extra if carrying .meta..... but this seems like shorter path. Do you think this work will get in the way of the multiwal feature?
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7213) Have HLog files for .META. edits only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HBASE-7213:
-------------------------------
Attachment: 7213-2.4.patch
Okay, this patch addresses most of the comments. The thing that I have left here not-handled is the (corner) case of the RS carrying the -root- region dying before its edits are persisted (splitMetaLog doesn't take into account -root- and the -root- edits are not written to a separate file either). The reason is HBASE-3171 is removing the -root- table/region and so not much point in trying to address -root- here ..
I have added an API getWAL(HRegionInfo) in RegionServerServices, and when the argument is null, the default/common WAL is returned (imagine a WAL for meta, and maybe some other regions, and a common WAL for everything else).
I have broken up the ServerShutdownHandler.process into two parts and moved some code into MetaServerShutdownHandler.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-2.4.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509537#comment-13509537 ]
stack commented on HBASE-7213:
------------------------------
.META. file ends in '.META' and not '.META.'? Can't it end in .log? xxxxxx.log.META looks a bit odd and I'm sure will trip up someone else trying to find WALs for whatever reason.
How does splitMetaLog work? It is passed a server name and if we find logs in the filesystem, then we'll split its logs? What if .META. is currently assigned? The splitting will be to no avail? When would we ever log this message:
{code}
+ LOG.info("No logs to split");
{code}
Should that be .META. logs to split?
Methods usually have a space between them rather than:
{code}
+ }
+ /**
{code}
... in hbase codebase boss.
Why emit two log lines when you could have one that varies with whether meta or not:
{code}
LOG.info("Done splitting " + path);
+ if (path.endsWith(HConstants.META_HLOG_FILE_EXTN)) {
+ LOG.info("Done splitting meta region");
+ }
{code}
Should be a isMetaWAL method? to do the looking for META_LOG_FILE_EXTN... since used in a few places.
In process, we have code for meta and root handling in the general ServerShutdownHandler. Should the root handling be up in RootServerShutdownHandler, in its process, which then calls through to the general SSH#process and ditto for the .META.? (Might mean ROOT needs to subclass Meta for now... ). We maybe should have done it before but was a bit tougher since had to do the log splitting first... could have broke up the process more ... but might be easier now these root and meta handlings are first thing done in the process method.
Is this needed?
{code}
- this.services.getExecutorService().submit(this);
+ //typecast to SSH so that we don't do the above meta log split again
+ this.services.getExecutorService().submit((ServerShutdownHandler)this);
{code}
Would make sense if you move the meta specific code to metaSSH, and ditto on ROOT?
MetaServices Interface doesn't seem right? Its all about a WAL. What else would ever go in here? Who uses this new Interface and its methods? I don't see anything else in this patch using it. I suppose MetaLogRoller does but it is sitting beside RegionServer -- could be protected or package private -- yet its public in the Interface. Hmm... i see that OpenRegionHandler uses it. Can't RegionServer notice when its a meta region and do the setup etc., internally? Ugh, maybe not looking at this OpenRegionHandler....how it needs the exact WAL? What if you passed in the regionname to getWAL? Then it internally would do the right thing. You wouldn't have to add new methods? Especially public ones?
When we lose the .META. region, does the RS clean up its .META. log?
I don't like extending the RegionServerServices because makes it harder mocking up an RSS... more stuff to add in.
Should it be MetaFSHLog rather than FSLog w/ boolean args in constructor? Just wondering.
You have this: + LOG.info("CREATED writer " + path); //REMOVETHIS
We have to add this method?
+ public static HLog createMetaHLog(final FileSystem fs, final Path root, final String logName,
+ final Configuration conf, final List<WALActionsListener> listeners,
+ final String prefix) throws IOException {
+ return new FSHLog(fs, root, logName, HConstants.HREGION_OLDLOGDIR_NAME,
+ conf, listeners, false, prefix, true);
+ }
Nothing in say the log name that would clue us its a .META. file? Could we preface the logName with .META. or something and then we don't have to pass flags to FSHLog? Could have static isMetaWAL method that looks at log name and can tell it .META.?
Import this in LogSplitter but not used: +import org.apache.hadoop.fs.PathFilter;
Is this used:
- private static final Pattern pattern = Pattern.compile(".*\\.\\d*");
+ private static final Pattern pattern = Pattern.compile(".*\\.\\d*(.META)*");
Good stuff.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511030#comment-13511030 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
I ran tests locally with the patch and they passed. I also ran some manual tests (killing region servers after making sure some edits weren't persisted, etc.).
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-2.4.patch, 7213-2.6.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505896#comment-13505896 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
[~stack], yes, I am trying to keep the patch as less intrusive as I can. The fact that the multiwal is being done for META only helps towards that (for example, replication won't need to be tweaked since replication is not done for META entries, etc.). I'll check on the flag comment and see if I can introduce new methods rather than use a flag..
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7213) Have HLog files for .META. edits only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HBASE-7213:
-------------------------------
Fix Version/s: 0.96.0
Status: Patch Available (was: Open)
Trying hadoopqa.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509575#comment-13509575 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
Sorry, I hit the submit button too soon. Ignore the last few lines in the last comment (starting from "Is this used:).
On the last bit:
bq. Is this used:
bq. private static final Pattern pattern = Pattern.compile(".\\.d");
bq. + private static final Pattern pattern = Pattern.compile(".\\.d(.META)*");
Yes, otherwise the master deletes the .META hlog files.. (this makes it so that files are accepted with .META extensions).
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509513#comment-13509513 ]
Hadoop QA commented on HBASE-7213:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12555829/7213-in-progress.2.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 99 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.catalog.TestMetaReaderEditor
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3443//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3443//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3443//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3443//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3443//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3443//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3443//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3443//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3443//console
This message is automatically generated.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506884#comment-13506884 ]
Ted Yu commented on HBASE-7213:
-------------------------------
We already have HLogFactory in trunk.
bq. to lump this with the multi-WAL work
Can I interpret the above as saying that multi-WAL work should be done at the same time, if not earlier ?
Since HLogFactory can hand out the unique instance for .META., it is not far from handing out different instances (for different regions) which is what HBASE-5699 tries to do.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506204#comment-13506204 ]
ramkrishna.s.vasudevan commented on HBASE-7213:
-----------------------------------------------
bq.we will see it wasn't carrying .META. so we won't try and recover it
Yes that is true...MetaServerServerShutDownHandler alone should be handling the split of META hlog. The normal SSH will handle only the other HLog.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506923#comment-13506923 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
[~tlipcon], good points there. But I'd like to separate the META SPOF work from the full fledged multiwal work (for the full multi-wal case, we'd need to fix up things like replication, and those can be skipped from the meta-only design that this jira attempts to do).
[~yuzhihong@gmail.com], I guess you are right. In theory, one could read the HLogFactory as the WALFactory...
Thoughts? (i am in the process of extending the patch I previously posted to a fully functional one)
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503342#comment-13503342 ]
ramkrishna.s.vasudevan commented on HBASE-7213:
-----------------------------------------------
Is the idea to write it in a seperate WAL file? So that i can be picked up first?
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508425#comment-13508425 ]
Ted Yu commented on HBASE-7213:
-------------------------------
{code}
+ List<ServerName> serverNames = new ArrayList<ServerName>();
+ serverNames.add(serverName);
+ List<Path> logDirs = getLogDirs(serverNames);
{code}
The first two lines above can be moved below the if block which checks logDirs.isEmpty()
{code}
+ * @throws IOException
+ * if there was an error while splitting any log file
+ * @return cumulative size of the logfiles split
+ * @throws IOException
+ */
+ public long splitMetatLogDistributed(final List<Path> logDirs) throws IOException {
{code}
Please group the @throws lines together. The method name is misspelled - I couldn't find where it is called.
I think the above method is eclipsed by splitMetaLogDistributed() which appears later.
{code}
+ if (isCarryingMeta() || isCarryingRoot()) {
+ try {
+ LOG.info("Splitting META logs for " + serverName);
+ if (this.shouldSplitHlog) {
+ this.services.getMasterFileSystem().splitMetaLog(serverName);
{code}
The check for ROOT region above is for assigning ROOT. Suggest making the check of ROOT and .META. separate so that the logic is clearer.
{code}
+ protected HLog instantiateMetaHLog(Path rootdir, String logName) throws IOException {
+ return HLogFactory.createMetaHLog(this.fs.getBackingFs(), rootdir, logName, this.conf,
+ getMETAWALActionListeners(), this.serverNameFromMasterPOV.toString());
+ }
{code}
The above method is only called by setupMetaWAL(). Can we merge it with setupMetaWAL() ?
nit: getMETAWALActionListeners(), do we need to make all 4 letters of Meta capitalized ?
{code}
+ LOG.info("ISALIVE: " + leases.isAlive() + " " +cacheFlusher.isAlive() + " "+ this.compactionChecker.isAlive() + " " + hlogRoller.isAlive() + " " + metaHlogRoller.isAlive()); //REMOVETHIS
{code}
As the comment says: remove in next patch.
{code}
+class MetaLogRoller extends LogRoller {
{code}
Consider putting MetaLogRoller in its own class.
For MetaServices.java, please add license and audience annotation.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "nkeywal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507250#comment-13507250 ]
nkeywal commented on HBASE-7213:
--------------------------------
To me, there are some stuff that are nice to have in multiwal but very useful here:
- it's better to have only one table in the wal with ".meta."
- it's would be great to be able to configure the replication factor for this wal
- having a separate wal for meta should be the only option (it could make sense short term to have it as an option for safety, but it would be temporary)
- there is no real split of the .meta. wal (at least conceptually: all regions ends up on the same server..). There is some room for (premature?) optimization there.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510706#comment-13510706 ]
stack commented on HBASE-7213:
------------------------------
Do other logs have a '.log' suffix? You add .hlog with the below?
+ /** The META region's HLog filename extension */
+ public static final String META_HLOG_FILE_EXTN = ".meta.hlog";
The refactoring in MasterFileSystem to make a new getLogDirs method is nice.
The SSH changes look good. There is a hole though at the moment around -ROOT- handling? We need to wait on -ROOT- removal before this can go in, right? (The RootServerShutdownHandler would go....or should RSSH be calling MSSH when it is done w/ its process?)
This below is given to an executor? Should have a better name than handler?
+ private UncaughtExceptionHandler handler;
Will we always create the metahlog just because getMetaHLog was called? Even if we are NOT carrying .META.:
+ private HLog getMetaWAL() throws IOException {
+ if (this.hlogForMeta == null) {
If .META. moves away from a RS and then comes back, we'll just use the already made meta log roller, etc.?
Replication will skip this .META. log?
It is intentional that MetaServices is still in the patch?
Changing param name from logName to logDir is good stuff
In SplitLogManager, we have added splitMetaLogDistributed. A more generic method might have been splitLogDistributed that took a file path filter instead...
There is a define for .META. HRegionInfo in HRegionInfo#FIRST_META_REGIONINFO so you don't have to make it each time, FYI.
Don't have to deprecate the below I'd say. This is for 0.96 the singularity and this is on a class with annotation @InterfaceAudience.Private
/** @return the HLog */
+ @Deprecated
public HLog getWAL();
+
+ /** @return the HLog for a particular region. Pass null for getting the
+ * default (common) WAL */
+ public HLog getWAL(HRegionInfo regionInfo) throws IOException;
Here....
- return new Path(dir, prefix + "." + filenum);
+ String child = prefix + "." + filenum;
+ if (forMeta) {
+ child += HConstants.META_HLOG_FILE_EXTN;
+ }
+ return new Path(dir, child);
the 'normal' WALs do not seem to pick up the suffix in here. Is it possible that the '.log' is appended elsewhere? Are your .META. logs getting double '.log'? Just wondering.
Good stuff
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-2.4.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510843#comment-13510843 ]
stack commented on HBASE-7213:
------------------------------
bq. So I am wondering whether I should just have ".meta" as the suffix for the meta log files, and be done with it (as I had it in the previous patch).
Yes. The '.hlog' is extraneous (I thought all log files had .log suffix -- sorry if I mislead)
One more spin and I'd say this patch is ready to go in. Good on you DD.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-2.4.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505253#comment-13505253 ]
ramkrishna.s.vasudevan commented on HBASE-7213:
-----------------------------------------------
Thanks Deva. Normally not possible. But seeing the 0.94 code atleast we dont block in issuing an assign or move command to META. !!
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505261#comment-13505261 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
Thanks for the clarifications, [~ram_krish] and [~nkeywal]. The thought of moving meta sounds scary to me though :-) [but this patch doesn't intend to affect that part].
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7213) Have HLog files for .META. edits only
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-7213:
-------------------------
Priority: Critical (was: Major)
Marking critical so we don't forget it, so it goes in.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Priority: Critical
> Fix For: 0.96.0
>
> Attachments: 7213-2.4.patch, 7213-2.6.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504413#comment-13504413 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
Will take a look [~ram_krish], but is there any particular reason why that should be reused? (are you referring to the field {noformat}final HLog log;{noformat}).
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "nkeywal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505254#comment-13505254 ]
nkeywal commented on HBASE-7213:
--------------------------------
Is it something forbidden? Because there is a (fixed) jira that says we should be able to do it (HBASE-3756).
It's obviously critical, but I think it's requires for rolling restart, where we move all the regions from a server before stopping/starting it.
We could also imagine balancers doing this...
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506730#comment-13506730 ]
Todd Lipcon commented on HBASE-7213:
------------------------------------
I agree it seems to make sense to lump this with the multi-WAL work. Perhaps an interface like "WALFactory" or "WALProvider", which, given a region name, gives back a WAL instance? The basic implementation would always provide the single WAL. Then, we could add the feature that returns a different WAL for META alone. More complex implementations could choose to give different tenants of a cluster separate WALs, etc.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509573#comment-13509573 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
bq. .META. file ends in '.META' and not '.META.'? Can't it end in .log? xxxxxx.log.META looks a bit odd and I'm sure will trip up someone else trying to find WALs for whatever reason.
Hmm.. "xxxx.log.META" sounds better..
bq. How does splitMetaLog work? It is passed a server name and if we find logs in the filesystem, then we'll split its logs?
Yes.. (much like how it's currently done for the non-meta logs)
bq. What if .META. is currently assigned? The splitting will be to no avail?
Can you please explain when this can happen (that a RS hosting the .META. crashes, and before we do the log splitting, the .META. gets assigned to someone else)? But in general, the behavior shouldn't change in the patch from what the behavior currently is...
bq. When would we ever log this message:
bq. LOG.info("No logs to split");
This is the same log that gets emitted for the general case (in the current codebase). Look at splitLog method in MasterFileSystem. Seems like when the logdir (.logs) doesn't exist, getLogDirs returns an empty list.
bq. Should that be .META. logs to split?
Yes..
bq. Methods usually have a space between them rather than:
Ack. I missed adding a space. I usually always add spaces without fail (smile)
bq. Why emit two log lines when you could have one that varies with whether meta or not:
This was mostly for my debugging. I'll update this..
bq. Should be a isMetaWAL method? to do the looking for META_LOG_FILE_EXTN... since used in a few places.
Yeah..
bq. We maybe should have done it before but was a bit tougher since had to do the log splitting first... could have broke up the process more ..
I went that route and backtracked.. The problem is that when a RS crashes, we will probably have that RS hosting many regions (including the catalog ones). To recover we have to call process() methods of both SSH and MSSH as a pair. That seemed harder to maintain.. So I kept the existing semantics of one SSH process doing the work for both META and non-meta regions. As I have noted in a previous comment, I have explicitly not handled -ROOT- since the root region is going to disappear post HBASE-3171.
bq. Is this needed?
Yeah, since the process() method handles both .META. and non-meta regions. My earlier comment explains why I am not very inclined to separate the handling in process(). But I'll reconsider this part and see if I can do something better.
bq. MetaServices Interface doesn't seem right? Its all about a WAL. You wouldn't have to add new methods? Especially public ones?
Yeah, I shouldn't have made that a public class, etc. and maybe called it MetaWALServices or something. Would doing that alleviate this concern?
bq. When we lose the .META. region, does the RS clean up its .META. log?
I need to check on this.
bq. I don't like extending the RegionServerServices because makes it harder mocking up an RSS... more stuff to add in.
The new methods could just return null (and that's what I have done in the existing mock classes). But I'll consider this.
bq. Should it be MetaFSHLog rather than FSLog w/ boolean args in constructor? Just wondering.
You mean another class.. Seems overkill to me..
bq. We have to add this method?
bq. public static HLog createMetaHLog(final FileSystem fs, final Path root, final String logName,
bq. Nothing in say the log name that would clue us its a .META. file? Could we preface the logName with .META. or something and then we don't have to pass flags to FSHLog? Could have static isMetaWAL method that looks at log name and can tell it .META.?
The "logname" (and this is there in the current trunk) is kind of misleading. It actually signifies the directory name (and maybe I should update the code to call it logdir everywhere)... So in that sense, there is nothing indicative of "meta". Also, I'd prefer passing explicit booleans rather than string based conditionals..
bq. Is this used:
private static final Pattern pattern = Pattern.compile(".\\.
d");
+ private static final Pattern pattern = Pattern.compile(".\\.
d(.META)*");
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506314#comment-13506314 ]
Ted Yu commented on HBASE-7213:
-------------------------------
I wonder what kind of data structure should be in place for the FSHLog instances (two for the region server hosting .META.) so that multiple WAL implementation would be more intuitive.
Previously, close / deletion of HLog operates on the single instance. Now things start to get more interesting.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504230#comment-13504230 ]
Enis Soztutar commented on HBASE-7213:
--------------------------------------
This can also allow us to directly deploy META without waiting for log splitting (if we have separate WAL for every META region, there is nothing to split).
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504398#comment-13504398 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
Apologies for not responding earlier.. But yes, the high level idea is to make the recovery of META table fast and hence reduce the downtime of META should the regionserver hosting the RS crash. By having a dedicated HLog for META, the log splitting will be really simple (maybe just simple rename operations of files; need to look at the code here)...
I am in the middle of plumbing through the code to make this work. Will hopefully have something in a day or two.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505247#comment-13505247 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
bq. If an RS that had the META once and again gets back the META in that case we should be pointing to the same HLog right?
Yes, [~ram_krish], it should be pointing to the same HLog. I am wondering though when could this happen. I can see the obvious case where a RS dies and the meta gets assigned to some other RS, but when does the former get it back?
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510829#comment-13510829 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
bq. Do other logs have a '.log' suffix? You add .hlog with the below?
Currently (in the trunk codebase), the log files don't have extension like .log or something (the files look like 192.168.1.8%2C60020%2C1354695930505.1354695931168). They are in directory called ".logs/<region-server>". So I am wondering whether I should just have ".meta" as the suffix for the meta log files, and be done with it (as I had it in the previous patch). In the current patch, the meta logs have .meta.hlog as the extension. Or, would it make sense to change the log files to have ".hlog" extension as well? Please let me know..
bq. The SSH changes look good. There is a hole though at the moment around ROOT handling? We need to wait on ROOT removal before this can go in, right? (The RootServerShutdownHandler would go....or should RSSH be calling MSSH when it is done w/ its process?)
It can be done either way in theory (HBASE-3171 first or this first). On the hole, yes, if this patch goes in first, and if there is a case where the same RS hosts both the root and meta regions, and that RS goes down, and the root's edits weren't already persisted, there will be a problem. But given that the root doesn't have edits other than the one row to do with .META., it is unlikely that we will run into the hole in practice. But am okay to wait till HBASE-3171 goes in (and that's a better way to line the commits, but would like to close on on the other comments).
bq. This below is given to an executor? Should have a better name than handler?
bq. + private UncaughtExceptionHandler handler;
Will take a look and update.
bq. Will we always create the metahlog just because getMetaHLog was called? Even if we are NOT carrying .META.:
No. It'll be created on the first call to getMetaWAL (and getMetaWAL gets indirectly called only when we are asked to open the meta region in OpenRegionHandler)
bq. If .META. moves away from a RS and then comes back, we'll just use the already made meta log roller, etc.?
Yes.
bq. Replication will skip this .META. log?
Yes. The MetaLogRoller doesn't "register" with the Replication folks when instantiated.
bq. It is intentional that MetaServices is still in the patch?
Mistake.. Will update.
bq. In SplitLogManager, we have added splitMetaLogDistributed. A more generic method might have been splitLogDistributed that took a file path filter instead...
Hmm.. Let me see..
bq. There is a define for .META. HRegionInfo in HRegionInfo#FIRST_META_REGIONINFO so you don't have to make it each time, FYI.
I had seen this one but had forgotten to update code to use it.. will update.
bq. Don't have to deprecate the below I'd say. This is for 0.96 the singularity and this is on a class with annotation @InterfaceAudience.Private
bq. /** @return the HLog */
bq. + @Deprecated
bq. public HLog getWAL();
Okay..
{quote}
Here....
return new Path(dir, prefix + "." + filenum);
+ String child = prefix + "." + filenum;
+ return new Path(dir, child);
the 'normal' WALs do not seem to pick up the suffix in here. Is it possible that the '.log' is appended elsewhere? Are your .META. logs getting double '.log'? Just wondering.
{quote}
No.. as I said before, the log files don't have the string extensions.. So what do you think - should I just remove the .hlog from the .meta.hlog extension that I did for meta hlogs? Or, leave .meta.hlog as it is and add .hlog to the non-meta hlog files, or, leave the non-meta hlog names as they are now?
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-2.4.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508999#comment-13508999 ]
Enis Soztutar commented on HBASE-7213:
--------------------------------------
bq. and in case folks feel that ROOT should be handled as well, it probably can be done in a follow up
I think JD has updated the patch for removing ROOT in favor of ZK.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511109#comment-13511109 ]
stack commented on HBASE-7213:
------------------------------
This looks new (and important):
+ splitLog(serverNames, META_FILTER);
+ splitLog(serverNames, NON_META_FILTER);
I can remove this on commit:
/** @return the HLog */
public HLog getWAL();
Patch looks good to me.
Lets wait some time on hbase-3171. If it doesn't show up soon, we'll commit this.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-2.4.patch, 7213-2.6.patch, 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509536#comment-13509536 ]
Devaraj Das commented on HBASE-7213:
------------------------------------
TestMetaReaderEditor (timed out in the hadoopqa run) passes locally..
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7213) Have HLog files for .META. edits
only
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505217#comment-13505217 ]
ramkrishna.s.vasudevan commented on HBASE-7213:
-----------------------------------------------
Deva,
I could not apply the code to trunk as i don have the trunk code with me now.. (will apply it later once i go home).
Just one question i have here..
If the META region moves newly to an RS we instantiate the HLog for META.
If an RS that had the META once and again gets back the META in that case we should be pointing to the same HLog right?
May be if the code does it already pardon me....because i could not apply the code i had this doubt.
> Have HLog files for .META. edits only
> -------------------------------------
>
> Key: HBASE-7213
> URL: https://issues.apache.org/jira/browse/HBASE-7213
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions from the other regions' edits w.r.t where the edits are written. This jira is to track an implementation of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira