[jira] [Commented] (HBASE-4344) Persist memstoreTS to disk

["stack (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-4344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13099244#comment-13099244 ] 

stack commented on HBASE-4344:
------------------------------

Nice work Amit.

So, you'd just prepend the memstore to the front of the KV that is in hfile?  Is the reason you put it here rather than integrate it better by inserting, say, after the current version/timestamp because you are trying to minimize changes?

Hmm... I see.  You have hfile decorate the KV bytes it picks from the file with the memstoreTS found in the metadata for hfile.

Its kinda ugly the fact that hfile knows about KV internals but its not your fault this is the case -- it has always been so.  Your addition here making the KV from bytes plus the metainfo that has the memstoreTs requires that hfile know about KV since its doing this merge of data from two sources (hfile data and hfile metadata).

Looks like you have to much logging going on.

Why '+      long maxMemstoreTS = -1;'?  Could do with a comment.

This is good '+     rwcc.initialize(maxMemstoreTS + 1);'

You have a note to yourself:

{code}
+    /* Amit: Move this into the HFileReaderV2
+     * 
+     */
+    b = metadataMap.get(HFileWriterV2.MAX_MEMSTORE_KEY);
+    if (b != null) {
+      this.maxMemstoreTS = Bytes.toLong(b);
+    }
{code}

Remove the commented out line:

{code}
+      //new KeyValue(kv.getBuffer(), kv.getOffset(), kv.getLength());
{code}

What you did w/ clone is improvement in line above.

So, this patch is not the prettiest I've seen but i think it could grow on me.  You have done the difficult task of getting the memstoreTS into the filesystem BUT WITHOUT REDOING KV so change is minimally invasive.  I'm in favor of this patch.  A more radical redo of KV that includes the memstoreTS can happen later.

Good stuff Amit.

> Persist memstoreTS to disk
> --------------------------
>
>                 Key: HBASE-4344
>                 URL: https://issues.apache.org/jira/browse/HBASE-4344
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Amitanand Aiyer
>            Assignee: Amitanand Aiyer
>             Fix For: 0.89.20100924
>
>         Attachments: patch-2
>
>
> Atomicity can be achieved in two ways -- (i) by using  a multiversion concurrency system (MVCC), or (ii) by ensuring that "new" writes do not complete, until the "old" reads complete.
> Currently, Memstore uses something along the lines of MVCC (called RWCC for read-write-consistency-control). But, this mechanism is not incorporated for the key-values written to the disk, as they do not include the memstore TS.
> Let us make the two approaches be similar, by persisting the memstoreTS along with the key-value when it is written to the disk.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira