[jira] [Resolved] (PHOENIX-569) Adding custom memstore implemenation for indexing

["Gabriel Reid (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PHOENIX-569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gabriel Reid resolved PHOENIX-569.
----------------------------------

    Resolution: Fixed

Bulk resolve of closed issues imported from GitHub. This status was reached by first re-opening all closed imported issues and then resolving them in bulk.

> Adding custom memstore implemenation for indexing
> -------------------------------------------------
>
>                 Key: PHOENIX-569
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-569
>             Project: Phoenix
>          Issue Type: Task
>            Reporter: Jason Yates
>
> We basically wrap a KeyValueSkipListSet, just
> like a regular MemStore, except we are (1) not
> dealing with (a) space considerations or (b) a snapshot
> set, and (2) ignoring memstore timestamps in favor of
> deciding when we want to overwrite keys based on how we
> obtain them.
> We can ignore the memstore timestamps because we know that
> anything we get from the local region is going to be MVCC
> visible - so it should just go in. However, we also want
> overwrite any existing state with our pending write that
> we are indexing, so that needs to clobber the KVs we get
> from the HRegion. This got really messy with a regular
> memstore as each KV from the MemStore frequently has a
> higher MemStoreTS, but we can't just up the pending KVs'
> MemStoreTs b/c a memstore relies on the MVCC readpoint,
> which generally is < Long.MAX_VALUE.
> By realizing that we don't need the snapshot or space
> requirements, we can go much faster than the previous
> implementation. Further, by being smart about how we
> manage the KVs, we can drop the extra object creation
> we were doing to wrap the pending KVs (which we did
> previously to ensure they sorted before the ones we got
> from the HRegion).



--
This message was sent by Atlassian JIRA
(v6.2#6252)