You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Sean Busbey (Jira)" <ji...@apache.org> on 2019/09/12 18:53:00 UTC
[jira] [Comment Edited] (HBASE-22749) Distributed MOB compactions
[ https://issues.apache.org/jira/browse/HBASE-22749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928823#comment-16928823 ]
Sean Busbey edited comment on HBASE-22749 at 9/12/19 6:52 PM:
--------------------------------------------------------------
I just got a clean build off of branch-2.2 ref {{a010bf154ae31c7aa4cb165a34bc7d42a6b70f2f}}.
Since branch-2.2 is going through release candidates ATM, the version numbers don't have SNAPSHOT in them, so you have to be careful to keep Maven from using an artifact from an old build. I use the following line, but you have to be careful because it will forcibly delete any untracked changes in the local working directory:
{code}
git clean -xdf && rm -rf ~/.m2/repository/org/apache/hbase/* && mvn -Dtest=NoTests -Dit.test=NOITs package install verify
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10:41 min
[INFO] Finished at: 2019-09-12T13:51:11-05:00
[INFO] ------------------------------------------------------------------------
{code}
was (Author: busbey):
I just got a clean build off of branch-2.2 ref {{ }}.
Since branch-2.2 is going through release candidates ATM, the version numbers don't have SNAPSHOT in them, so you have to be careful to keep Maven from using an artifact from an old build. I use the following line, but you have to be careful because it will forcibly delete any untracked changes in the local working directory:
{code}
git clean -xdf && rm -rf ~/.m2/repository/org/apache/hbase/* && mvn -Dtest=NoTests -Dit.test=NOITs package install verify
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10:41 min
[INFO] Finished at: 2019-09-12T13:51:11-05:00
[INFO] ------------------------------------------------------------------------
{code}
> Distributed MOB compactions
> ----------------------------
>
> Key: HBASE-22749
> URL: https://issues.apache.org/jira/browse/HBASE-22749
> Project: HBase
> Issue Type: New Feature
> Components: mob
> Reporter: Vladimir Rodionov
> Assignee: Vladimir Rodionov
> Priority: Major
> Attachments: HBASE-22749-branch-2.2-v3.patch, HBase-MOB-2.0-v1.pdf, HBase-MOB-2.0-v2.1.pdf, HBase-MOB-2.0-v2.2.pdf, HBase-MOB-2.0-v2.pdf
>
>
> There are several drawbacks in the original MOB 1.0 (Moderate Object Storage) implementation, which can limit the adoption of the MOB feature:
> # MOB compactions are executed in a Master as a chore, which limits scalability because all I/O goes through a single HBase Master server.
> # Yarn/Mapreduce framework is required to run MOB compactions in a scalable way, but this won’t work in a stand-alone HBase cluster.
> # Two separate compactors for MOB and for regular store files and their interactions can result in a data loss (see HBASE-22075)
> The design goals for MOB 2.0 were to provide 100% MOB 1.0 - compatible implementation, which is free of the above drawbacks and can be used as a drop in replacement in existing MOB deployments. So, these are design goals of a MOB 2.0:
> # Make MOB compactions scalable without relying on Yarn/Mapreduce framework
> # Provide unified compactor for both MOB and regular store files
> # Make it more robust especially w.r.t. to data losses.
> # Simplify and reduce the overall MOB code.
> # Provide 100% compatible implementation with MOB 1.0.
> # No migration of data should be required between MOB 1.0 and MOB 2.0 - just software upgrade.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)