You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@kudu.apache.org by "Adar Dembo (Code Review)" <ge...@cloudera.org> on 2017/03/13 06:56:12 UTC

[kudu-CR] KUDU-1769: fs check action with rudimentary data block GC

Hello Dinesh Bhat, Mike Percy, Todd Lipcon,

I'd like you to do a code review.  Please visit

    http://gerrit.cloudera.org:8080/6361

to review the following change.

Change subject: KUDU-1769: fs check action with rudimentary data block GC
......................................................................

KUDU-1769: fs check action with rudimentary data block GC

This commit introduces a new "fs check" action to the CLI tool. For the time
being its capabilities are slim:
- Detecting missing blocks.
- Detecting orphaned blocks, optionally "repairing" (i.e. deleting) them.

I'm sure it'll be augmented to perform more consistency checks and repairs
in the future.

Why build this in the CLI tool and not in the block manager? A few reasons:
1. This "poor man" approach to GCing orphaned data blocks is extremely
   heavy-weight. Doing it at server startup is probably a bad idea.
2. Performance aside, finding a good time at server startup to compare the
   block lists will require additional refactoring and reordering.
3. If we change our minds, we can always move it later.

I do expect that the block manager will take on some check or repair duties,
but they'll be relegated to startup time, and will be much smaller in scope
(e.g. the LBM could repunch holes that previously failed to punch).

Change-Id: I82da8e234c338cd5a7540a22cebfa0a4958388ec
---
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tablet/tablet_metadata.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_fs.cc
M src/kudu/tserver/tablet_copy_source_session.cc
5 files changed, 214 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/6361/1
-- 
To view, visit http://gerrit.cloudera.org:8080/6361
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I82da8e234c338cd5a7540a22cebfa0a4958388ec
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-1769: fs check action with rudimentary data block GC

Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.

Mike Percy has posted comments on this change.

Change subject: KUDU-1769: fs check action with rudimentary data block GC
......................................................................


Patch Set 3:

> It doesn't; see the NO_FATALS() call on RunFsCheck() just before.

Oh, great. I thought read-only mode still took the lock but just enforced read-only behavior.

-- 
To view, visit http://gerrit.cloudera.org:8080/6361
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I82da8e234c338cd5a7540a22cebfa0a4958388ec
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No