You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Adar Dembo (Code Review)" <ge...@cloudera.org> on 2017/06/23 21:01:27 UTC
[kudu-CR] KUDU-2052: disable hole repunching at startup on xfs filesystems
Hello Jean-Daniel Cryans, Todd Lipcon,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/7279
to review the following change.
Change subject: KUDU-2052: disable hole repunching at startup on xfs filesystems
......................................................................
KUDU-2052: disable hole repunching at startup on xfs filesystems
As a quick and dirty workaround for (what we believe to be) the root cause
of KUDU-2052, let's disable hole repunching at startup for containers that
reside on xfs filesystems. This toggle is gated behind a gflag which is
disabled (i.e. hole repunching is enabled) in the CLI and unit tests.
Once the real fix for KUDU-2052 (i.e. use the faster ioctl-based hole punch
method) is merged, this will be reverted.
To test, I ran ctest on a loopback mounted xfs filesystem. I also created a
single node deployment on this filesystem, loaded ~10k (live and dead)
blocks into it across ~100 containers. Then I restarted the tserver under
strace, with metadata compaction disabled and with
--log-container-max-blocks set to a special value so that the LBM would
consider all of the containers full. The strace log showed that all of the
container data files were all truncated at startup (because they were full
and had extra space at the end), but there were no hole punches.
Change-Id: Ifc0e935775da0817452ee90f98eea4edea3259d4
---
M src/kudu/fs/log_block_manager-test.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tools/tool_action_fs.cc
3 files changed, 22 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/7279/1
--
To view, visit http://gerrit.cloudera.org:8080/7279
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifc0e935775da0817452ee90f98eea4edea3259d4
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] KUDU-2052: disable hole repunching at startup on xfs filesystems
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7279
to look at the new patch set (#2).
Change subject: KUDU-2052: disable hole repunching at startup on xfs filesystems
......................................................................
KUDU-2052: disable hole repunching at startup on xfs filesystems
As a quick and dirty workaround for (what we believe to be) the root cause
of KUDU-2052, let's disable hole repunching at startup for containers that
reside on xfs filesystems. This toggle is gated behind a gflag which is
disabled (i.e. hole repunching is enabled) in the CLI and unit tests.
Once the real fix for KUDU-2052 (i.e. use the faster ioctl-based hole punch
method) is merged, this will be reverted.
To test, I ran ctest on a loopback mounted xfs filesystem. I also created a
single node deployment on this filesystem, loaded ~10k (live and dead)
blocks into it across ~100 containers. Then I restarted the tserver under
strace, with metadata compaction disabled and with
--log-container-max-blocks set to a special value so that the LBM would
consider all of the containers full. The strace log showed that all of the
container data files were all truncated at startup (because they were full
and had extra space at the end), but there were no hole punches.
Change-Id: Ifc0e935775da0817452ee90f98eea4edea3259d4
---
M src/kudu/fs/log_block_manager-test.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tools/tool_action_fs.cc
3 files changed, 23 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/7279/2
--
To view, visit http://gerrit.cloudera.org:8080/7279
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifc0e935775da0817452ee90f98eea4edea3259d4
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] KUDU-2052: disable hole repunching at startup on xfs filesystems
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.
Change subject: KUDU-2052: disable hole repunching at startup on xfs filesystems
......................................................................
Patch Set 1:
Note: I've also published a revert of this commit at https://gerrit.cloudera.org/#/c/7280. The idea is that although the fix for KUDU-2052 is ready to go (http://gerrit.cloudera.org:8080/7269), it may be too risky for a point release. This patch serves as a less risky workaround that can be backported into a point release.
--
To view, visit http://gerrit.cloudera.org:8080/7279
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifc0e935775da0817452ee90f98eea4edea3259d4
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No
[kudu-CR] KUDU-2052: disable hole repunching at startup on xfs filesystems
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.
Change subject: KUDU-2052: disable hole repunching at startup on xfs filesystems
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7279/1/src/kudu/tools/tool_action_fs.cc
File src/kudu/tools/tool_action_fs.cc:
Line 78: FLAGS_log_block_manager_disable_hole_repunching_on_xfs = false;
> i think we should only do this if it's not set by the CLI. There is an API
Done
--
To view, visit http://gerrit.cloudera.org:8080/7279
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifc0e935775da0817452ee90f98eea4edea3259d4
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2052: disable hole repunching at startup on xfs filesystems
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: KUDU-2052: disable hole repunching at startup on xfs filesystems
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7279/1/src/kudu/tools/tool_action_fs.cc
File src/kudu/tools/tool_action_fs.cc:
Line 78: FLAGS_log_block_manager_disable_hole_repunching_on_xfs = false;
i think we should only do this if it's not set by the CLI. There is an API like SET_DEFAULT which we could use to do so. This way if someone wants to perform the other portions of check without doing the repunching, they could do so by explicitly setting the flag true
--
To view, visit http://gerrit.cloudera.org:8080/7279
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifc0e935775da0817452ee90f98eea4edea3259d4
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2052: disable hole repunching at startup on xfs filesystems
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has submitted this change and it was merged.
Change subject: KUDU-2052: disable hole repunching at startup on xfs filesystems
......................................................................
KUDU-2052: disable hole repunching at startup on xfs filesystems
As a quick and dirty workaround for (what we believe to be) the root cause
of KUDU-2052, let's disable hole repunching at startup for containers that
reside on xfs filesystems. This toggle is gated behind a gflag which is
disabled (i.e. hole repunching is enabled) in the CLI and unit tests.
Once the real fix for KUDU-2052 (i.e. use the faster ioctl-based hole punch
method) is merged, this will be reverted.
To test, I ran ctest on a loopback mounted xfs filesystem. I also created a
single node deployment on this filesystem, loaded ~10k (live and dead)
blocks into it across ~100 containers. Then I restarted the tserver under
strace, with metadata compaction disabled and with
--log-container-max-blocks set to a special value so that the LBM would
consider all of the containers full. The strace log showed that all of the
container data files were all truncated at startup (because they were full
and had extra space at the end), but there were no hole punches.
Change-Id: Ifc0e935775da0817452ee90f98eea4edea3259d4
Reviewed-on: http://gerrit.cloudera.org:8080/7279
Reviewed-by: Todd Lipcon <to...@apache.org>
Tested-by: Kudu Jenkins
---
M src/kudu/fs/log_block_manager-test.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tools/tool_action_fs.cc
3 files changed, 23 insertions(+), 1 deletion(-)
Approvals:
Todd Lipcon: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/7279
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ifc0e935775da0817452ee90f98eea4edea3259d4
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] KUDU-2052: disable hole repunching at startup on xfs filesystems
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: KUDU-2052: disable hole repunching at startup on xfs filesystems
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/7279
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifc0e935775da0817452ee90f98eea4edea3259d4
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No