You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Andrew Wong (Code Review)" <ge...@cloudera.org> on 2017/12/06 17:55:02 UTC
[kudu-CR] docs: updates to data dir behavior
Andrew Wong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/8778
Change subject: docs: updates to data dir behavior
......................................................................
docs: updates to data dir behavior
Kudu tservers are now able to survive select disk failures, as well as
start up with new data dirs.
For a rendered version, see:
https://github.com/andrwng/kudu/blob/e49bd7059ba88f1584664a48f69a0aa6a7adefe0/docs/administration.adoc#change_dir_config
Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
---
M docs/administration.adoc
1 file changed, 84 insertions(+), 28 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/78/8778/1
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8778 )
Change subject: docs: updates to data dir behavior
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Wed, 06 Dec 2017 23:58:02 +0000
Gerrit-HasComments: No
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8778 )
Change subject: docs: updates to data dir behavior
......................................................................
Patch Set 2: Verified+1
This is a docs change.
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Wed, 06 Dec 2017 19:17:52 +0000
Gerrit-HasComments: No
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has removed a vote on this change.
Change subject: docs: updates to data dir behavior
......................................................................
Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8778 )
Change subject: docs: updates to data dir behavior
......................................................................
Patch Set 3:
(12 comments)
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc
File docs/administration.adoc:
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@665
PS2, Line 665: to configure servers to store data in multiple
> For a bit extra clarity, especially for less savvy users, can we say someth
Done
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@666
PS2, Line 666: devices. O
> started? What exactly does "configured" mean?
Done
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@671
PS2, Line 671: o an existing master or tablet se
> nit: I think this breaks up the sentence. Can it be made into a second sent
Done
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@683
PS2, Line 683: it reorderi
> nit: remove
Done
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@684
PS2, Line 684: etadata directory". For example if
> Should we warn that all of the tablets might get evicted if the window last
Done
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@703
PS2, Line 703:
> Maybe this is a techie minefield but I thought `service` was old init style
Hmm, I'm not particularly familiar with that, I just kept what was there in the original doc. Think I should use sysctl/systemd instead? Or maybe take out the command (since they can start it however they so want!)?
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@712
PS2, Line 712: r. If using CM, make sure to only update the configurations of the updated
: server, rather than of the entire Kudu service.
:
> Oh hey you answered my question. Can you move this info to where you first
Done
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@721
PS2, Line 721:
> Even though it's an implementation detail, maybe we should say why this res
Done
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@722
PS2, Line 722:
: +
> Can you make this clear that the data on it will be lost and must be re-rep
Done
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@741
PS2, Line 741: rs=/data/1/kudu-tserver,/data/2/kudu-tserver`, the following
: command
> Do you mean you need to update the config in CM? Or CM has a "update the di
The former. Done
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@746
PS2, Line 746:
: $ rm -rf /data/0/kudu-tserver-wal/* /data/1/ku
> After reading this section I'm confused on what I need to do when I use CM,
Done
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@760
PS2, Line 760: .
: === Recovering fr
> How would I know if this happened to one of my tablet servers? Just log mes
Yup, logs. Done
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Wed, 06 Dec 2017 22:37:28 +0000
Gerrit-HasComments: Yes
[kudu-CR] docs: updates to data dir behavior
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change. ( http://gerrit.cloudera.org:8080/8778 )
Change subject: docs: updates to data dir behavior
......................................................................
Patch Set 4: Code-Review+1
(1 comment)
just one moretypo nit
http://gerrit.cloudera.org:8080/#/c/8778/4/docs/administration.adoc
File docs/administration.adoc:
http://gerrit.cloudera.org:8080/#/c/8778/4/docs/administration.adoc@779
PS4, Line 779: directories
directory
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Fri, 08 Dec 2017 00:26:28 +0000
Gerrit-HasComments: Yes
[kudu-CR] docs: updates to data dir behavior
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change. ( http://gerrit.cloudera.org:8080/8778 )
Change subject: docs: updates to data dir behavior
......................................................................
Patch Set 3:
(11 comments)
looks good. added a few suggestions / thoughts, lmk what you think
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc
File docs/administration.adoc:
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@731
PS3, Line 731: the server must be emptied
the wal and data directories on the server must be deleted
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@732
PS3, Line 732: .
, destroying the local copy of the data for each tablet replica hosted on the local server.
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@733
PS3, Line 733: .
if the replication factor is at least 3 and all other tablet servers are online and healthy
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@738
PS3, Line 738: server
this will work on masters too, right? might want to add a note that these are instructions for tablet servers but the process is similar for master servers as well
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@747
PS3, Line 747: rm -rf /data/0/kudu-tserver-wal/* /data/1/kudu-tserver/* /data/2/kudu-tserver/*
add comment:
# Note: this will delete all of the data from the local tablet server
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@755
PS3, Line 755: emptied
deleted
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@756
PS3, Line 756: created with the appropriate permissions
You didn't mention this. shouldn't this happen automatically? Perhaps we should mention that it's automatic. How can they tell whether it worked?
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@762
PS3, Line 762: All directories on Kudu masters and select directories (the WAL directory and
: the metadata directory) of tablet servers are not resilient to disk failures.
How about: As of Kudu 1.6.0, Kudu master servers are not resilient to any types of disk failures. Kudu tablet servers are only resilient to disk failures if they occur on a disk storing data blocks, so the failure of a disk where the write-ahead logs or tablet metadata are stored will still result in data loss of the entire tablet server.
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@764
PS3, Line 764: these
non-resilient
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@769
PS3, Line 769: failures of the other directories
maybe: a disk failure on one of the failure-resilient directories (?)
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@769
PS3, Line 769: automatically
: re-replicate tablets with data on the affected disk, and the affected server
: will remain alive with the logs indicating the failure, for example:
:
automatically stop using the affected disk, shut down tablets with blocks on the affected disk, and re-replicate the affected tablets to other tablet servers. The affected server will remain alive and will print messages to the log indicating the disk failure, for example:
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Thu, 07 Dec 2017 20:33:42 +0000
Gerrit-HasComments: Yes
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Will Berkeley, Mike Percy, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8778
to look at the new patch set (#5).
Change subject: docs: updates to data dir behavior
......................................................................
docs: updates to data dir behavior
Kudu tservers are now able to survive select disk failures, as well as
start up with new data dirs.
For a rendered version, see:
https://github.com/andrwng/kudu/blob/df_docs/docs/administration.adoc#change_dir_config
Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
---
M docs/administration.adoc
1 file changed, 112 insertions(+), 28 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/78/8778/5
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] docs: updates to data dir behavior
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change. ( http://gerrit.cloudera.org:8080/8778 )
Change subject: docs: updates to data dir behavior
......................................................................
Patch Set 5: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Fri, 08 Dec 2017 03:39:39 +0000
Gerrit-HasComments: No
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8778 )
Change subject: docs: updates to data dir behavior
......................................................................
Patch Set 3:
(11 comments)
Yeah, I think the changes make sense.
Also going with failure-tolerant/failure-intolerant instead of "resilient".
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc
File docs/administration.adoc:
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@731
PS3, Line 731: the server must be emptied
> the wal and data directories on the server must be deleted
Done
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@732
PS3, Line 732: .
> , destroying the local copy of the data for each tablet replica hosted on t
Done
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@733
PS3, Line 733: .
> if the replication factor is at least 3 and all other tablet servers are on
Done
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@738
PS3, Line 738: server
> this will work on masters too, right? might want to add a note that these a
Done
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@747
PS3, Line 747: rm -rf /data/0/kudu-tserver-wal/* /data/1/kudu-tserver/* /data/2/kudu-tserver/*
> add comment:
Done
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@755
PS3, Line 755: emptied
> deleted
Done
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@756
PS3, Line 756: created with the appropriate permissions
> You didn't mention this. shouldn't this happen automatically? Perhaps we sh
Ah, good point. The directories are created automatically when they start up so this needn't be in the docs.
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@762
PS3, Line 762: All directories on Kudu masters and select directories (the WAL directory and
: the metadata directory) of tablet servers are not resilient to disk failures.
> How about: As of Kudu 1.6.0, Kudu master servers are not resilient to any t
Done
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@764
PS3, Line 764: these
> non-resilient
Done
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@769
PS3, Line 769: failures of the other directories
> maybe: a disk failure on one of the failure-resilient directories (?)
failure-tolerant?
http://gerrit.cloudera.org:8080/#/c/8778/3/docs/administration.adoc@769
PS3, Line 769: automatically
: re-replicate tablets with data on the affected disk, and the affected server
: will remain alive with the logs indicating the failure, for example:
:
> automatically stop using the affected disk, shut down tablets with blocks o
Done
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Thu, 07 Dec 2017 22:52:33 +0000
Gerrit-HasComments: Yes
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8778 )
Change subject: docs: updates to data dir behavior
......................................................................
Patch Set 5:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/8778/4/docs/administration.adoc
File docs/administration.adoc:
http://gerrit.cloudera.org:8080/#/c/8778/4/docs/administration.adoc@779
PS4, Line 779: directory,
> directory
Done
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Fri, 08 Dec 2017 00:28:44 +0000
Gerrit-HasComments: Yes
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has removed a vote on this change.
Change subject: docs: updates to data dir behavior
......................................................................
Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8778 )
Change subject: docs: updates to data dir behavior
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Thu, 07 Dec 2017 23:16:19 +0000
Gerrit-HasComments: No
[kudu-CR] docs: updates to data dir behavior
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8778 )
Change subject: docs: updates to data dir behavior
......................................................................
docs: updates to data dir behavior
Kudu tservers are now able to survive select disk failures, as well as
start up with new data dirs.
For a rendered version, see:
https://github.com/andrwng/kudu/blob/df_docs/docs/administration.adoc#change_dir_config
Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Reviewed-on: http://gerrit.cloudera.org:8080/8778
Tested-by: Kudu Jenkins
Reviewed-by: Mike Percy <mp...@apache.org>
---
M docs/administration.adoc
1 file changed, 112 insertions(+), 28 deletions(-)
Approvals:
Kudu Jenkins: Verified
Mike Percy: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Will Berkeley, Mike Percy, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8778
to look at the new patch set (#4).
Change subject: docs: updates to data dir behavior
......................................................................
docs: updates to data dir behavior
Kudu tservers are now able to survive select disk failures, as well as
start up with new data dirs.
For a rendered version, see:
https://github.com/andrwng/kudu/blob/df_docs/docs/administration.adoc#change_dir_config
Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
---
M docs/administration.adoc
1 file changed, 112 insertions(+), 28 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/78/8778/4
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8778
to look at the new patch set (#2).
Change subject: docs: updates to data dir behavior
......................................................................
docs: updates to data dir behavior
Kudu tservers are now able to survive select disk failures, as well as
start up with new data dirs.
For a rendered version, see:
https://github.com/andrwng/kudu/blob/df_docs/docs/administration.adoc#change_dir_config
Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
---
M docs/administration.adoc
1 file changed, 84 insertions(+), 28 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/78/8778/2
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] docs: updates to data dir behavior
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/8778 )
Change subject: docs: updates to data dir behavior
......................................................................
Patch Set 2:
(12 comments)
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc
File docs/administration.adoc:
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@665
PS2, Line 665: to configure servers with multiple directories
For a bit extra clarity, especially for less savvy users, can we say something like "to configure servers to store data in multiple directories on different devices"
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@666
PS2, Line 666: configured
started? What exactly does "configured" mean?
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@671
PS2, Line 671: across which data will be striped
nit: I think this breaks up the sentence. Can it be made into a second sentence after this one instead?
"across which data will be striped" -> "Data is striped across data directories, and when a new data directory is added, new data will be striped across the union of the old and new directories"
(I'm not 100% sure how new data dirs get data. Is it only from new tablets? New data? Will compactions on old data put compacted data on new data dirs?)
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@683
PS2, Line 683: relatively
nit: remove
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@684
PS2, Line 684: this offline window should be brief
Should we warn that all of the tablets might get evicted if the window lasts too long?
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@703
PS2, Line 703: service
Maybe this is a techie minefield but I thought `service` was old init style maintained for backwards compat and the new thing is to use sysctl / systemd?
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@712
PS2, Line 712: Only new tablet replicas (i.e. brand new tablets' replicas and replicas
: that are copied to the server for high availability) will use the new
: directory
Oh hey you answered my question. Can you move this info to where you first talk about adding data dirs?
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@721
PS2, Line 721: first configured data directory
Even though it's an implementation detail, maybe we should say why this restriction exists since this restriction is implementation leaking? People will ask
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@722
PS2, Line 722: the server must be
: rebuilt
Can you make this clear that the data on it will be lost and must be re-replicated?
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@741
PS2, Line 741: use update the directory configurations for the server to
: rebuild
Do you mean you need to update the config in CM? Or CM has a "update the directory configurations" command that you use?
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@746
PS2, Line 746: the server process can be
: started with the new directory configuration
After reading this section I'm confused on what I need to do when I use CM, and when I don't use CM. Does CM clear out the data dirs when I use the Update Dir Config command? Can you add an explicit step where you change the wal/data dirs when not using CM?
http://gerrit.cloudera.org:8080/#/c/8778/2/docs/administration.adoc@760
PS2, Line 760: and the affected server
: will remain alive
How would I know if this happened to one of my tablet servers? Just log messages? Can you give a "smoking gun" log message as an example?
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Wed, 06 Dec 2017 21:35:34 +0000
Gerrit-HasComments: Yes
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Will Berkeley, Mike Percy, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8778
to look at the new patch set (#3).
Change subject: docs: updates to data dir behavior
......................................................................
docs: updates to data dir behavior
Kudu tservers are now able to survive select disk failures, as well as
start up with new data dirs.
For a rendered version, see:
https://github.com/andrwng/kudu/blob/df_docs/docs/administration.adoc#change_dir_config
Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
---
M docs/administration.adoc
1 file changed, 100 insertions(+), 28 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/78/8778/3
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] docs: updates to data dir behavior
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has removed a vote on this change.
Change subject: docs: updates to data dir behavior
......................................................................
Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/8778
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I7cfef4aeaba92228d2e0a77c7596847a6a3137e3
Gerrit-Change-Number: 8778
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>