You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Will Berkeley (Code Review)" <ge...@cloudera.org> on 2019/05/16 21:10:26 UTC
[kudu-CR] [backup] Add a basic CLI tool
Will Berkeley has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13356
Change subject: [backup] Add a basic CLI tool
......................................................................
[backup] Add a basic CLI tool
This adds a CLI tool. For a set of tables, it can
- Print out the latest backup
- Print out the sequence of backups that would be used by a restore job
- Print out all backups, ordered by end time
An example invocation looks like
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backups \
list_restore_sequence \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e foo
The current restore path for 2 table(s):
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T09:38:54.642-0700 | 0 | 1557765534642 | full
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T10:33:23.669-0700 | 1557765534642 | 1557768803669 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
No backups were found for 1 table(s):
foo
Notice how, due to table renames, more than one table can match a table
name, and, of course, there may be no table for a name.
Another example:
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backup-tests-again \
list_latest \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e
Latest backup for 2 table(s):
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
OK
Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
---
M java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/BackupGraph.scala
A java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala
2 files changed, 149 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13356/1
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 1
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has removed a vote on this change.
Change subject: [backup] Add a basic CLI tool
......................................................................
Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 6
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Kudu Jenkins, Adar Dembo, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13356
to look at the new patch set (#3).
Change subject: [backup] Add a basic CLI tool
......................................................................
[backup] Add a basic CLI tool
This adds a CLI tool. For a set of tables, it can
- Print out the latest backup
- Print out the sequence of backups that would be used by a restore job
- Print out all backups, ordered by end time
An example invocation looks like
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backups \
list_restore_sequence \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e foo
The current restore path for 2 table(s):
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T09:38:54.642-0700 | 0 | 1557765534642 | full
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T10:33:23.669-0700 | 1557765534642 | 1557768803669 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
No backups were found for 1 table(s):
foo
Notice how, due to table renames, more than one table can match a table
name, and, of course, there may be no table for a name.
Another example:
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backup-tests-again \
list_latest \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e
Latest backup for 2 table(s):
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
---
M java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/BackupGraph.scala
A java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala
2 files changed, 164 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13356/3
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 3
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13356
to look at the new patch set (#2).
Change subject: [backup] Add a basic CLI tool
......................................................................
[backup] Add a basic CLI tool
This adds a CLI tool. For a set of tables, it can
- Print out the latest backup
- Print out the sequence of backups that would be used by a restore job
- Print out all backups, ordered by end time
An example invocation looks like
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backups \
list_restore_sequence \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e foo
The current restore path for 2 table(s):
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T09:38:54.642-0700 | 0 | 1557765534642 | full
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T10:33:23.669-0700 | 1557765534642 | 1557768803669 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
No backups were found for 1 table(s):
foo
Notice how, due to table renames, more than one table can match a table
name, and, of course, there may be no table for a name.
Another example:
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backup-tests-again \
list_latest \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e
Latest backup for 2 table(s):
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
OK
Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
---
M java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/BackupGraph.scala
A java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala
2 files changed, 164 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13356/2
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 2
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Kudu Jenkins, Adar Dembo, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13356
to look at the new patch set (#5).
Change subject: [backup] Add a basic CLI tool
......................................................................
[backup] Add a basic CLI tool
This adds a CLI tool. For a set of tables, it can
- Print out the latest backup
- Print out the sequence of backups that would be used by a restore job
- Print out all backups, ordered by end time
An example invocation looks like
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backups \
list_restore_sequence \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e foo
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T09:38:54.642-0700 | 0 | 1557765534642 | full
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T10:33:23.669-0700 | 1557765534642 | 1557768803669 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
No backups were found for 1 table(s):
foo
Notice how, due to table renames, more than one table can match a table
name, and, of course, there may be no table for a name.
Another example:
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backup-tests-again \
list_latest \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
The tool also supports TSV and CSV output, and listing out information
about all backups by omitting an explicit list of tables.
Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
---
M java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/BackupGraph.scala
M java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/BackupIO.scala
A java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala
A java/kudu-backup-tools/src/test/scala/org/apache/kudu/backup/TestKuduBackupCLI.scala
4 files changed, 423 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13356/5
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 5
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change. ( http://gerrit.cloudera.org:8080/13356 )
Change subject: [backup] Add a basic CLI tool
......................................................................
Patch Set 4:
Looks good. Can you add a simple unit test to just verify that the tool basically works? I'm OK with not meticulously validating everything, but basically triggering a couple of backups and then running the tool in tsv mode and parsing the output to show that the tables we expected are shown in the output seems simple to do and would warn us if the tool ever became totally broken for some reason.
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 4
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Tue, 21 May 2019 20:54:29 +0000
Gerrit-HasComments: No
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/13356 )
Change subject: [backup] Add a basic CLI tool
......................................................................
Patch Set 4:
> Looks good. Can you add a simple unit test to just verify that the
> tool basically works? I'm OK with not meticulously validating
> everything, but basically triggering a couple of backups and then
> running the tool in tsv mode and parsing the output to show that
> the tables we expected are shown in the output seems simple to do
> and would warn us if the tool ever became totally broken for some
> reason.
Done
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 4
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Wed, 22 May 2019 08:53:43 +0000
Gerrit-HasComments: No
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/13356 )
Change subject: [backup] Add a basic CLI tool
......................................................................
Patch Set 3:
Done. Need to test out on a real cluster (we lost the one where i had a bunch of backups handy).
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 3
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Tue, 21 May 2019 16:58:22 +0000
Gerrit-HasComments: No
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Kudu Jenkins, Adar Dembo, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13356
to look at the new patch set (#6).
Change subject: [backup] Add a basic CLI tool
......................................................................
[backup] Add a basic CLI tool
This adds a CLI tool. For a set of tables, it can
- Print out the latest backup
- Print out the sequence of backups that would be used by a restore job
- Print out all backups, ordered by end time
An example invocation looks like
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backups \
list_restore_sequence \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e foo
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T09:38:54.642-0700 | 0 | 1557765534642 | full
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T10:33:23.669-0700 | 1557765534642 | 1557768803669 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
No backups were found for 1 table(s):
foo
Notice how, due to table renames, more than one table can match a table
name, and, of course, there may be no table for a name.
Another example:
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backup-tests-again \
list_latest \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
The tool also supports TSV and CSV output, and listing out information
about all backups by omitting an explicit list of tables.
Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
---
M java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/BackupGraph.scala
M java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/BackupIO.scala
A java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala
A java/kudu-backup-tools/src/test/scala/org/apache/kudu/backup/TestKuduBackupCLI.scala
4 files changed, 430 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13356/6
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 6
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change. ( http://gerrit.cloudera.org:8080/13356 )
Change subject: [backup] Add a basic CLI tool
......................................................................
Patch Set 6: Code-Review+2
Looks great
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 6
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Wed, 22 May 2019 17:29:49 +0000
Gerrit-HasComments: No
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/13356 )
Change subject: [backup] Add a basic CLI tool
......................................................................
Patch Set 6: Verified+1
Unrelated flakes in TestSecureKuduSink and TestHybridTime. I have a patch out to fix the latter.
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 6
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Wed, 22 May 2019 17:53:41 +0000
Gerrit-HasComments: No
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13356 )
Change subject: [backup] Add a basic CLI tool
......................................................................
[backup] Add a basic CLI tool
This adds a CLI tool. For a set of tables, it can
- Print out the latest backup
- Print out the sequence of backups that would be used by a restore job
- Print out all backups, ordered by end time
An example invocation looks like
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backups \
list_restore_sequence \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e foo
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T09:38:54.642-0700 | 0 | 1557765534642 | full
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T10:33:23.669-0700 | 1557765534642 | 1557768803669 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
No backups were found for 1 table(s):
foo
Notice how, due to table renames, more than one table can match a table
name, and, of course, there may be no table for a name.
Another example:
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backup-tests-again \
list_latest \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
The tool also supports TSV and CSV output, and listing out information
about all backups by omitting an explicit list of tables.
Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Reviewed-on: http://gerrit.cloudera.org:8080/13356
Reviewed-by: Mike Percy <mp...@apache.org>
Tested-by: Will Berkeley <wd...@gmail.com>
---
M java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/BackupGraph.scala
M java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/BackupIO.scala
A java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala
A java/kudu-backup-tools/src/test/scala/org/apache/kudu/backup/TestKuduBackupCLI.scala
4 files changed, 430 insertions(+), 4 deletions(-)
Approvals:
Mike Percy: Looks good to me, approved
Will Berkeley: Verified
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 7
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Kudu Jenkins, Adar Dembo, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13356
to look at the new patch set (#4).
Change subject: [backup] Add a basic CLI tool
......................................................................
[backup] Add a basic CLI tool
This adds a CLI tool. For a set of tables, it can
- Print out the latest backup
- Print out the sequence of backups that would be used by a restore job
- Print out all backups, ordered by end time
An example invocation looks like
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backups \
list_restore_sequence \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e foo
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T09:38:54.642-0700 | 0 | 1557765534642 | full
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T10:33:23.669-0700 | 1557765534642 | 1557768803669 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
No backups were found for 1 table(s):
foo
Notice how, due to table renames, more than one table can match a table
name, and, of course, there may be no table for a name.
Another example:
$ java -cp <elided> org.apache.kudu.backup.KuduBackupCLI \
--rootPath=hdfs:///kudu-backup-tests-again \
list_latest \
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e
table name | table id | end time | start timestamp | end timestamp | type
-------------------------------------------------------+----------------------------------+------------------------------+-----------------+---------------+-------------
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | b46191d75c8d46bab6dc76ba0bd345ba | 2019-05-13T11:07:25.358-0700 | 1557768803669 | 1557770845358 | incremental
default.loadgen_auto_69c4838542ea481d81e03e8f66f7731e | 1e0ab43cb925473fbc5acf86db12eb56 | 2019-05-13T14:09:01.518-0700 | 0 | 1557781741518 | full
The tool also supports TSV and CSV output, and listing out information
about all backups by omitting and explicit list of tables.
Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
---
M java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/BackupGraph.scala
M java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/BackupIO.scala
A java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala
3 files changed, 202 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13356/4
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 4
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13356 )
Change subject: [backup] Add a basic CLI tool
......................................................................
Patch Set 1:
(2 comments)
Just passing through...
http://gerrit.cloudera.org:8080/#/c/13356/1/java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala
File java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala:
PS1:
Copyright header.
http://gerrit.cloudera.org:8080/#/c/13356/1/java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala@91
PS1, Line 91: println("OK")
Can we drop this? The exit status can always be derived from $? after running the shell command.
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 1
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 17 May 2019 18:13:10 +0000
Gerrit-HasComments: Yes
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change. ( http://gerrit.cloudera.org:8080/13356 )
Change subject: [backup] Add a basic CLI tool
......................................................................
Patch Set 3:
Looks good. Feature requests:
1. Make the list of tables optional, or provide another command to list all tables.
2. Provide the option for headerless TSV output that is easy to parse with a script, which would make it easy to write a quick test for this CLI tool.
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 3
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Mon, 20 May 2019 21:00:34 +0000
Gerrit-HasComments: No
[kudu-CR] [backup] Add a basic CLI tool
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/13356 )
Change subject: [backup] Add a basic CLI tool
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13356/1/java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala
File java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala:
PS1:
> Copyright header.
Done
http://gerrit.cloudera.org:8080/#/c/13356/1/java/kudu-backup-tools/src/main/scala/org/apache/kudu/backup/KuduBackupCLI.scala@91
PS1, Line 91: println("OK")
> Can we drop this? The exit status can always be derived from $? after runni
Done
--
To view, visit http://gerrit.cloudera.org:8080/13356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5a4d3b44c77a06fa1b5d2b09506ba5a6e1b52c1
Gerrit-Change-Number: 13356
Gerrit-PatchSet: 1
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Mon, 20 May 2019 16:38:21 +0000
Gerrit-HasComments: Yes