You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Grant Henke (Code Review)" <ge...@cloudera.org> on 2019/06/21 03:02:17 UTC
[kudu-CR](branch-1.10.x) [backup] Deflake TestKuduBackup.testRandomBackupAndRestore
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13700
Change subject: [backup] Deflake TestKuduBackup.testRandomBackupAndRestore
......................................................................
[backup] Deflake TestKuduBackup.testRandomBackupAndRestore
TestKuduBackup.testRandomBackupAndRestore creates a random table with
random partitions. In a run with TSAN binaries, I saw the following
partition schema:
partition_schema {
hash_bucket_schemas {
columns {
name: "int160"
}
num_buckets: 8
seed: 395460932
}
hash_bucket_schemas {
columns {
name: "int321"
}
num_buckets: 6
seed: 2072428334
}
hash_bucket_schemas {
columns {
name: "int82"
}
num_buckets: 9
seed: 138096156
}
}
So there were 8 * 6 * 9 = 432 tablets being created on a 3-tserver
minicluster running TSAN. As a result, the test timed out waiting for
the table to finish creating.
To prevent creating tables with too many tablets, this patch reduces the
maximum number of hash buckets to 3 and it reduces the maximum nesting of
hash partitioning to 2. Combined with the existing maximum of 8 range
partitions, this means no table will consist of more than 9 * 8 = 72
tablets. Hopefully this is few enough that create table will always
finish in a timely manner.
Change-Id: Id937f4513bb0ebaf6dec345a141ed9aef2c44388
Reviewed-on: http://gerrit.cloudera.org:8080/13693
Reviewed-by: Grant Henke <gr...@apache.org>
Tested-by: Kudu Jenkins
(cherry picked from commit 648b5b85aad0df92ab86ac1a9130d3ffdc7d92d9)
---
M java/kudu-client/src/main/java/org/apache/kudu/util/SchemaGenerator.java
1 file changed, 2 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/00/13700/1
--
To view, visit http://gerrit.cloudera.org:8080/13700
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: branch-1.10.x
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id937f4513bb0ebaf6dec345a141ed9aef2c44388
Gerrit-Change-Number: 13700
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR](branch-1.10.x) [backup] Deflake TestKuduBackup.testRandomBackupAndRestore
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13700 )
Change subject: [backup] Deflake TestKuduBackup.testRandomBackupAndRestore
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/13700
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: branch-1.10.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Id937f4513bb0ebaf6dec345a141ed9aef2c44388
Gerrit-Change-Number: 13700
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Fri, 21 Jun 2019 15:53:48 +0000
Gerrit-HasComments: No
[kudu-CR](branch-1.10.x) [backup] Deflake TestKuduBackup.testRandomBackupAndRestore
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13700 )
Change subject: [backup] Deflake TestKuduBackup.testRandomBackupAndRestore
......................................................................
[backup] Deflake TestKuduBackup.testRandomBackupAndRestore
TestKuduBackup.testRandomBackupAndRestore creates a random table with
random partitions. In a run with TSAN binaries, I saw the following
partition schema:
partition_schema {
hash_bucket_schemas {
columns {
name: "int160"
}
num_buckets: 8
seed: 395460932
}
hash_bucket_schemas {
columns {
name: "int321"
}
num_buckets: 6
seed: 2072428334
}
hash_bucket_schemas {
columns {
name: "int82"
}
num_buckets: 9
seed: 138096156
}
}
So there were 8 * 6 * 9 = 432 tablets being created on a 3-tserver
minicluster running TSAN. As a result, the test timed out waiting for
the table to finish creating.
To prevent creating tables with too many tablets, this patch reduces the
maximum number of hash buckets to 3 and it reduces the maximum nesting of
hash partitioning to 2. Combined with the existing maximum of 8 range
partitions, this means no table will consist of more than 9 * 8 = 72
tablets. Hopefully this is few enough that create table will always
finish in a timely manner.
Change-Id: Id937f4513bb0ebaf6dec345a141ed9aef2c44388
Reviewed-on: http://gerrit.cloudera.org:8080/13693
Reviewed-by: Grant Henke <gr...@apache.org>
Tested-by: Kudu Jenkins
(cherry picked from commit 648b5b85aad0df92ab86ac1a9130d3ffdc7d92d9)
Reviewed-on: http://gerrit.cloudera.org:8080/13700
Reviewed-by: Andrew Wong <aw...@cloudera.com>
---
M java/kudu-client/src/main/java/org/apache/kudu/util/SchemaGenerator.java
1 file changed, 2 insertions(+), 2 deletions(-)
Approvals:
Kudu Jenkins: Verified
Andrew Wong: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/13700
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: branch-1.10.x
Gerrit-MessageType: merged
Gerrit-Change-Id: Id937f4513bb0ebaf6dec345a141ed9aef2c44388
Gerrit-Change-Number: 13700
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>