[kudu] 01/02: [backup] Deflake TestKuduBackup.testRandomBackupAndRestore

[aw...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

awong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git

commit 648b5b85aad0df92ab86ac1a9130d3ffdc7d92d9
Author: Will Berkeley <wd...@gmail.com>
AuthorDate: Thu Jun 20 14:12:23 2019 -0700

    [backup] Deflake TestKuduBackup.testRandomBackupAndRestore
    
    TestKuduBackup.testRandomBackupAndRestore creates a random table with
    random partitions. In a run with TSAN binaries, I saw the following
    partition schema:
    
    partition_schema {
      hash_bucket_schemas {
        columns {
          name: "int160"
        }
        num_buckets: 8
        seed: 395460932
      }
      hash_bucket_schemas {
        columns {
          name: "int321"
        }
        num_buckets: 6
       seed: 2072428334
      }
      hash_bucket_schemas {
        columns {
          name: "int82"
        }
        num_buckets: 9
        seed: 138096156
      }
    }
    
    So there were 8 * 6 * 9 = 432 tablets being created on a 3-tserver
    minicluster running TSAN. As a result, the test timed out waiting for
    the table to finish creating.
    
    To prevent creating tables with too many tablets, this patch reduces the
    maximum number of hash buckets to 3 and it reduces the maximum nesting of
    hash partitioning to 2. Combined with the existing maximum of 8 range
    partitions, this means no table will consist of more than 9 * 8 = 72
    tablets. Hopefully this is few enough that create table will always
    finish in a timely manner.
    
    Change-Id: Id937f4513bb0ebaf6dec345a141ed9aef2c44388
    Reviewed-on: http://gerrit.cloudera.org:8080/13693
    Reviewed-by: Grant Henke <gr...@apache.org>
    Tested-by: Kudu Jenkins
---
 .../src/main/java/org/apache/kudu/util/SchemaGenerator.java           | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/java/kudu-client/src/main/java/org/apache/kudu/util/SchemaGenerator.java b/java/kudu-client/src/main/java/org/apache/kudu/util/SchemaGenerator.java
index 4116da8..6a97382 100644
--- a/java/kudu-client/src/main/java/org/apache/kudu/util/SchemaGenerator.java
+++ b/java/kudu-client/src/main/java/org/apache/kudu/util/SchemaGenerator.java
@@ -235,11 +235,11 @@ public class SchemaGenerator {
     final List<ColumnSchema> keyColumns = schema.getPrimaryKeyColumns();
 
     // Add hash partitioning (Max out at 3 levels to avoid being excessive).
-    int hashPartitionLevels = random.nextInt(Math.min(keyColumns.size(), 3)) + 1;
+    int hashPartitionLevels = random.nextInt(Math.min(keyColumns.size(), 2)) + 1;
     for (int i = 0; i < hashPartitionLevels; i++) {
       final ColumnSchema hashColumn = keyColumns.get(i);
       // TODO(ghenke): Make buckets configurable.
-      final int hashBuckets = random.nextInt(8) + MIN_HASH_BUCKETS;
+      final int hashBuckets = random.nextInt(2) + MIN_HASH_BUCKETS;
       final int hashSeed = random.nextInt();
       options.addHashPartitions(Arrays.asList(hashColumn.getName()), hashBuckets, hashSeed);
     }