You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Grant Henke (Code Review)" <ge...@cloudera.org> on 2019/04/24 22:56:41 UTC

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test 

Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13109


Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................

[test] Add testOpenScanWithDroppedPartition test

Adds a test that verifies that the Java client correctly
handles partitions being dropped while a scanner
is open and reads all the data.

Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
---
M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java
1 file changed, 48 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/13109/1
-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13109 )

Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/13109/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java:

http://gerrit.cloudera.org:8080/#/c/13109/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java@182
PS1, Line 182: ta
a


http://gerrit.cloudera.org:8080/#/c/13109/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java@211
PS1, Line 211:         // Drop the partition.
             :         client.alterTable(tableName,
             :             new AlterTableOptions().dropRangePartition(bottom, middle));
Is there any way to wait until the partition is actually dropped from the tserver? In C++ tests we have various helpers to do that; not sure if we do here. It'd help cement the point that the tserver retains enough state to finish the scan; as is, we could conceivably finish the scan before the tserver has even dropped the tablet.



-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Comment-Date: Wed, 24 Apr 2019 23:17:38 +0000
Gerrit-HasComments: Yes

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13109

to look at the new patch set (#2).

Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................

[test] Add testOpenScanWithDroppedPartition test

Adds a test that verifies that the Java client correctly
handles partitions being dropped while a scanner
is open and reads all the data.

Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
---
M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java
1 file changed, 57 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/13109/2
-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13109 )

Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Comment-Date: Thu, 25 Apr 2019 14:14:02 +0000
Gerrit-HasComments: No

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13109 )

Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................


Patch Set 2: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Comment-Date: Thu, 25 Apr 2019 02:26:12 +0000
Gerrit-HasComments: No

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13109 )

Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Comment-Date: Thu, 25 Apr 2019 19:54:33 +0000
Gerrit-HasComments: No

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has removed a vote on this change.

Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................


Removed Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13109 )

Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................

[test] Add testOpenScanWithDroppedPartition test

Adds a test that verifies that the Java client correctly
handles partitions being dropped while a scanner
is open and reads all the data.

Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Reviewed-on: http://gerrit.cloudera.org:8080/13109
Tested-by: Grant Henke <gr...@apache.org>
Reviewed-by: Adar Dembo <ad...@cloudera.com>
---
M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java
1 file changed, 57 insertions(+), 2 deletions(-)

Approvals:
  Grant Henke: Verified
  Adar Dembo: Looks good to me, approved

-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13109 )

Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13109/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java:

http://gerrit.cloudera.org:8080/#/c/13109/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java@211
PS1, Line 211:         // Drop the partition.
             :         client.alterTable(tableName,
             :             new AlterTableOptions().dropRangePartition(bottom, middle));
> Is there any way to wait until the partition is actually dropped from the t
I added a change that interrogates the logs.



-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Comment-Date: Thu, 25 Apr 2019 01:48:27 +0000
Gerrit-HasComments: Yes

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13109 )

Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/13109/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java:

http://gerrit.cloudera.org:8080/#/c/13109/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java@182
PS1, Line 182: ta
> a
Missed this?


http://gerrit.cloudera.org:8080/#/c/13109/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java:

http://gerrit.cloudera.org:8080/#/c/13109/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java@216
PS2, Line 216:           // Give time for the background drop operations.
             :           Thread.sleep(1000);
Will this be enough time for ASAN/TSAN builds?


http://gerrit.cloudera.org:8080/#/c/13109/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java@223
PS2, Line 223:         assertTrue(capture.getAppendedText().contains("Deleting tablet data"));
             :         assertTrue(capture.getAppendedText().contains("successfully deleted"));
These are from the cluster itself? Very clever.



-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Comment-Date: Thu, 25 Apr 2019 03:48:23 +0000
Gerrit-HasComments: Yes

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13109

to look at the new patch set (#3).

Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................

[test] Add testOpenScanWithDroppedPartition test

Adds a test that verifies that the Java client correctly
handles partitions being dropped while a scanner
is open and reads all the data.

Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
---
M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java
1 file changed, 57 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/13109/3
-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has removed a vote on this change.

Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................


Removed Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>

[kudu-CR] [test] Add testOpenScanWithDroppedPartition test

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13109 )

Change subject: [test] Add testOpenScanWithDroppedPartition test
......................................................................


Patch Set 2:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/13109/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java:

http://gerrit.cloudera.org:8080/#/c/13109/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java@182
PS1, Line 182: ta
> a
Done


http://gerrit.cloudera.org:8080/#/c/13109/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java@182
PS1, Line 182: ta
> Missed this?
Done


http://gerrit.cloudera.org:8080/#/c/13109/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java:

http://gerrit.cloudera.org:8080/#/c/13109/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java@216
PS2, Line 216:           // Give time for the background drop operations.
             :           Thread.sleep(1000);
> Will this be enough time for ASAN/TSAN builds?
I think so, we can always bump it. On my laptop it takes 40 milliseconds to fully delete the tablet.


http://gerrit.cloudera.org:8080/#/c/13109/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java@223
PS2, Line 223:         assertTrue(capture.getAppendedText().contains("Deleting tablet data"));
             :         assertTrue(capture.getAppendedText().contains("successfully deleted"));
> These are from the cluster itself? Very clever.
Yes, the cluster logs also go to the root logger which the CapturingLogAppender attaches to.



-- 
To view, visit http://gerrit.cloudera.org:8080/13109
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7538260aff7439fa9a5534d680cd2c2b60006549
Gerrit-Change-Number: 13109
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Comment-Date: Thu, 25 Apr 2019 13:11:55 +0000
Gerrit-HasComments: Yes