You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2018/03/09 00:49:40 UTC
[kudu-CR] KUDU-2320 apply exp back-off while deleting replica
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/9561
Change subject: KUDU-2320 apply exp back-off while deleting replica
......................................................................
KUDU-2320 apply exp back-off while deleting replica
In some scenarios, the replica to remove might be on a tablet
server which hasn't yet registered with the master. For example,
that happens when a tablet server where the replica had been hosted
went down and stays so when master is restarted. Such a scenario
is exercised by RaftConsensusNonVoterITest::RestartClusterWithNonVoter.
I ran the RaftConsensusNonVoterITest::RestartClusterWithNonVoter
scenario before and after the fix. Before the fix there was a steady
high rate of messages, and after the fix the rate of messages stated
following the exponential back-off pattern.
An example of the output before the fix:
I0309 00:07:34.972404 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 13 ms (attempt = 0)
W0309 00:07:34.972436 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
I0309 00:07:34.985633 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 28 ms (attempt = 0)
W0309 00:07:34.985673 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
I0309 00:07:35.014024 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 26 ms (attempt = 0)
W0309 00:07:35.014062 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
I0309 00:07:35.040323 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 19 ms (attempt = 0)
W0309 00:07:35.040377 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
I0309 00:07:35.059588 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 50 ms (attempt = 0)
W0309 00:07:35.059628 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
An example of the output after the fix:
I0308 22:36:59.251387 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 37 ms (attempt = 2)
W0308 22:36:59.251437 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:36:59.288799 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 84 ms (attempt = 3)
W0308 22:36:59.288851 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:36:59.373152 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 146 ms (attempt = 4)
W0308 22:36:59.373209 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:36:59.519738 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 267 ms (attempt = 5)
W0308 22:36:59.519806 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:36:59.787600 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 554 ms (attempt = 6)
W0308 22:36:59.787657 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:37:00.342607 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 1056 ms (attempt = 7)
W0308 22:37:00.342682 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:37:01.399219 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 2094 ms (attempt = 8)
W0308 22:37:01.399274 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:37:03.494213 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 4125 ms (attempt = 9)
Change-Id: Ia12d261d7270aae7fafe877780b547d262aef16d
---
M src/kudu/master/catalog_manager.cc
1 file changed, 4 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/9561/1
--
To view, visit http://gerrit.cloudera.org:8080/9561
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia12d261d7270aae7fafe877780b547d262aef16d
Gerrit-Change-Number: 9561
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
[kudu-CR] KUDU-2320 apply exponential back-off while deleting replica
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/9561 )
Change subject: KUDU-2320 apply exponential back-off while deleting replica
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/9561
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia12d261d7270aae7fafe877780b547d262aef16d
Gerrit-Change-Number: 9561
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 12 Mar 2018 17:59:53 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-2320 apply exponential back-off while deleting replica
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Kudu Jenkins, Adar Dembo, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9561
to look at the new patch set (#2).
Change subject: KUDU-2320 apply exponential back-off while deleting replica
......................................................................
KUDU-2320 apply exponential back-off while deleting replica
In some scenarios, the replica to remove might be on a tablet
server which hasn't yet registered with the master. For example,
that happens when a tablet server where the replica had been hosted
went down and stays so when master is restarted. Such a scenario
is exercised by RaftConsensusNonVoterITest::RestartClusterWithNonVoter.
I ran the RaftConsensusNonVoterITest::RestartClusterWithNonVoter
scenario before and after the fix. Before the fix there was a steady
high rate of messages, and after the fix the rate of messages stated
following the exponential back-off pattern.
An example of the output before the fix:
I0309 00:07:34.972404 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 13 ms (attempt = 0)
W0309 00:07:34.972436 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
I0309 00:07:34.985633 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 28 ms (attempt = 0)
W0309 00:07:34.985673 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
I0309 00:07:35.014024 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 26 ms (attempt = 0)
W0309 00:07:35.014062 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
I0309 00:07:35.040323 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 19 ms (attempt = 0)
W0309 00:07:35.040377 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
I0309 00:07:35.059588 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 50 ms (attempt = 0)
W0309 00:07:35.059628 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
An example of the output after the fix:
I0308 22:36:59.251387 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 37 ms (attempt = 2)
W0308 22:36:59.251437 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:36:59.288799 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 84 ms (attempt = 3)
W0308 22:36:59.288851 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:36:59.373152 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 146 ms (attempt = 4)
W0308 22:36:59.373209 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:36:59.519738 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 267 ms (attempt = 5)
W0308 22:36:59.519806 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:36:59.787600 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 554 ms (attempt = 6)
W0308 22:36:59.787657 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:37:00.342607 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 1056 ms (attempt = 7)
W0308 22:37:00.342682 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:37:01.399219 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 2094 ms (attempt = 8)
W0308 22:37:01.399274 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:37:03.494213 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 4125 ms (attempt = 9)
Change-Id: Ia12d261d7270aae7fafe877780b547d262aef16d
---
M src/kudu/master/catalog_manager.cc
1 file changed, 4 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/9561/2
--
To view, visit http://gerrit.cloudera.org:8080/9561
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia12d261d7270aae7fafe877780b547d262aef16d
Gerrit-Change-Number: 9561
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] KUDU-2320 apply exponential back-off while deleting replica
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/9561 )
Change subject: KUDU-2320 apply exponential back-off while deleting replica
......................................................................
KUDU-2320 apply exponential back-off while deleting replica
In some scenarios, the replica to remove might be on a tablet
server which hasn't yet registered with the master. For example,
that happens when a tablet server where the replica had been hosted
went down and stays so when master is restarted. Such a scenario
is exercised by RaftConsensusNonVoterITest::RestartClusterWithNonVoter.
I ran the RaftConsensusNonVoterITest::RestartClusterWithNonVoter
scenario before and after the fix. Before the fix there was a steady
high rate of messages, and after the fix the rate of messages stated
following the exponential back-off pattern.
An example of the output before the fix:
I0309 00:07:34.972404 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 13 ms (attempt = 0)
W0309 00:07:34.972436 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
I0309 00:07:34.985633 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 28 ms (attempt = 0)
W0309 00:07:34.985673 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
I0309 00:07:35.014024 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 26 ms (attempt = 0)
W0309 00:07:35.014062 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
I0309 00:07:35.040323 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 19 ms (attempt = 0)
W0309 00:07:35.040377 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
I0309 00:07:35.059588 2029 catalog_manager.cc:2697] Scheduling retry of 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a with a delay of 50 ms (attempt = 0)
W0309 00:07:35.059628 2029 catalog_manager.cc:2716] Async tablet task 832f394938da40ca954da7a842e2279b Delete Tablet RPC for TS=76ea4539475745e8983bab0e501d803a failed: Not found: failed to reset TS proxy: Could not find TS for UUID 76ea4539475745e8983bab0e501d803a
An example of the output after the fix:
I0308 22:36:59.251387 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 37 ms (attempt = 2)
W0308 22:36:59.251437 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:36:59.288799 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 84 ms (attempt = 3)
W0308 22:36:59.288851 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:36:59.373152 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 146 ms (attempt = 4)
W0308 22:36:59.373209 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:36:59.519738 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 267 ms (attempt = 5)
W0308 22:36:59.519806 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:36:59.787600 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 554 ms (attempt = 6)
W0308 22:36:59.787657 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:37:00.342607 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 1056 ms (attempt = 7)
W0308 22:37:00.342682 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:37:01.399219 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 2094 ms (attempt = 8)
W0308 22:37:01.399274 5428 catalog_manager.cc:2719] Async tablet task f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb failed: Not found: failed to reset TS proxy: Could not find TS for UUID 46d5f24c1096492d83b909cd0116edbb
I0308 22:37:03.494213 5428 catalog_manager.cc:2700] Scheduling retry of f259598750084d1db309c1659ee818f9 Delete Tablet RPC for TS=46d5f24c1096492d83b909cd0116edbb with a delay of 4125 ms (attempt = 9)
Change-Id: Ia12d261d7270aae7fafe877780b547d262aef16d
Reviewed-on: http://gerrit.cloudera.org:8080/9561
Reviewed-by: Todd Lipcon <to...@apache.org>
Tested-by: Alexey Serbin <as...@cloudera.com>
---
M src/kudu/master/catalog_manager.cc
1 file changed, 4 insertions(+), 1 deletion(-)
Approvals:
Todd Lipcon: Looks good to me, approved
Alexey Serbin: Verified
--
To view, visit http://gerrit.cloudera.org:8080/9561
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia12d261d7270aae7fafe877780b547d262aef16d
Gerrit-Change-Number: 9561
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] KUDU-2320 apply exp back-off while deleting replica
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/9561 )
Change subject: KUDU-2320 apply exp back-off while deleting replica
......................................................................
Patch Set 1: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/9561/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/9561/1//COMMIT_MSG@7
PS1, Line 7: KUDU-2320 apply exp back-off while deleting replica
nit: please don't abbreviate "exponential" - makes it harder to search the git log, even if it means it extends past the recommended character limit of the first line
--
To view, visit http://gerrit.cloudera.org:8080/9561
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia12d261d7270aae7fafe877780b547d262aef16d
Gerrit-Change-Number: 9561
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 12 Mar 2018 17:41:08 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2320 apply exponential back-off while deleting replica
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/9561 )
Change subject: KUDU-2320 apply exponential back-off while deleting replica
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/9561/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/9561/1//COMMIT_MSG@7
PS1, Line 7: KUDU-2320 apply exponential back-off while deleting replica
> nit: please don't abbreviate "exponential" - makes it harder to search the
Done.
--
To view, visit http://gerrit.cloudera.org:8080/9561
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia12d261d7270aae7fafe877780b547d262aef16d
Gerrit-Change-Number: 9561
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 12 Mar 2018 17:49:01 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2320 apply exponential back-off while deleting replica
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/9561 )
Change subject: KUDU-2320 apply exponential back-off while deleting replica
......................................................................
Patch Set 2: Verified+1
Seems to be a flake in CreateTableStressTest.CreateAndDeleteBigTable. I'll will take a that separately -- most likely it's unrelated to this change.
--
To view, visit http://gerrit.cloudera.org:8080/9561
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia12d261d7270aae7fafe877780b547d262aef16d
Gerrit-Change-Number: 9561
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 12 Mar 2018 18:19:42 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-2320 apply exponential back-off while deleting replica
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/9561 )
Change subject: KUDU-2320 apply exponential back-off while deleting replica
......................................................................
Removed reviewer Kudu Jenkins with the following votes:
* Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/9561
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Ia12d261d7270aae7fafe877780b547d262aef16d
Gerrit-Change-Number: 9561
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>