You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kudu.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2016/05/17 06:48:12 UTC
[jira] [Commented] (KUDU-1342)
LinkedListTest.TestLoadWhileOneServerDownAndVerify flakiness
[ https://issues.apache.org/jira/browse/KUDU-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15286138#comment-15286138 ]
Todd Lipcon commented on KUDU-1342:
-----------------------------------
I believe the root cause of this issue was KUDU-1387
> LinkedListTest.TestLoadWhileOneServerDownAndVerify flakiness
> ------------------------------------------------------------
>
> Key: KUDU-1342
> URL: https://issues.apache.org/jira/browse/KUDU-1342
> Project: Kudu
> Issue Type: Bug
> Components: integration
> Affects Versions: 0.7.0
> Reporter: Jean-Daniel Cryans
> Attachments: llt-1342.log
>
>
> I'm hitting a case where the client fails to scan from the node that remains after we kill two:
> {noformat}
> I0219 09:27:25.552407 8830 meta_cache.cc:635] Marking tablet server 92c6616aee764f2bafdeb5ece5816102 (127.34.126.1:56329) as failed.
> W0219 09:27:25.552430 8830 meta_cache.cc:191] Tablet 2bd2a0aa8c0d4f2890106408638d7860: Replica 92c6616aee764f2bafdeb5ece5816102 (127.34.126.1:56329) has failed: Network error: TS failed: Client connection negotiation failed: client connection to 127.34.126.1:56329: connect: Connection refused (error 111)
> I0219 09:27:25.552924 8830 meta_cache.cc:635] Marking tablet server cf874235214a4471b761e84bad1fdd03 (127.34.126.2:36921) as failed.
> W0219 09:27:25.552945 8830 meta_cache.cc:191] Tablet 2bd2a0aa8c0d4f2890106408638d7860: Replica cf874235214a4471b761e84bad1fdd03 (127.34.126.2:36921) has failed: Network error: TS failed: Client connection negotiation failed: client connection to 127.34.126.2:36921: connect: Connection refused (error 111)
> I0219 09:27:25.553062 8830 meta_cache.cc:635] Marking tablet server 92c6616aee764f2bafdeb5ece5816102 (127.34.126.1:56329) as failed.
> W0219 09:27:25.553074 8830 meta_cache.cc:191] Tablet 2bd2a0aa8c0d4f2890106408638d7860: Replica 92c6616aee764f2bafdeb5ece5816102 (127.34.126.1:56329) has failed: Network error: TS failed: Client connection negotiation failed: client connection to 127.34.126.1:56329: connect: Connection refused (error 111)
> I0219 09:27:25.553458 8830 meta_cache.cc:635] Marking tablet server 92c6616aee764f2bafdeb5ece5816102 (127.34.126.1:56329) as failed.
> W0219 09:27:25.553478 8830 meta_cache.cc:191] Tablet 2bd2a0aa8c0d4f2890106408638d7860: Replica 92c6616aee764f2bafdeb5ece5816102 (127.34.126.1:56329) has failed: Network error: TS failed: Client connection negotiation failed: client connection to 127.34.126.1:56329: connect: Connection refused (error 111)
> I0219 09:27:25.554150 8830 linked_list-test-util.h:826] Done collecting results (0 rows in 0.001179ms)
> {noformat}
> You can see it's trying to hit the two dead nodes. Meanwhile, the survivor 5863a398b4c340aea712e4097c355457 is trying to run a leader election.
> It reproes about 50% of the time on this fast machine I'm using, but setting the verbose logging higher makes it work 100% of the time I tried.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)