You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Dariusz Aniszewski (JIRA)" <ji...@apache.org> on 2018/05/18 14:45:00 UTC
[jira] [Commented] (BEAM-4041) Performance tests fail due to
kubernetes load balancer problems
[ https://issues.apache.org/jira/browse/BEAM-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480754#comment-16480754 ]
Dariusz Aniszewski commented on BEAM-4041:
------------------------------------------
Since those issues are still appearing, and [this PR to PerfKit|https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/1641] was merged, I submitted [PR 5425|https://github.com/apache/beam/pull/5425] to increase retry timeout from 3 to 6 minutes - please check it and merge it if it makes sense. If increasing timeout is pointless please discard the PR.
> Performance tests fail due to kubernetes load balancer problems
> ---------------------------------------------------------------
>
> Key: BEAM-4041
> URL: https://issues.apache.org/jira/browse/BEAM-4041
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Łukasz Gajowy
> Assignee: Jason Kuster
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Recently, as we added more IOITs to be run on jenkins using kubernetes, some of them started to fail randomly, because they couldn't retrieve LoadBalancer address. Normally obtaining the address took about one minute. Perfkit waits for the address (actively checking for it) for 3 minutes. This should be enough for getting the address, yet it recently started to exceed the 3 minutes limit. I also noticed that this error didn't happen when there were fewer tests.
> Example logs:
> https://builds.apache.org/view/A-D/view/Beam/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/31/console
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)