You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Aljoscha Krettek (Jira)" <ji...@apache.org> on 2019/11/27 08:30:00 UTC
[jira] [Commented] (FLINK-14968) Kerberized YARN on Docker test
(custom fs plugin) fails on Travis
[ https://issues.apache.org/jira/browse/FLINK-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16983279#comment-16983279 ]
Aljoscha Krettek commented on FLINK-14968:
------------------------------------------
I disabled the tests for now to get a better CI signal: https://github.com/apache/flink/commit/5374cfec2231fb14a70bd786424a18daee383231
> Kerberized YARN on Docker test (custom fs plugin) fails on Travis
> -----------------------------------------------------------------
>
> Key: FLINK-14968
> URL: https://issues.apache.org/jira/browse/FLINK-14968
> Project: Flink
> Issue Type: Bug
> Components: FileSystems, Tests
> Affects Versions: 1.10.0
> Reporter: Gary Yao
> Priority: Blocker
> Labels: test-stability
> Fix For: 1.10.0
>
>
> This change made the test flaky: https://github.com/apache/flink/commit/749965348170e4608ff2a23c9617f67b8c341df5. It changes the job to have two sources instead of one which, under normal circumstances, requires too many slots to run and therefore the job will fail.
> The setup of this test is very intricate, we configure YARN to have two NodeManagers with 2500mb memory each: https://github.com/apache/flink/blob/413a77157caf25dbbfb8b0caaf2c9e12c7374d98/flink-end-to-end-tests/test-scripts/docker-hadoop-secure-cluster/config/yarn-site.xml#L39. We run the job with parallelism 3 and configure Flink to use 1000mb as TaskManager memory and 1000mb of JobManager memory. This means that the job fits into the YARN memory budget but more TaskManagers would not fit. We also don't simply increase the YARN resources because we want the Flink job to use TMs on different NMs because we had a bug where Kerberos config file shipping was not working correctly but the bug was not materialising if all TMs where on the same NM.
> https://api.travis-ci.org/v3/job/612782888/log.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)