You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Martijn Visser (Jira)" <ji...@apache.org> on 2022/07/01 14:21:00 UTC
[jira] [Commented] (FLINK-28355) Python Bash e2e tests don't clean-up after they've ran, causing disk space issues
[ https://issues.apache.org/jira/browse/FLINK-28355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561472#comment-17561472 ]
Martijn Visser commented on FLINK-28355:
----------------------------------------
[~hxbks2ks] [~dianfu] Can either of you help with this?
> Python Bash e2e tests don't clean-up after they've ran, causing disk space issues
> ---------------------------------------------------------------------------------
>
> Key: FLINK-28355
> URL: https://issues.apache.org/jira/browse/FLINK-28355
> Project: Flink
> Issue Type: Bug
> Components: API / Python, Test Infrastructure
> Affects Versions: 1.16.0, 1.15.2, 1.14.6
> Reporter: Martijn Visser
> Priority: Critical
>
> The Bash based E2E tests that are used in Python aren't cleaned-up after they've ran. These cause disk space issues further downstream.
> See the CI run from https://github.com/apache/flink/pull/20114 for results, for example:
> -- When starting with the Bash e2e tests
> {code:java}
> 08:47:10 ##[group]Top 15 biggest directories in terms of used disk space
> Jul 01 08:47:12 3983560 .
> Jul 01 08:47:12 1266692 ./flink-end-to-end-tests
> Jul 01 08:47:12 624568 ./flink-dist
> Jul 01 08:47:12 624180 ./flink-dist/target
> Jul 01 08:47:12 500076 ./flink-dist/target/flink-1.16-SNAPSHOT-bin
> Jul 01 08:47:12 500072 ./flink-dist/target/flink-1.16-SNAPSHOT-bin/flink-1.16-SNAPSHOT
> Jul 01 08:47:12 460812 ./flink-connectors
> Jul 01 08:47:12 392588 ./.git
> Jul 01 08:47:12 366396 ./.git/objects
> Jul 01 08:47:12 366388 ./.git/objects/pack
> Jul 01 08:47:12 349272 ./flink-table
> Jul 01 08:47:12 335592 ./.git/objects/pack/pack-38d46915823ebec2bc660fd160e5cfca5bc3e567.pack
> Jul 01 08:47:12 293044 ./flink-dist/target/flink-1.16-SNAPSHOT-bin/flink-1.16-SNAPSHOT/opt
> Jul 01 08:47:12 251272 ./flink-filesystems
> Jul 01 08:47:12 246596 ./flink-end-to-end-tests/flink-streaming-kinesis-test
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=37425&view=logs&j=ef799394-2d67-5ff4-b2e5-410b80c9c0af&t=860bfb5d-81b0-5968-f128-2a8b5362110d&l=664
> -- After completing all Bash bashed e2e tests:
> {code:java}
> 2022-07-01T10:20:17.3594718Z Jul 01 10:20:17 ##[group]Top 15 biggest directories in terms of used disk space
> 2022-07-01T10:20:18.7520631Z Jul 01 10:20:18 5425892 .
> 2022-07-01T10:20:18.7521823Z Jul 01 10:20:18 1521472 ./flink-end-to-end-tests
> 2022-07-01T10:20:18.7522566Z Jul 01 10:20:18 1242528 ./flink-python
> 2022-07-01T10:20:18.7523244Z Jul 01 10:20:18 952336 ./flink-python/dev
> 2022-07-01T10:20:18.7524159Z Jul 01 10:20:18 878764 ./flink-python/dev/.conda
> 2022-07-01T10:20:18.7524870Z Jul 01 10:20:18 834200 ./flink-python/dev/.conda/lib
> 2022-07-01T10:20:18.7525619Z Jul 01 10:20:18 726528 ./flink-python/dev/.conda/lib/python3.7
> 2022-07-01T10:20:18.7526397Z Jul 01 10:20:18 683256 ./flink-python/dev/.conda/lib/python3.7/site-packages
> 2022-07-01T10:20:18.7527101Z Jul 01 10:20:18 624568 ./flink-dist
> 2022-07-01T10:20:18.7527768Z Jul 01 10:20:18 624180 ./flink-dist/target
> 2022-07-01T10:20:18.7528494Z Jul 01 10:20:18 500076 ./flink-dist/target/flink-1.16-SNAPSHOT-bin
> 2022-07-01T10:20:18.7529298Z Jul 01 10:20:18 500072 ./flink-dist/target/flink-1.16-SNAPSHOT-bin/flink-1.16-SNAPSHOT
> 2022-07-01T10:20:18.7530046Z Jul 01 10:20:18 460812 ./flink-connectors
> 2022-07-01T10:20:18.7530546Z Jul 01 10:20:18 392588 ./.git
> 2022-07-01T10:20:18.7531014Z Jul 01 10:20:18 366396 ./.git/objects
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=37425&view=logs&j=ef799394-2d67-5ff4-b2e5-410b80c9c0af&t=860bfb5d-81b0-5968-f128-2a8b5362110d&l=9631
--
This message was sent by Atlassian Jira
(v8.20.10#820010)