[jira] [Created] (FLINK-38949) unpack_build_artifact.sh script speed improvement

["Tom Ho (Jira)" <ji...@apache.org>]

Tom Ho created FLINK-38949:
------------------------------

             Summary: unpack_build_artifact.sh script speed improvement
                 Key: FLINK-38949
                 URL: https://issues.apache.org/jira/browse/FLINK-38949
             Project: Flink
          Issue Type: Improvement
          Components: Build System / CI
    Affects Versions: 2.1.1, 2.2.0, 2.0.1, 2.1.0
            Reporter: Tom Ho


Current unpack_build_artifact.sh script use hardcoded sleep time, find+chmod approach is slow because it will spin up one thread for chmod for every file. Spinning up thread will take very long if the number of file found is large. To address this, we can use code like this, which will join all file as a list and pass those file list into one thread, which is tested in flink repo, it will save 4 min of time with the same result.
{code:java}
find . -type f -name '*.proto' -exec touch -d "$PROTO_TIME" {} + {code}
https://github.com/apache/flink/blob/9e2d2b8ed0d90d1c6277d5eb630b47f9a4d5fc54/tools/azure-pipelines/unpack_build_artifact.sh#L38-L48



--
This message was sent by Atlassian Jira
(v8.20.10#820010)