[jira] [Closed] (FLINK-11208) Decrease build/testing time of modules

["Till Rohrmann (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Till Rohrmann closed FLINK-11208.
---------------------------------
    Resolution: Abandoned

Closing for inactivity.

> Decrease build/testing time of modules
> --------------------------------------
>
>                 Key: FLINK-11208
>                 URL: https://issues.apache.org/jira/browse/FLINK-11208
>             Project: Flink
>          Issue Type: Improvement
>          Components: Build System
>    Affects Versions: 1.8.0
>            Reporter: Till Rohrmann
>            Priority: Major
>
> This issue is intended to be the umbrella issue for speeding up build and test times of our modules. At the moment building Flink takes more than 1 hour on my machine. Here are the modules sorted in descending order with respect to build+test times (second column in seconds):
> {code}
> flink-connector-kafka-0.11 766
> flink-tests 425
> flink-runtime 391
> flink-yarn-tests 331
> flink-table 207
> flink-connector-kafka-0.9 189
> flink-connector-cassandra 135
> flink-connector-kafka-0.8 132
> flink-connector-kafka-0.10 130
> flink-gelly-examples 76
> flink-hbase 60
> flink-core 58.251
> flink-storm-examples 56.796
> flink-streaming-java 56.080
> flink-ml 53.349
> flink-connector-filesystem 47.698
> flink-connector-elasticsearch5 47.143
> flink-connector-elasticsearch2 45.406
> flink-connector-elasticsearch 40.804
> flink-streaming-scala 40.493
> flink-storm 40.107
> flink-scala-shell 38.566
> flink-scala 34.964
> flink-gelly 33.714
> flink-fs-tests 30.466
> flink-python 26.919d 
> flink-java 24.899
> flink-gelly-scala 23.406
> flink-dist 21.778
> flink-mesos 21.524
> flink-hadoop-compatibility 20.106
> flink-examples-streaming 19.951
> flink-shaded-hadoop2 19.782
> flink-s3-fs-presto 18.731
> flink-s3-fs-hadoop 17.070
> flink-optimizer 17.034
> flink-hadoop-fs 15.857
> flink-avro 15.409
> flink-shaded-yarn-tests 15.147
> flink-yarn 14.962
> flink-swift-fs-hadoop 13.709
> flink-streaming-python 13.163
> flink-connector-wikiedits 12.515
> flink-queryable-state-runtime 12.474
> flink-statebackend-rocksdb 11.987
> flink-shaded-hadoop2-uber 11.527
> flink-connector-elasticsearch6 10.880
> flink-runtime-web 10.384
> flink-orc 9.916
> flink-cep 8.963
> flink-sql-client 8.944
> flink-examples-batch 8.555
> flink-cep-scala 8.270
> flink-clients 5.966
> flink-examples-table 5.769
> flink-connector-kafka-base 5.754
> flink-high-parallelism-iterations-test 4.707
> flink-parquet 3.640
> flink-hcatalog 3.436
> flink-jdbc 3.296
> flink 3.257
> flink-connector-rabbitmq 2.779
> flink-test-utils 2.686
> flink-metrics-statsd 2.667
> flink-elasticsearch6-test 2.251
> flink-metrics-jmx 2.167
> flink-elasticsearch5-test 2.144
> flink-elasticsearch2-test 2.114
> flink-metrics-dropwizard 1.898
> flink-connector-elasticsearch-base 1.726
> flink-metrics-datadog 1.676
> flink-elasticsearch1-test 1.652
> flink-mapr-fs 1.551
> flink-json 1.526
> flink-docs 1.441
> flink-metrics-core 1.373
> flink-avro-confluent-registry 1.345
> flink-metrics-prometheus 1.340
> flink-connector-twitter 1.276
> flink-annotations 1.183
> flink-queryable-state-test 1.121
> force-shading 1.114
> flink-queryable-state-client-java 1.085
> flink-test-utils-junit 1.058
> flink-sql-client-test 1.049
> flink-confluent-schema-registry 1.001
> flink-quickstart 0.939
> flink-datastream-allround-test 0.924
> flink-container 0.740
> flink-stream-state-ttl-test 0.679
> flink-metrics-slf4j 0.663
> flink-stream-stateful-job-upgrade-test 0.623
> flink-shaded-curator 0.544
> flink-connector-nifi 0.487
> flink-quickstart-java 0.451
> flink-bucketing-sink-test 0.369
> flink-local-recovery-and-allocation-test 0.220
> flink-stream-sql-test 0.191
> flink-quickstart-test 0.191
> flink-streaming-file-sink-test 0.182
> flink-dataset-allround-test 0.175
> flink-distributed-cache-via-blob 0.163
> flink-metrics-graphite 0.160
> flink-examples 0.155
> flink-parent-child-classloading-test 0.154
> flink-quickstart-scala 0.135
> flink-shaded-hadoop 0.120
> flink-filesystems 0.081
> flink-metrics 0.076
> flink-test-utils-parent 0.074
> flink-queryable-state 0.067
> flink-formats 0.061
> flink-state-backends 0.058
> flink-end-to-end-tests 0.058
> flink-libraries 0.056
> flink-connectors 0.054
> flink-contrib 0.052
> {code}
> As one can see {{flink-connector-kafka-0.11}} alone takes more than 12 minutes to build and run all tests. I think it would be good to revisit some of these tests to see whether they can be sped up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)