You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by az...@apache.org on 2020/04/21 13:13:24 UTC

[flink] branch flip116 updated (44b10ab -> 1626008)

This is an automated email from the ASF dual-hosted git repository.

azagrebin pushed a change to branch flip116
in repository https://gitbox.apache.org/repos/asf/flink.git.


 discard 44b10ab  change type to cron
 discard 1e4c0b7  [FLINK-17048] Add memory related JVM args to Mesos JM startup scripts
 discard 774dba9  fixup! [FLINK-16745][k8s] Start Kubernetes JM with FLIP-116 JVM memory args
 discard 3ba8e78  [FLINK-16746][conf] Deprecate jobmanager.heap.size and expose new memory options
 discard b737e73  [FLINK-16746] Add default 'jobmanager.memory.process.size: 1600m' to flink-conf.yaml
 discard 2f359ca  [hotfix] Remove 'TaskManager' from MemoryBackwardsCompatibilityUtils#getLegacyTaskManagerHeapMemoryIfExplicitlyConfigured
 discard 3cbf22f  [FLINK-16745][coordination] Remove unused container cutoff
 discard 94feed4  [FLINK-16745][k8s] Start Kubernetes JM with FLIP-116 JVM memory args
 discard 29a3baf  [FLINK-16745][yarn] Start Yarn JM with FLIP-116 JVM memory args
 discard b1577a2  [FLINK-16745][clients] Parse JobManagerProcessSpec from Configuration into ClusterSpecification
 discard e2ab453  [FLINK-16742][runtime] Refactor BashJavaUtils, move to separated package and extract configuration loading logics.
 discard 1f7dd29  [FLINK-16472][dist] Remove unused memory configuration logics from config.sh.
 discard 795bafc  [FLINK-16742][dist] Extend and use BashJavaUtils for JM memory configuration in start-up scripts.
 discard 31dd9b6  [FLINK-16742][runtime] Ignore unknown command line options for BashJavaUtils.
 discard 92e8c69  [FLINK-16742][dist] config.sh does not assume BashJavaUtils always return two lines of results.
 discard 3df3f60  [hotfix][runtime] Code clean-ups in BashJavaUtils and its test cases.
     add a0f349f  [FLINK-16572] [pubsub,e2e] Only acknowledge list of messages if the list is not empty + small style fixes (removal of uncommented code etc)]
     add c89df66  [FLINK-16656] Introduce the DispatcherBootstrap
     add 7676360  [FLINK-16660] Add stop() to DispatcherBootstrap
     add e600dc4  [FLINK-16660] Separate the code-paths of web submission and application mode
     add 3ff4cee  [FLINK-16660] Introduce the ApplicationDispatcherBootstrap
     add 9d4fdcb  [FLINK-16658] Add ApplicationDispatcherGatewayServiceFactory for Application Mode execution
     add 52bcb21  [FLINK-16658] Add PackagedProgramRetriever
     add 74fc5a7  [FLINK-16658] Wire the ApplicationDispatcherBootstrap to the StandaloneJobClusterEntryPoint
     add e15da89  [minor] Remove unused ClassPathJobGraphRetriever
     add 8af9e21  [FLINK-16658] Make StandaloneJobClusterEntryPoint extend directly ClusterEntrypoint
     add 3ab3b57  [FLINK-16660] Make rpc timeout and retry period configurable for application mode
     add 4f55b88  [FLINK-16658] Refactor ApplicationDispatcherBootstrap and test
     add e191220  [FLINK-17249] Bump universal Kafka connector to Kafka 2.2.2
     add 2b23c4a  [hotfix][table sql planner/table sql legacy planner]fix icu license in NOTICE file
     add 9b35c28  [hotfix][docs] fix broken SVG figure and include license this time
     add 00f8bf5  [hotfix][runtime] Code clean-ups in BashJavaUtils and its test cases.
     add d2c6d9b  [FLINK-16742][dist] config.sh does not assume BashJavaUtils always return two lines of results.
     add 49d1214  [FLINK-16742][runtime] Ignore unknown command line options for BashJavaUtils.
     add 9a921b1  [FLINK-16742][dist] Extend and use BashJavaUtils for JM memory configuration in start-up scripts.
     add a1b4d49  [FLINK-16472][dist] Remove unused memory configuration logics from config.sh.
     add a237df3  [FLINK-16742][runtime] Refactor BashJavaUtils, move to separated package and extract configuration loading logics.
     add 90cd324  [FLINK-16745][clients] Parse JobManagerProcessSpec from Configuration into ClusterSpecification
     add aa64dc5  [FLINK-16745][yarn] Start Yarn JM with FLIP-116 JVM memory args
     add af27aa5  [FLINK-16745][k8s] Start Kubernetes JM with FLIP-116 JVM memory args
     add 05dd7a9  [FLINK-16745][coordination] Remove unused container cutoff
     add 3283496  [hotfix] Remove 'TaskManager' from MemoryBackwardsCompatibilityUtils#getLegacyTaskManagerHeapMemoryIfExplicitlyConfigured
     add 3b47c1b  fixup! [FLINK-16745][k8s] Start Kubernetes JM with FLIP-116 JVM memory args
     add c7ba8e5  [hotfix] Fix ProcessMemoryUtilsTestBase#testExceptionShouldContainRequiredConfigOptions to run also for TM
     add 966a10d  fixup! [FLINK-16745][clients] Parse JobManagerProcessSpec from Configuration into ClusterSpecification
     add 11f4f7e  fixup! [FLINK-16745][yarn] Start Yarn JM with FLIP-116 JVM memory args
     add ca3e9c5  fixup! [FLINK-16745][k8s] Start Kubernetes JM with FLIP-116 JVM memory args
     add bb4d7b6  [FLINK-16746] Add default 'jobmanager.memory.process.size: 1600m' to flink-conf.yaml
     add eebcb0f  [FLINK-16746][conf] Deprecate jobmanager.heap.size and expose new memory options
     new a98a008  [FLINK-17048] Add memory related JVM args to Mesos JM startup scripts
     new 1626008  change type to cron

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (44b10ab)
            \
             N -- N -- N   refs/heads/flip116 (1626008)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../generated/execution_configuration.html         |   12 +
 docs/fig/program_dataflow.svg                      | 1042 ++++++++++----------
 .../java/org/apache/flink/client/ClientUtils.java  |    6 +-
 .../AbstractContainerizedClusterClientFactory.java |    5 +-
 .../ApplicationDispatcherBootstrap.java            |  270 +++++
 ...ApplicationDispatcherGatewayServiceFactory.java |   44 +-
 ...ationDispatcherLeaderProcessFactoryFactory.java |   87 ++
 .../application/ApplicationExecutionException.java |   33 +-
 .../application/DetachedApplicationRunner.java     |   13 +-
 .../deployment/application/EmbeddedJobClient.java  |   12 +-
 .../application/JobStatusPollingUtils.java         |  104 ++
 .../WebSubmissionJobClient.java}                   |   19 +-
 .../application/executors/EmbeddedExecutor.java    |   61 +-
 .../executors/EmbeddedExecutorFactory.java         |   18 +-
 .../executors/EmbeddedExecutorServiceLoader.java   |    8 +-
 .../executors/EmbeddedJobClientCreator.java        |   33 +-
 ...tory.java => WebSubmissionExecutorFactory.java} |   23 +-
 ...ava => WebSubmissionExecutorServiceLoader.java} |   12 +-
 .../flink/client/program/ContextEnvironment.java   |   20 +-
 .../client/program/PackagedProgramRetriever.java   |   33 +-
 .../client/program/StreamContextEnvironment.java   |   20 +-
 .../ApplicationDispatcherBootstrapTest.java        |  380 +++++++
 .../application/JobStatusPollingUtilsTest.java     |  169 ++++
 .../apache/flink/client/program/ClientTest.java    |   20 +-
 .../flink/client/testjar/MultiExecuteJob.java      |   46 +-
 flink-connectors/flink-connector-kafka/pom.xml     |    4 +-
 .../src/main/resources/META-INF/NOTICE             |    2 +-
 ...java => ClassPathPackagedProgramRetriever.java} |   91 +-
 .../entrypoint/StandaloneJobClusterEntryPoint.java |  134 ++-
 ... => ClassPathPackagedProgramRetrieverTest.java} |  107 +-
 ...neJobClusterConfigurationParserFactoryTest.java |   55 ++
 .../StandaloneJobClusterEntryPointTest.java        |   37 -
 .../apache/flink/container/entrypoint/TestJob.java |    2 +-
 .../flink/configuration/ExecutionOptions.java      |   15 +
 flink-dist/src/main/flink-bin/bin/config.sh        |    4 +-
 .../gcp/pubsub/CheckPubSubEmulatorTest.java        |    2 +-
 .../gcp/pubsub/emulator/PubsubHelper.java          |   55 +-
 .../tests/util/kafka/StreamingKafkaITCase.java     |    2 +-
 .../flink-sql-client-test/pom.xml                  |    2 +-
 .../test-scripts/test_sql_client.sh                |    2 +-
 .../src/main/resources/META-INF/NOTICE             |    2 +-
 flink-jepsen/src/jepsen/flink/flink.clj            |    2 +-
 .../decorators/JavaCmdJobManagerDecorator.java     |    7 +-
 .../runtime/webmonitor/WebSubmissionExtension.java |    2 +-
 .../runtime/webmonitor/handlers/JarHandlers.java   |    2 +-
 .../handlers/JarRunHandlerParameterTest.java       |    2 +-
 ...ctory.java => AbstractDispatcherBootstrap.java} |   34 +-
 ...actory.java => DefaultDispatcherBootstrap.java} |   39 +-
 .../flink/runtime/dispatcher/Dispatcher.java       |   25 +-
 ...atcherFactory.java => DispatcherBootstrap.java} |   37 +-
 .../runtime/dispatcher/DispatcherFactory.java      |    4 +-
 .../runtime/dispatcher/JobDispatcherFactory.java   |   11 +-
 .../flink/runtime/dispatcher/MiniDispatcher.java   |    5 +-
 .../dispatcher/SessionDispatcherFactory.java       |    7 +-
 .../runtime/dispatcher/StandaloneDispatcher.java   |    6 +-
 .../runner/AbstractDispatcherLeaderProcess.java    |   20 +-
 .../runner/DefaultDispatcherGatewayService.java    |    7 +-
 .../DefaultDispatcherGatewayServiceFactory.java    |    8 +-
 .../runner/DefaultDispatcherRunnerFactory.java     |    2 +-
 .../DispatcherLeaderProcessFactoryFactory.java     |    2 +
 .../SessionDispatcherLeaderProcessFactory.java     |    6 +-
 ...tDispatcherResourceManagerComponentFactory.java |    2 +-
 .../runtime/jobmanager/JobManagerProcessUtils.java |   13 +-
 .../flink/runtime/util/bash/BashJavaUtils.java     |   11 +-
 .../dispatcher/DispatcherResourceCleanupTest.java  |    2 +-
 .../flink/runtime/dispatcher/DispatcherTest.java   |   12 +-
 .../runtime/dispatcher/MiniDispatcherTest.java     |    3 +-
 .../runtime/dispatcher/TestingDispatcher.java      |    6 +-
 .../runner/DefaultDispatcherRunnerITCase.java      |    5 +-
 .../SessionDispatcherWithUUIDFactory.java          |    8 +-
 .../config/memory/ProcessMemoryUtilsTestBase.java  |    3 +-
 .../webmonitor/TestingDispatcherGateway.java       |    9 +-
 .../runtime/webmonitor/TestingRestfulGateway.java  |   25 +-
 .../src/main/resources/META-INF/NOTICE             |    4 +
 .../src/main/resources/META-INF/NOTICE             |    6 +-
 .../apache/flink/yarn/YarnClusterDescriptor.java   |    5 +-
 76 files changed, 2275 insertions(+), 1073 deletions(-)
 create mode 100644 flink-clients/src/main/java/org/apache/flink/client/deployment/application/ApplicationDispatcherBootstrap.java
 copy flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/runner/DefaultDispatcherGatewayServiceFactory.java => flink-clients/src/main/java/org/apache/flink/client/deployment/application/ApplicationDispatcherGatewayServiceFactory.java (54%)
 create mode 100644 flink-clients/src/main/java/org/apache/flink/client/deployment/application/ApplicationDispatcherLeaderProcessFactoryFactory.java
 copy flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/DispatcherFactory.java => flink-clients/src/main/java/org/apache/flink/client/deployment/application/ApplicationExecutionException.java (53%)
 create mode 100644 flink-clients/src/main/java/org/apache/flink/client/deployment/application/JobStatusPollingUtils.java
 rename flink-clients/src/main/java/org/apache/flink/client/deployment/{DetachedOnlyJobClientAdapter.java => application/WebSubmissionJobClient.java} (85%)
 copy flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/DispatcherFactory.java => flink-clients/src/main/java/org/apache/flink/client/deployment/application/executors/EmbeddedJobClientCreator.java (53%)
 copy flink-clients/src/main/java/org/apache/flink/client/deployment/application/executors/{EmbeddedExecutorFactory.java => WebSubmissionExecutorFactory.java} (74%)
 copy flink-clients/src/main/java/org/apache/flink/client/deployment/application/executors/{EmbeddedExecutorServiceLoader.java => WebSubmissionExecutorServiceLoader.java} (87%)
 copy flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/DispatcherFactory.java => flink-clients/src/main/java/org/apache/flink/client/program/PackagedProgramRetriever.java (53%)
 create mode 100644 flink-clients/src/test/java/org/apache/flink/client/deployment/application/ApplicationDispatcherBootstrapTest.java
 create mode 100644 flink-clients/src/test/java/org/apache/flink/client/deployment/application/JobStatusPollingUtilsTest.java
 copy flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/DispatcherFactory.java => flink-clients/src/test/java/org/apache/flink/client/testjar/MultiExecuteJob.java (50%)
 rename flink-container/src/main/java/org/apache/flink/container/entrypoint/{ClassPathJobGraphRetriever.java => ClassPathPackagedProgramRetriever.java} (71%)
 rename flink-container/src/test/java/org/apache/flink/container/entrypoint/{ClassPathJobGraphRetrieverTest.java => ClassPathPackagedProgramRetrieverTest.java} (66%)
 copy flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/{DispatcherFactory.java => AbstractDispatcherBootstrap.java} (56%)
 copy flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/{SessionDispatcherFactory.java => DefaultDispatcherBootstrap.java} (52%)
 copy flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/{DispatcherFactory.java => DispatcherBootstrap.java} (53%)


[flink] 01/02: [FLINK-17048] Add memory related JVM args to Mesos JM startup scripts

Posted by az...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

azagrebin pushed a commit to branch flip116
in repository https://gitbox.apache.org/repos/asf/flink.git

commit a98a008ae210ef1b50b7a67274a626b544e8a7d7
Author: Andrey Zagrebin <az...@apache.org>
AuthorDate: Mon Apr 20 19:46:08 2020 +0300

    [FLINK-17048] Add memory related JVM args to Mesos JM startup scripts
---
 .../main/flink-bin/mesos-bin/mesos-appmaster-job.sh | 19 +------------------
 .../src/main/flink-bin/mesos-bin/mesos-appmaster.sh | 21 +--------------------
 .../{mesos-appmaster.sh => mesos-jobmanager.sh}     | 17 +++++++++++++++--
 3 files changed, 17 insertions(+), 40 deletions(-)

diff --git a/flink-dist/src/main/flink-bin/mesos-bin/mesos-appmaster-job.sh b/flink-dist/src/main/flink-bin/mesos-bin/mesos-appmaster-job.sh
index f2e419f..d928631 100755
--- a/flink-dist/src/main/flink-bin/mesos-bin/mesos-appmaster-job.sh
+++ b/flink-dist/src/main/flink-bin/mesos-bin/mesos-appmaster-job.sh
@@ -23,21 +23,4 @@ bin=`cd "$bin"; pwd`
 # get Flink config
 . "$bin"/config.sh
 
-if [ "$FLINK_IDENT_STRING" = "" ]; then
-    FLINK_IDENT_STRING="$USER"
-fi
-
-CC_CLASSPATH=`manglePathList $(constructFlinkClassPath):$INTERNAL_HADOOP_CLASSPATHS`
-
-log="${FLINK_LOG_DIR}/flink-${FLINK_IDENT_STRING}-mesos-appmaster-${HOSTNAME}.log"
-log_setting="-Dlog.file="$log" -Dlog4j.configuration=file:"$FLINK_CONF_DIR"/log4j.properties -Dlog4j.configurationFile=file:"$FLINK_CONF_DIR"/log4j.properties -Dlogback.configurationFile=file:"$FLINK_CONF_DIR"/logback.xml"
-
-exec $JAVA_RUN $JVM_ARGS -classpath "$CC_CLASSPATH" $log_setting org.apache.flink.mesos.entrypoint.MesosJobClusterEntrypoint "$@"
-
-rc=$?
-
-if [[ $rc -ne 0 ]]; then
-    echo "Error while starting the mesos application master. Please check ${log} for more details."
-fi
-
-exit $rc
+exec "${FLINK_BIN_DIR}"/mesos-jobmanager.sh "org.apache.flink.mesos.entrypoint.MesosJobClusterEntrypoint"
diff --git a/flink-dist/src/main/flink-bin/mesos-bin/mesos-appmaster.sh b/flink-dist/src/main/flink-bin/mesos-bin/mesos-appmaster.sh
index 20c88ea..ab68d1b 100755
--- a/flink-dist/src/main/flink-bin/mesos-bin/mesos-appmaster.sh
+++ b/flink-dist/src/main/flink-bin/mesos-bin/mesos-appmaster.sh
@@ -23,23 +23,4 @@ bin=`cd "$bin"; pwd`
 # get Flink config
 . "$bin"/config.sh
 
-if [ "$FLINK_IDENT_STRING" = "" ]; then
-    FLINK_IDENT_STRING="$USER"
-fi
-
-CC_CLASSPATH=`manglePathList $(constructFlinkClassPath):$INTERNAL_HADOOP_CLASSPATHS`
-
-log="${FLINK_LOG_DIR}/flink-${FLINK_IDENT_STRING}-mesos-appmaster-${HOSTNAME}.log"
-log_setting="-Dlog.file="$log" -Dlog4j.configuration=file:"$FLINK_CONF_DIR"/log4j.properties -Dlog4j.configurationFile=file:"$FLINK_CONF_DIR"/log4j.properties -Dlogback.configurationFile=file:"$FLINK_CONF_DIR"/logback.xml"
-
-ENTRY_POINT=org.apache.flink.mesos.entrypoint.MesosSessionClusterEntrypoint
-
-exec $JAVA_RUN $JVM_ARGS -classpath "$CC_CLASSPATH" $log_setting ${ENTRY_POINT} "$@"
-
-rc=$?
-
-if [[ $rc -ne 0 ]]; then
-    echo "Error while starting the mesos application master. Please check ${log} for more details."
-fi
-
-exit $rc
+exec "${FLINK_BIN_DIR}"/mesos-jobmanager.sh "org.apache.flink.mesos.entrypoint.MesosSessionClusterEntrypoint"
diff --git a/flink-dist/src/main/flink-bin/mesos-bin/mesos-appmaster.sh b/flink-dist/src/main/flink-bin/mesos-bin/mesos-jobmanager.sh
similarity index 78%
copy from flink-dist/src/main/flink-bin/mesos-bin/mesos-appmaster.sh
copy to flink-dist/src/main/flink-bin/mesos-bin/mesos-jobmanager.sh
index 20c88ea..64cbe7d 100755
--- a/flink-dist/src/main/flink-bin/mesos-bin/mesos-appmaster.sh
+++ b/flink-dist/src/main/flink-bin/mesos-bin/mesos-jobmanager.sh
@@ -17,12 +17,27 @@
 # limitations under the License.
 ################################################################################
 
+ENTRY_POINT=$1
+
 bin=`dirname "$0"`
 bin=`cd "$bin"; pwd`
 
 # get Flink config
 . "$bin"/config.sh
 
+java_utils_output=$(runBashJavaUtilsCmd GET_JM_RESOURCE_PARAMS ${FLINK_CONF_DIR} $FLINK_BIN_DIR/bash-java-utils.jar:$(findFlinkDistJar) "${ARGS[@]}")
+logging_output=$(extractLoggingOutputs "${java_utils_output}")
+jvm_params=$(extractExecutionResults "${java_utils_output}" 1)
+export JVM_ARGS="${JVM_ARGS} ${jvm_params}"
+
+export FLINK_INHERITED_LOGS="
+$FLINK_INHERITED_LOGS
+
+JM_RESOURCE_PARAMS extraction logs:
+jvm_params: $jvm_params
+logs: $logging_output
+"
+
 if [ "$FLINK_IDENT_STRING" = "" ]; then
     FLINK_IDENT_STRING="$USER"
 fi
@@ -32,8 +47,6 @@ CC_CLASSPATH=`manglePathList $(constructFlinkClassPath):$INTERNAL_HADOOP_CLASSPA
 log="${FLINK_LOG_DIR}/flink-${FLINK_IDENT_STRING}-mesos-appmaster-${HOSTNAME}.log"
 log_setting="-Dlog.file="$log" -Dlog4j.configuration=file:"$FLINK_CONF_DIR"/log4j.properties -Dlog4j.configurationFile=file:"$FLINK_CONF_DIR"/log4j.properties -Dlogback.configurationFile=file:"$FLINK_CONF_DIR"/logback.xml"
 
-ENTRY_POINT=org.apache.flink.mesos.entrypoint.MesosSessionClusterEntrypoint
-
 exec $JAVA_RUN $JVM_ARGS -classpath "$CC_CLASSPATH" $log_setting ${ENTRY_POINT} "$@"
 
 rc=$?


[flink] 02/02: change type to cron

Posted by az...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

azagrebin pushed a commit to branch flip116
in repository https://gitbox.apache.org/repos/asf/flink.git

commit 1626008be84370fa6e3b4a56c03d6a3f3cddab0f
Author: Andrey Zagrebin <az...@apache.org>
AuthorDate: Wed Apr 15 13:01:39 2020 +0300

    change type to cron
---
 .travis.yml | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 3b4d212..029e31d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -74,6 +74,8 @@ notifications:
     on_success: change
     on_pull_requests: false
 
+type: cron
+
 stages:
   - name: compile
   - name: test
@@ -85,53 +87,53 @@ jdk: "openjdk8"
 jobs:
   include:
     # main profile
-    - if: type in (pull_request, push)
+    - if: type in (pull_request, push, cron)
       stage: compile
       script: ./tools/travis_controller.sh compile
       env: PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11"
       name: compile
-    - if: type in (pull_request, push)
+    - if: type in (pull_request, push, cron)
       stage: test
       script: ./tools/travis_controller.sh core
       env: PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11"
       name: core
-    - if: type in (pull_request, push)
+    - if: type in (pull_request, push, cron)
       script: ./tools/travis_controller.sh python
       env: PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11"
       name: python
-    - if: type in (pull_request, push)
+    - if: type in (pull_request, push, cron)
       script: ./tools/travis_controller.sh libraries
       env: PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11"
       name: libraries
-    - if: type in (pull_request, push)
+    - if: type in (pull_request, push, cron)
       script: ./tools/travis_controller.sh blink_planner
       env: PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11"
       name: blink_planner
-    - if: type in (pull_request, push)
+    - if: type in (pull_request, push, cron)
       script: ./tools/travis_controller.sh connectors
       env: PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11"
       name: connectors
-    - if: type in (pull_request, push)
+    - if: type in (pull_request, push, cron)
       script: ./tools/travis_controller.sh kafka/gelly
       env: PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11"
       name: kafka/gelly
-    - if: type in (pull_request, push)
+    - if: type in (pull_request, push, cron)
       script: ./tools/travis_controller.sh tests
       env: PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11"
       name: tests
-    - if: type in (pull_request, push)
+    - if: type in (pull_request, push, cron)
       script: ./tools/travis_controller.sh legacy_scheduler_core
       env: PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11"
       name: core - legacy_scheduler
-    - if: type in (pull_request, push)
+    - if: type in (pull_request, push, cron)
       script: ./tools/travis_controller.sh legacy_scheduler_tests
       env: PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11"
       name: tests - legacy_scheduler
-    - if: type in (pull_request, push)
+    - if: type in (pull_request, push, cron)
       script: ./tools/travis_controller.sh misc
       env: PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11"
       name: misc
-    - if: type in (pull_request, push)
+    - if: type in (pull_request, push, cron)
       stage: cleanup
       script: ./tools/travis_controller.sh cleanup
       env: PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11"