You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Amit Jain <aj...@gmail.com> on 2018/04/17 07:35:37 UTC

Testing on Flink 1.5

Hi,


We have started testing our workload on Flink 1.5.0 (Commit: 2af481a)
and faced few issues. Here, we are using YARN as cluster manager.


   - Few of our jobs are stuck with logs [1], these jobs are only able
to allocate JM and couldn't get any TM, however, there are ample
resource on our cluster.
   - All completed application on YARN are marked as KILLED, instead
of FINISHED. It creates problem on tracking jobs as we require to
navigate on each application logs and also history server is not
logging any completed application.
   - We are running our system on AWS EMR and using CORE nodes at its
bare minimum, we see with new paradigm of application submission we
are hitting Out of space error in our HDFS Data Node. Do you think, we
should have configuration to automatically evict these .flink/ files?
   - Lastly, we are also observing Akka.ask.timeout error for few of
our jobs (JM's logs[2]), we tried to increase this parameter to 2 min
from default 10 s and still observe the same.

Please help us guide through these issues.

--
Thanks,
Amit

[1] JM logs where jobs are not even started

2018-04-17 04:26:16,165 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
--------------------------------------------------------------------------------
2018-04-17 04:26:16,166 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Starting YarnSessionClusterEntrypoint (Version: 1.5.0, Rev:2af481a,
Date:02.04.2018 @ 15:17:35 UTC)
2018-04-17 04:26:16,166 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  OS
current user: yarn
2018-04-17 04:26:16,843 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Current Hadoop/Kerberos user: hadoop
2018-04-17 04:26:16,843 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM:
OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.161-b14
2018-04-17 04:26:16,843 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Maximum heap size: 406 MiBytes
2018-04-17 04:26:16,843 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
JAVA_HOME: /usr/lib/jvm/java-openjdk
2018-04-17 04:26:16,844 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Hadoop version: 2.8.3
2018-04-17 04:26:16,844 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM
Options:
2018-04-17 04:26:16,844 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
-Xmx424m
2018-04-17 04:26:16,845 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
-Dlog.file=/var/log/hadoop-yarn/containers/application_1523726493647_0503/container_1523726493647_0503_01_000001/jobmanager.log
2018-04-17 04:26:16,845 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
-Dlogback.configurationFile=file:logback.xml
2018-04-17 04:26:16,845 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
-Dlog4j.configuration=file:log4j.properties
2018-04-17 04:26:16,845 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Program Arguments: (none)
2018-04-17 04:26:16,845 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Classpath: lib/flink-dist_2.11-1.5.0.jar:lib/flink-python_2.11-1.5.0.jar:lib/flink-shaded-hadoop2-uber-1.5.0.jar:lib/flink-shaded-include-yarn-0.9.1.jar:lib/flink-yarn_2.11-1.5-SNAPSHOT.jar:lib/guava-18.0.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.7.jar:log4j.properties:logback.xml:flink.jar:flink-conf.yaml::/etc/hadoop/conf:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-streaming.jar:/usr/lib/hadoop/hadoop-distcp.jar:/usr/lib/hadoop/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archives.jar:/usr/lib/hadoop/hadoop-ant.jar:/usr/lib/hadoop/hadoop-aws.jar:/usr/lib/hadoop/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-rumen.jar:/usr/lib/hadoop/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack.jar:/usr/lib/hadoop/hadoop-azure.jar:/usr/lib/hadoop/hadoop-extras.jar:/usr/lib/hadoop/hadoop-azure-datalake.jar:/usr/lib/hadoop/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix.jar:/usr/lib/hadoop/hadoop-nfs.jar:/usr/lib/hadoop/hadoop-datajoin.jar:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop/hadoop-sls.jar:/usr/lib/hadoop/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/lib/curator-framework-2.7.1.jar:/usr/lib/hadoop/lib/json-smart-1.1.1.jar:/usr/lib/hadoop/lib/curator-recipes-2.7.1.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/lib/jcip-annotations-1.0.jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/lib/avro-1.7.4.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop/lib/gson-2.2.4.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop/lib/jsch-0.1.54.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar:/usr/lib/hadoop/lib/jersey-server-1.9.jar:/usr/lib/hadoop/lib/httpcore-4.4.4.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/curator-client-2.7.1.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop/lib/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/jsr305-3.0.0.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/httpclient-4.5.3.jar:/usr/lib/hadoop/lib/commons-lang-2.6.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/slf4j-api-1.7.10.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/xercesImpl-2.9.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-hdfs/lib/netty-all-4.0.23.Final.jar:/usr/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-hdfs/lib/okio-1.4.0.jar:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/xml-apis-1.3.04.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/okhttp-2.4.0.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/ion-java-1.0.2.jar:/usr/lib/hadoop-mapreduce/curator-framework-2.7.1.jar:/usr/lib/hadoop-mapreduce/json-smart-1.1.1.jar:/usr/lib/hadoop-mapreduce/curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-core-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/zookeeper-3.4.10.jar:/usr/lib/hadoop-mapreduce/jmespath-java-1.11.267.jar:/usr/lib/hadoop-mapreduce/jettison-1.1.jar:/usr/lib/hadoop-mapreduce/java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/jcip-annotations-1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs.jar:/usr/lib/hadoop-mapreduce/hadoop-auth.jar:/usr/lib/hadoop-mapreduce/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-2.6.7.jar:/usr/lib/hadoop-mapreduce/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-mapreduce/gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/azure-data-lake-store-sdk-2.2.3.jar:/usr/lib/hadoop-mapreduce/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/jsch-0.1.54.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/commons-lang3-3.3.2.jar:/usr/lib/hadoop-mapreduce/xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/commons-collections-3.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/hadoop-archives.jar:/usr/lib/hadoop-mapreduce/commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/hadoop-ant.jar:/usr/lib/hadoop-mapreduce/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/jackson-databind-2.6.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-aws.jar:/usr/lib/hadoop-mapreduce/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/jackson-annotations-2.6.7.jar:/usr/lib/hadoop-mapreduce/httpcore-4.4.4.jar:/usr/lib/hadoop-mapreduce/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jetty-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop-mapreduce/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/xz-1.0.jar:/usr/lib/hadoop-mapreduce/joda-time-2.9.4.jar:/usr/lib/hadoop-mapreduce/commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-mapreduce/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/activation-1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/hadoop-azure.jar:/usr/lib/hadoop-mapreduce/hadoop-extras.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/okio-1.4.0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/azure-storage-2.2.0.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-s3-1.11.267.jar:/usr/lib/hadoop-mapreduce/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/jackson-dataformat-cbor-2.6.7.jar:/usr/lib/hadoop-mapreduce/jackson-xc-1.9.13.jar:/usr/lib/hadoop-mapreduce/okhttp-2.4.0.jar:/usr/lib/hadoop-mapreduce/asm-3.2.jar:/usr/lib/hadoop-mapreduce/hadoop-sls.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/httpclient-4.5.3.jar:/usr/lib/hadoop-mapreduce/commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-kms-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/lib/curator-test-2.7.1.jar:/usr/lib/hadoop-yarn/lib/java-util-1.9.0.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10-tests.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop-yarn/lib/jettison-1.1.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/lib/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/lib/hadoop-yarn/lib/commons-math-2.2.jar:/usr/lib/hadoop-yarn/lib/curator-client-2.7.1.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-yarn/lib/activation-1.1.jar:/usr/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-yarn/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-yarn/lib/guava-11.0.2.jar:/usr/lib/hadoop-yarn/lib/json-io-2.5.1.jar:/usr/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/lib/hadoop-yarn/lib/fst-2.50.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/javassist-3.18.1-GA.jar:/usr/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo-0.4.19.jar:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/jcl-over-slf4j-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/ion-java-1.0.2.jar:/usr/share/aws/emr/emrfs/lib/emrfs-hadoop-assembly-2.21.0.jar:/usr/share/aws/emr/emrfs/lib/jmespath-java-1.11.267.jar:/usr/share/aws/emr/emrfs/lib/javax.inject-1.jar:/usr/share/aws/emr/emrfs/lib/bcpkix-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/lib/slf4j-api-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/aopalliance-1.0.jar:/usr/share/aws/emr/emrfs/lib/bcprov-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/auxlib/*:/usr/share/aws/emr/lib/*:/usr/share/aws/emr/ddb/lib/emr-ddb-hadoop.jar:/usr/share/aws/emr/goodies/lib/emr-hadoop-goodies.jar:/usr/share/aws/emr/kinesis/lib/emr-kinesis-hadoop.jar:/usr/lib/spark/yarn/lib/datanucleus-api-jdo.jar:/usr/lib/spark/yarn/lib/datanucleus-core.jar:/usr/lib/spark/yarn/lib/datanucleus-rdbms.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink-1.1.0.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacecommerceanalytics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rds-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-core-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-clouddirectory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsmv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-emr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediaconvert-1.11.267.jar:/usr/share/aws/aws-java-sdk/jmespath-java-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appsync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastore-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iot-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costandusagereport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpleworkflow-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-comprehend-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidentity-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ec2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudtrail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servermigration-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesis-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workmail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-polly-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpledb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscalingplans-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elastictranscoder-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pinpoint-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lexmodelbuilding-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacemeteringservice-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dynamodb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-storagegateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codegen-maven-plugin-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediapackage-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroupstaggingapi-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-acm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-machinelearning-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mobile-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dax-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lightsail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sns-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codedeploy-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-serverlessapplicationrepository-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-shield-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-applicationautoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastoredata-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloud9-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-budgets-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iotjobsdataplane-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-medialive-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-inspector-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-importexport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-test-utils-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitosync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-api-gateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codebuild-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workspaces-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opensdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-snowball-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pricing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-code-generator-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glacier-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-guardduty-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicediscovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codestar-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appstream-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancingv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-greengrass-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lex-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroups-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-support-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticbeanstalk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-route53-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-datapipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rekognition-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-organizations-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iam-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codepipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-logs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworkscm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-xray-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplaceentitlement-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-translate-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-events-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-alexaforbusiness-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemaker-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-s3-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costexplorer-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directconnect-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticache-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-gamelift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sqs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemakerruntime-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glue-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-waf-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mechanicalturkrequester-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatchmetrics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-migrationhub-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lambda-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-discovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidp-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-athena-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicecatalog-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mq-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-batch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-devicefarm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sts-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-stepfunctions-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesisvideo-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworks-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workdocs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-redshift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudfront-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-config-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-efs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-models-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ssm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codecommit-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-health-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudformation-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ses-1.11.267.jar
2018-04-17 04:26:16,846 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
--------------------------------------------------------------------------------
2018-04-17 04:26:16,848 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Registered UNIX signal handlers for [TERM, HUP, INT]
2018-04-17 04:26:16,851 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - YARN
daemon is running as: hadoop Yarn client user obtainer: hadoop
2018-04-17 04:26:16,854 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: jobmanager.archive.fs.dir,
hdfs:///completed-jobs/
2018-04-17 04:26:16,854 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: fs.default-scheme, s3a
2018-04-17 04:26:16,854 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: classloader.resolve-order,
parent-first
2018-04-17 04:26:16,855 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: high-availability.cluster-id,
application_1523726493647_0503
2018-04-17 04:26:16,855 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: jobmanager.heap.mb, 1024
2018-04-17 04:26:16,855 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: yarn.heap-cutoff-ratio, 0.5
2018-04-17 04:26:16,855 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: taskmanager.memory.preallocate, true
2018-04-17 04:26:16,855 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: yarn.maximum-failed-containers, -1
2018-04-17 04:26:16,856 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: akka.watch.heartbeat.pause, 600 s
2018-04-17 04:26:16,856 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: internal.cluster.execution-mode,
NORMAL
2018-04-17 04:26:16,856 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: taskmanager.memory.off-heap, on
2018-04-17 04:26:16,856 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: taskmanager.memory.fraction, 0.1
2018-04-17 04:26:16,856 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: taskmanager.numberOfTaskSlots, 1
2018-04-17 04:26:16,857 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf
2018-04-17 04:26:16,857 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: akka.ask.timeout, 2 min
2018-04-17 04:26:16,857 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: taskmanager.heap.mb, 10772
2018-04-17 04:26:16,857 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: historyserver.archive.fs.dir,
hdfs:///completed-jobs/
2018-04-17 04:26:16,874 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Setting directories for temporary files to:
/mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503
2018-04-17 04:26:16,896 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Starting YarnSessionClusterEntrypoint.
2018-04-17 04:26:16,897 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Install default filesystem.
2018-04-17 04:26:17,014 INFO
org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop
user set to hadoop (auth:SIMPLE)
2018-04-17 04:26:17,034 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Initializing cluster services.
2018-04-17 04:26:17,041 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Trying
to start actor system at
ip-10-0-22-186.ap-southeast-1.compute.internal:38669
2018-04-17 04:26:17,671 INFO  akka.event.slf4j.Slf4jLogger
                     - Slf4jLogger started
2018-04-17 04:26:17,794 INFO  akka.remote.Remoting
                     - Starting remoting
2018-04-17 04:26:17,937 INFO  akka.remote.Remoting
                     - Remoting started; listening on addresses
:[akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669]
2018-04-17 04:26:17,948 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Actor
system started at
akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669
2018-04-17 04:26:17,980 INFO  org.apache.flink.runtime.blob.BlobServer
                     - Created BLOB server storage directory
/mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503/blobStore-bca451c7-977d-425a-9e8f-4fe324fa8623
2018-04-17 04:26:17,983 INFO  org.apache.flink.runtime.blob.BlobServer
                     - Started BLOB server at 0.0.0.0:44217 - max
concurrent requests: 50 - max backlog: 1000
2018-04-17 04:26:18,009 INFO
org.apache.flink.runtime.metrics.MetricRegistryImpl           - No
metrics reporter configured, no metrics will be exposed/reported.
2018-04-17 04:26:18,015 INFO
org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore  -
Initializing FileArchivedExecutionGraphStore: Storage directory
/mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503/executionGraphStore-28bf6537-0bbf-486c-a346-084bd7517d7d,
expiration time 3600000, maximum cache size 52428800 bytes.
2018-04-17 04:26:18,047 INFO
org.apache.flink.runtime.blob.TransientBlobCache              -
Created BLOB cache storage directory
/mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503/blobStore-f1de0cad-77b0-4bbd-b218-63e32b10bf6a
2018-04-17 04:26:18,057 WARN
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Upload
directory /tmp/flink-web-1fd19ea7-ae19-4880-8787-92c5ec6ee509/flink-web-upload
does not exist, or has been deleted externally. Previously uploaded
files are no longer available.
2018-04-17 04:26:18,059 INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    -
Created directory
/tmp/flink-web-1fd19ea7-ae19-4880-8787-92c5ec6ee509/flink-web-upload
for file uploads.
2018-04-17 04:26:18,063 INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    -
Starting rest endpoint.
2018-04-17 04:26:18,386 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils           -
Determined location of main cluster component log file:
/var/log/hadoop-yarn/containers/application_1523726493647_0503/container_1523726493647_0503_01_000001/jobmanager.log
2018-04-17 04:26:18,386 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils           -
Determined location of main cluster component stdout file:
/var/log/hadoop-yarn/containers/application_1523726493647_0503/container_1523726493647_0503_01_000001/jobmanager.out
2018-04-17 04:26:18,454 INFO  org.apache.flink.runtime.rest.RestClient
                     - Rest client endpoint started.
2018-04-17 04:26:18,538 INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Rest
endpoint listening at
ip-10-0-22-186.ap-southeast-1.compute.internal:44431
2018-04-17 04:26:18,542 INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    -
http://ip-10-0-22-186.ap-southeast-1.compute.internal:44431 was
granted leadership with
leaderSessionID=00000000-0000-0000-0000-000000000000
2018-04-17 04:26:18,558 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService              -
Starting RPC endpoint for org.apache.flink.yarn.YarnResourceManager at
akka://flink/user/resourcemanager .
2018-04-17 04:26:18,713 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService              -
Starting RPC endpoint for
org.apache.flink.runtime.dispatcher.StandaloneDispatcher at
akka://flink/user/dispatcher .
2018-04-17 04:26:18,739 INFO
org.apache.flink.yarn.YarnResourceManager                     -
ResourceManager
akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669/user/resourcemanager
was granted leadership with fencing token
00000000000000000000000000000000
2018-04-17 04:26:18,740 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager  -
Starting the SlotManager.
2018-04-17 04:26:18,769 INFO  org.apache.hadoop.yarn.client.RMProxy
                     - Connecting to ResourceManager at
ip-10-0-22-169.ap-southeast-1.compute.internal/10.0.22.169:8030
2018-04-17 04:26:19,083 INFO
org.apache.flink.yarn.YarnResourceManager                     -
Recovered 0 containers from previous attempts ([]).
2018-04-17 04:26:19,086 INFO
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy
 - yarn.client.max-cached-nodemanagers-proxies : 0
2018-04-17 04:26:19,101 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher      -
Dispatcher akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669/user/dispatcher
was granted leadership with fencing token
00000000000000000000000000000000
2018-04-17 04:26:19,101 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher      -
Recovering all persisted jobs.




[2] JM's logs

2018-04-16 12:53:15,825 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
--------------------------------------------------------------------------------
2018-04-16 12:53:15,827 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Starting YarnSessionClusterEntrypoint (Version: 1.5.0, Rev:2af481a,
Date:02.04.2018 @ 15:17:35 UTC)
2018-04-16 12:53:15,827 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  OS
current user: yarn
2018-04-16 12:53:16,798 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Current Hadoop/Kerberos user: hadoop
2018-04-16 12:53:16,799 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM:
OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.161-b14
2018-04-16 12:53:16,799 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Maximum heap size: 406 MiBytes
2018-04-16 12:53:16,799 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
JAVA_HOME: /usr/lib/jvm/java-openjdk
2018-04-16 12:53:16,800 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Hadoop version: 2.8.3
2018-04-16 12:53:16,801 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM
Options:
2018-04-16 12:53:16,801 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
-Xmx424m
2018-04-16 12:53:16,801 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
-Dlog.file=/var/log/hadoop-yarn/containers/application_1523726493647_0302/container_1523726493647_0302_01_000001/jobmanager.log
2018-04-16 12:53:16,801 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
-Dlogback.configurationFile=file:logback.xml
2018-04-16 12:53:16,801 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
-Dlog4j.configuration=file:log4j.properties
2018-04-16 12:53:16,801 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Program Arguments: (none)
2018-04-16 12:53:16,801 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Classpath: lib/flink-dist_2.11-1.5.0.jar:lib/flink-python_2.11-1.5.0.jar:lib/flink-shaded-hadoop2-uber-1.5.0.jar:lib/flink-shaded-include-yarn-0.9.1.jar:lib/flink-yarn_2.11-1.5-SNAPSHOT.jar:lib/guava-18.0.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.7.jar:log4j.properties:logback.xml:flink.jar:flink-conf.yaml::/etc/hadoop/conf:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-streaming.jar:/usr/lib/hadoop/hadoop-distcp.jar:/usr/lib/hadoop/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archives.jar:/usr/lib/hadoop/hadoop-ant.jar:/usr/lib/hadoop/hadoop-aws.jar:/usr/lib/hadoop/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-rumen.jar:/usr/lib/hadoop/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack.jar:/usr/lib/hadoop/hadoop-azure.jar:/usr/lib/hadoop/hadoop-extras.jar:/usr/lib/hadoop/hadoop-azure-datalake.jar:/usr/lib/hadoop/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix.jar:/usr/lib/hadoop/hadoop-nfs.jar:/usr/lib/hadoop/hadoop-datajoin.jar:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop/hadoop-sls.jar:/usr/lib/hadoop/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/lib/curator-framework-2.7.1.jar:/usr/lib/hadoop/lib/json-smart-1.1.1.jar:/usr/lib/hadoop/lib/curator-recipes-2.7.1.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/lib/jcip-annotations-1.0.jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/lib/avro-1.7.4.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop/lib/gson-2.2.4.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop/lib/jsch-0.1.54.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar:/usr/lib/hadoop/lib/jersey-server-1.9.jar:/usr/lib/hadoop/lib/httpcore-4.4.4.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/curator-client-2.7.1.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop/lib/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/jsr305-3.0.0.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/httpclient-4.5.3.jar:/usr/lib/hadoop/lib/commons-lang-2.6.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/slf4j-api-1.7.10.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/xercesImpl-2.9.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-hdfs/lib/netty-all-4.0.23.Final.jar:/usr/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-hdfs/lib/okio-1.4.0.jar:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/xml-apis-1.3.04.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/okhttp-2.4.0.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/ion-java-1.0.2.jar:/usr/lib/hadoop-mapreduce/curator-framework-2.7.1.jar:/usr/lib/hadoop-mapreduce/json-smart-1.1.1.jar:/usr/lib/hadoop-mapreduce/curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-core-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/zookeeper-3.4.10.jar:/usr/lib/hadoop-mapreduce/jmespath-java-1.11.267.jar:/usr/lib/hadoop-mapreduce/jettison-1.1.jar:/usr/lib/hadoop-mapreduce/java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/jcip-annotations-1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs.jar:/usr/lib/hadoop-mapreduce/hadoop-auth.jar:/usr/lib/hadoop-mapreduce/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-2.6.7.jar:/usr/lib/hadoop-mapreduce/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-mapreduce/gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/azure-data-lake-store-sdk-2.2.3.jar:/usr/lib/hadoop-mapreduce/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/jsch-0.1.54.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/commons-lang3-3.3.2.jar:/usr/lib/hadoop-mapreduce/xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/commons-collections-3.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/hadoop-archives.jar:/usr/lib/hadoop-mapreduce/commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/hadoop-ant.jar:/usr/lib/hadoop-mapreduce/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/jackson-databind-2.6.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-aws.jar:/usr/lib/hadoop-mapreduce/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/jackson-annotations-2.6.7.jar:/usr/lib/hadoop-mapreduce/httpcore-4.4.4.jar:/usr/lib/hadoop-mapreduce/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jetty-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop-mapreduce/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/xz-1.0.jar:/usr/lib/hadoop-mapreduce/joda-time-2.9.4.jar:/usr/lib/hadoop-mapreduce/commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-mapreduce/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/activation-1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/hadoop-azure.jar:/usr/lib/hadoop-mapreduce/hadoop-extras.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/okio-1.4.0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/azure-storage-2.2.0.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-s3-1.11.267.jar:/usr/lib/hadoop-mapreduce/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/jackson-dataformat-cbor-2.6.7.jar:/usr/lib/hadoop-mapreduce/jackson-xc-1.9.13.jar:/usr/lib/hadoop-mapreduce/okhttp-2.4.0.jar:/usr/lib/hadoop-mapreduce/asm-3.2.jar:/usr/lib/hadoop-mapreduce/hadoop-sls.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/httpclient-4.5.3.jar:/usr/lib/hadoop-mapreduce/commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-kms-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/lib/curator-test-2.7.1.jar:/usr/lib/hadoop-yarn/lib/java-util-1.9.0.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10-tests.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop-yarn/lib/jettison-1.1.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/lib/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/lib/hadoop-yarn/lib/commons-math-2.2.jar:/usr/lib/hadoop-yarn/lib/curator-client-2.7.1.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-yarn/lib/activation-1.1.jar:/usr/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-yarn/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-yarn/lib/guava-11.0.2.jar:/usr/lib/hadoop-yarn/lib/json-io-2.5.1.jar:/usr/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/lib/hadoop-yarn/lib/fst-2.50.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/javassist-3.18.1-GA.jar:/usr/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo-0.4.19.jar:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/jcl-over-slf4j-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/ion-java-1.0.2.jar:/usr/share/aws/emr/emrfs/lib/emrfs-hadoop-assembly-2.21.0.jar:/usr/share/aws/emr/emrfs/lib/jmespath-java-1.11.267.jar:/usr/share/aws/emr/emrfs/lib/javax.inject-1.jar:/usr/share/aws/emr/emrfs/lib/bcpkix-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/lib/slf4j-api-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/aopalliance-1.0.jar:/usr/share/aws/emr/emrfs/lib/bcprov-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/auxlib/*:/usr/share/aws/emr/lib/*:/usr/share/aws/emr/ddb/lib/emr-ddb-hadoop.jar:/usr/share/aws/emr/goodies/lib/emr-hadoop-goodies.jar:/usr/share/aws/emr/kinesis/lib/emr-kinesis-hadoop.jar:/usr/lib/spark/yarn/lib/datanucleus-api-jdo.jar:/usr/lib/spark/yarn/lib/datanucleus-core.jar:/usr/lib/spark/yarn/lib/datanucleus-rdbms.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink-1.1.0.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacecommerceanalytics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rds-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-core-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-clouddirectory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsmv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-emr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediaconvert-1.11.267.jar:/usr/share/aws/aws-java-sdk/jmespath-java-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appsync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastore-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iot-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costandusagereport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpleworkflow-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-comprehend-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidentity-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ec2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudtrail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servermigration-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesis-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workmail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-polly-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpledb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscalingplans-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elastictranscoder-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pinpoint-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lexmodelbuilding-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacemeteringservice-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dynamodb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-storagegateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codegen-maven-plugin-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediapackage-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroupstaggingapi-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-acm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-machinelearning-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mobile-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dax-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lightsail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sns-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codedeploy-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-serverlessapplicationrepository-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-shield-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-applicationautoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastoredata-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloud9-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-budgets-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iotjobsdataplane-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-medialive-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-inspector-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-importexport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-test-utils-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitosync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-api-gateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codebuild-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workspaces-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opensdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-snowball-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pricing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-code-generator-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glacier-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-guardduty-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicediscovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codestar-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appstream-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancingv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-greengrass-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lex-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroups-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-support-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticbeanstalk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-route53-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-datapipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rekognition-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-organizations-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iam-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codepipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-logs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworkscm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-xray-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplaceentitlement-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-translate-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-events-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-alexaforbusiness-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemaker-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-s3-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costexplorer-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directconnect-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticache-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-gamelift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sqs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemakerruntime-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glue-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-waf-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mechanicalturkrequester-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatchmetrics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-migrationhub-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lambda-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-discovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidp-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-athena-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicecatalog-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mq-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-batch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-devicefarm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sts-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-stepfunctions-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesisvideo-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworks-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workdocs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-redshift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudfront-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-config-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-efs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-models-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ssm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codecommit-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-health-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudformation-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ses-1.11.267.jar
2018-04-16 12:53:16,803 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
--------------------------------------------------------------------------------
2018-04-16 12:53:16,805 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Registered UNIX signal handlers for [TERM, HUP, INT]
2018-04-16 12:53:16,809 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - YARN
daemon is running as: hadoop Yarn client user obtainer: hadoop
2018-04-16 12:53:16,813 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: jobmanager.archive.fs.dir,
hdfs:///completed-jobs/
2018-04-16 12:53:16,814 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: fs.default-scheme, s3a
2018-04-16 12:53:16,814 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: classloader.resolve-order,
parent-first
2018-04-16 12:53:16,814 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: high-availability.cluster-id,
application_1523726493647_0302
2018-04-16 12:53:16,815 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: jobmanager.heap.mb, 1024
2018-04-16 12:53:16,815 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: yarn.heap-cutoff-ratio, 0.5
2018-04-16 12:53:16,815 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: taskmanager.memory.preallocate, true
2018-04-16 12:53:16,815 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: yarn.maximum-failed-containers, -1
2018-04-16 12:53:16,815 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: akka.watch.heartbeat.pause, 600 s
2018-04-16 12:53:16,815 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: internal.cluster.execution-mode,
NORMAL
2018-04-16 12:53:16,816 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: taskmanager.memory.off-heap, on
2018-04-16 12:53:16,816 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: taskmanager.memory.fraction, 0.1
2018-04-16 12:53:16,816 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: taskmanager.numberOfTaskSlots, 1
2018-04-16 12:53:16,816 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf
2018-04-16 12:53:16,816 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: taskmanager.heap.mb, 9216
2018-04-16 12:53:16,817 INFO
org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: historyserver.archive.fs.dir,
hdfs:///completed-jobs/
2018-04-16 12:53:16,833 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Setting directories for temporary files to:
/mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302
2018-04-16 12:53:16,859 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Starting YarnSessionClusterEntrypoint.
2018-04-16 12:53:16,859 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Install default filesystem.
2018-04-16 12:53:17,036 INFO
org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop
user set to hadoop (auth:SIMPLE)
2018-04-16 12:53:17,071 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
Initializing cluster services.
2018-04-16 12:53:17,081 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Trying
to start actor system at
ip-10-0-22-186.ap-southeast-1.compute.internal:36411
2018-04-16 12:53:18,022 INFO  akka.event.slf4j.Slf4jLogger
                     - Slf4jLogger started
2018-04-16 12:53:18,254 INFO  akka.remote.Remoting
                     - Starting remoting
2018-04-16 12:53:18,461 INFO  akka.remote.Remoting
                     - Remoting started; listening on addresses
:[akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411]
2018-04-16 12:53:18,488 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Actor
system started at
akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411
2018-04-16 12:53:18,559 INFO  org.apache.flink.runtime.blob.BlobServer
                     - Created BLOB server storage directory
/mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302/blobStore-9ec0a59f-c03a-446f-a841-60a65892b383
2018-04-16 12:53:18,566 INFO  org.apache.flink.runtime.blob.BlobServer
                     - Started BLOB server at 0.0.0.0:46869 - max
concurrent requests: 50 - max backlog: 1000
2018-04-16 12:53:18,632 INFO
org.apache.flink.runtime.metrics.MetricRegistryImpl           - No
metrics reporter configured, no metrics will be exposed/reported.
2018-04-16 12:53:18,638 INFO
org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore  -
Initializing FileArchivedExecutionGraphStore: Storage directory
/mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302/executionGraphStore-ee071b9a-ad86-4b36-860c-6501f509ecf9,
expiration time 3600000, maximum cache size 52428800 bytes.
2018-04-16 12:53:18,726 INFO
org.apache.flink.runtime.blob.TransientBlobCache              -
Created BLOB cache storage directory
/mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302/blobStore-c15647ee-31c1-451d-84d2-729a67e065d8
2018-04-16 12:53:18,750 WARN
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Upload
directory /tmp/flink-web-6d14c830-9074-4a2f-8e1e-7344545e34e5/flink-web-upload
does not exist, or has been deleted externally. Previously uploaded
files are no longer available.
2018-04-16 12:53:18,757 INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    -
Created directory
/tmp/flink-web-6d14c830-9074-4a2f-8e1e-7344545e34e5/flink-web-upload
for file uploads.
2018-04-16 12:53:18,765 INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    -
Starting rest endpoint.
2018-04-16 12:53:19,361 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils           -
Determined location of main cluster component log file:
/var/log/hadoop-yarn/containers/application_1523726493647_0302/container_1523726493647_0302_01_000001/jobmanager.log
2018-04-16 12:53:19,362 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils           -
Determined location of main cluster component stdout file:
/var/log/hadoop-yarn/containers/application_1523726493647_0302/container_1523726493647_0302_01_000001/jobmanager.out
2018-04-16 12:53:19,442 INFO  org.apache.flink.runtime.rest.RestClient
                     - Rest client endpoint started.
2018-04-16 12:53:19,589 INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Rest
endpoint listening at
ip-10-0-22-186.ap-southeast-1.compute.internal:45697
2018-04-16 12:53:19,589 INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    -
http://ip-10-0-22-186.ap-southeast-1.compute.internal:45697 was
granted leadership with
leaderSessionID=00000000-0000-0000-0000-000000000000
2018-04-16 12:53:19,614 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService              -
Starting RPC endpoint for org.apache.flink.yarn.YarnResourceManager at
akka://flink/user/resourcemanager .
2018-04-16 12:53:19,979 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService              -
Starting RPC endpoint for
org.apache.flink.runtime.dispatcher.StandaloneDispatcher at
akka://flink/user/dispatcher .
2018-04-16 12:53:20,049 INFO
org.apache.flink.yarn.YarnResourceManager                     -
ResourceManager
akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411/user/resourcemanager
was granted leadership with fencing token
00000000000000000000000000000000
2018-04-16 12:53:20,050 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager  -
Starting the SlotManager.
2018-04-16 12:53:20,101 INFO  org.apache.hadoop.yarn.client.RMProxy
                     - Connecting to ResourceManager at
ip-10-0-22-169.ap-southeast-1.compute.internal/10.0.22.169:8030
2018-04-16 12:53:20,649 INFO
org.apache.flink.yarn.YarnResourceManager                     -
Recovered 0 containers from previous attempts ([]).
2018-04-16 12:53:20,655 INFO
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy
 - yarn.client.max-cached-nodemanagers-proxies : 0
2018-04-16 12:53:20,706 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher      -
Dispatcher akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411/user/dispatcher
was granted leadership with fencing token
00000000000000000000000000000000
2018-04-16 12:53:20,706 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher      -
Recovering all persisted jobs.
2018-04-16 12:53:26,207 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher      -
Submitting job 58a1b149b5b69de66bb931836c459d2b
(mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job).
2018-04-16 12:53:26,223 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService              -
Starting RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster
at akka://flink/user/jobmanager_0 .
2018-04-16 12:53:26,237 INFO
org.apache.flink.runtime.jobmaster.JobMaster                  -
Initializing job
mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job
(58a1b149b5b69de66bb931836c459d2b).
2018-04-16 12:53:26,279 INFO
org.apache.flink.runtime.jobmaster.JobMaster                  - Using
restart strategy NoRestartStrategy for
mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job
(58a1b149b5b69de66bb931836c459d2b).
2018-04-16 12:53:26,285 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService              -
Starting RPC endpoint for
org.apache.flink.runtime.jobmaster.slotpool.SlotPool at
akka://flink/user/820d207e-fddd-4e24-b327-1f3c1cb25024 .
2018-04-16 12:53:26,320 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph        - Job
recovers via failover strategy: full graph restart
2018-04-16 12:53:26,386 INFO
org.apache.flink.runtime.jobmaster.JobMaster                  -
Running initialization on master for job
mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job
(58a1b149b5b69de66bb931836c459d2b).
2018-04-16 12:53:26,671 INFO
org.apache.flink.runtime.jobmaster.JobMaster                  -
Successfully ran initialization on master in 284 ms.
2018-04-16 12:53:36,204 ERROR
org.apache.flink.runtime.rest.handler.job.JobSubmitHandler    -
Implementation error: Unhandled exception.
akka.pattern.AskTimeoutException: Ask timed out on
[Actor[akka://flink/user/dispatcher#1710592138]] after [10000 ms].
Sender[null] sent message of type
"org.apache.flink.runtime.rpc.messages.LocalFencedMessage".
	at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:604)
	at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:126)
	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
	at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
	at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
	at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:329)
	at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:280)
	at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:284)
	at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:236)
	at java.lang.Thread.run(Thread.java:748)
2018-04-16 12:53:36,300 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
RECEIVED SIGNAL 15: SIGTERM. Shutting down as requested.
2018-04-16 12:53:36,311 INFO
org.apache.flink.runtime.blob.TransientBlobCache              -
Shutting down BLOB cache
2018-04-16 12:53:36,314 INFO  org.apache.flink.runtime.blob.BlobServer
                     - Stopped BLOB server at 0.0.0.0:46869

Re: Testing on Flink 1.5

Posted by Gary Yao <ga...@data-artisans.com>.
Hi Amit,

web.timeout should only affect RPC calls originating from the REST API. In
FLIP-6, the submission of the job graph happens via HTTP. The value under
akka.ask.timeout is still used as the default timeout for RPC calls [1][2].
Since you also had custom heartbeats settings, you should consider setting
heartbeat.interval and heartbeat.timeout when using FLIP-6 mode [3]. AFAIK
Akka's DeathWatch is not used anymore to detect TaskManager failures. Hence,
akka.watch.heartbeat.interval should have no effect.

Best,
Gary

[1]
https://github.com/apache/flink/blob/fb254763c00df5d336c6defa1ae960e32c97b2ae/flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.java#L389
[2]
https://github.com/apache/flink/blob/fb254763c00df5d336c6defa1ae960e32c97b2ae/flink-runtime/src/main/scala/org/apache/flink/runtime/akka/AkkaUtils.scala#L606
[3]
https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#heartbeat-manager

On Fri, Apr 20, 2018 at 12:22 PM, Amit Jain <aj...@gmail.com> wrote:

> Hi Gary,
>
> This setting has resolved the issue. Does it increase timeout for all the
> RPC or specific components?
>
> We had following settings in Flink 1.3.2 and they did the job for us.
>
> akka.watch.heartbeat.pause: 600 s
> akka.client.timeout: 5 min
> akka.ask.timeout: 120 s
>
>
> --
> Thanks,
> Amit
>

Re: Testing on Flink 1.5

Posted by Amit Jain <aj...@gmail.com>.
Hi Gary,

This setting has resolved the issue. Does it increase timeout for all the
RPC or specific components?

We had following settings in Flink 1.3.2 and they did the job for us.

akka.watch.heartbeat.pause: 600 s
akka.client.timeout: 5 min
akka.ask.timeout: 120 s


--
Thanks,
Amit

Re: Testing on Flink 1.5

Posted by Gary Yao <ga...@data-artisans.com>.
Hi Amit,

Can you try setting:

    web.timeout: 120000

This will increase the timeout of the RPC call to 2 minutes. Did you need to
raise the timeouts prior to FLIP-6 as well?

Best,
Gary

On Wed, Apr 18, 2018 at 3:09 PM, Amit Jain <aj...@gmail.com> wrote:

> Hi Gary,
>
> Please find my reply in place. We have also logged https://issues.apache.
> org/jira/browse/FLINK-9207, please have a look on this.
>
>
> --
> Thanks,
> Amit
>
>
> On Tue, Apr 17, 2018 at 9:10 PM, Gary Yao <ga...@data-artisans.com> wrote:
>
>> Hi Amit,
>>
>> Thank you for testing the 1.5 Release Candidate. Find my answers below.
>>
>> > Few of our jobs are stuck with logs [1], these jobs are only able to
>> allocate JM and couldn't get any TM, however, there are ample resource on
>> our cluster.
>>
>> In the logs you attached I cannot see that a job was submitted. If a job
>> is
>> submitted, the dispatcher should log: "Submitting job <job_id>
>> (<job_name>)"
>> [1]. Could it be that the attached logs are incomplete, or the client
>> already
>> fails to submit the job? Can you also attach the client logs?
>>
>> I agree with you, we should have these logs however it is not happening.
> I'll work on it further and provide more info, right now, I can provide
> logs which got generated later.
>
> 2018-04-16 04:36:43,747 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Dispatcher akka.tcp://flink@ip-10-0-22-43.ap-southeast-1.compute.internal:39329/user/dispatcher was granted leadership with fencing token 00000000000000000000000000000000
> 2018-04-16 04:36:43,747 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Recovering all persisted jobs.
> 2018-04-16 17:21:37,800 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:21:42,803 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:21:47,805 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:21:52,806 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:21:57,807 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:22:02,808 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:22:07,810 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:22:12,812 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:22:17,813 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:22:22,814 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:22:27,816 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:22:32,817 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:22:37,818 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:22:42,821 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:22:47,822 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:22:52,825 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:22:57,826 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:23:02,828 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:23:07,829 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
> 2018-04-16 17:23:12,832 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Updating with new AMRMToken
>
>
>
>> > All completed application on YARN are marked as KILLED, instead of
>> FINISHED. It creates problem on tracking jobs as we require to navigate on
>> each application logs and also history server is not logging any completed
>> application.
>>
>> I was able to reproduce this: In non-detached mode, the State and
>> FinalStatus
>> are set to KILLED. In detached mode, the Status and FinalStatus are set
>> correctly. I have re-opened this ticket:
>>   https://issues.apache.org/jira/browse/FLINK-8900
>>
>> You are also right about the History Server, which is currently not
>> implemented
>> for FLIP-6. Here is the ticket for reference:
>>   https://issues.apache.org/jira/browse/FLINK-9194
>>
>>
>> > We are running our system on AWS EMR and using CORE nodes at its bare
>> minimum, we see with new paradigm of application submission we are hitting
>> Out of space error in our HDFS Data Node. Do you think, we should have
>> configuration to automatically evict these .flink/ files?
>>
>> This seems to be a regression. In Flink 1.4., the files are cleaned up
>> automatically. I opened this ticket:
>> https://issues.apache.org/jira/browse/FLINK-9196
>>
>>
>> > Lastly, we are also observing Akka.ask.timeout error for few of our
>> jobs (JM's logs[2]), we tried to increase this parameter to 2 min from
>> default 10 s and still observe the same.
>>
>> Does this always happen for the same jobs? I cannot reproduce this.
>>
>> Yes, it happening for same set of jobs. Specifically, these jobs are
> having multiple S3 objects to work with. I tried increasing
> akka.ask.timeout which is not getting reflected on JM's log also and same
> has been experienced while running the jobs.
> I also tried one of these jobs on empty cluster(EMR TASK 15 node,
> r3.xlarge type) and still face same.
>
> Best,
>> Gary
>>
>> [1] https://github.com/apache/flink/blob/master/flink-runtime/
>> src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java#L235
>>
>>
>> On Tue, Apr 17, 2018 at 9:35 AM, Amit Jain <aj...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>>
>>> We have started testing our workload on Flink 1.5.0 (Commit: 2af481a) and faced few issues. Here, we are using YARN as cluster manager.
>>>
>>>
>>>    - Few of our jobs are stuck with logs [1], these jobs are only able to allocate JM and couldn't get any TM, however, there are ample resource on our cluster.
>>>    - All completed application on YARN are marked as KILLED, instead of FINISHED. It creates problem on tracking jobs as we require to navigate on each application logs and also history server is not logging any completed application.
>>>    - We are running our system on AWS EMR and using CORE nodes at its bare minimum, we see with new paradigm of application submission we are hitting Out of space error in our HDFS Data Node. Do you think, we should have configuration to automatically evict these .flink/ files?
>>>    - Lastly, we are also observing Akka.ask.timeout error for few of our jobs (JM's logs[2]), we tried to increase this parameter to 2 min from default 10 s and still observe the same.
>>>
>>> Please help us guide through these issues.
>>>
>>> --
>>> Thanks,
>>> Amit
>>>
>>> [1] JM logs where jobs are not even started
>>>
>>> 2018-04-17 04:26:16,165 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
>>> 2018-04-17 04:26:16,166 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Starting YarnSessionClusterEntrypoint (Version: 1.5.0, Rev:2af481a, Date:02.04.2018 @ 15:17:35 UTC)
>>> 2018-04-17 04:26:16,166 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  OS current user: yarn
>>> 2018-04-17 04:26:16,843 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Current Hadoop/Kerberos user: hadoop
>>> 2018-04-17 04:26:16,843 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.161-b14
>>> 2018-04-17 04:26:16,843 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Maximum heap size: 406 MiBytes
>>> 2018-04-17 04:26:16,843 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JAVA_HOME: /usr/lib/jvm/java-openjdk
>>> 2018-04-17 04:26:16,844 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Hadoop version: 2.8.3
>>> 2018-04-17 04:26:16,844 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM Options:
>>> 2018-04-17 04:26:16,844 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Xmx424m
>>> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog.file=/var/log/hadoop-yarn/containers/application_1523726493647_0503/container_1523726493647_0503_01_000001/jobmanager.log
>>> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlogback.configurationFile=file:logback.xml
>>> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog4j.configuration=file:log4j.properties
>>> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Program Arguments: (none)
>>> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Classpath: lib/flink-dist_2.11-1.5.0.jar:lib/flink-python_2.11-1.5.0.jar:lib/flink-shaded-hadoop2-uber-1.5.0.jar:lib/flink-shaded-include-yarn-0.9.1.jar:lib/flink-yarn_2.11-1.5-SNAPSHOT.jar:lib/guava-18.0.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.7.jar:log4j.properties:logback.xml:flink.jar:flink-conf.yaml::/etc/hadoop/conf:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-streaming.jar:/usr/lib/hadoop/hadoop-distcp.jar:/usr/lib/hadoop/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archives.jar:/usr/lib/hadoop/hadoop-ant.jar:/usr/lib/hadoop/hadoop-aws.jar:/usr/lib/hadoop/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-rumen.jar:/usr/lib/hadoop/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack.jar:/usr/lib/hadoop/hadoop-azure.jar:/usr/lib/hadoop/hadoop-extras.jar:/usr/lib/hadoop/hadoop-azure-datalake.jar:/usr/lib/hadoop/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix.jar:/usr/lib/hadoop/hadoop-nfs.jar:/usr/lib/hadoop/hadoop-datajoin.jar:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop/hadoop-sls.jar:/usr/lib/hadoop/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/lib/curator-framework-2.7.1.jar:/usr/lib/hadoop/lib/json-smart-1.1.1.jar:/usr/lib/hadoop/lib/curator-recipes-2.7.1.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/lib/jcip-annotations-1.0.jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/lib/avro-1.7.4.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop/lib/gson-2.2.4.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop/lib/jsch-0.1.54.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar:/usr/lib/hadoop/lib/jersey-server-1.9.jar:/usr/lib/hadoop/lib/httpcore-4.4.4.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/curator-client-2.7.1.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop/lib/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/jsr305-3.0.0.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/httpclient-4.5.3.jar:/usr/lib/hadoop/lib/commons-lang-2.6.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/slf4j-api-1.7.10.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/xercesImpl-2.9.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-hdfs/lib/netty-all-4.0.23.Final.jar:/usr/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-hdfs/lib/okio-1.4.0.jar:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/xml-apis-1.3.04.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/okhttp-2.4.0.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/ion-java-1.0.2.jar:/usr/lib/hadoop-mapreduce/curator-framework-2.7.1.jar:/usr/lib/hadoop-mapreduce/json-smart-1.1.1.jar:/usr/lib/hadoop-mapreduce/curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-core-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/zookeeper-3.4.10.jar:/usr/lib/hadoop-mapreduce/jmespath-java-1.11.267.jar:/usr/lib/hadoop-mapreduce/jettison-1.1.jar:/usr/lib/hadoop-mapreduce/java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/jcip-annotations-1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs.jar:/usr/lib/hadoop-mapreduce/hadoop-auth.jar:/usr/lib/hadoop-mapreduce/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-2.6.7.jar:/usr/lib/hadoop-mapreduce/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-mapreduce/gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/azure-data-lake-store-sdk-2.2.3.jar:/usr/lib/hadoop-mapreduce/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/jsch-0.1.54.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/commons-lang3-3.3.2.jar:/usr/lib/hadoop-mapreduce/xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/commons-collections-3.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/hadoop-archives.jar:/usr/lib/hadoop-mapreduce/commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/hadoop-ant.jar:/usr/lib/hadoop-mapreduce/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/jackson-databind-2.6.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-aws.jar:/usr/lib/hadoop-mapreduce/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/jackson-annotations-2.6.7.jar:/usr/lib/hadoop-mapreduce/httpcore-4.4.4.jar:/usr/lib/hadoop-mapreduce/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jetty-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop-mapreduce/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/xz-1.0.jar:/usr/lib/hadoop-mapreduce/joda-time-2.9.4.jar:/usr/lib/hadoop-mapreduce/commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-mapreduce/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/activation-1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/hadoop-azure.jar:/usr/lib/hadoop-mapreduce/hadoop-extras.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/okio-1.4.0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/azure-storage-2.2.0.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-s3-1.11.267.jar:/usr/lib/hadoop-mapreduce/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/jackson-dataformat-cbor-2.6.7.jar:/usr/lib/hadoop-mapreduce/jackson-xc-1.9.13.jar:/usr/lib/hadoop-mapreduce/okhttp-2.4.0.jar:/usr/lib/hadoop-mapreduce/asm-3.2.jar:/usr/lib/hadoop-mapreduce/hadoop-sls.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/httpclient-4.5.3.jar:/usr/lib/hadoop-mapreduce/commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-kms-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/lib/curator-test-2.7.1.jar:/usr/lib/hadoop-yarn/lib/java-util-1.9.0.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10-tests.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop-yarn/lib/jettison-1.1.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/lib/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/lib/hadoop-yarn/lib/commons-math-2.2.jar:/usr/lib/hadoop-yarn/lib/curator-client-2.7.1.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-yarn/lib/activation-1.1.jar:/usr/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-yarn/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-yarn/lib/guava-11.0.2.jar:/usr/lib/hadoop-yarn/lib/json-io-2.5.1.jar:/usr/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/lib/hadoop-yarn/lib/fst-2.50.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/javassist-3.18.1-GA.jar:/usr/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo-0.4.19.jar:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/jcl-over-slf4j-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/ion-java-1.0.2.jar:/usr/share/aws/emr/emrfs/lib/emrfs-hadoop-assembly-2.21.0.jar:/usr/share/aws/emr/emrfs/lib/jmespath-java-1.11.267.jar:/usr/share/aws/emr/emrfs/lib/javax.inject-1.jar:/usr/share/aws/emr/emrfs/lib/bcpkix-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/lib/slf4j-api-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/aopalliance-1.0.jar:/usr/share/aws/emr/emrfs/lib/bcprov-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/auxlib/*:/usr/share/aws/emr/lib/*:/usr/share/aws/emr/ddb/lib/emr-ddb-hadoop.jar:/usr/share/aws/emr/goodies/lib/emr-hadoop-goodies.jar:/usr/share/aws/emr/kinesis/lib/emr-kinesis-hadoop.jar:/usr/lib/spark/yarn/lib/datanucleus-api-jdo.jar:/usr/lib/spark/yarn/lib/datanucleus-core.jar:/usr/lib/spark/yarn/lib/datanucleus-rdbms.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink-1.1.0.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacecommerceanalytics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rds-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-core-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-clouddirectory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsmv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-emr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediaconvert-1.11.267.jar:/usr/share/aws/aws-java-sdk/jmespath-java-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appsync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastore-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iot-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costandusagereport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpleworkflow-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-comprehend-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidentity-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ec2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudtrail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servermigration-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesis-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workmail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-polly-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpledb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscalingplans-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elastictranscoder-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pinpoint-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lexmodelbuilding-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacemeteringservice-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dynamodb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-storagegateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codegen-maven-plugin-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediapackage-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroupstaggingapi-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-acm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-machinelearning-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mobile-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dax-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lightsail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sns-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codedeploy-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-serverlessapplicationrepository-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-shield-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-applicationautoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastoredata-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloud9-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-budgets-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iotjobsdataplane-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-medialive-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-inspector-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-importexport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-test-utils-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitosync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-api-gateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codebuild-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workspaces-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opensdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-snowball-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pricing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-code-generator-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glacier-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-guardduty-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicediscovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codestar-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appstream-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancingv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-greengrass-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lex-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroups-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-support-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticbeanstalk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-route53-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-datapipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rekognition-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-organizations-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iam-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codepipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-logs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworkscm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-xray-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplaceentitlement-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-translate-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-events-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-alexaforbusiness-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemaker-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-s3-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costexplorer-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directconnect-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticache-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-gamelift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sqs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemakerruntime-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glue-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-waf-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mechanicalturkrequester-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatchmetrics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-migrationhub-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lambda-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-discovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidp-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-athena-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicecatalog-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mq-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-batch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-devicefarm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sts-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-stepfunctions-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesisvideo-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworks-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workdocs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-redshift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudfront-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-config-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-efs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-models-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ssm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codecommit-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-health-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudformation-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ses-1.11.267.jar
>>> 2018-04-17 04:26:16,846 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
>>> 2018-04-17 04:26:16,848 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Registered UNIX signal handlers for [TERM, HUP, INT]
>>> 2018-04-17 04:26:16,851 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - YARN daemon is running as: hadoop Yarn client user obtainer: hadoop
>>> 2018-04-17 04:26:16,854 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.archive.fs.dir, hdfs:///completed-jobs/
>>> 2018-04-17 04:26:16,854 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.default-scheme, s3a
>>> 2018-04-17 04:26:16,854 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: classloader.resolve-order, parent-first
>>> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: high-availability.cluster-id, application_1523726493647_0503
>>> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.heap.mb, 1024
>>> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: yarn.heap-cutoff-ratio, 0.5
>>> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.preallocate, true
>>> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: yarn.maximum-failed-containers, -1
>>> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: akka.watch.heartbeat.pause, 600 s
>>> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: internal.cluster.execution-mode, NORMAL
>>> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.off-heap, on
>>> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.fraction, 0.1
>>> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.numberOfTaskSlots, 1
>>> 2018-04-17 04:26:16,857 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf
>>> 2018-04-17 04:26:16,857 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: akka.ask.timeout, 2 min
>>> 2018-04-17 04:26:16,857 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.heap.mb, 10772
>>> 2018-04-17 04:26:16,857 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: historyserver.archive.fs.dir, hdfs:///completed-jobs/
>>> 2018-04-17 04:26:16,874 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Setting directories for temporary files to: /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503
>>> 2018-04-17 04:26:16,896 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Starting YarnSessionClusterEntrypoint.
>>> 2018-04-17 04:26:16,897 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Install default filesystem.
>>> 2018-04-17 04:26:17,014 INFO  org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop user set to hadoop (auth:SIMPLE)
>>> 2018-04-17 04:26:17,034 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Initializing cluster services.
>>> 2018-04-17 04:26:17,041 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Trying to start actor system at ip-10-0-22-186.ap-southeast-1.compute.internal:38669
>>> 2018-04-17 04:26:17,671 INFO  akka.event.slf4j.Slf4jLogger                                  - Slf4jLogger started
>>> 2018-04-17 04:26:17,794 INFO  akka.remote.Remoting                                          - Starting remoting
>>> 2018-04-17 04:26:17,937 INFO  akka.remote.Remoting                                          - Remoting started; listening on addresses :[akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669]
>>> 2018-04-17 04:26:17,948 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Actor system started at akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669
>>> 2018-04-17 04:26:17,980 INFO  org.apache.flink.runtime.blob.BlobServer                      - Created BLOB server storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503/blobStore-bca451c7-977d-425a-9e8f-4fe324fa8623
>>> 2018-04-17 04:26:17,983 INFO  org.apache.flink.runtime.blob.BlobServer                      - Started BLOB server at 0.0.0.0:44217 - max concurrent requests: 50 - max backlog: 1000
>>> 2018-04-17 04:26:18,009 INFO  org.apache.flink.runtime.metrics.MetricRegistryImpl           - No metrics reporter configured, no metrics will be exposed/reported.
>>> 2018-04-17 04:26:18,015 INFO  org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore  - Initializing FileArchivedExecutionGraphStore: Storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503/executionGraphStore-28bf6537-0bbf-486c-a346-084bd7517d7d, expiration time 3600000, maximum cache size 52428800 bytes.
>>> 2018-04-17 04:26:18,047 INFO  org.apache.flink.runtime.blob.TransientBlobCache              - Created BLOB cache storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503/blobStore-f1de0cad-77b0-4bbd-b218-63e32b10bf6a
>>> 2018-04-17 04:26:18,057 WARN  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Upload directory /tmp/flink-web-1fd19ea7-ae19-4880-8787-92c5ec6ee509/flink-web-upload does not exist, or has been deleted externally. Previously uploaded files are no longer available.
>>> 2018-04-17 04:26:18,059 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Created directory /tmp/flink-web-1fd19ea7-ae19-4880-8787-92c5ec6ee509/flink-web-upload for file uploads.
>>> 2018-04-17 04:26:18,063 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Starting rest endpoint.
>>> 2018-04-17 04:26:18,386 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component log file: /var/log/hadoop-yarn/containers/application_1523726493647_0503/container_1523726493647_0503_01_000001/jobmanager.log
>>> 2018-04-17 04:26:18,386 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component stdout file: /var/log/hadoop-yarn/containers/application_1523726493647_0503/container_1523726493647_0503_01_000001/jobmanager.out
>>> 2018-04-17 04:26:18,454 INFO  org.apache.flink.runtime.rest.RestClient                      - Rest client endpoint started.
>>> 2018-04-17 04:26:18,538 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Rest endpoint listening at ip-10-0-22-186.ap-southeast-1.compute.internal:44431
>>> 2018-04-17 04:26:18,542 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - http://ip-10-0-22-186.ap-southeast-1.compute.internal:44431 was granted leadership with leaderSessionID=00000000-0000-0000-0000-000000000000
>>> 2018-04-17 04:26:18,558 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.yarn.YarnResourceManager at akka://flink/user/resourcemanager .
>>> 2018-04-17 04:26:18,713 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at akka://flink/user/dispatcher .
>>> 2018-04-17 04:26:18,739 INFO  org.apache.flink.yarn.YarnResourceManager                     - ResourceManager akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669/user/resourcemanager was granted leadership with fencing token 00000000000000000000000000000000
>>> 2018-04-17 04:26:18,740 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager  - Starting the SlotManager.
>>> 2018-04-17 04:26:18,769 INFO  org.apache.hadoop.yarn.client.RMProxy                         - Connecting to ResourceManager at ip-10-0-22-169.ap-southeast-1.compute.internal/10.0.22.169:8030
>>> 2018-04-17 04:26:19,083 INFO  org.apache.flink.yarn.YarnResourceManager                     - Recovered 0 containers from previous attempts ([]).
>>> 2018-04-17 04:26:19,086 INFO  org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  - yarn.client.max-cached-nodemanagers-proxies : 0
>>> 2018-04-17 04:26:19,101 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Dispatcher akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669/user/dispatcher was granted leadership with fencing token 00000000000000000000000000000000
>>> 2018-04-17 04:26:19,101 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Recovering all persisted jobs.
>>>
>>>
>>>
>>>
>>> [2] JM's logs
>>>
>>> 2018-04-16 12:53:15,825 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
>>> 2018-04-16 12:53:15,827 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Starting YarnSessionClusterEntrypoint (Version: 1.5.0, Rev:2af481a, Date:02.04.2018 @ 15:17:35 UTC)
>>> 2018-04-16 12:53:15,827 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  OS current user: yarn
>>> 2018-04-16 12:53:16,798 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Current Hadoop/Kerberos user: hadoop
>>> 2018-04-16 12:53:16,799 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.161-b14
>>> 2018-04-16 12:53:16,799 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Maximum heap size: 406 MiBytes
>>> 2018-04-16 12:53:16,799 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JAVA_HOME: /usr/lib/jvm/java-openjdk
>>> 2018-04-16 12:53:16,800 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Hadoop version: 2.8.3
>>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM Options:
>>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Xmx424m
>>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog.file=/var/log/hadoop-yarn/containers/application_1523726493647_0302/container_1523726493647_0302_01_000001/jobmanager.log
>>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlogback.configurationFile=file:logback.xml
>>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog4j.configuration=file:log4j.properties
>>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Program Arguments: (none)
>>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Classpath: lib/flink-dist_2.11-1.5.0.jar:lib/flink-python_2.11-1.5.0.jar:lib/flink-shaded-hadoop2-uber-1.5.0.jar:lib/flink-shaded-include-yarn-0.9.1.jar:lib/flink-yarn_2.11-1.5-SNAPSHOT.jar:lib/guava-18.0.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.7.jar:log4j.properties:logback.xml:flink.jar:flink-conf.yaml::/etc/hadoop/conf:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-streaming.jar:/usr/lib/hadoop/hadoop-distcp.jar:/usr/lib/hadoop/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archives.jar:/usr/lib/hadoop/hadoop-ant.jar:/usr/lib/hadoop/hadoop-aws.jar:/usr/lib/hadoop/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-rumen.jar:/usr/lib/hadoop/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack.jar:/usr/lib/hadoop/hadoop-azure.jar:/usr/lib/hadoop/hadoop-extras.jar:/usr/lib/hadoop/hadoop-azure-datalake.jar:/usr/lib/hadoop/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix.jar:/usr/lib/hadoop/hadoop-nfs.jar:/usr/lib/hadoop/hadoop-datajoin.jar:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop/hadoop-sls.jar:/usr/lib/hadoop/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/lib/curator-framework-2.7.1.jar:/usr/lib/hadoop/lib/json-smart-1.1.1.jar:/usr/lib/hadoop/lib/curator-recipes-2.7.1.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/lib/jcip-annotations-1.0.jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/lib/avro-1.7.4.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop/lib/gson-2.2.4.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop/lib/jsch-0.1.54.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar:/usr/lib/hadoop/lib/jersey-server-1.9.jar:/usr/lib/hadoop/lib/httpcore-4.4.4.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/curator-client-2.7.1.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop/lib/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/jsr305-3.0.0.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/httpclient-4.5.3.jar:/usr/lib/hadoop/lib/commons-lang-2.6.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/slf4j-api-1.7.10.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/xercesImpl-2.9.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-hdfs/lib/netty-all-4.0.23.Final.jar:/usr/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-hdfs/lib/okio-1.4.0.jar:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/xml-apis-1.3.04.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/okhttp-2.4.0.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/ion-java-1.0.2.jar:/usr/lib/hadoop-mapreduce/curator-framework-2.7.1.jar:/usr/lib/hadoop-mapreduce/json-smart-1.1.1.jar:/usr/lib/hadoop-mapreduce/curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-core-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/zookeeper-3.4.10.jar:/usr/lib/hadoop-mapreduce/jmespath-java-1.11.267.jar:/usr/lib/hadoop-mapreduce/jettison-1.1.jar:/usr/lib/hadoop-mapreduce/java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/jcip-annotations-1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs.jar:/usr/lib/hadoop-mapreduce/hadoop-auth.jar:/usr/lib/hadoop-mapreduce/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-2.6.7.jar:/usr/lib/hadoop-mapreduce/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-mapreduce/gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/azure-data-lake-store-sdk-2.2.3.jar:/usr/lib/hadoop-mapreduce/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/jsch-0.1.54.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/commons-lang3-3.3.2.jar:/usr/lib/hadoop-mapreduce/xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/commons-collections-3.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/hadoop-archives.jar:/usr/lib/hadoop-mapreduce/commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/hadoop-ant.jar:/usr/lib/hadoop-mapreduce/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/jackson-databind-2.6.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-aws.jar:/usr/lib/hadoop-mapreduce/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/jackson-annotations-2.6.7.jar:/usr/lib/hadoop-mapreduce/httpcore-4.4.4.jar:/usr/lib/hadoop-mapreduce/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jetty-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop-mapreduce/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/xz-1.0.jar:/usr/lib/hadoop-mapreduce/joda-time-2.9.4.jar:/usr/lib/hadoop-mapreduce/commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-mapreduce/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/activation-1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/hadoop-azure.jar:/usr/lib/hadoop-mapreduce/hadoop-extras.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/okio-1.4.0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/azure-storage-2.2.0.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-s3-1.11.267.jar:/usr/lib/hadoop-mapreduce/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/jackson-dataformat-cbor-2.6.7.jar:/usr/lib/hadoop-mapreduce/jackson-xc-1.9.13.jar:/usr/lib/hadoop-mapreduce/okhttp-2.4.0.jar:/usr/lib/hadoop-mapreduce/asm-3.2.jar:/usr/lib/hadoop-mapreduce/hadoop-sls.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/httpclient-4.5.3.jar:/usr/lib/hadoop-mapreduce/commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-kms-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/lib/curator-test-2.7.1.jar:/usr/lib/hadoop-yarn/lib/java-util-1.9.0.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10-tests.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop-yarn/lib/jettison-1.1.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/lib/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/lib/hadoop-yarn/lib/commons-math-2.2.jar:/usr/lib/hadoop-yarn/lib/curator-client-2.7.1.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-yarn/lib/activation-1.1.jar:/usr/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-yarn/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-yarn/lib/guava-11.0.2.jar:/usr/lib/hadoop-yarn/lib/json-io-2.5.1.jar:/usr/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/lib/hadoop-yarn/lib/fst-2.50.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/javassist-3.18.1-GA.jar:/usr/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo-0.4.19.jar:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/jcl-over-slf4j-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/ion-java-1.0.2.jar:/usr/share/aws/emr/emrfs/lib/emrfs-hadoop-assembly-2.21.0.jar:/usr/share/aws/emr/emrfs/lib/jmespath-java-1.11.267.jar:/usr/share/aws/emr/emrfs/lib/javax.inject-1.jar:/usr/share/aws/emr/emrfs/lib/bcpkix-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/lib/slf4j-api-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/aopalliance-1.0.jar:/usr/share/aws/emr/emrfs/lib/bcprov-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/auxlib/*:/usr/share/aws/emr/lib/*:/usr/share/aws/emr/ddb/lib/emr-ddb-hadoop.jar:/usr/share/aws/emr/goodies/lib/emr-hadoop-goodies.jar:/usr/share/aws/emr/kinesis/lib/emr-kinesis-hadoop.jar:/usr/lib/spark/yarn/lib/datanucleus-api-jdo.jar:/usr/lib/spark/yarn/lib/datanucleus-core.jar:/usr/lib/spark/yarn/lib/datanucleus-rdbms.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink-1.1.0.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacecommerceanalytics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rds-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-core-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-clouddirectory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsmv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-emr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediaconvert-1.11.267.jar:/usr/share/aws/aws-java-sdk/jmespath-java-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appsync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastore-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iot-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costandusagereport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpleworkflow-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-comprehend-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidentity-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ec2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudtrail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servermigration-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesis-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workmail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-polly-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpledb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscalingplans-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elastictranscoder-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pinpoint-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lexmodelbuilding-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacemeteringservice-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dynamodb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-storagegateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codegen-maven-plugin-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediapackage-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroupstaggingapi-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-acm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-machinelearning-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mobile-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dax-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lightsail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sns-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codedeploy-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-serverlessapplicationrepository-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-shield-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-applicationautoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastoredata-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloud9-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-budgets-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iotjobsdataplane-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-medialive-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-inspector-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-importexport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-test-utils-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitosync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-api-gateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codebuild-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workspaces-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opensdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-snowball-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pricing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-code-generator-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glacier-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-guardduty-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicediscovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codestar-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appstream-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancingv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-greengrass-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lex-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroups-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-support-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticbeanstalk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-route53-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-datapipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rekognition-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-organizations-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iam-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codepipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-logs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworkscm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-xray-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplaceentitlement-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-translate-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-events-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-alexaforbusiness-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemaker-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-s3-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costexplorer-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directconnect-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticache-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-gamelift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sqs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemakerruntime-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glue-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-waf-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mechanicalturkrequester-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatchmetrics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-migrationhub-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lambda-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-discovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidp-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-athena-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicecatalog-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mq-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-batch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-devicefarm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sts-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-stepfunctions-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesisvideo-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworks-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workdocs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-redshift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudfront-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-config-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-efs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-models-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ssm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codecommit-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-health-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudformation-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ses-1.11.267.jar
>>> 2018-04-16 12:53:16,803 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
>>> 2018-04-16 12:53:16,805 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Registered UNIX signal handlers for [TERM, HUP, INT]
>>> 2018-04-16 12:53:16,809 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - YARN daemon is running as: hadoop Yarn client user obtainer: hadoop
>>> 2018-04-16 12:53:16,813 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.archive.fs.dir, hdfs:///completed-jobs/
>>> 2018-04-16 12:53:16,814 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.default-scheme, s3a
>>> 2018-04-16 12:53:16,814 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: classloader.resolve-order, parent-first
>>> 2018-04-16 12:53:16,814 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: high-availability.cluster-id, application_1523726493647_0302
>>> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.heap.mb, 1024
>>> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: yarn.heap-cutoff-ratio, 0.5
>>> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.preallocate, true
>>> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: yarn.maximum-failed-containers, -1
>>> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: akka.watch.heartbeat.pause, 600 s
>>> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: internal.cluster.execution-mode, NORMAL
>>> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.off-heap, on
>>> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.fraction, 0.1
>>> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.numberOfTaskSlots, 1
>>> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf
>>> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.heap.mb, 9216
>>> 2018-04-16 12:53:16,817 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: historyserver.archive.fs.dir, hdfs:///completed-jobs/
>>> 2018-04-16 12:53:16,833 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Setting directories for temporary files to: /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302
>>> 2018-04-16 12:53:16,859 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Starting YarnSessionClusterEntrypoint.
>>> 2018-04-16 12:53:16,859 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Install default filesystem.
>>> 2018-04-16 12:53:17,036 INFO  org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop user set to hadoop (auth:SIMPLE)
>>> 2018-04-16 12:53:17,071 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Initializing cluster services.
>>> 2018-04-16 12:53:17,081 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Trying to start actor system at ip-10-0-22-186.ap-southeast-1.compute.internal:36411
>>> 2018-04-16 12:53:18,022 INFO  akka.event.slf4j.Slf4jLogger                                  - Slf4jLogger started
>>> 2018-04-16 12:53:18,254 INFO  akka.remote.Remoting                                          - Starting remoting
>>> 2018-04-16 12:53:18,461 INFO  akka.remote.Remoting                                          - Remoting started; listening on addresses :[akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411]
>>> 2018-04-16 12:53:18,488 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Actor system started at akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411
>>> 2018-04-16 12:53:18,559 INFO  org.apache.flink.runtime.blob.BlobServer                      - Created BLOB server storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302/blobStore-9ec0a59f-c03a-446f-a841-60a65892b383
>>> 2018-04-16 12:53:18,566 INFO  org.apache.flink.runtime.blob.BlobServer                      - Started BLOB server at 0.0.0.0:46869 - max concurrent requests: 50 - max backlog: 1000
>>> 2018-04-16 12:53:18,632 INFO  org.apache.flink.runtime.metrics.MetricRegistryImpl           - No metrics reporter configured, no metrics will be exposed/reported.
>>> 2018-04-16 12:53:18,638 INFO  org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore  - Initializing FileArchivedExecutionGraphStore: Storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302/executionGraphStore-ee071b9a-ad86-4b36-860c-6501f509ecf9, expiration time 3600000, maximum cache size 52428800 bytes.
>>> 2018-04-16 12:53:18,726 INFO  org.apache.flink.runtime.blob.TransientBlobCache              - Created BLOB cache storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302/blobStore-c15647ee-31c1-451d-84d2-729a67e065d8
>>> 2018-04-16 12:53:18,750 WARN  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Upload directory /tmp/flink-web-6d14c830-9074-4a2f-8e1e-7344545e34e5/flink-web-upload does not exist, or has been deleted externally. Previously uploaded files are no longer available.
>>> 2018-04-16 12:53:18,757 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Created directory /tmp/flink-web-6d14c830-9074-4a2f-8e1e-7344545e34e5/flink-web-upload for file uploads.
>>> 2018-04-16 12:53:18,765 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Starting rest endpoint.
>>> 2018-04-16 12:53:19,361 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component log file: /var/log/hadoop-yarn/containers/application_1523726493647_0302/container_1523726493647_0302_01_000001/jobmanager.log
>>> 2018-04-16 12:53:19,362 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component stdout file: /var/log/hadoop-yarn/containers/application_1523726493647_0302/container_1523726493647_0302_01_000001/jobmanager.out
>>> 2018-04-16 12:53:19,442 INFO  org.apache.flink.runtime.rest.RestClient                      - Rest client endpoint started.
>>> 2018-04-16 12:53:19,589 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Rest endpoint listening at ip-10-0-22-186.ap-southeast-1.compute.internal:45697
>>> 2018-04-16 12:53:19,589 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - http://ip-10-0-22-186.ap-southeast-1.compute.internal:45697 was granted leadership with leaderSessionID=00000000-0000-0000-0000-000000000000
>>> 2018-04-16 12:53:19,614 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.yarn.YarnResourceManager at akka://flink/user/resourcemanager .
>>> 2018-04-16 12:53:19,979 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at akka://flink/user/dispatcher .
>>> 2018-04-16 12:53:20,049 INFO  org.apache.flink.yarn.YarnResourceManager                     - ResourceManager akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411/user/resourcemanager was granted leadership with fencing token 00000000000000000000000000000000
>>> 2018-04-16 12:53:20,050 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager  - Starting the SlotManager.
>>> 2018-04-16 12:53:20,101 INFO  org.apache.hadoop.yarn.client.RMProxy                         - Connecting to ResourceManager at ip-10-0-22-169.ap-southeast-1.compute.internal/10.0.22.169:8030
>>> 2018-04-16 12:53:20,649 INFO  org.apache.flink.yarn.YarnResourceManager                     - Recovered 0 containers from previous attempts ([]).
>>> 2018-04-16 12:53:20,655 INFO  org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  - yarn.client.max-cached-nodemanagers-proxies : 0
>>> 2018-04-16 12:53:20,706 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Dispatcher akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411/user/dispatcher was granted leadership with fencing token 00000000000000000000000000000000
>>> 2018-04-16 12:53:20,706 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Recovering all persisted jobs.
>>> 2018-04-16 12:53:26,207 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Submitting job 58a1b149b5b69de66bb931836c459d2b (mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job).
>>> 2018-04-16 12:53:26,223 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster at akka://flink/user/jobmanager_0 .
>>> 2018-04-16 12:53:26,237 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Initializing job mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job (58a1b149b5b69de66bb931836c459d2b).
>>> 2018-04-16 12:53:26,279 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Using restart strategy NoRestartStrategy for mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job (58a1b149b5b69de66bb931836c459d2b).
>>> 2018-04-16 12:53:26,285 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.slotpool.SlotPool at akka://flink/user/820d207e-fddd-4e24-b327-1f3c1cb25024 .
>>> 2018-04-16 12:53:26,320 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Job recovers via failover strategy: full graph restart
>>> 2018-04-16 12:53:26,386 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Running initialization on master for job mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job (58a1b149b5b69de66bb931836c459d2b).
>>> 2018-04-16 12:53:26,671 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Successfully ran initialization on master in 284 ms.
>>> 2018-04-16 12:53:36,204 ERROR org.apache.flink.runtime.rest.handler.job.JobSubmitHandler    - Implementation error: Unhandled exception.
>>> akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://flink/user/dispatcher#1710592138]] after [10000 ms]. Sender[null] sent message of type "org.apache.flink.runtime.rpc.messages.LocalFencedMessage".
>>> 	at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:604)
>>> 	at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:126)
>>> 	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
>>> 	at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
>>> 	at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
>>> 	at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:329)
>>> 	at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:280)
>>> 	at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:284)
>>> 	at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:236)
>>> 	at java.lang.Thread.run(Thread.java:748)
>>> 2018-04-16 12:53:36,300 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - RECEIVED SIGNAL 15: SIGTERM. Shutting down as requested.
>>> 2018-04-16 12:53:36,311 INFO  org.apache.flink.runtime.blob.TransientBlobCache              - Shutting down BLOB cache
>>> 2018-04-16 12:53:36,314 INFO  org.apache.flink.runtime.blob.BlobServer                      - Stopped BLOB server at 0.0.0.0:46869
>>>
>>>
>>>
>>>
>>
>

Re: Testing on Flink 1.5

Posted by Gary Yao <ga...@data-artisans.com>.
Hi Amit,

Thank you for the follow up. What you describe sounds like a bug but I am
not
able to reproduce it. Can you open an issue in Jira with an outline of your
code
and how you submit the job?

> Could you also recommend us the best practice in FLIP6, should we use
YARN session or submit jobs in non-detached mode?

This should be independent of FLIP-6. We are aiming at backwards
compatibility
but apparently there are still things that require polishing.

Best,
Gary

On Thu, Apr 19, 2018 at 12:49 PM, Amit Jain <aj...@gmail.com> wrote:
>
> Hi Gary,
>
> We found the underlying issue with the following problem.
> Few of our jobs are stuck with logs [1], these jobs are only able to
allocate JM and couldn't get any TM, however, there are ample resource on
our cluster.
>
> We are running ETL merge job here. In this job, we first find new deltas
and if there is no delta detected then we make exit without actually
executing the job. I think this is the reason we see no TM allocation is
happening.
>
> I believe in above case (non-detached mode) we should mark the submitted
application as complete compare to running. Please share your thoughts on
this.
> Should I log this improvement in JIRA?
>
> Could you also recommend us the best practice in FLIP6, should we use
YARN session or submit jobs in non-detached mode?
>
> --
> Thanks,
> Amit
>

Re: Testing on Flink 1.5

Posted by Amit Jain <aj...@gmail.com>.
Hi Gary,

We found the underlying issue with the following problem.
Few of our jobs are stuck with logs [1], these jobs are only able to
allocate JM and couldn't get any TM, however, there are ample resource on
our cluster.

We are running ETL merge job here. In this job, we first find new deltas
and if there is no delta detected then we make exit without actually
executing the job. I think this is the reason we see no TM allocation is
happening.

I believe in above case (non-detached mode) we should mark the submitted
application as complete compare to running. Please share your thoughts on
this.
Should I log this improvement in JIRA?

Could you also recommend us the best practice in FLIP6, should we use YARN
session or submit jobs in non-detached mode?

--
Thanks,
Amit

Re: Testing on Flink 1.5

Posted by Amit Jain <aj...@gmail.com>.
Hi Gary,

Please find my reply in place. We have also logged
https://issues.apache.org/jira/browse/FLINK-9207, please have a look on
this.


--
Thanks,
Amit


On Tue, Apr 17, 2018 at 9:10 PM, Gary Yao <ga...@data-artisans.com> wrote:

> Hi Amit,
>
> Thank you for testing the 1.5 Release Candidate. Find my answers below.
>
> > Few of our jobs are stuck with logs [1], these jobs are only able to
> allocate JM and couldn't get any TM, however, there are ample resource on
> our cluster.
>
> In the logs you attached I cannot see that a job was submitted. If a job is
> submitted, the dispatcher should log: "Submitting job <job_id>
> (<job_name>)"
> [1]. Could it be that the attached logs are incomplete, or the client
> already
> fails to submit the job? Can you also attach the client logs?
>
> I agree with you, we should have these logs however it is not happening.
I'll work on it further and provide more info, right now, I can provide
logs which got generated later.

2018-04-16 04:36:43,747 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher      -
Dispatcher akka.tcp://flink@ip-10-0-22-43.ap-southeast-1.compute.internal:39329/user/dispatcher
was granted leadership with fencing token
00000000000000000000000000000000
2018-04-16 04:36:43,747 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher      -
Recovering all persisted jobs.
2018-04-16 17:21:37,800 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:21:42,803 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:21:47,805 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:21:52,806 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:21:57,807 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:22:02,808 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:22:07,810 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:22:12,812 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:22:17,813 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:22:22,814 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:22:27,816 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:22:32,817 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:22:37,818 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:22:42,821 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:22:47,822 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:22:52,825 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:22:57,826 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:23:02,828 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:23:07,829 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken
2018-04-16 17:23:12,832 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         -
Updating with new AMRMToken



> > All completed application on YARN are marked as KILLED, instead of
> FINISHED. It creates problem on tracking jobs as we require to navigate on
> each application logs and also history server is not logging any completed
> application.
>
> I was able to reproduce this: In non-detached mode, the State and
> FinalStatus
> are set to KILLED. In detached mode, the Status and FinalStatus are set
> correctly. I have re-opened this ticket:
>   https://issues.apache.org/jira/browse/FLINK-8900
>
> You are also right about the History Server, which is currently not
> implemented
> for FLIP-6. Here is the ticket for reference:
>   https://issues.apache.org/jira/browse/FLINK-9194
>
>
> > We are running our system on AWS EMR and using CORE nodes at its bare
> minimum, we see with new paradigm of application submission we are hitting
> Out of space error in our HDFS Data Node. Do you think, we should have
> configuration to automatically evict these .flink/ files?
>
> This seems to be a regression. In Flink 1.4., the files are cleaned up
> automatically. I opened this ticket:
> https://issues.apache.org/jira/browse/FLINK-9196
>
>
> > Lastly, we are also observing Akka.ask.timeout error for few of our jobs
> (JM's logs[2]), we tried to increase this parameter to 2 min from default
> 10 s and still observe the same.
>
> Does this always happen for the same jobs? I cannot reproduce this.
>
> Yes, it happening for same set of jobs. Specifically, these jobs are
having multiple S3 objects to work with. I tried increasing
akka.ask.timeout which is not getting reflected on JM's log also and same
has been experienced while running the jobs.
I also tried one of these jobs on empty cluster(EMR TASK 15 node, r3.xlarge
type) and still face same.

Best,
> Gary
>
> [1] https://github.com/apache/flink/blob/master/flink-
> runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java#
> L235
>
>
> On Tue, Apr 17, 2018 at 9:35 AM, Amit Jain <aj...@gmail.com> wrote:
>
>> Hi,
>>
>>
>> We have started testing our workload on Flink 1.5.0 (Commit: 2af481a) and faced few issues. Here, we are using YARN as cluster manager.
>>
>>
>>    - Few of our jobs are stuck with logs [1], these jobs are only able to allocate JM and couldn't get any TM, however, there are ample resource on our cluster.
>>    - All completed application on YARN are marked as KILLED, instead of FINISHED. It creates problem on tracking jobs as we require to navigate on each application logs and also history server is not logging any completed application.
>>    - We are running our system on AWS EMR and using CORE nodes at its bare minimum, we see with new paradigm of application submission we are hitting Out of space error in our HDFS Data Node. Do you think, we should have configuration to automatically evict these .flink/ files?
>>    - Lastly, we are also observing Akka.ask.timeout error for few of our jobs (JM's logs[2]), we tried to increase this parameter to 2 min from default 10 s and still observe the same.
>>
>> Please help us guide through these issues.
>>
>> --
>> Thanks,
>> Amit
>>
>> [1] JM logs where jobs are not even started
>>
>> 2018-04-17 04:26:16,165 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
>> 2018-04-17 04:26:16,166 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Starting YarnSessionClusterEntrypoint (Version: 1.5.0, Rev:2af481a, Date:02.04.2018 @ 15:17:35 UTC)
>> 2018-04-17 04:26:16,166 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  OS current user: yarn
>> 2018-04-17 04:26:16,843 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Current Hadoop/Kerberos user: hadoop
>> 2018-04-17 04:26:16,843 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.161-b14
>> 2018-04-17 04:26:16,843 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Maximum heap size: 406 MiBytes
>> 2018-04-17 04:26:16,843 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JAVA_HOME: /usr/lib/jvm/java-openjdk
>> 2018-04-17 04:26:16,844 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Hadoop version: 2.8.3
>> 2018-04-17 04:26:16,844 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM Options:
>> 2018-04-17 04:26:16,844 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Xmx424m
>> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog.file=/var/log/hadoop-yarn/containers/application_1523726493647_0503/container_1523726493647_0503_01_000001/jobmanager.log
>> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlogback.configurationFile=file:logback.xml
>> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog4j.configuration=file:log4j.properties
>> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Program Arguments: (none)
>> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Classpath: lib/flink-dist_2.11-1.5.0.jar:lib/flink-python_2.11-1.5.0.jar:lib/flink-shaded-hadoop2-uber-1.5.0.jar:lib/flink-shaded-include-yarn-0.9.1.jar:lib/flink-yarn_2.11-1.5-SNAPSHOT.jar:lib/guava-18.0.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.7.jar:log4j.properties:logback.xml:flink.jar:flink-conf.yaml::/etc/hadoop/conf:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-streaming.jar:/usr/lib/hadoop/hadoop-distcp.jar:/usr/lib/hadoop/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archives.jar:/usr/lib/hadoop/hadoop-ant.jar:/usr/lib/hadoop/hadoop-aws.jar:/usr/lib/hadoop/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-rumen.jar:/usr/lib/hadoop/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack.jar:/usr/lib/hadoop/hadoop-azure.jar:/usr/lib/hadoop/hadoop-extras.jar:/usr/lib/hadoop/hadoop-azure-datalake.jar:/usr/lib/hadoop/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix.jar:/usr/lib/hadoop/hadoop-nfs.jar:/usr/lib/hadoop/hadoop-datajoin.jar:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop/hadoop-sls.jar:/usr/lib/hadoop/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/lib/curator-framework-2.7.1.jar:/usr/lib/hadoop/lib/json-smart-1.1.1.jar:/usr/lib/hadoop/lib/curator-recipes-2.7.1.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/lib/jcip-annotations-1.0.jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/lib/avro-1.7.4.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop/lib/gson-2.2.4.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop/lib/jsch-0.1.54.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar:/usr/lib/hadoop/lib/jersey-server-1.9.jar:/usr/lib/hadoop/lib/httpcore-4.4.4.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/curator-client-2.7.1.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop/lib/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/jsr305-3.0.0.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/httpclient-4.5.3.jar:/usr/lib/hadoop/lib/commons-lang-2.6.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/slf4j-api-1.7.10.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/xercesImpl-2.9.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-hdfs/lib/netty-all-4.0.23.Final.jar:/usr/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-hdfs/lib/okio-1.4.0.jar:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/xml-apis-1.3.04.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/okhttp-2.4.0.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/ion-java-1.0.2.jar:/usr/lib/hadoop-mapreduce/curator-framework-2.7.1.jar:/usr/lib/hadoop-mapreduce/json-smart-1.1.1.jar:/usr/lib/hadoop-mapreduce/curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-core-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/zookeeper-3.4.10.jar:/usr/lib/hadoop-mapreduce/jmespath-java-1.11.267.jar:/usr/lib/hadoop-mapreduce/jettison-1.1.jar:/usr/lib/hadoop-mapreduce/java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/jcip-annotations-1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs.jar:/usr/lib/hadoop-mapreduce/hadoop-auth.jar:/usr/lib/hadoop-mapreduce/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-2.6.7.jar:/usr/lib/hadoop-mapreduce/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-mapreduce/gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/azure-data-lake-store-sdk-2.2.3.jar:/usr/lib/hadoop-mapreduce/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/jsch-0.1.54.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/commons-lang3-3.3.2.jar:/usr/lib/hadoop-mapreduce/xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/commons-collections-3.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/hadoop-archives.jar:/usr/lib/hadoop-mapreduce/commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/hadoop-ant.jar:/usr/lib/hadoop-mapreduce/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/jackson-databind-2.6.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-aws.jar:/usr/lib/hadoop-mapreduce/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/jackson-annotations-2.6.7.jar:/usr/lib/hadoop-mapreduce/httpcore-4.4.4.jar:/usr/lib/hadoop-mapreduce/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jetty-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop-mapreduce/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/xz-1.0.jar:/usr/lib/hadoop-mapreduce/joda-time-2.9.4.jar:/usr/lib/hadoop-mapreduce/commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-mapreduce/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/activation-1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/hadoop-azure.jar:/usr/lib/hadoop-mapreduce/hadoop-extras.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/okio-1.4.0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/azure-storage-2.2.0.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-s3-1.11.267.jar:/usr/lib/hadoop-mapreduce/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/jackson-dataformat-cbor-2.6.7.jar:/usr/lib/hadoop-mapreduce/jackson-xc-1.9.13.jar:/usr/lib/hadoop-mapreduce/okhttp-2.4.0.jar:/usr/lib/hadoop-mapreduce/asm-3.2.jar:/usr/lib/hadoop-mapreduce/hadoop-sls.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/httpclient-4.5.3.jar:/usr/lib/hadoop-mapreduce/commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-kms-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/lib/curator-test-2.7.1.jar:/usr/lib/hadoop-yarn/lib/java-util-1.9.0.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10-tests.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop-yarn/lib/jettison-1.1.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/lib/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/lib/hadoop-yarn/lib/commons-math-2.2.jar:/usr/lib/hadoop-yarn/lib/curator-client-2.7.1.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-yarn/lib/activation-1.1.jar:/usr/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-yarn/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-yarn/lib/guava-11.0.2.jar:/usr/lib/hadoop-yarn/lib/json-io-2.5.1.jar:/usr/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/lib/hadoop-yarn/lib/fst-2.50.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/javassist-3.18.1-GA.jar:/usr/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo-0.4.19.jar:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/jcl-over-slf4j-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/ion-java-1.0.2.jar:/usr/share/aws/emr/emrfs/lib/emrfs-hadoop-assembly-2.21.0.jar:/usr/share/aws/emr/emrfs/lib/jmespath-java-1.11.267.jar:/usr/share/aws/emr/emrfs/lib/javax.inject-1.jar:/usr/share/aws/emr/emrfs/lib/bcpkix-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/lib/slf4j-api-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/aopalliance-1.0.jar:/usr/share/aws/emr/emrfs/lib/bcprov-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/auxlib/*:/usr/share/aws/emr/lib/*:/usr/share/aws/emr/ddb/lib/emr-ddb-hadoop.jar:/usr/share/aws/emr/goodies/lib/emr-hadoop-goodies.jar:/usr/share/aws/emr/kinesis/lib/emr-kinesis-hadoop.jar:/usr/lib/spark/yarn/lib/datanucleus-api-jdo.jar:/usr/lib/spark/yarn/lib/datanucleus-core.jar:/usr/lib/spark/yarn/lib/datanucleus-rdbms.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink-1.1.0.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacecommerceanalytics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rds-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-core-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-clouddirectory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsmv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-emr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediaconvert-1.11.267.jar:/usr/share/aws/aws-java-sdk/jmespath-java-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appsync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastore-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iot-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costandusagereport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpleworkflow-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-comprehend-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidentity-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ec2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudtrail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servermigration-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesis-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workmail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-polly-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpledb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscalingplans-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elastictranscoder-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pinpoint-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lexmodelbuilding-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacemeteringservice-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dynamodb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-storagegateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codegen-maven-plugin-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediapackage-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroupstaggingapi-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-acm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-machinelearning-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mobile-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dax-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lightsail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sns-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codedeploy-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-serverlessapplicationrepository-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-shield-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-applicationautoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastoredata-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloud9-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-budgets-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iotjobsdataplane-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-medialive-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-inspector-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-importexport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-test-utils-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitosync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-api-gateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codebuild-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workspaces-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opensdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-snowball-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pricing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-code-generator-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glacier-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-guardduty-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicediscovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codestar-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appstream-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancingv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-greengrass-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lex-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroups-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-support-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticbeanstalk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-route53-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-datapipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rekognition-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-organizations-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iam-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codepipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-logs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworkscm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-xray-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplaceentitlement-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-translate-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-events-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-alexaforbusiness-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemaker-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-s3-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costexplorer-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directconnect-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticache-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-gamelift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sqs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemakerruntime-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glue-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-waf-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mechanicalturkrequester-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatchmetrics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-migrationhub-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lambda-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-discovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidp-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-athena-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicecatalog-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mq-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-batch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-devicefarm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sts-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-stepfunctions-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesisvideo-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworks-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workdocs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-redshift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudfront-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-config-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-efs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-models-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ssm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codecommit-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-health-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudformation-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ses-1.11.267.jar
>> 2018-04-17 04:26:16,846 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
>> 2018-04-17 04:26:16,848 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Registered UNIX signal handlers for [TERM, HUP, INT]
>> 2018-04-17 04:26:16,851 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - YARN daemon is running as: hadoop Yarn client user obtainer: hadoop
>> 2018-04-17 04:26:16,854 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.archive.fs.dir, hdfs:///completed-jobs/
>> 2018-04-17 04:26:16,854 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.default-scheme, s3a
>> 2018-04-17 04:26:16,854 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: classloader.resolve-order, parent-first
>> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: high-availability.cluster-id, application_1523726493647_0503
>> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.heap.mb, 1024
>> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: yarn.heap-cutoff-ratio, 0.5
>> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.preallocate, true
>> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: yarn.maximum-failed-containers, -1
>> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: akka.watch.heartbeat.pause, 600 s
>> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: internal.cluster.execution-mode, NORMAL
>> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.off-heap, on
>> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.fraction, 0.1
>> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.numberOfTaskSlots, 1
>> 2018-04-17 04:26:16,857 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf
>> 2018-04-17 04:26:16,857 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: akka.ask.timeout, 2 min
>> 2018-04-17 04:26:16,857 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.heap.mb, 10772
>> 2018-04-17 04:26:16,857 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: historyserver.archive.fs.dir, hdfs:///completed-jobs/
>> 2018-04-17 04:26:16,874 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Setting directories for temporary files to: /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503
>> 2018-04-17 04:26:16,896 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Starting YarnSessionClusterEntrypoint.
>> 2018-04-17 04:26:16,897 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Install default filesystem.
>> 2018-04-17 04:26:17,014 INFO  org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop user set to hadoop (auth:SIMPLE)
>> 2018-04-17 04:26:17,034 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Initializing cluster services.
>> 2018-04-17 04:26:17,041 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Trying to start actor system at ip-10-0-22-186.ap-southeast-1.compute.internal:38669
>> 2018-04-17 04:26:17,671 INFO  akka.event.slf4j.Slf4jLogger                                  - Slf4jLogger started
>> 2018-04-17 04:26:17,794 INFO  akka.remote.Remoting                                          - Starting remoting
>> 2018-04-17 04:26:17,937 INFO  akka.remote.Remoting                                          - Remoting started; listening on addresses :[akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669]
>> 2018-04-17 04:26:17,948 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Actor system started at akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669
>> 2018-04-17 04:26:17,980 INFO  org.apache.flink.runtime.blob.BlobServer                      - Created BLOB server storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503/blobStore-bca451c7-977d-425a-9e8f-4fe324fa8623
>> 2018-04-17 04:26:17,983 INFO  org.apache.flink.runtime.blob.BlobServer                      - Started BLOB server at 0.0.0.0:44217 - max concurrent requests: 50 - max backlog: 1000
>> 2018-04-17 04:26:18,009 INFO  org.apache.flink.runtime.metrics.MetricRegistryImpl           - No metrics reporter configured, no metrics will be exposed/reported.
>> 2018-04-17 04:26:18,015 INFO  org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore  - Initializing FileArchivedExecutionGraphStore: Storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503/executionGraphStore-28bf6537-0bbf-486c-a346-084bd7517d7d, expiration time 3600000, maximum cache size 52428800 bytes.
>> 2018-04-17 04:26:18,047 INFO  org.apache.flink.runtime.blob.TransientBlobCache              - Created BLOB cache storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503/blobStore-f1de0cad-77b0-4bbd-b218-63e32b10bf6a
>> 2018-04-17 04:26:18,057 WARN  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Upload directory /tmp/flink-web-1fd19ea7-ae19-4880-8787-92c5ec6ee509/flink-web-upload does not exist, or has been deleted externally. Previously uploaded files are no longer available.
>> 2018-04-17 04:26:18,059 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Created directory /tmp/flink-web-1fd19ea7-ae19-4880-8787-92c5ec6ee509/flink-web-upload for file uploads.
>> 2018-04-17 04:26:18,063 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Starting rest endpoint.
>> 2018-04-17 04:26:18,386 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component log file: /var/log/hadoop-yarn/containers/application_1523726493647_0503/container_1523726493647_0503_01_000001/jobmanager.log
>> 2018-04-17 04:26:18,386 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component stdout file: /var/log/hadoop-yarn/containers/application_1523726493647_0503/container_1523726493647_0503_01_000001/jobmanager.out
>> 2018-04-17 04:26:18,454 INFO  org.apache.flink.runtime.rest.RestClient                      - Rest client endpoint started.
>> 2018-04-17 04:26:18,538 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Rest endpoint listening at ip-10-0-22-186.ap-southeast-1.compute.internal:44431
>> 2018-04-17 04:26:18,542 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - http://ip-10-0-22-186.ap-southeast-1.compute.internal:44431 was granted leadership with leaderSessionID=00000000-0000-0000-0000-000000000000
>> 2018-04-17 04:26:18,558 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.yarn.YarnResourceManager at akka://flink/user/resourcemanager .
>> 2018-04-17 04:26:18,713 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at akka://flink/user/dispatcher .
>> 2018-04-17 04:26:18,739 INFO  org.apache.flink.yarn.YarnResourceManager                     - ResourceManager akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669/user/resourcemanager was granted leadership with fencing token 00000000000000000000000000000000
>> 2018-04-17 04:26:18,740 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager  - Starting the SlotManager.
>> 2018-04-17 04:26:18,769 INFO  org.apache.hadoop.yarn.client.RMProxy                         - Connecting to ResourceManager at ip-10-0-22-169.ap-southeast-1.compute.internal/10.0.22.169:8030
>> 2018-04-17 04:26:19,083 INFO  org.apache.flink.yarn.YarnResourceManager                     - Recovered 0 containers from previous attempts ([]).
>> 2018-04-17 04:26:19,086 INFO  org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  - yarn.client.max-cached-nodemanagers-proxies : 0
>> 2018-04-17 04:26:19,101 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Dispatcher akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669/user/dispatcher was granted leadership with fencing token 00000000000000000000000000000000
>> 2018-04-17 04:26:19,101 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Recovering all persisted jobs.
>>
>>
>>
>>
>> [2] JM's logs
>>
>> 2018-04-16 12:53:15,825 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
>> 2018-04-16 12:53:15,827 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Starting YarnSessionClusterEntrypoint (Version: 1.5.0, Rev:2af481a, Date:02.04.2018 @ 15:17:35 UTC)
>> 2018-04-16 12:53:15,827 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  OS current user: yarn
>> 2018-04-16 12:53:16,798 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Current Hadoop/Kerberos user: hadoop
>> 2018-04-16 12:53:16,799 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.161-b14
>> 2018-04-16 12:53:16,799 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Maximum heap size: 406 MiBytes
>> 2018-04-16 12:53:16,799 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JAVA_HOME: /usr/lib/jvm/java-openjdk
>> 2018-04-16 12:53:16,800 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Hadoop version: 2.8.3
>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM Options:
>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Xmx424m
>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog.file=/var/log/hadoop-yarn/containers/application_1523726493647_0302/container_1523726493647_0302_01_000001/jobmanager.log
>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlogback.configurationFile=file:logback.xml
>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog4j.configuration=file:log4j.properties
>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Program Arguments: (none)
>> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Classpath: lib/flink-dist_2.11-1.5.0.jar:lib/flink-python_2.11-1.5.0.jar:lib/flink-shaded-hadoop2-uber-1.5.0.jar:lib/flink-shaded-include-yarn-0.9.1.jar:lib/flink-yarn_2.11-1.5-SNAPSHOT.jar:lib/guava-18.0.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.7.jar:log4j.properties:logback.xml:flink.jar:flink-conf.yaml::/etc/hadoop/conf:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-streaming.jar:/usr/lib/hadoop/hadoop-distcp.jar:/usr/lib/hadoop/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archives.jar:/usr/lib/hadoop/hadoop-ant.jar:/usr/lib/hadoop/hadoop-aws.jar:/usr/lib/hadoop/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-rumen.jar:/usr/lib/hadoop/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack.jar:/usr/lib/hadoop/hadoop-azure.jar:/usr/lib/hadoop/hadoop-extras.jar:/usr/lib/hadoop/hadoop-azure-datalake.jar:/usr/lib/hadoop/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix.jar:/usr/lib/hadoop/hadoop-nfs.jar:/usr/lib/hadoop/hadoop-datajoin.jar:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop/hadoop-sls.jar:/usr/lib/hadoop/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/lib/curator-framework-2.7.1.jar:/usr/lib/hadoop/lib/json-smart-1.1.1.jar:/usr/lib/hadoop/lib/curator-recipes-2.7.1.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/lib/jcip-annotations-1.0.jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/lib/avro-1.7.4.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop/lib/gson-2.2.4.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop/lib/jsch-0.1.54.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar:/usr/lib/hadoop/lib/jersey-server-1.9.jar:/usr/lib/hadoop/lib/httpcore-4.4.4.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/curator-client-2.7.1.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop/lib/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/jsr305-3.0.0.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/httpclient-4.5.3.jar:/usr/lib/hadoop/lib/commons-lang-2.6.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/slf4j-api-1.7.10.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/xercesImpl-2.9.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-hdfs/lib/netty-all-4.0.23.Final.jar:/usr/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-hdfs/lib/okio-1.4.0.jar:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/xml-apis-1.3.04.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/okhttp-2.4.0.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/ion-java-1.0.2.jar:/usr/lib/hadoop-mapreduce/curator-framework-2.7.1.jar:/usr/lib/hadoop-mapreduce/json-smart-1.1.1.jar:/usr/lib/hadoop-mapreduce/curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-core-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/zookeeper-3.4.10.jar:/usr/lib/hadoop-mapreduce/jmespath-java-1.11.267.jar:/usr/lib/hadoop-mapreduce/jettison-1.1.jar:/usr/lib/hadoop-mapreduce/java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/jcip-annotations-1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs.jar:/usr/lib/hadoop-mapreduce/hadoop-auth.jar:/usr/lib/hadoop-mapreduce/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-2.6.7.jar:/usr/lib/hadoop-mapreduce/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-mapreduce/gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/azure-data-lake-store-sdk-2.2.3.jar:/usr/lib/hadoop-mapreduce/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/jsch-0.1.54.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/commons-lang3-3.3.2.jar:/usr/lib/hadoop-mapreduce/xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/commons-collections-3.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/hadoop-archives.jar:/usr/lib/hadoop-mapreduce/commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/hadoop-ant.jar:/usr/lib/hadoop-mapreduce/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/jackson-databind-2.6.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-aws.jar:/usr/lib/hadoop-mapreduce/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/jackson-annotations-2.6.7.jar:/usr/lib/hadoop-mapreduce/httpcore-4.4.4.jar:/usr/lib/hadoop-mapreduce/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jetty-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop-mapreduce/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/xz-1.0.jar:/usr/lib/hadoop-mapreduce/joda-time-2.9.4.jar:/usr/lib/hadoop-mapreduce/commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-mapreduce/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/activation-1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/hadoop-azure.jar:/usr/lib/hadoop-mapreduce/hadoop-extras.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/okio-1.4.0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/azure-storage-2.2.0.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-s3-1.11.267.jar:/usr/lib/hadoop-mapreduce/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/jackson-dataformat-cbor-2.6.7.jar:/usr/lib/hadoop-mapreduce/jackson-xc-1.9.13.jar:/usr/lib/hadoop-mapreduce/okhttp-2.4.0.jar:/usr/lib/hadoop-mapreduce/asm-3.2.jar:/usr/lib/hadoop-mapreduce/hadoop-sls.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/httpclient-4.5.3.jar:/usr/lib/hadoop-mapreduce/commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-kms-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/lib/curator-test-2.7.1.jar:/usr/lib/hadoop-yarn/lib/java-util-1.9.0.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10-tests.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop-yarn/lib/jettison-1.1.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/lib/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/lib/hadoop-yarn/lib/commons-math-2.2.jar:/usr/lib/hadoop-yarn/lib/curator-client-2.7.1.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-yarn/lib/activation-1.1.jar:/usr/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-yarn/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-yarn/lib/guava-11.0.2.jar:/usr/lib/hadoop-yarn/lib/json-io-2.5.1.jar:/usr/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/lib/hadoop-yarn/lib/fst-2.50.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/javassist-3.18.1-GA.jar:/usr/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo-0.4.19.jar:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/jcl-over-slf4j-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/ion-java-1.0.2.jar:/usr/share/aws/emr/emrfs/lib/emrfs-hadoop-assembly-2.21.0.jar:/usr/share/aws/emr/emrfs/lib/jmespath-java-1.11.267.jar:/usr/share/aws/emr/emrfs/lib/javax.inject-1.jar:/usr/share/aws/emr/emrfs/lib/bcpkix-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/lib/slf4j-api-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/aopalliance-1.0.jar:/usr/share/aws/emr/emrfs/lib/bcprov-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/auxlib/*:/usr/share/aws/emr/lib/*:/usr/share/aws/emr/ddb/lib/emr-ddb-hadoop.jar:/usr/share/aws/emr/goodies/lib/emr-hadoop-goodies.jar:/usr/share/aws/emr/kinesis/lib/emr-kinesis-hadoop.jar:/usr/lib/spark/yarn/lib/datanucleus-api-jdo.jar:/usr/lib/spark/yarn/lib/datanucleus-core.jar:/usr/lib/spark/yarn/lib/datanucleus-rdbms.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink-1.1.0.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacecommerceanalytics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rds-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-core-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-clouddirectory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsmv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-emr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediaconvert-1.11.267.jar:/usr/share/aws/aws-java-sdk/jmespath-java-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appsync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastore-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iot-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costandusagereport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpleworkflow-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-comprehend-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidentity-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ec2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudtrail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servermigration-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesis-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workmail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-polly-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpledb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscalingplans-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elastictranscoder-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pinpoint-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lexmodelbuilding-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacemeteringservice-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dynamodb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-storagegateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codegen-maven-plugin-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediapackage-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroupstaggingapi-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-acm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-machinelearning-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mobile-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dax-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lightsail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sns-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codedeploy-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-serverlessapplicationrepository-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-shield-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-applicationautoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastoredata-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloud9-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-budgets-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iotjobsdataplane-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-medialive-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-inspector-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-importexport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-test-utils-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitosync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-api-gateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codebuild-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workspaces-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opensdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-snowball-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pricing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-code-generator-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glacier-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-guardduty-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicediscovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codestar-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appstream-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancingv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-greengrass-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lex-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroups-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-support-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticbeanstalk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-route53-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-datapipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rekognition-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-organizations-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iam-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codepipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-logs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworkscm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-xray-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplaceentitlement-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-translate-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-events-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-alexaforbusiness-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemaker-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-s3-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costexplorer-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directconnect-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticache-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-gamelift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sqs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemakerruntime-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glue-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-waf-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mechanicalturkrequester-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatchmetrics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-migrationhub-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lambda-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-discovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidp-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-athena-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicecatalog-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mq-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-batch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-devicefarm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sts-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-stepfunctions-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesisvideo-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworks-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workdocs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-redshift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudfront-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-config-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-efs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-models-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ssm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codecommit-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-health-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudformation-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ses-1.11.267.jar
>> 2018-04-16 12:53:16,803 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
>> 2018-04-16 12:53:16,805 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Registered UNIX signal handlers for [TERM, HUP, INT]
>> 2018-04-16 12:53:16,809 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - YARN daemon is running as: hadoop Yarn client user obtainer: hadoop
>> 2018-04-16 12:53:16,813 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.archive.fs.dir, hdfs:///completed-jobs/
>> 2018-04-16 12:53:16,814 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.default-scheme, s3a
>> 2018-04-16 12:53:16,814 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: classloader.resolve-order, parent-first
>> 2018-04-16 12:53:16,814 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: high-availability.cluster-id, application_1523726493647_0302
>> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.heap.mb, 1024
>> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: yarn.heap-cutoff-ratio, 0.5
>> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.preallocate, true
>> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: yarn.maximum-failed-containers, -1
>> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: akka.watch.heartbeat.pause, 600 s
>> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: internal.cluster.execution-mode, NORMAL
>> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.off-heap, on
>> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.fraction, 0.1
>> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.numberOfTaskSlots, 1
>> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf
>> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.heap.mb, 9216
>> 2018-04-16 12:53:16,817 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: historyserver.archive.fs.dir, hdfs:///completed-jobs/
>> 2018-04-16 12:53:16,833 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Setting directories for temporary files to: /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302
>> 2018-04-16 12:53:16,859 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Starting YarnSessionClusterEntrypoint.
>> 2018-04-16 12:53:16,859 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Install default filesystem.
>> 2018-04-16 12:53:17,036 INFO  org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop user set to hadoop (auth:SIMPLE)
>> 2018-04-16 12:53:17,071 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Initializing cluster services.
>> 2018-04-16 12:53:17,081 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Trying to start actor system at ip-10-0-22-186.ap-southeast-1.compute.internal:36411
>> 2018-04-16 12:53:18,022 INFO  akka.event.slf4j.Slf4jLogger                                  - Slf4jLogger started
>> 2018-04-16 12:53:18,254 INFO  akka.remote.Remoting                                          - Starting remoting
>> 2018-04-16 12:53:18,461 INFO  akka.remote.Remoting                                          - Remoting started; listening on addresses :[akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411]
>> 2018-04-16 12:53:18,488 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Actor system started at akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411
>> 2018-04-16 12:53:18,559 INFO  org.apache.flink.runtime.blob.BlobServer                      - Created BLOB server storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302/blobStore-9ec0a59f-c03a-446f-a841-60a65892b383
>> 2018-04-16 12:53:18,566 INFO  org.apache.flink.runtime.blob.BlobServer                      - Started BLOB server at 0.0.0.0:46869 - max concurrent requests: 50 - max backlog: 1000
>> 2018-04-16 12:53:18,632 INFO  org.apache.flink.runtime.metrics.MetricRegistryImpl           - No metrics reporter configured, no metrics will be exposed/reported.
>> 2018-04-16 12:53:18,638 INFO  org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore  - Initializing FileArchivedExecutionGraphStore: Storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302/executionGraphStore-ee071b9a-ad86-4b36-860c-6501f509ecf9, expiration time 3600000, maximum cache size 52428800 bytes.
>> 2018-04-16 12:53:18,726 INFO  org.apache.flink.runtime.blob.TransientBlobCache              - Created BLOB cache storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302/blobStore-c15647ee-31c1-451d-84d2-729a67e065d8
>> 2018-04-16 12:53:18,750 WARN  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Upload directory /tmp/flink-web-6d14c830-9074-4a2f-8e1e-7344545e34e5/flink-web-upload does not exist, or has been deleted externally. Previously uploaded files are no longer available.
>> 2018-04-16 12:53:18,757 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Created directory /tmp/flink-web-6d14c830-9074-4a2f-8e1e-7344545e34e5/flink-web-upload for file uploads.
>> 2018-04-16 12:53:18,765 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Starting rest endpoint.
>> 2018-04-16 12:53:19,361 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component log file: /var/log/hadoop-yarn/containers/application_1523726493647_0302/container_1523726493647_0302_01_000001/jobmanager.log
>> 2018-04-16 12:53:19,362 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component stdout file: /var/log/hadoop-yarn/containers/application_1523726493647_0302/container_1523726493647_0302_01_000001/jobmanager.out
>> 2018-04-16 12:53:19,442 INFO  org.apache.flink.runtime.rest.RestClient                      - Rest client endpoint started.
>> 2018-04-16 12:53:19,589 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Rest endpoint listening at ip-10-0-22-186.ap-southeast-1.compute.internal:45697
>> 2018-04-16 12:53:19,589 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - http://ip-10-0-22-186.ap-southeast-1.compute.internal:45697 was granted leadership with leaderSessionID=00000000-0000-0000-0000-000000000000
>> 2018-04-16 12:53:19,614 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.yarn.YarnResourceManager at akka://flink/user/resourcemanager .
>> 2018-04-16 12:53:19,979 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at akka://flink/user/dispatcher .
>> 2018-04-16 12:53:20,049 INFO  org.apache.flink.yarn.YarnResourceManager                     - ResourceManager akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411/user/resourcemanager was granted leadership with fencing token 00000000000000000000000000000000
>> 2018-04-16 12:53:20,050 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager  - Starting the SlotManager.
>> 2018-04-16 12:53:20,101 INFO  org.apache.hadoop.yarn.client.RMProxy                         - Connecting to ResourceManager at ip-10-0-22-169.ap-southeast-1.compute.internal/10.0.22.169:8030
>> 2018-04-16 12:53:20,649 INFO  org.apache.flink.yarn.YarnResourceManager                     - Recovered 0 containers from previous attempts ([]).
>> 2018-04-16 12:53:20,655 INFO  org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  - yarn.client.max-cached-nodemanagers-proxies : 0
>> 2018-04-16 12:53:20,706 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Dispatcher akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411/user/dispatcher was granted leadership with fencing token 00000000000000000000000000000000
>> 2018-04-16 12:53:20,706 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Recovering all persisted jobs.
>> 2018-04-16 12:53:26,207 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Submitting job 58a1b149b5b69de66bb931836c459d2b (mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job).
>> 2018-04-16 12:53:26,223 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster at akka://flink/user/jobmanager_0 .
>> 2018-04-16 12:53:26,237 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Initializing job mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job (58a1b149b5b69de66bb931836c459d2b).
>> 2018-04-16 12:53:26,279 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Using restart strategy NoRestartStrategy for mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job (58a1b149b5b69de66bb931836c459d2b).
>> 2018-04-16 12:53:26,285 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.slotpool.SlotPool at akka://flink/user/820d207e-fddd-4e24-b327-1f3c1cb25024 .
>> 2018-04-16 12:53:26,320 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Job recovers via failover strategy: full graph restart
>> 2018-04-16 12:53:26,386 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Running initialization on master for job mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job (58a1b149b5b69de66bb931836c459d2b).
>> 2018-04-16 12:53:26,671 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Successfully ran initialization on master in 284 ms.
>> 2018-04-16 12:53:36,204 ERROR org.apache.flink.runtime.rest.handler.job.JobSubmitHandler    - Implementation error: Unhandled exception.
>> akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://flink/user/dispatcher#1710592138]] after [10000 ms]. Sender[null] sent message of type "org.apache.flink.runtime.rpc.messages.LocalFencedMessage".
>> 	at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:604)
>> 	at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:126)
>> 	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
>> 	at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
>> 	at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
>> 	at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:329)
>> 	at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:280)
>> 	at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:284)
>> 	at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:236)
>> 	at java.lang.Thread.run(Thread.java:748)
>> 2018-04-16 12:53:36,300 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - RECEIVED SIGNAL 15: SIGTERM. Shutting down as requested.
>> 2018-04-16 12:53:36,311 INFO  org.apache.flink.runtime.blob.TransientBlobCache              - Shutting down BLOB cache
>> 2018-04-16 12:53:36,314 INFO  org.apache.flink.runtime.blob.BlobServer                      - Stopped BLOB server at 0.0.0.0:46869
>>
>>
>>
>>
>

Re: Testing on Flink 1.5

Posted by Gary Yao <ga...@data-artisans.com>.
Hi Amit,

Thank you for testing the 1.5 Release Candidate. Find my answers below.

> Few of our jobs are stuck with logs [1], these jobs are only able to
allocate JM and couldn't get any TM, however, there are ample resource on
our cluster.

In the logs you attached I cannot see that a job was submitted. If a job is
submitted, the dispatcher should log: "Submitting job <job_id> (<job_name>)"
[1]. Could it be that the attached logs are incomplete, or the client
already
fails to submit the job? Can you also attach the client logs?


> All completed application on YARN are marked as KILLED, instead of
FINISHED. It creates problem on tracking jobs as we require to navigate on
each application logs and also history server is not logging any completed
application.

I was able to reproduce this: In non-detached mode, the State and
FinalStatus
are set to KILLED. In detached mode, the Status and FinalStatus are set
correctly. I have re-opened this ticket:
  https://issues.apache.org/jira/browse/FLINK-8900

You are also right about the History Server, which is currently not
implemented
for FLIP-6. Here is the ticket for reference:
  https://issues.apache.org/jira/browse/FLINK-9194


> We are running our system on AWS EMR and using CORE nodes at its bare
minimum, we see with new paradigm of application submission we are hitting
Out of space error in our HDFS Data Node. Do you think, we should have
configuration to automatically evict these .flink/ files?

This seems to be a regression. In Flink 1.4., the files are cleaned up
automatically. I opened this ticket:
https://issues.apache.org/jira/browse/FLINK-9196


> Lastly, we are also observing Akka.ask.timeout error for few of our jobs
(JM's logs[2]), we tried to increase this parameter to 2 min from default
10 s and still observe the same.

Does this always happen for the same jobs? I cannot reproduce this.

Best,
Gary

[1]
https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java#L235

On Tue, Apr 17, 2018 at 9:35 AM, Amit Jain <aj...@gmail.com> wrote:

> Hi,
>
>
> We have started testing our workload on Flink 1.5.0 (Commit: 2af481a) and faced few issues. Here, we are using YARN as cluster manager.
>
>
>    - Few of our jobs are stuck with logs [1], these jobs are only able to allocate JM and couldn't get any TM, however, there are ample resource on our cluster.
>    - All completed application on YARN are marked as KILLED, instead of FINISHED. It creates problem on tracking jobs as we require to navigate on each application logs and also history server is not logging any completed application.
>    - We are running our system on AWS EMR and using CORE nodes at its bare minimum, we see with new paradigm of application submission we are hitting Out of space error in our HDFS Data Node. Do you think, we should have configuration to automatically evict these .flink/ files?
>    - Lastly, we are also observing Akka.ask.timeout error for few of our jobs (JM's logs[2]), we tried to increase this parameter to 2 min from default 10 s and still observe the same.
>
> Please help us guide through these issues.
>
> --
> Thanks,
> Amit
>
> [1] JM logs where jobs are not even started
>
> 2018-04-17 04:26:16,165 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
> 2018-04-17 04:26:16,166 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Starting YarnSessionClusterEntrypoint (Version: 1.5.0, Rev:2af481a, Date:02.04.2018 @ 15:17:35 UTC)
> 2018-04-17 04:26:16,166 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  OS current user: yarn
> 2018-04-17 04:26:16,843 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Current Hadoop/Kerberos user: hadoop
> 2018-04-17 04:26:16,843 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.161-b14
> 2018-04-17 04:26:16,843 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Maximum heap size: 406 MiBytes
> 2018-04-17 04:26:16,843 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JAVA_HOME: /usr/lib/jvm/java-openjdk
> 2018-04-17 04:26:16,844 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Hadoop version: 2.8.3
> 2018-04-17 04:26:16,844 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM Options:
> 2018-04-17 04:26:16,844 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Xmx424m
> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog.file=/var/log/hadoop-yarn/containers/application_1523726493647_0503/container_1523726493647_0503_01_000001/jobmanager.log
> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlogback.configurationFile=file:logback.xml
> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog4j.configuration=file:log4j.properties
> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Program Arguments: (none)
> 2018-04-17 04:26:16,845 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Classpath: lib/flink-dist_2.11-1.5.0.jar:lib/flink-python_2.11-1.5.0.jar:lib/flink-shaded-hadoop2-uber-1.5.0.jar:lib/flink-shaded-include-yarn-0.9.1.jar:lib/flink-yarn_2.11-1.5-SNAPSHOT.jar:lib/guava-18.0.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.7.jar:log4j.properties:logback.xml:flink.jar:flink-conf.yaml::/etc/hadoop/conf:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-streaming.jar:/usr/lib/hadoop/hadoop-distcp.jar:/usr/lib/hadoop/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archives.jar:/usr/lib/hadoop/hadoop-ant.jar:/usr/lib/hadoop/hadoop-aws.jar:/usr/lib/hadoop/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-rumen.jar:/usr/lib/hadoop/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack.jar:/usr/lib/hadoop/hadoop-azure.jar:/usr/lib/hadoop/hadoop-extras.jar:/usr/lib/hadoop/hadoop-azure-datalake.jar:/usr/lib/hadoop/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix.jar:/usr/lib/hadoop/hadoop-nfs.jar:/usr/lib/hadoop/hadoop-datajoin.jar:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop/hadoop-sls.jar:/usr/lib/hadoop/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/lib/curator-framework-2.7.1.jar:/usr/lib/hadoop/lib/json-smart-1.1.1.jar:/usr/lib/hadoop/lib/curator-recipes-2.7.1.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/lib/jcip-annotations-1.0.jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/lib/avro-1.7.4.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop/lib/gson-2.2.4.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop/lib/jsch-0.1.54.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar:/usr/lib/hadoop/lib/jersey-server-1.9.jar:/usr/lib/hadoop/lib/httpcore-4.4.4.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/curator-client-2.7.1.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop/lib/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/jsr305-3.0.0.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/httpclient-4.5.3.jar:/usr/lib/hadoop/lib/commons-lang-2.6.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/slf4j-api-1.7.10.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/xercesImpl-2.9.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-hdfs/lib/netty-all-4.0.23.Final.jar:/usr/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-hdfs/lib/okio-1.4.0.jar:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/xml-apis-1.3.04.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/okhttp-2.4.0.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/ion-java-1.0.2.jar:/usr/lib/hadoop-mapreduce/curator-framework-2.7.1.jar:/usr/lib/hadoop-mapreduce/json-smart-1.1.1.jar:/usr/lib/hadoop-mapreduce/curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-core-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/zookeeper-3.4.10.jar:/usr/lib/hadoop-mapreduce/jmespath-java-1.11.267.jar:/usr/lib/hadoop-mapreduce/jettison-1.1.jar:/usr/lib/hadoop-mapreduce/java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/jcip-annotations-1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs.jar:/usr/lib/hadoop-mapreduce/hadoop-auth.jar:/usr/lib/hadoop-mapreduce/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-2.6.7.jar:/usr/lib/hadoop-mapreduce/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-mapreduce/gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/azure-data-lake-store-sdk-2.2.3.jar:/usr/lib/hadoop-mapreduce/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/jsch-0.1.54.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/commons-lang3-3.3.2.jar:/usr/lib/hadoop-mapreduce/xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/commons-collections-3.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/hadoop-archives.jar:/usr/lib/hadoop-mapreduce/commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/hadoop-ant.jar:/usr/lib/hadoop-mapreduce/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/jackson-databind-2.6.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-aws.jar:/usr/lib/hadoop-mapreduce/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/jackson-annotations-2.6.7.jar:/usr/lib/hadoop-mapreduce/httpcore-4.4.4.jar:/usr/lib/hadoop-mapreduce/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jetty-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop-mapreduce/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/xz-1.0.jar:/usr/lib/hadoop-mapreduce/joda-time-2.9.4.jar:/usr/lib/hadoop-mapreduce/commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-mapreduce/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/activation-1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/hadoop-azure.jar:/usr/lib/hadoop-mapreduce/hadoop-extras.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/okio-1.4.0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/azure-storage-2.2.0.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-s3-1.11.267.jar:/usr/lib/hadoop-mapreduce/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/jackson-dataformat-cbor-2.6.7.jar:/usr/lib/hadoop-mapreduce/jackson-xc-1.9.13.jar:/usr/lib/hadoop-mapreduce/okhttp-2.4.0.jar:/usr/lib/hadoop-mapreduce/asm-3.2.jar:/usr/lib/hadoop-mapreduce/hadoop-sls.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/httpclient-4.5.3.jar:/usr/lib/hadoop-mapreduce/commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-kms-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/lib/curator-test-2.7.1.jar:/usr/lib/hadoop-yarn/lib/java-util-1.9.0.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10-tests.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop-yarn/lib/jettison-1.1.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/lib/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/lib/hadoop-yarn/lib/commons-math-2.2.jar:/usr/lib/hadoop-yarn/lib/curator-client-2.7.1.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-yarn/lib/activation-1.1.jar:/usr/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-yarn/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-yarn/lib/guava-11.0.2.jar:/usr/lib/hadoop-yarn/lib/json-io-2.5.1.jar:/usr/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/lib/hadoop-yarn/lib/fst-2.50.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/javassist-3.18.1-GA.jar:/usr/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo-0.4.19.jar:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/jcl-over-slf4j-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/ion-java-1.0.2.jar:/usr/share/aws/emr/emrfs/lib/emrfs-hadoop-assembly-2.21.0.jar:/usr/share/aws/emr/emrfs/lib/jmespath-java-1.11.267.jar:/usr/share/aws/emr/emrfs/lib/javax.inject-1.jar:/usr/share/aws/emr/emrfs/lib/bcpkix-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/lib/slf4j-api-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/aopalliance-1.0.jar:/usr/share/aws/emr/emrfs/lib/bcprov-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/auxlib/*:/usr/share/aws/emr/lib/*:/usr/share/aws/emr/ddb/lib/emr-ddb-hadoop.jar:/usr/share/aws/emr/goodies/lib/emr-hadoop-goodies.jar:/usr/share/aws/emr/kinesis/lib/emr-kinesis-hadoop.jar:/usr/lib/spark/yarn/lib/datanucleus-api-jdo.jar:/usr/lib/spark/yarn/lib/datanucleus-core.jar:/usr/lib/spark/yarn/lib/datanucleus-rdbms.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink-1.1.0.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacecommerceanalytics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rds-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-core-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-clouddirectory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsmv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-emr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediaconvert-1.11.267.jar:/usr/share/aws/aws-java-sdk/jmespath-java-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appsync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastore-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iot-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costandusagereport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpleworkflow-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-comprehend-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidentity-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ec2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudtrail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servermigration-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesis-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workmail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-polly-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpledb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscalingplans-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elastictranscoder-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pinpoint-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lexmodelbuilding-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacemeteringservice-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dynamodb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-storagegateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codegen-maven-plugin-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediapackage-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroupstaggingapi-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-acm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-machinelearning-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mobile-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dax-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lightsail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sns-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codedeploy-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-serverlessapplicationrepository-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-shield-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-applicationautoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastoredata-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloud9-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-budgets-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iotjobsdataplane-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-medialive-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-inspector-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-importexport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-test-utils-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitosync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-api-gateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codebuild-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workspaces-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opensdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-snowball-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pricing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-code-generator-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glacier-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-guardduty-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicediscovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codestar-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appstream-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancingv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-greengrass-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lex-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroups-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-support-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticbeanstalk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-route53-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-datapipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rekognition-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-organizations-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iam-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codepipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-logs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworkscm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-xray-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplaceentitlement-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-translate-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-events-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-alexaforbusiness-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemaker-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-s3-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costexplorer-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directconnect-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticache-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-gamelift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sqs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemakerruntime-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glue-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-waf-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mechanicalturkrequester-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatchmetrics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-migrationhub-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lambda-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-discovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidp-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-athena-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicecatalog-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mq-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-batch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-devicefarm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sts-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-stepfunctions-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesisvideo-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworks-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workdocs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-redshift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudfront-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-config-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-efs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-models-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ssm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codecommit-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-health-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudformation-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ses-1.11.267.jar
> 2018-04-17 04:26:16,846 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
> 2018-04-17 04:26:16,848 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Registered UNIX signal handlers for [TERM, HUP, INT]
> 2018-04-17 04:26:16,851 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - YARN daemon is running as: hadoop Yarn client user obtainer: hadoop
> 2018-04-17 04:26:16,854 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.archive.fs.dir, hdfs:///completed-jobs/
> 2018-04-17 04:26:16,854 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.default-scheme, s3a
> 2018-04-17 04:26:16,854 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: classloader.resolve-order, parent-first
> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: high-availability.cluster-id, application_1523726493647_0503
> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.heap.mb, 1024
> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: yarn.heap-cutoff-ratio, 0.5
> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.preallocate, true
> 2018-04-17 04:26:16,855 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: yarn.maximum-failed-containers, -1
> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: akka.watch.heartbeat.pause, 600 s
> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: internal.cluster.execution-mode, NORMAL
> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.off-heap, on
> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.fraction, 0.1
> 2018-04-17 04:26:16,856 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.numberOfTaskSlots, 1
> 2018-04-17 04:26:16,857 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf
> 2018-04-17 04:26:16,857 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: akka.ask.timeout, 2 min
> 2018-04-17 04:26:16,857 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.heap.mb, 10772
> 2018-04-17 04:26:16,857 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: historyserver.archive.fs.dir, hdfs:///completed-jobs/
> 2018-04-17 04:26:16,874 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Setting directories for temporary files to: /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503
> 2018-04-17 04:26:16,896 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Starting YarnSessionClusterEntrypoint.
> 2018-04-17 04:26:16,897 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Install default filesystem.
> 2018-04-17 04:26:17,014 INFO  org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop user set to hadoop (auth:SIMPLE)
> 2018-04-17 04:26:17,034 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Initializing cluster services.
> 2018-04-17 04:26:17,041 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Trying to start actor system at ip-10-0-22-186.ap-southeast-1.compute.internal:38669
> 2018-04-17 04:26:17,671 INFO  akka.event.slf4j.Slf4jLogger                                  - Slf4jLogger started
> 2018-04-17 04:26:17,794 INFO  akka.remote.Remoting                                          - Starting remoting
> 2018-04-17 04:26:17,937 INFO  akka.remote.Remoting                                          - Remoting started; listening on addresses :[akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669]
> 2018-04-17 04:26:17,948 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Actor system started at akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669
> 2018-04-17 04:26:17,980 INFO  org.apache.flink.runtime.blob.BlobServer                      - Created BLOB server storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503/blobStore-bca451c7-977d-425a-9e8f-4fe324fa8623
> 2018-04-17 04:26:17,983 INFO  org.apache.flink.runtime.blob.BlobServer                      - Started BLOB server at 0.0.0.0:44217 - max concurrent requests: 50 - max backlog: 1000
> 2018-04-17 04:26:18,009 INFO  org.apache.flink.runtime.metrics.MetricRegistryImpl           - No metrics reporter configured, no metrics will be exposed/reported.
> 2018-04-17 04:26:18,015 INFO  org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore  - Initializing FileArchivedExecutionGraphStore: Storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503/executionGraphStore-28bf6537-0bbf-486c-a346-084bd7517d7d, expiration time 3600000, maximum cache size 52428800 bytes.
> 2018-04-17 04:26:18,047 INFO  org.apache.flink.runtime.blob.TransientBlobCache              - Created BLOB cache storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0503/blobStore-f1de0cad-77b0-4bbd-b218-63e32b10bf6a
> 2018-04-17 04:26:18,057 WARN  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Upload directory /tmp/flink-web-1fd19ea7-ae19-4880-8787-92c5ec6ee509/flink-web-upload does not exist, or has been deleted externally. Previously uploaded files are no longer available.
> 2018-04-17 04:26:18,059 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Created directory /tmp/flink-web-1fd19ea7-ae19-4880-8787-92c5ec6ee509/flink-web-upload for file uploads.
> 2018-04-17 04:26:18,063 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Starting rest endpoint.
> 2018-04-17 04:26:18,386 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component log file: /var/log/hadoop-yarn/containers/application_1523726493647_0503/container_1523726493647_0503_01_000001/jobmanager.log
> 2018-04-17 04:26:18,386 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component stdout file: /var/log/hadoop-yarn/containers/application_1523726493647_0503/container_1523726493647_0503_01_000001/jobmanager.out
> 2018-04-17 04:26:18,454 INFO  org.apache.flink.runtime.rest.RestClient                      - Rest client endpoint started.
> 2018-04-17 04:26:18,538 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Rest endpoint listening at ip-10-0-22-186.ap-southeast-1.compute.internal:44431
> 2018-04-17 04:26:18,542 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - http://ip-10-0-22-186.ap-southeast-1.compute.internal:44431 was granted leadership with leaderSessionID=00000000-0000-0000-0000-000000000000
> 2018-04-17 04:26:18,558 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.yarn.YarnResourceManager at akka://flink/user/resourcemanager .
> 2018-04-17 04:26:18,713 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at akka://flink/user/dispatcher .
> 2018-04-17 04:26:18,739 INFO  org.apache.flink.yarn.YarnResourceManager                     - ResourceManager akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669/user/resourcemanager was granted leadership with fencing token 00000000000000000000000000000000
> 2018-04-17 04:26:18,740 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager  - Starting the SlotManager.
> 2018-04-17 04:26:18,769 INFO  org.apache.hadoop.yarn.client.RMProxy                         - Connecting to ResourceManager at ip-10-0-22-169.ap-southeast-1.compute.internal/10.0.22.169:8030
> 2018-04-17 04:26:19,083 INFO  org.apache.flink.yarn.YarnResourceManager                     - Recovered 0 containers from previous attempts ([]).
> 2018-04-17 04:26:19,086 INFO  org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  - yarn.client.max-cached-nodemanagers-proxies : 0
> 2018-04-17 04:26:19,101 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Dispatcher akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:38669/user/dispatcher was granted leadership with fencing token 00000000000000000000000000000000
> 2018-04-17 04:26:19,101 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Recovering all persisted jobs.
>
>
>
>
> [2] JM's logs
>
> 2018-04-16 12:53:15,825 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
> 2018-04-16 12:53:15,827 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Starting YarnSessionClusterEntrypoint (Version: 1.5.0, Rev:2af481a, Date:02.04.2018 @ 15:17:35 UTC)
> 2018-04-16 12:53:15,827 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  OS current user: yarn
> 2018-04-16 12:53:16,798 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Current Hadoop/Kerberos user: hadoop
> 2018-04-16 12:53:16,799 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.161-b14
> 2018-04-16 12:53:16,799 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Maximum heap size: 406 MiBytes
> 2018-04-16 12:53:16,799 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JAVA_HOME: /usr/lib/jvm/java-openjdk
> 2018-04-16 12:53:16,800 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Hadoop version: 2.8.3
> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM Options:
> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Xmx424m
> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog.file=/var/log/hadoop-yarn/containers/application_1523726493647_0302/container_1523726493647_0302_01_000001/jobmanager.log
> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlogback.configurationFile=file:logback.xml
> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog4j.configuration=file:log4j.properties
> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Program Arguments: (none)
> 2018-04-16 12:53:16,801 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Classpath: lib/flink-dist_2.11-1.5.0.jar:lib/flink-python_2.11-1.5.0.jar:lib/flink-shaded-hadoop2-uber-1.5.0.jar:lib/flink-shaded-include-yarn-0.9.1.jar:lib/flink-yarn_2.11-1.5-SNAPSHOT.jar:lib/guava-18.0.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.7.jar:log4j.properties:logback.xml:flink.jar:flink-conf.yaml::/etc/hadoop/conf:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-streaming.jar:/usr/lib/hadoop/hadoop-distcp.jar:/usr/lib/hadoop/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archives.jar:/usr/lib/hadoop/hadoop-ant.jar:/usr/lib/hadoop/hadoop-aws.jar:/usr/lib/hadoop/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-rumen.jar:/usr/lib/hadoop/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack.jar:/usr/lib/hadoop/hadoop-azure.jar:/usr/lib/hadoop/hadoop-extras.jar:/usr/lib/hadoop/hadoop-azure-datalake.jar:/usr/lib/hadoop/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix.jar:/usr/lib/hadoop/hadoop-nfs.jar:/usr/lib/hadoop/hadoop-datajoin.jar:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop/hadoop-sls.jar:/usr/lib/hadoop/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/lib/curator-framework-2.7.1.jar:/usr/lib/hadoop/lib/json-smart-1.1.1.jar:/usr/lib/hadoop/lib/curator-recipes-2.7.1.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/lib/jcip-annotations-1.0.jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/lib/avro-1.7.4.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop/lib/gson-2.2.4.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop/lib/jsch-0.1.54.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar:/usr/lib/hadoop/lib/jersey-server-1.9.jar:/usr/lib/hadoop/lib/httpcore-4.4.4.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/curator-client-2.7.1.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop/lib/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/jsr305-3.0.0.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/httpclient-4.5.3.jar:/usr/lib/hadoop/lib/commons-lang-2.6.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/slf4j-api-1.7.10.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/xercesImpl-2.9.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-hdfs/lib/netty-all-4.0.23.Final.jar:/usr/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-hdfs/lib/okio-1.4.0.jar:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/xml-apis-1.3.04.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/okhttp-2.4.0.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/ion-java-1.0.2.jar:/usr/lib/hadoop-mapreduce/curator-framework-2.7.1.jar:/usr/lib/hadoop-mapreduce/json-smart-1.1.1.jar:/usr/lib/hadoop-mapreduce/curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-core-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/zookeeper-3.4.10.jar:/usr/lib/hadoop-mapreduce/jmespath-java-1.11.267.jar:/usr/lib/hadoop-mapreduce/jettison-1.1.jar:/usr/lib/hadoop-mapreduce/java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/jcip-annotations-1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs.jar:/usr/lib/hadoop-mapreduce/hadoop-auth.jar:/usr/lib/hadoop-mapreduce/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-2.6.7.jar:/usr/lib/hadoop-mapreduce/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-mapreduce/gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/azure-data-lake-store-sdk-2.2.3.jar:/usr/lib/hadoop-mapreduce/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/jsch-0.1.54.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/commons-lang3-3.3.2.jar:/usr/lib/hadoop-mapreduce/xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/commons-collections-3.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/hadoop-archives.jar:/usr/lib/hadoop-mapreduce/commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/hadoop-ant.jar:/usr/lib/hadoop-mapreduce/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/jackson-databind-2.6.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-aws.jar:/usr/lib/hadoop-mapreduce/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/jackson-annotations-2.6.7.jar:/usr/lib/hadoop-mapreduce/httpcore-4.4.4.jar:/usr/lib/hadoop-mapreduce/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jetty-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop-mapreduce/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/xz-1.0.jar:/usr/lib/hadoop-mapreduce/joda-time-2.9.4.jar:/usr/lib/hadoop-mapreduce/commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-mapreduce/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/activation-1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/hadoop-azure.jar:/usr/lib/hadoop-mapreduce/hadoop-extras.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/okio-1.4.0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/azure-storage-2.2.0.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-s3-1.11.267.jar:/usr/lib/hadoop-mapreduce/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/jackson-dataformat-cbor-2.6.7.jar:/usr/lib/hadoop-mapreduce/jackson-xc-1.9.13.jar:/usr/lib/hadoop-mapreduce/okhttp-2.4.0.jar:/usr/lib/hadoop-mapreduce/asm-3.2.jar:/usr/lib/hadoop-mapreduce/hadoop-sls.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/httpclient-4.5.3.jar:/usr/lib/hadoop-mapreduce/commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-kms-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/lib/curator-test-2.7.1.jar:/usr/lib/hadoop-yarn/lib/java-util-1.9.0.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10-tests.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop-yarn/lib/jettison-1.1.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/lib/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/lib/hadoop-yarn/lib/commons-math-2.2.jar:/usr/lib/hadoop-yarn/lib/curator-client-2.7.1.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-yarn/lib/activation-1.1.jar:/usr/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-yarn/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-yarn/lib/guava-11.0.2.jar:/usr/lib/hadoop-yarn/lib/json-io-2.5.1.jar:/usr/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/lib/hadoop-yarn/lib/fst-2.50.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/javassist-3.18.1-GA.jar:/usr/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo-0.4.19.jar:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/jcl-over-slf4j-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/ion-java-1.0.2.jar:/usr/share/aws/emr/emrfs/lib/emrfs-hadoop-assembly-2.21.0.jar:/usr/share/aws/emr/emrfs/lib/jmespath-java-1.11.267.jar:/usr/share/aws/emr/emrfs/lib/javax.inject-1.jar:/usr/share/aws/emr/emrfs/lib/bcpkix-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/lib/slf4j-api-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/aopalliance-1.0.jar:/usr/share/aws/emr/emrfs/lib/bcprov-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/auxlib/*:/usr/share/aws/emr/lib/*:/usr/share/aws/emr/ddb/lib/emr-ddb-hadoop.jar:/usr/share/aws/emr/goodies/lib/emr-hadoop-goodies.jar:/usr/share/aws/emr/kinesis/lib/emr-kinesis-hadoop.jar:/usr/lib/spark/yarn/lib/datanucleus-api-jdo.jar:/usr/lib/spark/yarn/lib/datanucleus-core.jar:/usr/lib/spark/yarn/lib/datanucleus-rdbms.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink-1.1.0.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacecommerceanalytics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rds-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-core-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-clouddirectory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsmv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-emr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediaconvert-1.11.267.jar:/usr/share/aws/aws-java-sdk/jmespath-java-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appsync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastore-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iot-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costandusagereport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpleworkflow-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-comprehend-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidentity-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ec2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudtrail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servermigration-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesis-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workmail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-polly-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpledb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscalingplans-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elastictranscoder-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pinpoint-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lexmodelbuilding-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacemeteringservice-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dynamodb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-storagegateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codegen-maven-plugin-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediapackage-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroupstaggingapi-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-acm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-machinelearning-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mobile-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dax-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lightsail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sns-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codedeploy-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-serverlessapplicationrepository-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-shield-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-applicationautoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastoredata-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloud9-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-budgets-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iotjobsdataplane-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-medialive-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-inspector-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-importexport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-test-utils-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitosync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-api-gateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codebuild-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workspaces-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opensdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-snowball-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pricing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-code-generator-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glacier-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-guardduty-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicediscovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codestar-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appstream-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancingv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-greengrass-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lex-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroups-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-support-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticbeanstalk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-route53-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-datapipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rekognition-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-organizations-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iam-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codepipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-logs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworkscm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-xray-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplaceentitlement-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-translate-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-events-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-alexaforbusiness-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemaker-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-s3-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costexplorer-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directconnect-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticache-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-gamelift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sqs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemakerruntime-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glue-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-waf-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mechanicalturkrequester-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatchmetrics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-migrationhub-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lambda-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-discovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidp-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-athena-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicecatalog-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mq-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-batch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-devicefarm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sts-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-stepfunctions-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesisvideo-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworks-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workdocs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-redshift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudfront-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-config-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-efs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-models-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ssm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codecommit-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-health-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudformation-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ses-1.11.267.jar
> 2018-04-16 12:53:16,803 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
> 2018-04-16 12:53:16,805 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Registered UNIX signal handlers for [TERM, HUP, INT]
> 2018-04-16 12:53:16,809 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - YARN daemon is running as: hadoop Yarn client user obtainer: hadoop
> 2018-04-16 12:53:16,813 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.archive.fs.dir, hdfs:///completed-jobs/
> 2018-04-16 12:53:16,814 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.default-scheme, s3a
> 2018-04-16 12:53:16,814 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: classloader.resolve-order, parent-first
> 2018-04-16 12:53:16,814 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: high-availability.cluster-id, application_1523726493647_0302
> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.heap.mb, 1024
> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: yarn.heap-cutoff-ratio, 0.5
> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.preallocate, true
> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: yarn.maximum-failed-containers, -1
> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: akka.watch.heartbeat.pause, 600 s
> 2018-04-16 12:53:16,815 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: internal.cluster.execution-mode, NORMAL
> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.off-heap, on
> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.fraction, 0.1
> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.numberOfTaskSlots, 1
> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf
> 2018-04-16 12:53:16,816 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.heap.mb, 9216
> 2018-04-16 12:53:16,817 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: historyserver.archive.fs.dir, hdfs:///completed-jobs/
> 2018-04-16 12:53:16,833 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Setting directories for temporary files to: /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302
> 2018-04-16 12:53:16,859 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Starting YarnSessionClusterEntrypoint.
> 2018-04-16 12:53:16,859 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Install default filesystem.
> 2018-04-16 12:53:17,036 INFO  org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop user set to hadoop (auth:SIMPLE)
> 2018-04-16 12:53:17,071 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Initializing cluster services.
> 2018-04-16 12:53:17,081 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Trying to start actor system at ip-10-0-22-186.ap-southeast-1.compute.internal:36411
> 2018-04-16 12:53:18,022 INFO  akka.event.slf4j.Slf4jLogger                                  - Slf4jLogger started
> 2018-04-16 12:53:18,254 INFO  akka.remote.Remoting                                          - Starting remoting
> 2018-04-16 12:53:18,461 INFO  akka.remote.Remoting                                          - Remoting started; listening on addresses :[akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411]
> 2018-04-16 12:53:18,488 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Actor system started at akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411
> 2018-04-16 12:53:18,559 INFO  org.apache.flink.runtime.blob.BlobServer                      - Created BLOB server storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302/blobStore-9ec0a59f-c03a-446f-a841-60a65892b383
> 2018-04-16 12:53:18,566 INFO  org.apache.flink.runtime.blob.BlobServer                      - Started BLOB server at 0.0.0.0:46869 - max concurrent requests: 50 - max backlog: 1000
> 2018-04-16 12:53:18,632 INFO  org.apache.flink.runtime.metrics.MetricRegistryImpl           - No metrics reporter configured, no metrics will be exposed/reported.
> 2018-04-16 12:53:18,638 INFO  org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore  - Initializing FileArchivedExecutionGraphStore: Storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302/executionGraphStore-ee071b9a-ad86-4b36-860c-6501f509ecf9, expiration time 3600000, maximum cache size 52428800 bytes.
> 2018-04-16 12:53:18,726 INFO  org.apache.flink.runtime.blob.TransientBlobCache              - Created BLOB cache storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_0302/blobStore-c15647ee-31c1-451d-84d2-729a67e065d8
> 2018-04-16 12:53:18,750 WARN  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Upload directory /tmp/flink-web-6d14c830-9074-4a2f-8e1e-7344545e34e5/flink-web-upload does not exist, or has been deleted externally. Previously uploaded files are no longer available.
> 2018-04-16 12:53:18,757 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Created directory /tmp/flink-web-6d14c830-9074-4a2f-8e1e-7344545e34e5/flink-web-upload for file uploads.
> 2018-04-16 12:53:18,765 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Starting rest endpoint.
> 2018-04-16 12:53:19,361 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component log file: /var/log/hadoop-yarn/containers/application_1523726493647_0302/container_1523726493647_0302_01_000001/jobmanager.log
> 2018-04-16 12:53:19,362 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component stdout file: /var/log/hadoop-yarn/containers/application_1523726493647_0302/container_1523726493647_0302_01_000001/jobmanager.out
> 2018-04-16 12:53:19,442 INFO  org.apache.flink.runtime.rest.RestClient                      - Rest client endpoint started.
> 2018-04-16 12:53:19,589 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Rest endpoint listening at ip-10-0-22-186.ap-southeast-1.compute.internal:45697
> 2018-04-16 12:53:19,589 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - http://ip-10-0-22-186.ap-southeast-1.compute.internal:45697 was granted leadership with leaderSessionID=00000000-0000-0000-0000-000000000000
> 2018-04-16 12:53:19,614 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.yarn.YarnResourceManager at akka://flink/user/resourcemanager .
> 2018-04-16 12:53:19,979 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at akka://flink/user/dispatcher .
> 2018-04-16 12:53:20,049 INFO  org.apache.flink.yarn.YarnResourceManager                     - ResourceManager akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411/user/resourcemanager was granted leadership with fencing token 00000000000000000000000000000000
> 2018-04-16 12:53:20,050 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager  - Starting the SlotManager.
> 2018-04-16 12:53:20,101 INFO  org.apache.hadoop.yarn.client.RMProxy                         - Connecting to ResourceManager at ip-10-0-22-169.ap-southeast-1.compute.internal/10.0.22.169:8030
> 2018-04-16 12:53:20,649 INFO  org.apache.flink.yarn.YarnResourceManager                     - Recovered 0 containers from previous attempts ([]).
> 2018-04-16 12:53:20,655 INFO  org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  - yarn.client.max-cached-nodemanagers-proxies : 0
> 2018-04-16 12:53:20,706 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Dispatcher akka.tcp://flink@ip-10-0-22-186.ap-southeast-1.compute.internal:36411/user/dispatcher was granted leadership with fencing token 00000000000000000000000000000000
> 2018-04-16 12:53:20,706 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Recovering all persisted jobs.
> 2018-04-16 12:53:26,207 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Submitting job 58a1b149b5b69de66bb931836c459d2b (mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job).
> 2018-04-16 12:53:26,223 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster at akka://flink/user/jobmanager_0 .
> 2018-04-16 12:53:26,237 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Initializing job mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job (58a1b149b5b69de66bb931836c459d2b).
> 2018-04-16 12:53:26,279 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Using restart strategy NoRestartStrategy for mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job (58a1b149b5b69de66bb931836c459d2b).
> 2018-04-16 12:53:26,285 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.slotpool.SlotPool at akka://flink/user/820d207e-fddd-4e24-b327-1f3c1cb25024 .
> 2018-04-16 12:53:26,320 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Job recovers via failover strategy: full graph restart
> 2018-04-16 12:53:26,386 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Running initialization on master for job mongo_notificationDB.userdeviceidentifiers_userdeviceidentifiers_merge_job (58a1b149b5b69de66bb931836c459d2b).
> 2018-04-16 12:53:26,671 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Successfully ran initialization on master in 284 ms.
> 2018-04-16 12:53:36,204 ERROR org.apache.flink.runtime.rest.handler.job.JobSubmitHandler    - Implementation error: Unhandled exception.
> akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://flink/user/dispatcher#1710592138]] after [10000 ms]. Sender[null] sent message of type "org.apache.flink.runtime.rpc.messages.LocalFencedMessage".
> 	at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:604)
> 	at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:126)
> 	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
> 	at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
> 	at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
> 	at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:329)
> 	at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:280)
> 	at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:284)
> 	at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:236)
> 	at java.lang.Thread.run(Thread.java:748)
> 2018-04-16 12:53:36,300 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - RECEIVED SIGNAL 15: SIGTERM. Shutting down as requested.
> 2018-04-16 12:53:36,311 INFO  org.apache.flink.runtime.blob.TransientBlobCache              - Shutting down BLOB cache
> 2018-04-16 12:53:36,314 INFO  org.apache.flink.runtime.blob.BlobServer                      - Stopped BLOB server at 0.0.0.0:46869
>
>
>
>