You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spot.apache.org by Deon Griessel <dg...@searchtechnologies.com> on 2017/08/01 15:04:20 UTC
RE: Spot Ingest Proxy Error
Ok, it seems like I got around the original error by enabling the hive service in Spark2 configuration in Cloudera Manager and restarting all services.
Now getting the following error:
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
Anybody seen this before? Full log below.
Thanks
---------------------------
2017-08-01 14:49:13,694 - SPOT.INGEST.WORKER - INFO - Initializing kafka instance
2017-08-01 14:49:13,694 - SPOT.INGEST.WRK.PROXY - INFO - Creating Spark Job for topic: SPOT-INGEST-proxy-14_49_11
2017-08-01 14:49:13,694 - SPOT.INGEST.WRK.PROXY - INFO - SPOT.Utils: Executing: spark2-submit --master yarn --driver-memory 2325m --num-executors 1 --conf spark.executor.memory=2325m --conf spark.executor.cores=1 --jars /home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar /home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py -zk ip-10-0-0-199.ec2.internal:2181 -t SPOT-INGEST-proxy-14_49_11 -db spotdb -dt proxy -w 1 -bs 1
17/08/01 14:49:14 INFO spark.SparkContext: Running Spark version 2.1.0.cloudera1
17/08/01 14:49:14 WARN spark.SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0
17/08/01 14:49:15 INFO spark.SecurityManager: Changing view acls to: spot-user
17/08/01 14:49:15 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/08/01 14:49:15 INFO spark.SecurityManager: Changing view acls groups to:
17/08/01 14:49:15 INFO spark.SecurityManager: Changing modify acls groups to:
17/08/01 14:49:15 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/08/01 14:49:15 INFO util.Utils: Successfully started service 'sparkDriver' on port 56925.
17/08/01 14:49:15 INFO spark.SparkEnv: Registering MapOutputTracker
17/08/01 14:49:15 INFO spark.SparkEnv: Registering BlockManagerMaster
17/08/01 14:49:15 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
17/08/01 14:49:15 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
17/08/01 14:49:15 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-23e4ad79-bd38-4e37-b688-a75f9565bda4
17/08/01 14:49:16 INFO memory.MemoryStore: MemoryStore started with capacity 1060.5 MB
17/08/01 14:49:16 INFO spark.SparkEnv: Registering OutputCommitCoordinator
17/08/01 14:49:16 INFO util.log: Logging initialized @2378ms
17/08/01 14:49:16 INFO server.Server: jetty-9.2.z-SNAPSHOT
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3b9882ec{/jobs,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19bb7285{/jobs/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@615b86f5{/jobs/job,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ff970c2{/jobs/job/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1159f15e{/stages,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27f55b38{/stages/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5d37ce06{/stages/stage,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ab71bb9{/stages/stage/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b7d4da0{/stages/pool,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f734944{/stages/pool/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19cf381c{/storage,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@736cc380{/storage/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@699743b3{/storage/rdd,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@49ddd2ed{/storage/rdd/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@242eb7e{/environment,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@77729e5a{/environment/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5fa63fa{/executors,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@79b9b526{/executors/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@f241039{/executors/threadDump,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@8a515f3{/executors/threadDump/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@e4408f4{/static,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14a7d27d{/,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@15ec1723{/api,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ee30613{/jobs/job/kill,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2aa3e9a6{/stages/stage/kill,null,AVAILABLE}
17/08/01 14:49:16 INFO server.ServerConnector: Started ServerConnector@37681deb{HTTP/1.1}{0.0.0.0:4040}
17/08/01 14:49:16 INFO server.Server: Started @2532ms
17/08/01 14:49:16 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
17/08/01 14:49:16 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.0.0.199:4040
17/08/01 14:49:16 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/08/01 14:49:17 INFO client.RMProxy: Connecting to ResourceManager at ip-10-0-0-199.ec2.internal/10.0.0.199:8032
17/08/01 14:49:17 INFO yarn.Client: Requesting a new application from cluster with 3 NodeManagers
17/08/01 14:49:17 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (2725 MB per container)
17/08/01 14:49:17 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/08/01 14:49:17 INFO yarn.Client: Setting up container launch context for our AM
17/08/01 14:49:17 INFO yarn.Client: Setting up the launch environment for our AM container
17/08/01 14:49:17 INFO yarn.Client: Preparing resources for our AM container
17/08/01 14:49:19 INFO yarn.Client: Uploading resource file:/home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501598591600_0003/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar
17/08/01 14:49:19 INFO yarn.Client: Uploading resource file:/tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25/__spark_conf__4374658135010889941.zip -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501598591600_0003/__spark_conf__.zip
17/08/01 14:49:19 INFO spark.SecurityManager: Changing view acls to: spot-user
17/08/01 14:49:19 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/08/01 14:49:19 INFO spark.SecurityManager: Changing view acls groups to:
17/08/01 14:49:19 INFO spark.SecurityManager: Changing modify acls groups to:
17/08/01 14:49:19 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/08/01 14:49:19 INFO yarn.Client: Submitting application application_1501598591600_0003 to ResourceManager
17/08/01 14:49:19 INFO impl.YarnClientImpl: Submitted application application_1501598591600_0003
17/08/01 14:49:19 INFO cluster.SchedulerExtensionServices: Starting Yarn extension services with app application_1501598591600_0003 and attemptId None
17/08/01 14:49:20 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:20 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.users.spot-user
start time: 1501598959525
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003/
user: spot-user
17/08/01 14:49:21 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:22 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:23 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/08/01 14:49:23 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> ip-10-0-0-199.ec2.internal, PROXY_URI_BASES -> http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003), /proxy/application_1501598591600_0003
17/08/01 14:49:23 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/08/01 14:49:23 INFO yarn.Client: Application report for application_1501598591600_0003 (state: RUNNING)
17/08/01 14:49:23 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 10.0.0.151
ApplicationMaster RPC port: 0
queue: root.users.spot-user
start time: 1501598959525
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003/
user: spot-user
17/08/01 14:49:23 INFO cluster.YarnClientSchedulerBackend: Application application_1501598591600_0003 has started running.
17/08/01 14:49:23 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 53355.
17/08/01 14:49:23 INFO netty.NettyBlockTransferService: Server created on 10.0.0.199:53355
17/08/01 14:49:23 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/08/01 14:49:23 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManagerMasterEndpoint: Registering block manager 10.0.0.199:53355 with 1060.5 MB RAM, BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManager: external shuffle service port = 7337
17/08/01 14:49:23 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@52c2dab5{/metrics/json,null,AVAILABLE}
17/08/01 14:49:23 INFO scheduler.EventLoggingListener: Logging events to hdfs://ip-10-0-0-199.ec2.internal:8020/user/spark/spark2ApplicationHistory/application_1501598591600_0003
17/08/01 14:49:23 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/08/01 14:49:27 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.229:37096) with ID 1
17/08/01 14:49:27 INFO spark.ExecutorAllocationManager: New executor 1 has registered (new total is 1)
17/08/01 14:49:27 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-229.ec2.internal:51770 with 1060.5 MB RAM, BlockManagerId(1, ip-10-0-0-229.ec2.internal, 51770, None)
17/08/01 14:49:27 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/08/01 14:49:27 INFO internal.SharedState: spark.sql.warehouse.dir is not set, but hive.metastore.warehouse.dir is set. Setting spark.sql.warehouse.dir to the value of hive.metastore.warehouse.dir ('/user/hive/warehouse').
17/08/01 14:49:27 INFO internal.SharedState: Warehouse path is '/user/hive/warehouse'.
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5554f0a0{/SQL,null,AVAILABLE}
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@44743601{/SQL/json,null,AVAILABLE}
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@144776d3{/SQL/execution,null,AVAILABLE}
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f3f8a4e{/SQL/execution/json,null,AVAILABLE}
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27946137{/static/sql,null,AVAILABLE}
17/08/01 14:49:27 INFO hive.HiveUtils: Initializing HiveMetastoreConnection version 1.1.0 using file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-logging-1.1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-exec-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-exec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-1.1.0-cdh5.12.0-standalone.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-standalone.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-metastore-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-metastore.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-serde-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-serde.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-service-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-service.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/libfb303-0.9.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/libthrift-0.9.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/log4j-1.2.16.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-hadoop-compat.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-hadoop2-compat.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-protocol.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-server.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/htrace-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ST4-4.0.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-core-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-fate-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-start-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-trace-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/activation-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ant-1.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ant-launcher-1.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/antlr-2.7.7.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/antlr-runtime-3.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/apache-log4j-extras-1.2.17.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-3.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-commons-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-tree-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/avro.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/bonecp-0.8.0.RELEASE.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-avatica-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-core-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-linq4j-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-beanutils-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-beanutils-core-1.8.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-cli-1.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-codec-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-collections-3.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-compiler-2.7.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-compress-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-configuration-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-dbcp-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-digester-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-el-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-httpclient-3.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-io-2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-lang-2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-lang3-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-math-2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-pool-1.5.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-vfs2-2.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-client-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-framework-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-recipes-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-api-jdo-3.2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-core-3.2.10.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-rdbms-3.2.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/derby-10.11.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/eigenbase-properties-1.1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/findbugs-annotations-1.3.9-1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-annotation_1.0_spec-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-jaspic_1.0_spec-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-jta_1.1_spec-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/groovy-all-2.4.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/gson-2.2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/guava-14.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hamcrest-core-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/high-scale-lib-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-accumulo-handler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-accumulo-handler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-ant-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-ant.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-beeline-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-beeline.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-cli-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-cli.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-common-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-contrib-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-contrib.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hbase-handler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hbase-handler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hwi-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hwi.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-0.23-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-0.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-common-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-scheduler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-scheduler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-testutils-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-testutils.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/httpclient-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/httpcore-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-core-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ivy-2.0.0-rc2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/parquet-hadoop-bundle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/stringtemplate-3.2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/regexp-1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/xz-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/tempus-fugit-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/super-csv-2.2.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/stax-api-1.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/servlet-api-2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/opencsv-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-jvm-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-json-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-core-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-provider-svnexe-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-provider-svn-commons-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-api-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/mail-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/logredactor-1.0.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/junit-4.11.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jta-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jsr305-3.0.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jsp-api-2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jpam-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/joda-time-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jline-2.12.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jetty-all-server-7.6.0.v20120127.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jetty-all-7.6.0.v20120127.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jersey-servlet-1.14.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jersey-server-1.14.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jdo-api-3.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jcommander-1.32.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jasper-runtime-5.5.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jasper-compiler-5.5.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/janino-2.7.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jamon-runtime-2.3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-xc-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-jaxrs-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-databind-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-annotations-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/zookeeper.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/velocity-1.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/snappy-java-1.0.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/plexus-utils-1.5.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/paranamer-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/oro-2.0.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/activation-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/activation.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-i18n-2.0.0-M15.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-i18n.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-kerberos-codec-2.0.0-M15.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-kerberos-codec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-asn1-api-1.0.0-M20.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-asn1-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-util-1.0.0-M20.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-util.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/avro.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/aws-java-sdk-bundle-1.11.134.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/aws-java-sdk-bundle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/azure-data-lake-store-sdk-2.1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/azure-data-lake-store-sdk.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-core-1.8.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-cli-1.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-cli.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-codec-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-codec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-collections-3.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-collections.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-compress-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-compress.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-configuration-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-configuration.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-digester-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-digester.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-httpclient-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-httpclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-io-2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-io.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-lang-2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-lang.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-logging-1.1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-logging.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-math3-3.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-math3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-net-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-net.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-client-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-framework-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-framework.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-recipes-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-recipes.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/gson-2.2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/gson.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/guava-11.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/guava.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-annotations-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-auth-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-auth.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-aws-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-aws.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-azure-datalake-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-azure-datalake.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-hdfs-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-hdfs.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-app-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-app.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-core-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-jobclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-shuffle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-api-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-client-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-server-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-server-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/htrace-core4-4.0.1-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/htrace-core4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpclient-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpcore-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpcore.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-annotations-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-core-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-databind-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-databind.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-jaxrs-1.8.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-jaxrs.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-xc-1.8.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-xc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jaxb-api-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jaxb-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-client-1.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-core-1.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/zookeeper.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xz.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xz-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xmlenc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xmlenc-0.52.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xml-apis.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xml-apis-1.3.04.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xercesImpl.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xercesImpl-2.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/stax-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/stax-api-1.0-2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/snappy-java.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/snappy-java-1.0.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-log4j12.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-api-1.7.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/servlet-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/servlet-api-2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/protobuf-java.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/protobuf-java-2.5.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/paranamer.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/paranamer-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/netty.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/netty-3.10.5.Final.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/log4j.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/log4j-1.2.17.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/leveldbjni-all.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/leveldbjni-all-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jsr305.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jsr305-3.0.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jetty-util.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jetty-util-6.1.26.cloudera.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-core.jar
17/08/01 14:49:28 INFO session.SessionState: Created local directory: /tmp/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2_resources
17/08/01 14:49:28 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2
17/08/01 14:49:28 INFO session.SessionState: Created local directory: /tmp/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2
17/08/01 14:49:28 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2/_tmp_space.db
17/08/01 14:49:28 INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
17/08/01 14:49:28 INFO client.HiveClientImpl: Warehouse location for Hive client (version 1.1.0) is /user/hive/warehouse
17/08/01 14:49:29 INFO hive.metastore: Trying to connect to metastore with URI thrift://ip-10-0-0-8.ec2.internal:9083
17/08/01 14:49:29 INFO hive.metastore: Opened a connection to metastore, current connections: 1
17/08/01 14:49:29 INFO hive.metastore: Connected to metastore.
17/08/01 14:49:29 WARN streaming.StreamingContext: Dynamic Allocation is enabled for this application. Enabling Dynamic allocation for Spark Streaming applications can cause data loss if Write Ahead Log is not enabled for non-replayable sources like Flume. See the programming guide for details on how to enable the Write Ahead Log.
17/08/01 14:49:29 INFO spark.SparkContext: Starting job: start at NativeMethodAccessorImpl.java:0
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Registering RDD 1 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Got job 0 (start at NativeMethodAccessorImpl.java:0) with 20 output partitions
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/08/01 14:49:29 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 3.1 KB, free 1060.5 MB)
17/08/01 14:49:29 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 1988.0 B, free 1060.5 MB)
17/08/01 14:49:29 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.0.0.199:53355 (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:49:29 INFO spark.SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Submitting 50 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO cluster.YarnScheduler: Adding task set 0.0 with 50 tasks
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, ip-10-0-0-229.ec2.internal, executor 1, partition 1, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 664 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, ip-10-0-0-229.ec2.internal, executor 1, partition 2, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 78 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3, ip-10-0-0-229.ec2.internal, executor 1, partition 3, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 68 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 0.0 (TID 4, ip-10-0-0-229.ec2.internal, executor 1, partition 4, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 65 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 0.0 (TID 5, ip-10-0-0-229.ec2.internal, executor 1, partition 5, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 59 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/50)
17/08/01 14:49:30 INFO spark.ExecutorAllocationManager: Requesting 1 new executor because tasks are backlogged (new desired total will be 2)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 0.0 (TID 6, ip-10-0-0-229.ec2.internal, executor 1, partition 6, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 0.0 (TID 7, ip-10-0-0-229.ec2.internal, executor 1, partition 7, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 60 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 0.0 (TID 8, ip-10-0-0-229.ec2.internal, executor 1, partition 8, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 0.0 (TID 7) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 0.0 (TID 9, ip-10-0-0-229.ec2.internal, executor 1, partition 9, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 0.0 (TID 8) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 0.0 (TID 10, ip-10-0-0-229.ec2.internal, executor 1, partition 10, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 0.0 (TID 9) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 0.0 (TID 11, ip-10-0-0-229.ec2.internal, executor 1, partition 11, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 0.0 (TID 10) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 0.0 (TID 12, ip-10-0-0-229.ec2.internal, executor 1, partition 12, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 0.0 (TID 11) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 0.0 (TID 13, ip-10-0-0-229.ec2.internal, executor 1, partition 13, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 0.0 (TID 12) in 60 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 0.0 (TID 14, ip-10-0-0-229.ec2.internal, executor 1, partition 14, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 0.0 (TID 13) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 0.0 (TID 15, ip-10-0-0-229.ec2.internal, executor 1, partition 15, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 0.0 (TID 14) in 51 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 0.0 (TID 16, ip-10-0-0-229.ec2.internal, executor 1, partition 16, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 0.0 (TID 15) in 63 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 0.0 (TID 17, ip-10-0-0-229.ec2.internal, executor 1, partition 17, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 0.0 (TID 16) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 0.0 (TID 18, ip-10-0-0-229.ec2.internal, executor 1, partition 18, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 0.0 (TID 17) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 0.0 (TID 19, ip-10-0-0-229.ec2.internal, executor 1, partition 19, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 0.0 (TID 18) in 56 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 0.0 (TID 20, ip-10-0-0-229.ec2.internal, executor 1, partition 20, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 0.0 (TID 19) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 0.0 (TID 21, ip-10-0-0-229.ec2.internal, executor 1, partition 21, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 0.0 (TID 20) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (21/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 0.0 (TID 22, ip-10-0-0-229.ec2.internal, executor 1, partition 22, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 0.0 (TID 21) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (22/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 0.0 (TID 23, ip-10-0-0-229.ec2.internal, executor 1, partition 23, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 0.0 (TID 22) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (23/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 0.0 (TID 24, ip-10-0-0-229.ec2.internal, executor 1, partition 24, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 0.0 (TID 23) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (24/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 0.0 (TID 25, ip-10-0-0-229.ec2.internal, executor 1, partition 25, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 0.0 (TID 24) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (25/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 0.0 (TID 26, ip-10-0-0-229.ec2.internal, executor 1, partition 26, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 0.0 (TID 25) in 46 ms on ip-10-0-0-229.ec2.internal (executor 1) (26/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 0.0 (TID 27, ip-10-0-0-229.ec2.internal, executor 1, partition 27, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 0.0 (TID 26) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (27/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 0.0 (TID 28, ip-10-0-0-229.ec2.internal, executor 1, partition 28, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 0.0 (TID 27) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (28/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 0.0 (TID 29, ip-10-0-0-229.ec2.internal, executor 1, partition 29, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 0.0 (TID 28) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (29/50)
17/08/01 14:49:31 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 4)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 0.0 (TID 30, ip-10-0-0-229.ec2.internal, executor 1, partition 30, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 0.0 (TID 29) in 42 ms on ip-10-0-0-229.ec2.internal (executor 1) (30/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 0.0 (TID 31, ip-10-0-0-229.ec2.internal, executor 1, partition 31, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 0.0 (TID 30) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (31/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 0.0 (TID 32, ip-10-0-0-229.ec2.internal, executor 1, partition 32, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 0.0 (TID 31) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (32/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 0.0 (TID 33, ip-10-0-0-229.ec2.internal, executor 1, partition 33, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 0.0 (TID 32) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (33/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 0.0 (TID 34, ip-10-0-0-229.ec2.internal, executor 1, partition 34, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 0.0 (TID 33) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (34/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 0.0 (TID 35, ip-10-0-0-229.ec2.internal, executor 1, partition 35, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 0.0 (TID 34) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (35/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 0.0 (TID 36, ip-10-0-0-229.ec2.internal, executor 1, partition 36, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 0.0 (TID 35) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (36/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 0.0 (TID 37, ip-10-0-0-229.ec2.internal, executor 1, partition 37, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 0.0 (TID 36) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (37/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 0.0 (TID 38, ip-10-0-0-229.ec2.internal, executor 1, partition 38, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 0.0 (TID 37) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (38/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 0.0 (TID 39, ip-10-0-0-229.ec2.internal, executor 1, partition 39, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 0.0 (TID 38) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (39/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 0.0 (TID 40, ip-10-0-0-229.ec2.internal, executor 1, partition 40, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 0.0 (TID 39) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (40/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 0.0 (TID 41, ip-10-0-0-229.ec2.internal, executor 1, partition 41, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 0.0 (TID 40) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (41/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 0.0 (TID 42, ip-10-0-0-229.ec2.internal, executor 1, partition 42, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 0.0 (TID 41) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (42/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 0.0 (TID 43, ip-10-0-0-229.ec2.internal, executor 1, partition 43, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 0.0 (TID 42) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (43/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 0.0 (TID 44, ip-10-0-0-229.ec2.internal, executor 1, partition 44, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 0.0 (TID 43) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (44/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 0.0 (TID 45, ip-10-0-0-229.ec2.internal, executor 1, partition 45, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 0.0 (TID 44) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (45/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 0.0 (TID 46, ip-10-0-0-229.ec2.internal, executor 1, partition 46, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 0.0 (TID 45) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (46/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 0.0 (TID 47, ip-10-0-0-229.ec2.internal, executor 1, partition 47, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 0.0 (TID 46) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (47/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 0.0 (TID 48, ip-10-0-0-229.ec2.internal, executor 1, partition 48, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 0.0 (TID 47) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (48/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 0.0 (TID 49, ip-10-0-0-229.ec2.internal, executor 1, partition 49, PROCESS_LOCAL, 6676 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 0.0 (TID 48) in 28 ms on ip-10-0-0-229.ec2.internal (executor 1) (49/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 0.0 (TID 49) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (50/50)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/08/01 14:49:32 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (start at NativeMethodAccessorImpl.java:0) finished in 2.517 s
17/08/01 14:49:32 INFO scheduler.DAGScheduler: looking for newly runnable stages
17/08/01 14:49:32 INFO scheduler.DAGScheduler: running: Set()
17/08/01 14:49:32 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 1)
17/08/01 14:49:32 INFO scheduler.DAGScheduler: failed: Set()
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/08/01 14:49:32 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.2 KB, free 1060.5 MB)
17/08/01 14:49:32 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 1956.0 B, free 1060.5 MB)
17/08/01 14:49:32 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.0.0.199:53355 (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:49:32 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Submitting 20 missing tasks from ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Adding task set 1.0 with 20 tasks
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 50, ip-10-0-0-229.ec2.internal, executor 1, partition 0, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:49:32 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.0.0.229:37096
17/08/01 14:49:32 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 283 bytes
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.0 (TID 51, ip-10-0-0-229.ec2.internal, executor 1, partition 1, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 50) in 89 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 1.0 (TID 52, ip-10-0-0-229.ec2.internal, executor 1, partition 2, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 1.0 (TID 51) in 23 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 1.0 (TID 53, ip-10-0-0-229.ec2.internal, executor 1, partition 3, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 1.0 (TID 52) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 1.0 (TID 54, ip-10-0-0-229.ec2.internal, executor 1, partition 4, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 1.0 (TID 53) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 1.0 (TID 55, ip-10-0-0-229.ec2.internal, executor 1, partition 5, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 1.0 (TID 54) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 1.0 (TID 56, ip-10-0-0-229.ec2.internal, executor 1, partition 6, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 1.0 (TID 55) in 25 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 1.0 (TID 57, ip-10-0-0-229.ec2.internal, executor 1, partition 7, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 1.0 (TID 56) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 1.0 (TID 58, ip-10-0-0-229.ec2.internal, executor 1, partition 8, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 1.0 (TID 57) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 1.0 (TID 59, ip-10-0-0-229.ec2.internal, executor 1, partition 9, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 1.0 (TID 58) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 1.0 (TID 60, ip-10-0-0-229.ec2.internal, executor 1, partition 10, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 1.0 (TID 59) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 1.0 (TID 61, ip-10-0-0-229.ec2.internal, executor 1, partition 11, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 1.0 (TID 60) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 1.0 (TID 62, ip-10-0-0-229.ec2.internal, executor 1, partition 12, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 1.0 (TID 61) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 1.0 (TID 63, ip-10-0-0-229.ec2.internal, executor 1, partition 13, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 1.0 (TID 62) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 1.0 (TID 64, ip-10-0-0-229.ec2.internal, executor 1, partition 14, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 1.0 (TID 63) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 1.0 (TID 65, ip-10-0-0-229.ec2.internal, executor 1, partition 15, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 1.0 (TID 64) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 1.0 (TID 66, ip-10-0-0-229.ec2.internal, executor 1, partition 16, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 1.0 (TID 65) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 1.0 (TID 67, ip-10-0-0-229.ec2.internal, executor 1, partition 17, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 1.0 (TID 66) in 22 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 1.0 (TID 68, ip-10-0-0-229.ec2.internal, executor 1, partition 18, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 1.0 (TID 67) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 1.0 (TID 69, ip-10-0-0-229.ec2.internal, executor 1, partition 19, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 1.0 (TID 68) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 1.0 (TID 69) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/20)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool
17/08/01 14:49:32 INFO scheduler.DAGScheduler: ResultStage 1 (start at NativeMethodAccessorImpl.java:0) finished in 0.420 s
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Job 0 finished: start at NativeMethodAccessorImpl.java:0, took 3.286639 s
17/08/01 14:49:32 INFO scheduler.ReceiverTracker: Starting 1 receivers
17/08/01 14:49:32 INFO scheduler.ReceiverTracker: ReceiverTracker started
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Initialized and validated org.apache.spark.streaming.kafka.KafkaInputDStream@71871773
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Initialized and validated org.apache.spark.streaming.api.python.PythonTransformedDStream@6593d619
17/08/01 14:49:32 INFO dstream.ForEachDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO dstream.ForEachDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO dstream.ForEachDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO dstream.ForEachDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO dstream.ForEachDStream: Initialized and validated org.apache.spark.streaming.dstream.ForEachDStream@26c8ea4b
17/08/01 14:49:33 INFO util.RecurringTimer: Started timer for JobGenerator at time 1501598973000
17/08/01 14:49:33 INFO scheduler.JobGenerator: Started JobGenerator at 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Started JobScheduler
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@625d3c2d{/streaming,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@48a256f9{/streaming/json,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@39edaf54{/streaming/batch,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@648a544d{/streaming/batch/json,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@c773c0d{/static/streaming,null,AVAILABLE}
17/08/01 14:49:33 INFO streaming.StreamingContext: StreamingContext started
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Got job 1 (start at NativeMethodAccessorImpl.java:0) with 1 output partitions
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Final stage: ResultStage 2 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Missing parents: List()
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Submitting ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620), which has no missing parents
17/08/01 14:49:33 INFO scheduler.ReceiverTracker: Receiver 0 started
17/08/01 14:49:33 INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 80.8 KB, free 1060.4 MB)
17/08/01 14:49:33 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 29.1 KB, free 1060.4 MB)
17/08/01 14:49:33 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.0.0.199:53355 (size: 29.1 KB, free: 1060.5 MB)
17/08/01 14:49:33 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620)
17/08/01 14:49:33 INFO cluster.YarnScheduler: Adding task set 2.0 with 1 tasks
17/08/01 14:49:33 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 2.0 (TID 70, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 7067 bytes)
17/08/01 14:49:33 INFO scheduler.JobScheduler: Added jobs for time 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Starting job streaming job 1501598973000 ms.0 from job set of time 1501598973000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:33 INFO scheduler.JobScheduler: Finished job streaming job 1501598973000 ms.0 from job set of time 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Total delay: 0.102 s for time 1501598973000 ms (execution: 0.026 s)
17/08/01 14:49:33 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 29.1 KB, free: 1060.5 MB)
17/08/01 14:49:33 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/08/01 14:49:33 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/08/01 14:49:33 INFO scheduler.ReceiverTracker: Registered receiver for stream 0 from 10.0.0.229:37096
17/08/01 14:49:34 INFO scheduler.JobScheduler: Added jobs for time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.JobScheduler: Starting job streaming job 1501598974000 ms.0 from job set of time 1501598974000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:34 INFO scheduler.JobScheduler: Finished job streaming job 1501598974000 ms.0 from job set of time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598974000 ms (execution: 0.009 s)
17/08/01 14:49:34 INFO python.PythonRDD: Removing RDD 5 from persistence list
17/08/01 14:49:34 INFO storage.BlockManager: Removing RDD 5
17/08/01 14:49:34 INFO rdd.BlockRDD: Removing RDD 4 from persistence list
17/08/01 14:49:34 INFO storage.BlockManager: Removing RDD 4
17/08/01 14:49:34 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[4] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/08/01 14:49:34 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/08/01 14:49:34 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.127:44695) with ID 2
17/08/01 14:49:34 INFO spark.ExecutorAllocationManager: New executor 2 has registered (new total is 2)
17/08/01 14:49:34 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-127.ec2.internal:59267 with 1060.5 MB RAM, BlockManagerId(2, ip-10-0-0-127.ec2.internal, 59267, None)
17/08/01 14:49:35 INFO scheduler.JobScheduler: Added jobs for time 1501598975000 ms
17/08/01 14:49:35 INFO scheduler.JobScheduler: Starting job streaming job 1501598975000 ms.0 from job set of time 1501598975000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:35 INFO scheduler.JobScheduler: Finished job streaming job 1501598975000 ms.0 from job set of time 1501598975000 ms
17/08/01 14:49:35 INFO python.PythonRDD: Removing RDD 7 from persistence list
17/08/01 14:49:35 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501598975000 ms (execution: 0.009 s)
17/08/01 14:49:35 INFO rdd.BlockRDD: Removing RDD 6 from persistence list
17/08/01 14:49:35 INFO storage.BlockManager: Removing RDD 6
17/08/01 14:49:35 INFO storage.BlockManager: Removing RDD 7
17/08/01 14:49:35 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[6] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598975000 ms
17/08/01 14:49:35 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598973000 ms
17/08/01 14:49:35 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598973000 ms
17/08/01 14:49:36 INFO scheduler.JobScheduler: Added jobs for time 1501598976000 ms
17/08/01 14:49:36 INFO scheduler.JobScheduler: Starting job streaming job 1501598976000 ms.0 from job set of time 1501598976000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:36 INFO scheduler.JobScheduler: Finished job streaming job 1501598976000 ms.0 from job set of time 1501598976000 ms
17/08/01 14:49:36 INFO python.PythonRDD: Removing RDD 9 from persistence list
17/08/01 14:49:36 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501598976000 ms (execution: 0.008 s)
17/08/01 14:49:36 INFO storage.BlockManager: Removing RDD 9
17/08/01 14:49:36 INFO rdd.BlockRDD: Removing RDD 8 from persistence list
17/08/01 14:49:36 INFO storage.BlockManager: Removing RDD 8
17/08/01 14:49:36 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[8] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598976000 ms
17/08/01 14:49:36 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598974000 ms
17/08/01 14:49:36 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598974000 ms
17/08/01 14:49:37 INFO scheduler.JobScheduler: Added jobs for time 1501598977000 ms
17/08/01 14:49:37 INFO scheduler.JobScheduler: Starting job streaming job 1501598977000 ms.0 from job set of time 1501598977000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:37 INFO scheduler.JobScheduler: Finished job streaming job 1501598977000 ms.0 from job set of time 1501598977000 ms
17/08/01 14:49:37 INFO python.PythonRDD: Removing RDD 11 from persistence list
17/08/01 14:49:37 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598977000 ms (execution: 0.008 s)
17/08/01 14:49:37 INFO storage.BlockManager: Removing RDD 11
17/08/01 14:49:37 INFO rdd.BlockRDD: Removing RDD 10 from persistence list
17/08/01 14:49:37 INFO storage.BlockManager: Removing RDD 10
17/08/01 14:49:37 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[10] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598977000 ms
17/08/01 14:49:37 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598975000 ms
17/08/01 14:49:37 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598975000 ms
17/08/01 14:49:38 INFO scheduler.JobScheduler: Added jobs for time 1501598978000 ms
17/08/01 14:49:38 INFO scheduler.JobScheduler: Starting job streaming job 1501598978000 ms.0 from job set of time 1501598978000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:38 INFO scheduler.JobScheduler: Finished job streaming job 1501598978000 ms.0 from job set of time 1501598978000 ms
17/08/01 14:49:38 INFO python.PythonRDD: Removing RDD 13 from persistence list
17/08/01 14:49:38 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501598978000 ms (execution: 0.007 s)
17/08/01 14:49:38 INFO storage.BlockManager: Removing RDD 13
17/08/01 14:49:38 INFO rdd.BlockRDD: Removing RDD 12 from persistence list
17/08/01 14:49:38 INFO storage.BlockManager: Removing RDD 12
17/08/01 14:49:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[12] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598978000 ms
17/08/01 14:49:38 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598976000 ms
17/08/01 14:49:38 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598976000 ms
17/08/01 14:49:39 INFO scheduler.JobScheduler: Added jobs for time 1501598979000 ms
17/08/01 14:49:39 INFO scheduler.JobScheduler: Starting job streaming job 1501598979000 ms.0 from job set of time 1501598979000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:39 INFO scheduler.JobScheduler: Finished job streaming job 1501598979000 ms.0 from job set of time 1501598979000 ms
17/08/01 14:49:39 INFO python.PythonRDD: Removing RDD 15 from persistence list
17/08/01 14:49:39 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598979000 ms (execution: 0.007 s)
17/08/01 14:49:39 INFO storage.BlockManager: Removing RDD 15
17/08/01 14:49:39 INFO rdd.BlockRDD: Removing RDD 14 from persistence list
17/08/01 14:49:39 INFO storage.BlockManager: Removing RDD 14
17/08/01 14:49:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[14] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598979000 ms
17/08/01 14:49:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598977000 ms
17/08/01 14:49:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598977000 ms
17/08/01 14:49:40 INFO scheduler.JobScheduler: Added jobs for time 1501598980000 ms
17/08/01 14:49:40 INFO scheduler.JobScheduler: Starting job streaming job 1501598980000 ms.0 from job set of time 1501598980000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:40 INFO scheduler.JobScheduler: Finished job streaming job 1501598980000 ms.0 from job set of time 1501598980000 ms
17/08/01 14:49:40 INFO python.PythonRDD: Removing RDD 17 from persistence list
17/08/01 14:49:40 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598980000 ms (execution: 0.015 s)
17/08/01 14:49:40 INFO storage.BlockManager: Removing RDD 17
17/08/01 14:49:40 INFO rdd.BlockRDD: Removing RDD 16 from persistence list
17/08/01 14:49:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[16] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598980000 ms
17/08/01 14:49:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598978000 ms
17/08/01 14:49:40 INFO storage.BlockManager: Removing RDD 16
17/08/01 14:49:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598978000 ms
17/08/01 14:49:41 INFO scheduler.JobScheduler: Added jobs for time 1501598981000 ms
17/08/01 14:49:41 INFO scheduler.JobScheduler: Starting job streaming job 1501598981000 ms.0 from job set of time 1501598981000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:41 INFO scheduler.JobScheduler: Finished job streaming job 1501598981000 ms.0 from job set of time 1501598981000 ms
17/08/01 14:49:41 INFO python.PythonRDD: Removing RDD 19 from persistence list
17/08/01 14:49:41 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501598981000 ms (execution: 0.006 s)
17/08/01 14:49:41 INFO storage.BlockManager: Removing RDD 19
17/08/01 14:49:41 INFO rdd.BlockRDD: Removing RDD 18 from persistence list
17/08/01 14:49:41 INFO storage.BlockManager: Removing RDD 18
17/08/01 14:49:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[18] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598981000 ms
17/08/01 14:49:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598979000 ms
17/08/01 14:49:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598979000 ms
17/08/01 14:49:42 INFO scheduler.JobScheduler: Added jobs for time 1501598982000 ms
17/08/01 14:49:42 INFO scheduler.JobScheduler: Starting job streaming job 1501598982000 ms.0 from job set of time 1501598982000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:42 INFO scheduler.JobScheduler: Finished job streaming job 1501598982000 ms.0 from job set of time 1501598982000 ms
17/08/01 14:49:42 INFO python.PythonRDD: Removing RDD 21 from persistence list
17/08/01 14:49:42 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598982000 ms (execution: 0.008 s)
17/08/01 14:49:42 INFO storage.BlockManager: Removing RDD 21
17/08/01 14:49:42 INFO rdd.BlockRDD: Removing RDD 20 from persistence list
17/08/01 14:49:42 INFO storage.BlockManager: Removing RDD 20
17/08/01 14:49:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[20] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598982000 ms
17/08/01 14:49:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598980000 ms
17/08/01 14:49:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598980000 ms
17/08/01 14:49:43 INFO scheduler.JobScheduler: Added jobs for time 1501598983000 ms
17/08/01 14:49:43 INFO scheduler.JobScheduler: Starting job streaming job 1501598983000 ms.0 from job set of time 1501598983000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:43 INFO scheduler.JobScheduler: Finished job streaming job 1501598983000 ms.0 from job set of time 1501598983000 ms
17/08/01 14:49:43 INFO python.PythonRDD: Removing RDD 23 from persistence list
17/08/01 14:49:43 INFO scheduler.JobScheduler: Total delay: 0.036 s for time 1501598983000 ms (execution: 0.009 s)
17/08/01 14:49:43 INFO storage.BlockManager: Removing RDD 23
17/08/01 14:49:43 INFO rdd.BlockRDD: Removing RDD 22 from persistence list
17/08/01 14:49:43 INFO storage.BlockManager: Removing RDD 22
17/08/01 14:49:43 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[22] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598983000 ms
17/08/01 14:49:43 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598981000 ms
17/08/01 14:49:43 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598981000 ms
17/08/01 14:49:44 INFO scheduler.JobScheduler: Added jobs for time 1501598984000 ms
17/08/01 14:49:44 INFO scheduler.JobScheduler: Starting job streaming job 1501598984000 ms.0 from job set of time 1501598984000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:44 INFO scheduler.JobScheduler: Finished job streaming job 1501598984000 ms.0 from job set of time 1501598984000 ms
17/08/01 14:49:44 INFO python.PythonRDD: Removing RDD 25 from persistence list
17/08/01 14:49:44 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598984000 ms (execution: 0.007 s)
17/08/01 14:49:44 INFO storage.BlockManager: Removing RDD 25
17/08/01 14:49:44 INFO rdd.BlockRDD: Removing RDD 24 from persistence list
17/08/01 14:49:44 INFO storage.BlockManager: Removing RDD 24
17/08/01 14:49:44 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[24] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598984000 ms
17/08/01 14:49:44 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598982000 ms
17/08/01 14:49:44 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598982000 ms
17/08/01 14:49:45 INFO scheduler.JobScheduler: Added jobs for time 1501598985000 ms
17/08/01 14:49:45 INFO scheduler.JobScheduler: Starting job streaming job 1501598985000 ms.0 from job set of time 1501598985000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:45 INFO scheduler.JobScheduler: Finished job streaming job 1501598985000 ms.0 from job set of time 1501598985000 ms
17/08/01 14:49:45 INFO python.PythonRDD: Removing RDD 27 from persistence list
17/08/01 14:49:45 INFO scheduler.JobScheduler: Total delay: 0.033 s for time 1501598985000 ms (execution: 0.007 s)
17/08/01 14:49:45 INFO storage.BlockManager: Removing RDD 27
17/08/01 14:49:45 INFO rdd.BlockRDD: Removing RDD 26 from persistence list
17/08/01 14:49:45 INFO storage.BlockManager: Removing RDD 26
17/08/01 14:49:45 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[26] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598985000 ms
17/08/01 14:49:45 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598983000 ms
17/08/01 14:49:45 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598983000 ms
17/08/01 14:49:46 INFO scheduler.JobScheduler: Added jobs for time 1501598986000 ms
17/08/01 14:49:46 INFO scheduler.JobScheduler: Starting job streaming job 1501598986000 ms.0 from job set of time 1501598986000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:46 INFO scheduler.JobScheduler: Finished job streaming job 1501598986000 ms.0 from job set of time 1501598986000 ms
17/08/01 14:49:46 INFO python.PythonRDD: Removing RDD 29 from persistence list
17/08/01 14:49:46 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598986000 ms (execution: 0.008 s)
17/08/01 14:49:46 INFO storage.BlockManager: Removing RDD 29
17/08/01 14:49:46 INFO rdd.BlockRDD: Removing RDD 28 from persistence list
17/08/01 14:49:46 INFO storage.BlockManager: Removing RDD 28
17/08/01 14:49:46 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[28] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598986000 ms
17/08/01 14:49:46 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598984000 ms
17/08/01 14:49:46 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598984000 ms
17/08/01 14:49:47 INFO scheduler.JobScheduler: Added jobs for time 1501598987000 ms
17/08/01 14:49:47 INFO scheduler.JobScheduler: Starting job streaming job 1501598987000 ms.0 from job set of time 1501598987000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:47 INFO scheduler.JobScheduler: Finished job streaming job 1501598987000 ms.0 from job set of time 1501598987000 ms
17/08/01 14:49:47 INFO python.PythonRDD: Removing RDD 31 from persistence list
17/08/01 14:49:47 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598987000 ms (execution: 0.014 s)
17/08/01 14:49:47 INFO storage.BlockManager: Removing RDD 31
17/08/01 14:49:47 INFO rdd.BlockRDD: Removing RDD 30 from persistence list
17/08/01 14:49:47 INFO storage.BlockManager: Removing RDD 30
17/08/01 14:49:47 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[30] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598987000 ms
17/08/01 14:49:47 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598985000 ms
17/08/01 14:49:47 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598985000 ms
17/08/01 14:49:48 INFO scheduler.JobScheduler: Added jobs for time 1501598988000 ms
17/08/01 14:49:48 INFO scheduler.JobScheduler: Starting job streaming job 1501598988000 ms.0 from job set of time 1501598988000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:48 INFO scheduler.JobScheduler: Finished job streaming job 1501598988000 ms.0 from job set of time 1501598988000 ms
17/08/01 14:49:48 INFO python.PythonRDD: Removing RDD 33 from persistence list
17/08/01 14:49:48 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501598988000 ms (execution: 0.008 s)
17/08/01 14:49:48 INFO storage.BlockManager: Removing RDD 33
17/08/01 14:49:48 INFO rdd.BlockRDD: Removing RDD 32 from persistence list
17/08/01 14:49:48 INFO storage.BlockManager: Removing RDD 32
17/08/01 14:49:48 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[32] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598988000 ms
17/08/01 14:49:48 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598986000 ms
17/08/01 14:49:48 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598986000 ms
17/08/01 14:49:49 INFO scheduler.JobScheduler: Added jobs for time 1501598989000 ms
17/08/01 14:49:49 INFO scheduler.JobScheduler: Starting job streaming job 1501598989000 ms.0 from job set of time 1501598989000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:49 INFO scheduler.JobScheduler: Finished job streaming job 1501598989000 ms.0 from job set of time 1501598989000 ms
17/08/01 14:49:49 INFO python.PythonRDD: Removing RDD 35 from persistence list
17/08/01 14:49:49 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501598989000 ms (execution: 0.006 s)
17/08/01 14:49:49 INFO storage.BlockManager: Removing RDD 35
17/08/01 14:49:49 INFO rdd.BlockRDD: Removing RDD 34 from persistence list
17/08/01 14:49:49 INFO storage.BlockManager: Removing RDD 34
17/08/01 14:49:49 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[34] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598989000 ms
17/08/01 14:49:49 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598987000 ms
17/08/01 14:49:49 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598987000 ms
17/08/01 14:49:50 INFO scheduler.JobScheduler: Added jobs for time 1501598990000 ms
17/08/01 14:49:50 INFO scheduler.JobScheduler: Starting job streaming job 1501598990000 ms.0 from job set of time 1501598990000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:50 INFO scheduler.JobScheduler: Finished job streaming job 1501598990000 ms.0 from job set of time 1501598990000 ms
17/08/01 14:49:50 INFO python.PythonRDD: Removing RDD 37 from persistence list
17/08/01 14:49:50 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501598990000 ms (execution: 0.008 s)
17/08/01 14:49:50 INFO storage.BlockManager: Removing RDD 37
17/08/01 14:49:50 INFO rdd.BlockRDD: Removing RDD 36 from persistence list
17/08/01 14:49:50 INFO storage.BlockManager: Removing RDD 36
17/08/01 14:49:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[36] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598990000 ms
17/08/01 14:49:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598988000 ms
17/08/01 14:49:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598988000 ms
17/08/01 14:49:51 INFO scheduler.JobScheduler: Added jobs for time 1501598991000 ms
17/08/01 14:49:51 INFO scheduler.JobScheduler: Starting job streaming job 1501598991000 ms.0 from job set of time 1501598991000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:51 INFO scheduler.JobScheduler: Finished job streaming job 1501598991000 ms.0 from job set of time 1501598991000 ms
17/08/01 14:49:51 INFO python.PythonRDD: Removing RDD 39 from persistence list
17/08/01 14:49:51 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501598991000 ms (execution: 0.006 s)
17/08/01 14:49:51 INFO storage.BlockManager: Removing RDD 39
17/08/01 14:49:51 INFO rdd.BlockRDD: Removing RDD 38 from persistence list
17/08/01 14:49:51 INFO storage.BlockManager: Removing RDD 38
17/08/01 14:49:51 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[38] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598991000 ms
17/08/01 14:49:51 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598989000 ms
17/08/01 14:49:51 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598989000 ms
17/08/01 14:49:52 INFO scheduler.JobScheduler: Added jobs for time 1501598992000 ms
17/08/01 14:49:52 INFO scheduler.JobScheduler: Starting job streaming job 1501598992000 ms.0 from job set of time 1501598992000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:52 INFO scheduler.JobScheduler: Finished job streaming job 1501598992000 ms.0 from job set of time 1501598992000 ms
17/08/01 14:49:52 INFO python.PythonRDD: Removing RDD 41 from persistence list
17/08/01 14:49:52 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501598992000 ms (execution: 0.007 s)
17/08/01 14:49:52 INFO storage.BlockManager: Removing RDD 41
17/08/01 14:49:52 INFO rdd.BlockRDD: Removing RDD 40 from persistence list
17/08/01 14:49:52 INFO storage.BlockManager: Removing RDD 40
17/08/01 14:49:52 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[40] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598992000 ms
17/08/01 14:49:52 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598990000 ms
17/08/01 14:49:52 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598990000 ms
17/08/01 14:49:53 INFO scheduler.JobScheduler: Added jobs for time 1501598993000 ms
17/08/01 14:49:53 INFO scheduler.JobScheduler: Starting job streaming job 1501598993000 ms.0 from job set of time 1501598993000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:53 INFO scheduler.JobScheduler: Finished job streaming job 1501598993000 ms.0 from job set of time 1501598993000 ms
17/08/01 14:49:53 INFO python.PythonRDD: Removing RDD 43 from persistence list
17/08/01 14:49:53 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598993000 ms (execution: 0.008 s)
17/08/01 14:49:53 INFO storage.BlockManager: Removing RDD 43
17/08/01 14:49:53 INFO rdd.BlockRDD: Removing RDD 42 from persistence list
17/08/01 14:49:53 INFO storage.BlockManager: Removing RDD 42
17/08/01 14:49:53 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[42] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598993000 ms
17/08/01 14:49:53 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598991000 ms
17/08/01 14:49:53 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598991000 ms
17/08/01 14:49:54 INFO scheduler.JobScheduler: Added jobs for time 1501598994000 ms
17/08/01 14:49:54 INFO scheduler.JobScheduler: Starting job streaming job 1501598994000 ms.0 from job set of time 1501598994000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:54 INFO scheduler.JobScheduler: Finished job streaming job 1501598994000 ms.0 from job set of time 1501598994000 ms
17/08/01 14:49:54 INFO python.PythonRDD: Removing RDD 45 from persistence list
17/08/01 14:49:54 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501598994000 ms (execution: 0.006 s)
17/08/01 14:49:54 INFO storage.BlockManager: Removing RDD 45
17/08/01 14:49:54 INFO rdd.BlockRDD: Removing RDD 44 from persistence list
17/08/01 14:49:54 INFO storage.BlockManager: Removing RDD 44
17/08/01 14:49:54 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[44] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598994000 ms
17/08/01 14:49:54 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598992000 ms
17/08/01 14:49:54 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598992000 ms
17/08/01 14:49:55 INFO scheduler.JobScheduler: Added jobs for time 1501598995000 ms
17/08/01 14:49:55 INFO scheduler.JobScheduler: Starting job streaming job 1501598995000 ms.0 from job set of time 1501598995000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:55 INFO scheduler.JobScheduler: Finished job streaming job 1501598995000 ms.0 from job set of time 1501598995000 ms
17/08/01 14:49:55 INFO python.PythonRDD: Removing RDD 47 from persistence list
17/08/01 14:49:55 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598995000 ms (execution: 0.008 s)
17/08/01 14:49:55 INFO storage.BlockManager: Removing RDD 47
17/08/01 14:49:55 INFO rdd.BlockRDD: Removing RDD 46 from persistence list
17/08/01 14:49:55 INFO storage.BlockManager: Removing RDD 46
17/08/01 14:49:55 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[46] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598995000 ms
17/08/01 14:49:55 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598993000 ms
17/08/01 14:49:55 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598993000 ms
17/08/01 14:49:56 INFO scheduler.JobScheduler: Added jobs for time 1501598996000 ms
17/08/01 14:49:56 INFO scheduler.JobScheduler: Starting job streaming job 1501598996000 ms.0 from job set of time 1501598996000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:56 INFO scheduler.JobScheduler: Finished job streaming job 1501598996000 ms.0 from job set of time 1501598996000 ms
17/08/01 14:49:56 INFO python.PythonRDD: Removing RDD 49 from persistence list
17/08/01 14:49:56 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501598996000 ms (execution: 0.020 s)
17/08/01 14:49:56 INFO storage.BlockManager: Removing RDD 49
17/08/01 14:49:56 INFO rdd.BlockRDD: Removing RDD 48 from persistence list
17/08/01 14:49:56 INFO storage.BlockManager: Removing RDD 48
17/08/01 14:49:56 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[48] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598996000 ms
17/08/01 14:49:56 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598994000 ms
17/08/01 14:49:56 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598994000 ms
17/08/01 14:49:57 INFO scheduler.JobScheduler: Added jobs for time 1501598997000 ms
17/08/01 14:49:57 INFO scheduler.JobScheduler: Starting job streaming job 1501598997000 ms.0 from job set of time 1501598997000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:57 INFO scheduler.JobScheduler: Finished job streaming job 1501598997000 ms.0 from job set of time 1501598997000 ms
17/08/01 14:49:57 INFO python.PythonRDD: Removing RDD 51 from persistence list
17/08/01 14:49:57 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598997000 ms (execution: 0.007 s)
17/08/01 14:49:57 INFO storage.BlockManager: Removing RDD 51
17/08/01 14:49:57 INFO rdd.BlockRDD: Removing RDD 50 from persistence list
17/08/01 14:49:57 INFO storage.BlockManager: Removing RDD 50
17/08/01 14:49:57 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[50] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598997000 ms
17/08/01 14:49:57 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598995000 ms
17/08/01 14:49:57 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598995000 ms
17/08/01 14:49:58 INFO scheduler.JobScheduler: Added jobs for time 1501598998000 ms
17/08/01 14:49:58 INFO scheduler.JobScheduler: Starting job streaming job 1501598998000 ms.0 from job set of time 1501598998000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:58 INFO scheduler.JobScheduler: Finished job streaming job 1501598998000 ms.0 from job set of time 1501598998000 ms
17/08/01 14:49:58 INFO python.PythonRDD: Removing RDD 53 from persistence list
17/08/01 14:49:58 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501598998000 ms (execution: 0.008 s)
17/08/01 14:49:58 INFO storage.BlockManager: Removing RDD 53
17/08/01 14:49:58 INFO rdd.BlockRDD: Removing RDD 52 from persistence list
17/08/01 14:49:58 INFO storage.BlockManager: Removing RDD 52
17/08/01 14:49:58 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[52] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598998000 ms
17/08/01 14:49:58 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598996000 ms
17/08/01 14:49:58 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598996000 ms
17/08/01 14:49:59 INFO scheduler.JobScheduler: Added jobs for time 1501598999000 ms
17/08/01 14:49:59 INFO scheduler.JobScheduler: Starting job streaming job 1501598999000 ms.0 from job set of time 1501598999000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:59 INFO scheduler.JobScheduler: Finished job streaming job 1501598999000 ms.0 from job set of time 1501598999000 ms
17/08/01 14:49:59 INFO python.PythonRDD: Removing RDD 55 from persistence list
17/08/01 14:49:59 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598999000 ms (execution: 0.007 s)
17/08/01 14:49:59 INFO storage.BlockManager: Removing RDD 55
17/08/01 14:49:59 INFO rdd.BlockRDD: Removing RDD 54 from persistence list
17/08/01 14:49:59 INFO storage.BlockManager: Removing RDD 54
17/08/01 14:49:59 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[54] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598999000 ms
17/08/01 14:49:59 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598997000 ms
17/08/01 14:49:59 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598997000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Added jobs for time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Starting job streaming job 1501599000000 ms.0 from job set of time 1501599000000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:00 INFO scheduler.JobScheduler: Finished job streaming job 1501599000000 ms.0 from job set of time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599000000 ms (execution: 0.006 s)
17/08/01 14:50:00 INFO python.PythonRDD: Removing RDD 57 from persistence list
17/08/01 14:50:00 INFO storage.BlockManager: Removing RDD 57
17/08/01 14:50:00 INFO rdd.BlockRDD: Removing RDD 56 from persistence list
17/08/01 14:50:00 INFO storage.BlockManager: Removing RDD 56
17/08/01 14:50:00 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[56] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598998000 ms
17/08/01 14:50:00 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598998000 ms
17/08/01 14:50:01 INFO scheduler.JobScheduler: Added jobs for time 1501599001000 ms
17/08/01 14:50:01 INFO scheduler.JobScheduler: Starting job streaming job 1501599001000 ms.0 from job set of time 1501599001000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:01 INFO scheduler.JobScheduler: Finished job streaming job 1501599001000 ms.0 from job set of time 1501599001000 ms
17/08/01 14:50:01 INFO python.PythonRDD: Removing RDD 59 from persistence list
17/08/01 14:50:01 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599001000 ms (execution: 0.007 s)
17/08/01 14:50:01 INFO storage.BlockManager: Removing RDD 59
17/08/01 14:50:01 INFO rdd.BlockRDD: Removing RDD 58 from persistence list
17/08/01 14:50:01 INFO storage.BlockManager: Removing RDD 58
17/08/01 14:50:01 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[58] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599001000 ms
17/08/01 14:50:01 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598999000 ms
17/08/01 14:50:01 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598999000 ms
17/08/01 14:50:02 INFO scheduler.JobScheduler: Added jobs for time 1501599002000 ms
17/08/01 14:50:02 INFO scheduler.JobScheduler: Starting job streaming job 1501599002000 ms.0 from job set of time 1501599002000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:02 INFO scheduler.JobScheduler: Finished job streaming job 1501599002000 ms.0 from job set of time 1501599002000 ms
17/08/01 14:50:02 INFO python.PythonRDD: Removing RDD 61 from persistence list
17/08/01 14:50:02 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599002000 ms (execution: 0.006 s)
17/08/01 14:50:02 INFO storage.BlockManager: Removing RDD 61
17/08/01 14:50:02 INFO rdd.BlockRDD: Removing RDD 60 from persistence list
17/08/01 14:50:02 INFO storage.BlockManager: Removing RDD 60
17/08/01 14:50:02 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[60] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599002000 ms
17/08/01 14:50:02 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599000000 ms
17/08/01 14:50:02 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599000000 ms
17/08/01 14:50:03 INFO scheduler.JobScheduler: Added jobs for time 1501599003000 ms
17/08/01 14:50:03 INFO scheduler.JobScheduler: Starting job streaming job 1501599003000 ms.0 from job set of time 1501599003000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:03 INFO scheduler.JobScheduler: Finished job streaming job 1501599003000 ms.0 from job set of time 1501599003000 ms
17/08/01 14:50:03 INFO python.PythonRDD: Removing RDD 63 from persistence list
17/08/01 14:50:03 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599003000 ms (execution: 0.007 s)
17/08/01 14:50:03 INFO storage.BlockManager: Removing RDD 63
17/08/01 14:50:03 INFO rdd.BlockRDD: Removing RDD 62 from persistence list
17/08/01 14:50:03 INFO storage.BlockManager: Removing RDD 62
17/08/01 14:50:03 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[62] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599003000 ms
17/08/01 14:50:03 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599001000 ms
17/08/01 14:50:03 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599001000 ms
17/08/01 14:50:04 INFO scheduler.JobScheduler: Added jobs for time 1501599004000 ms
17/08/01 14:50:04 INFO scheduler.JobScheduler: Starting job streaming job 1501599004000 ms.0 from job set of time 1501599004000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:04 INFO scheduler.JobScheduler: Finished job streaming job 1501599004000 ms.0 from job set of time 1501599004000 ms
17/08/01 14:50:04 INFO python.PythonRDD: Removing RDD 65 from persistence list
17/08/01 14:50:04 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599004000 ms (execution: 0.006 s)
17/08/01 14:50:04 INFO storage.BlockManager: Removing RDD 65
17/08/01 14:50:04 INFO rdd.BlockRDD: Removing RDD 64 from persistence list
17/08/01 14:50:04 INFO storage.BlockManager: Removing RDD 64
17/08/01 14:50:04 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[64] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599004000 ms
17/08/01 14:50:04 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599002000 ms
17/08/01 14:50:04 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599002000 ms
17/08/01 14:50:05 INFO scheduler.JobScheduler: Added jobs for time 1501599005000 ms
17/08/01 14:50:05 INFO scheduler.JobScheduler: Starting job streaming job 1501599005000 ms.0 from job set of time 1501599005000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:05 INFO scheduler.JobScheduler: Finished job streaming job 1501599005000 ms.0 from job set of time 1501599005000 ms
17/08/01 14:50:05 INFO python.PythonRDD: Removing RDD 67 from persistence list
17/08/01 14:50:05 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501599005000 ms (execution: 0.013 s)
17/08/01 14:50:05 INFO storage.BlockManager: Removing RDD 67
17/08/01 14:50:05 INFO rdd.BlockRDD: Removing RDD 66 from persistence list
17/08/01 14:50:05 INFO storage.BlockManager: Removing RDD 66
17/08/01 14:50:05 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[66] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599005000 ms
17/08/01 14:50:05 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599003000 ms
17/08/01 14:50:05 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599003000 ms
17/08/01 14:50:06 INFO scheduler.JobScheduler: Added jobs for time 1501599006000 ms
17/08/01 14:50:06 INFO scheduler.JobScheduler: Starting job streaming job 1501599006000 ms.0 from job set of time 1501599006000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:06 INFO scheduler.JobScheduler: Finished job streaming job 1501599006000 ms.0 from job set of time 1501599006000 ms
17/08/01 14:50:06 INFO python.PythonRDD: Removing RDD 69 from persistence list
17/08/01 14:50:06 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599006000 ms (execution: 0.008 s)
17/08/01 14:50:06 INFO storage.BlockManager: Removing RDD 69
17/08/01 14:50:06 INFO rdd.BlockRDD: Removing RDD 68 from persistence list
17/08/01 14:50:06 INFO storage.BlockManager: Removing RDD 68
17/08/01 14:50:06 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[68] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599006000 ms
17/08/01 14:50:06 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599004000 ms
17/08/01 14:50:06 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599004000 ms
17/08/01 14:50:07 INFO scheduler.JobScheduler: Added jobs for time 1501599007000 ms
17/08/01 14:50:07 INFO scheduler.JobScheduler: Starting job streaming job 1501599007000 ms.0 from job set of time 1501599007000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:07 INFO scheduler.JobScheduler: Finished job streaming job 1501599007000 ms.0 from job set of time 1501599007000 ms
17/08/01 14:50:07 INFO python.PythonRDD: Removing RDD 71 from persistence list
17/08/01 14:50:07 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599007000 ms (execution: 0.009 s)
17/08/01 14:50:07 INFO rdd.BlockRDD: Removing RDD 70 from persistence list
17/08/01 14:50:07 INFO storage.BlockManager: Removing RDD 71
17/08/01 14:50:07 INFO storage.BlockManager: Removing RDD 70
17/08/01 14:50:07 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[70] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599007000 ms
17/08/01 14:50:07 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599005000 ms
17/08/01 14:50:07 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599005000 ms
17/08/01 14:50:08 INFO scheduler.JobScheduler: Added jobs for time 1501599008000 ms
17/08/01 14:50:08 INFO scheduler.JobScheduler: Starting job streaming job 1501599008000 ms.0 from job set of time 1501599008000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:08 INFO scheduler.JobScheduler: Finished job streaming job 1501599008000 ms.0 from job set of time 1501599008000 ms
17/08/01 14:50:08 INFO python.PythonRDD: Removing RDD 73 from persistence list
17/08/01 14:50:08 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599008000 ms (execution: 0.006 s)
17/08/01 14:50:08 INFO storage.BlockManager: Removing RDD 73
17/08/01 14:50:08 INFO rdd.BlockRDD: Removing RDD 72 from persistence list
17/08/01 14:50:08 INFO storage.BlockManager: Removing RDD 72
17/08/01 14:50:08 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[72] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599008000 ms
17/08/01 14:50:08 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599006000 ms
17/08/01 14:50:08 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599006000 ms
17/08/01 14:50:09 INFO scheduler.JobScheduler: Added jobs for time 1501599009000 ms
17/08/01 14:50:09 INFO scheduler.JobScheduler: Starting job streaming job 1501599009000 ms.0 from job set of time 1501599009000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:09 INFO scheduler.JobScheduler: Finished job streaming job 1501599009000 ms.0 from job set of time 1501599009000 ms
17/08/01 14:50:09 INFO python.PythonRDD: Removing RDD 75 from persistence list
17/08/01 14:50:09 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599009000 ms (execution: 0.005 s)
17/08/01 14:50:09 INFO storage.BlockManager: Removing RDD 75
17/08/01 14:50:09 INFO rdd.BlockRDD: Removing RDD 74 from persistence list
17/08/01 14:50:09 INFO storage.BlockManager: Removing RDD 74
17/08/01 14:50:09 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[74] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599009000 ms
17/08/01 14:50:09 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599007000 ms
17/08/01 14:50:09 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599007000 ms
17/08/01 14:50:10 INFO scheduler.JobScheduler: Added jobs for time 1501599010000 ms
17/08/01 14:50:10 INFO scheduler.JobScheduler: Starting job streaming job 1501599010000 ms.0 from job set of time 1501599010000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:10 INFO scheduler.JobScheduler: Finished job streaming job 1501599010000 ms.0 from job set of time 1501599010000 ms
17/08/01 14:50:10 INFO python.PythonRDD: Removing RDD 77 from persistence list
17/08/01 14:50:10 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599010000 ms (execution: 0.005 s)
17/08/01 14:50:10 INFO rdd.BlockRDD: Removing RDD 76 from persistence list
17/08/01 14:50:10 INFO storage.BlockManager: Removing RDD 76
17/08/01 14:50:10 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[76] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599010000 ms
17/08/01 14:50:10 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599008000 ms
17/08/01 14:50:10 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599008000 ms
17/08/01 14:50:10 INFO storage.BlockManager: Removing RDD 77
17/08/01 14:50:11 INFO scheduler.JobScheduler: Added jobs for time 1501599011000 ms
17/08/01 14:50:11 INFO scheduler.JobScheduler: Starting job streaming job 1501599011000 ms.0 from job set of time 1501599011000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:11 INFO scheduler.JobScheduler: Finished job streaming job 1501599011000 ms.0 from job set of time 1501599011000 ms
17/08/01 14:50:11 INFO python.PythonRDD: Removing RDD 79 from persistence list
17/08/01 14:50:11 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599011000 ms (execution: 0.007 s)
17/08/01 14:50:11 INFO storage.BlockManager: Removing RDD 79
17/08/01 14:50:11 INFO rdd.BlockRDD: Removing RDD 78 from persistence list
17/08/01 14:50:11 INFO storage.BlockManager: Removing RDD 78
17/08/01 14:50:11 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[78] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599011000 ms
17/08/01 14:50:11 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599009000 ms
17/08/01 14:50:11 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599009000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Added jobs for time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Starting job streaming job 1501599012000 ms.0 from job set of time 1501599012000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:12 INFO scheduler.JobScheduler: Finished job streaming job 1501599012000 ms.0 from job set of time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599012000 ms (execution: 0.015 s)
17/08/01 14:50:12 INFO python.PythonRDD: Removing RDD 81 from persistence list
17/08/01 14:50:12 INFO storage.BlockManager: Removing RDD 81
17/08/01 14:50:12 INFO rdd.BlockRDD: Removing RDD 80 from persistence list
17/08/01 14:50:12 INFO storage.BlockManager: Removing RDD 80
17/08/01 14:50:12 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[80] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599010000 ms
17/08/01 14:50:12 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599010000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Added jobs for time 1501599013000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Starting job streaming job 1501599013000 ms.0 from job set of time 1501599013000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:13 INFO scheduler.JobScheduler: Finished job streaming job 1501599013000 ms.0 from job set of time 1501599013000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501599013000 ms (execution: 0.008 s)
17/08/01 14:50:13 INFO python.PythonRDD: Removing RDD 83 from persistence list
17/08/01 14:50:13 INFO storage.BlockManager: Removing RDD 83
17/08/01 14:50:13 INFO rdd.BlockRDD: Removing RDD 82 from persistence list
17/08/01 14:50:13 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[82] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599013000 ms
17/08/01 14:50:13 INFO storage.BlockManager: Removing RDD 82
17/08/01 14:50:13 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599011000 ms
17/08/01 14:50:13 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599011000 ms
17/08/01 14:50:14 INFO scheduler.JobScheduler: Added jobs for time 1501599014000 ms
17/08/01 14:50:14 INFO scheduler.JobScheduler: Starting job streaming job 1501599014000 ms.0 from job set of time 1501599014000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:14 INFO scheduler.JobScheduler: Finished job streaming job 1501599014000 ms.0 from job set of time 1501599014000 ms
17/08/01 14:50:14 INFO python.PythonRDD: Removing RDD 85 from persistence list
17/08/01 14:50:14 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599014000 ms (execution: 0.007 s)
17/08/01 14:50:14 INFO rdd.BlockRDD: Removing RDD 84 from persistence list
17/08/01 14:50:14 INFO storage.BlockManager: Removing RDD 85
17/08/01 14:50:14 INFO storage.BlockManager: Removing RDD 84
17/08/01 14:50:14 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[84] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599014000 ms
17/08/01 14:50:14 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599012000 ms
17/08/01 14:50:14 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599012000 ms
17/08/01 14:50:15 INFO scheduler.JobScheduler: Added jobs for time 1501599015000 ms
17/08/01 14:50:15 INFO scheduler.JobScheduler: Starting job streaming job 1501599015000 ms.0 from job set of time 1501599015000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:15 INFO scheduler.JobScheduler: Finished job streaming job 1501599015000 ms.0 from job set of time 1501599015000 ms
17/08/01 14:50:15 INFO python.PythonRDD: Removing RDD 87 from persistence list
17/08/01 14:50:15 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599015000 ms (execution: 0.007 s)
17/08/01 14:50:15 INFO storage.BlockManager: Removing RDD 87
17/08/01 14:50:15 INFO rdd.BlockRDD: Removing RDD 86 from persistence list
17/08/01 14:50:15 INFO storage.BlockManager: Removing RDD 86
17/08/01 14:50:15 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[86] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599015000 ms
17/08/01 14:50:15 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599013000 ms
17/08/01 14:50:15 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599013000 ms
17/08/01 14:50:16 INFO scheduler.JobScheduler: Added jobs for time 1501599016000 ms
17/08/01 14:50:16 INFO scheduler.JobScheduler: Starting job streaming job 1501599016000 ms.0 from job set of time 1501599016000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:16 INFO scheduler.JobScheduler: Finished job streaming job 1501599016000 ms.0 from job set of time 1501599016000 ms
17/08/01 14:50:16 INFO python.PythonRDD: Removing RDD 89 from persistence list
17/08/01 14:50:16 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501599016000 ms (execution: 0.008 s)
17/08/01 14:50:16 INFO storage.BlockManager: Removing RDD 89
17/08/01 14:50:16 INFO rdd.BlockRDD: Removing RDD 88 from persistence list
17/08/01 14:50:16 INFO storage.BlockManager: Removing RDD 88
17/08/01 14:50:16 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[88] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599016000 ms
17/08/01 14:50:16 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599014000 ms
17/08/01 14:50:16 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599014000 ms
17/08/01 14:50:17 INFO scheduler.JobScheduler: Added jobs for time 1501599017000 ms
17/08/01 14:50:17 INFO scheduler.JobScheduler: Starting job streaming job 1501599017000 ms.0 from job set of time 1501599017000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:17 INFO scheduler.JobScheduler: Finished job streaming job 1501599017000 ms.0 from job set of time 1501599017000 ms
17/08/01 14:50:17 INFO python.PythonRDD: Removing RDD 91 from persistence list
17/08/01 14:50:17 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501599017000 ms (execution: 0.008 s)
17/08/01 14:50:17 INFO storage.BlockManager: Removing RDD 91
17/08/01 14:50:17 INFO rdd.BlockRDD: Removing RDD 90 from persistence list
17/08/01 14:50:17 INFO storage.BlockManager: Removing RDD 90
17/08/01 14:50:17 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[90] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599017000 ms
17/08/01 14:50:17 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599015000 ms
17/08/01 14:50:17 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599015000 ms
17/08/01 14:50:18 INFO scheduler.JobScheduler: Added jobs for time 1501599018000 ms
17/08/01 14:50:18 INFO scheduler.JobScheduler: Starting job streaming job 1501599018000 ms.0 from job set of time 1501599018000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:18 INFO scheduler.JobScheduler: Finished job streaming job 1501599018000 ms.0 from job set of time 1501599018000 ms
17/08/01 14:50:18 INFO python.PythonRDD: Removing RDD 93 from persistence list
17/08/01 14:50:18 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599018000 ms (execution: 0.006 s)
17/08/01 14:50:18 INFO storage.BlockManager: Removing RDD 93
17/08/01 14:50:18 INFO rdd.BlockRDD: Removing RDD 92 from persistence list
17/08/01 14:50:18 INFO storage.BlockManager: Removing RDD 92
17/08/01 14:50:18 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[92] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599018000 ms
17/08/01 14:50:18 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599016000 ms
17/08/01 14:50:18 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599016000 ms
17/08/01 14:50:19 INFO scheduler.JobScheduler: Added jobs for time 1501599019000 ms
17/08/01 14:50:19 INFO scheduler.JobScheduler: Starting job streaming job 1501599019000 ms.0 from job set of time 1501599019000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:19 INFO scheduler.JobScheduler: Finished job streaming job 1501599019000 ms.0 from job set of time 1501599019000 ms
17/08/01 14:50:19 INFO python.PythonRDD: Removing RDD 95 from persistence list
17/08/01 14:50:19 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599019000 ms (execution: 0.015 s)
17/08/01 14:50:19 INFO storage.BlockManager: Removing RDD 95
17/08/01 14:50:19 INFO rdd.BlockRDD: Removing RDD 94 from persistence list
17/08/01 14:50:19 INFO storage.BlockManager: Removing RDD 94
17/08/01 14:50:19 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[94] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599019000 ms
17/08/01 14:50:19 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599017000 ms
17/08/01 14:50:19 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599017000 ms
17/08/01 14:50:20 INFO scheduler.JobScheduler: Added jobs for time 1501599020000 ms
17/08/01 14:50:20 INFO scheduler.JobScheduler: Starting job streaming job 1501599020000 ms.0 from job set of time 1501599020000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:20 INFO scheduler.JobScheduler: Finished job streaming job 1501599020000 ms.0 from job set of time 1501599020000 ms
17/08/01 14:50:20 INFO python.PythonRDD: Removing RDD 97 from persistence list
17/08/01 14:50:20 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599020000 ms (execution: 0.006 s)
17/08/01 14:50:20 INFO storage.BlockManager: Removing RDD 97
17/08/01 14:50:20 INFO rdd.BlockRDD: Removing RDD 96 from persistence list
17/08/01 14:50:20 INFO storage.BlockManager: Removing RDD 96
17/08/01 14:50:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[96] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599020000 ms
17/08/01 14:50:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599018000 ms
17/08/01 14:50:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599018000 ms
17/08/01 14:50:21 INFO scheduler.JobScheduler: Added jobs for time 1501599021000 ms
17/08/01 14:50:21 INFO scheduler.JobScheduler: Starting job streaming job 1501599021000 ms.0 from job set of time 1501599021000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:21 INFO scheduler.JobScheduler: Finished job streaming job 1501599021000 ms.0 from job set of time 1501599021000 ms
17/08/01 14:50:21 INFO python.PythonRDD: Removing RDD 99 from persistence list
17/08/01 14:50:21 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599021000 ms (execution: 0.007 s)
17/08/01 14:50:21 INFO storage.BlockManager: Removing RDD 99
17/08/01 14:50:21 INFO rdd.BlockRDD: Removing RDD 98 from persistence list
17/08/01 14:50:21 INFO storage.BlockManager: Removing RDD 98
17/08/01 14:50:21 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[98] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599021000 ms
17/08/01 14:50:21 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599019000 ms
17/08/01 14:50:21 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599019000 ms
17/08/01 14:50:22 INFO scheduler.JobScheduler: Added jobs for time 1501599022000 ms
17/08/01 14:50:22 INFO scheduler.JobScheduler: Starting job streaming job 1501599022000 ms.0 from job set of time 1501599022000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:22 INFO scheduler.JobScheduler: Finished job streaming job 1501599022000 ms.0 from job set of time 1501599022000 ms
17/08/01 14:50:22 INFO python.PythonRDD: Removing RDD 101 from persistence list
17/08/01 14:50:22 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501599022000 ms (execution: 0.007 s)
17/08/01 14:50:22 INFO storage.BlockManager: Removing RDD 101
17/08/01 14:50:22 INFO rdd.BlockRDD: Removing RDD 100 from persistence list
17/08/01 14:50:22 INFO storage.BlockManager: Removing RDD 100
17/08/01 14:50:22 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[100] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599022000 ms
17/08/01 14:50:22 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599020000 ms
17/08/01 14:50:22 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599020000 ms
17/08/01 14:50:23 INFO scheduler.JobScheduler: Added jobs for time 1501599023000 ms
17/08/01 14:50:23 INFO scheduler.JobScheduler: Starting job streaming job 1501599023000 ms.0 from job set of time 1501599023000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:23 INFO scheduler.JobScheduler: Finished job streaming job 1501599023000 ms.0 from job set of time 1501599023000 ms
17/08/01 14:50:23 INFO python.PythonRDD: Removing RDD 103 from persistence list
17/08/01 14:50:23 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599023000 ms (execution: 0.007 s)
17/08/01 14:50:23 INFO storage.BlockManager: Removing RDD 103
17/08/01 14:50:23 INFO rdd.BlockRDD: Removing RDD 102 from persistence list
17/08/01 14:50:23 INFO storage.BlockManager: Removing RDD 102
17/08/01 14:50:23 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[102] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599023000 ms
17/08/01 14:50:23 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599021000 ms
17/08/01 14:50:23 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599021000 ms
17/08/01 14:50:24 INFO scheduler.JobScheduler: Added jobs for time 1501599024000 ms
17/08/01 14:50:24 INFO scheduler.JobScheduler: Starting job streaming job 1501599024000 ms.0 from job set of time 1501599024000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:24 INFO scheduler.JobScheduler: Finished job streaming job 1501599024000 ms.0 from job set of time 1501599024000 ms
17/08/01 14:50:24 INFO python.PythonRDD: Removing RDD 105 from persistence list
17/08/01 14:50:24 INFO scheduler.JobScheduler: Total delay: 0.035 s for time 1501599024000 ms (execution: 0.007 s)
17/08/01 14:50:24 INFO storage.BlockManager: Removing RDD 105
17/08/01 14:50:24 INFO rdd.BlockRDD: Removing RDD 104 from persistence list
17/08/01 14:50:24 INFO storage.BlockManager: Removing RDD 104
17/08/01 14:50:24 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[104] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599024000 ms
17/08/01 14:50:24 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599022000 ms
17/08/01 14:50:24 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599022000 ms
17/08/01 14:50:25 INFO scheduler.JobScheduler: Added jobs for time 1501599025000 ms
17/08/01 14:50:25 INFO scheduler.JobScheduler: Starting job streaming job 1501599025000 ms.0 from job set of time 1501599025000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:25 INFO scheduler.JobScheduler: Finished job streaming job 1501599025000 ms.0 from job set of time 1501599025000 ms
17/08/01 14:50:25 INFO python.PythonRDD: Removing RDD 107 from persistence list
17/08/01 14:50:25 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599025000 ms (execution: 0.007 s)
17/08/01 14:50:25 INFO storage.BlockManager: Removing RDD 107
17/08/01 14:50:25 INFO rdd.BlockRDD: Removing RDD 106 from persistence list
17/08/01 14:50:25 INFO storage.BlockManager: Removing RDD 106
17/08/01 14:50:25 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[106] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599025000 ms
17/08/01 14:50:25 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599023000 ms
17/08/01 14:50:25 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599023000 ms
17/08/01 14:50:26 INFO scheduler.JobScheduler: Added jobs for time 1501599026000 ms
17/08/01 14:50:26 INFO scheduler.JobScheduler: Starting job streaming job 1501599026000 ms.0 from job set of time 1501599026000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:26 INFO scheduler.JobScheduler: Finished job streaming job 1501599026000 ms.0 from job set of time 1501599026000 ms
17/08/01 14:50:26 INFO python.PythonRDD: Removing RDD 109 from persistence list
17/08/01 14:50:26 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599026000 ms (execution: 0.007 s)
17/08/01 14:50:26 INFO storage.BlockManager: Removing RDD 109
17/08/01 14:50:26 INFO rdd.BlockRDD: Removing RDD 108 from persistence list
17/08/01 14:50:26 INFO storage.BlockManager: Removing RDD 108
17/08/01 14:50:26 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[108] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599026000 ms
17/08/01 14:50:26 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599024000 ms
17/08/01 14:50:26 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599024000 ms
17/08/01 14:50:27 INFO scheduler.JobScheduler: Added jobs for time 1501599027000 ms
17/08/01 14:50:27 INFO scheduler.JobScheduler: Starting job streaming job 1501599027000 ms.0 from job set of time 1501599027000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:27 INFO scheduler.JobScheduler: Finished job streaming job 1501599027000 ms.0 from job set of time 1501599027000 ms
17/08/01 14:50:27 INFO python.PythonRDD: Removing RDD 111 from persistence list
17/08/01 14:50:27 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599027000 ms (execution: 0.007 s)
17/08/01 14:50:27 INFO storage.BlockManager: Removing RDD 111
17/08/01 14:50:27 INFO rdd.BlockRDD: Removing RDD 110 from persistence list
17/08/01 14:50:27 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[110] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599027000 ms
17/08/01 14:50:27 INFO storage.BlockManager: Removing RDD 110
17/08/01 14:50:27 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599025000 ms
17/08/01 14:50:27 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599025000 ms
17/08/01 14:50:28 INFO scheduler.JobScheduler: Added jobs for time 1501599028000 ms
17/08/01 14:50:28 INFO scheduler.JobScheduler: Starting job streaming job 1501599028000 ms.0 from job set of time 1501599028000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:28 INFO scheduler.JobScheduler: Finished job streaming job 1501599028000 ms.0 from job set of time 1501599028000 ms
17/08/01 14:50:28 INFO python.PythonRDD: Removing RDD 113 from persistence list
17/08/01 14:50:28 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599028000 ms (execution: 0.017 s)
17/08/01 14:50:28 INFO storage.BlockManager: Removing RDD 113
17/08/01 14:50:28 INFO rdd.BlockRDD: Removing RDD 112 from persistence list
17/08/01 14:50:28 INFO storage.BlockManager: Removing RDD 112
17/08/01 14:50:28 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[112] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599028000 ms
17/08/01 14:50:28 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599026000 ms
17/08/01 14:50:28 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599026000 ms
17/08/01 14:50:29 INFO scheduler.JobScheduler: Added jobs for time 1501599029000 ms
17/08/01 14:50:29 INFO scheduler.JobScheduler: Starting job streaming job 1501599029000 ms.0 from job set of time 1501599029000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:29 INFO scheduler.JobScheduler: Finished job streaming job 1501599029000 ms.0 from job set of time 1501599029000 ms
17/08/01 14:50:29 INFO python.PythonRDD: Removing RDD 115 from persistence list
17/08/01 14:50:29 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599029000 ms (execution: 0.007 s)
17/08/01 14:50:29 INFO storage.BlockManager: Removing RDD 115
17/08/01 14:50:29 INFO rdd.BlockRDD: Removing RDD 114 from persistence list
17/08/01 14:50:29 INFO storage.BlockManager: Removing RDD 114
17/08/01 14:50:29 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[114] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599029000 ms
17/08/01 14:50:29 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599027000 ms
17/08/01 14:50:29 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599027000 ms
17/08/01 14:50:30 INFO scheduler.JobScheduler: Added jobs for time 1501599030000 ms
17/08/01 14:50:30 INFO scheduler.JobScheduler: Starting job streaming job 1501599030000 ms.0 from job set of time 1501599030000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:30 INFO scheduler.JobScheduler: Finished job streaming job 1501599030000 ms.0 from job set of time 1501599030000 ms
17/08/01 14:50:30 INFO python.PythonRDD: Removing RDD 117 from persistence list
17/08/01 14:50:30 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599030000 ms (execution: 0.007 s)
17/08/01 14:50:30 INFO storage.BlockManager: Removing RDD 117
17/08/01 14:50:30 INFO rdd.BlockRDD: Removing RDD 116 from persistence list
17/08/01 14:50:30 INFO storage.BlockManager: Removing RDD 116
17/08/01 14:50:30 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[116] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599030000 ms
17/08/01 14:50:30 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599028000 ms
17/08/01 14:50:30 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599028000 ms
17/08/01 14:50:31 INFO scheduler.JobScheduler: Added jobs for time 1501599031000 ms
17/08/01 14:50:31 INFO scheduler.JobScheduler: Starting job streaming job 1501599031000 ms.0 from job set of time 1501599031000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:31 INFO scheduler.JobScheduler: Finished job streaming job 1501599031000 ms.0 from job set of time 1501599031000 ms
17/08/01 14:50:31 INFO python.PythonRDD: Removing RDD 119 from persistence list
17/08/01 14:50:31 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599031000 ms (execution: 0.006 s)
17/08/01 14:50:31 INFO rdd.BlockRDD: Removing RDD 118 from persistence list
17/08/01 14:50:31 INFO storage.BlockManager: Removing RDD 119
17/08/01 14:50:31 INFO storage.BlockManager: Removing RDD 118
17/08/01 14:50:31 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[118] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599031000 ms
17/08/01 14:50:31 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599029000 ms
17/08/01 14:50:31 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599029000 ms
17/08/01 14:50:32 INFO scheduler.JobScheduler: Added jobs for time 1501599032000 ms
17/08/01 14:50:32 INFO scheduler.JobScheduler: Starting job streaming job 1501599032000 ms.0 from job set of time 1501599032000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:32 INFO scheduler.JobScheduler: Finished job streaming job 1501599032000 ms.0 from job set of time 1501599032000 ms
17/08/01 14:50:32 INFO python.PythonRDD: Removing RDD 121 from persistence list
17/08/01 14:50:32 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599032000 ms (execution: 0.007 s)
17/08/01 14:50:32 INFO storage.BlockManager: Removing RDD 121
17/08/01 14:50:32 INFO rdd.BlockRDD: Removing RDD 120 from persistence list
17/08/01 14:50:32 INFO storage.BlockManager: Removing RDD 120
17/08/01 14:50:32 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[120] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599032000 ms
17/08/01 14:50:32 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599030000 ms
17/08/01 14:50:32 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599030000 ms
17/08/01 14:50:33 INFO scheduler.JobScheduler: Added jobs for time 1501599033000 ms
17/08/01 14:50:33 INFO scheduler.JobScheduler: Starting job streaming job 1501599033000 ms.0 from job set of time 1501599033000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:33 INFO scheduler.JobScheduler: Finished job streaming job 1501599033000 ms.0 from job set of time 1501599033000 ms
17/08/01 14:50:33 INFO python.PythonRDD: Removing RDD 123 from persistence list
17/08/01 14:50:33 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599033000 ms (execution: 0.005 s)
17/08/01 14:50:33 INFO storage.BlockManager: Removing RDD 123
17/08/01 14:50:33 INFO rdd.BlockRDD: Removing RDD 122 from persistence list
17/08/01 14:50:33 INFO storage.BlockManager: Removing RDD 122
17/08/01 14:50:33 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[122] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599033000 ms
17/08/01 14:50:33 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599031000 ms
17/08/01 14:50:33 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599031000 ms
17/08/01 14:50:34 INFO scheduler.JobScheduler: Added jobs for time 1501599034000 ms
17/08/01 14:50:34 INFO scheduler.JobScheduler: Starting job streaming job 1501599034000 ms.0 from job set of time 1501599034000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:34 INFO scheduler.JobScheduler: Finished job streaming job 1501599034000 ms.0 from job set of time 1501599034000 ms
17/08/01 14:50:34 INFO python.PythonRDD: Removing RDD 125 from persistence list
17/08/01 14:50:34 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599034000 ms (execution: 0.006 s)
17/08/01 14:50:34 INFO storage.BlockManager: Removing RDD 125
17/08/01 14:50:34 INFO rdd.BlockRDD: Removing RDD 124 from persistence list
17/08/01 14:50:34 INFO storage.BlockManager: Removing RDD 124
17/08/01 14:50:34 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[124] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599034000 ms
17/08/01 14:50:34 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599032000 ms
17/08/01 14:50:34 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599032000 ms
17/08/01 14:50:34 INFO spark.ExecutorAllocationManager: Request to remove executorIds: 2
17/08/01 14:50:34 INFO cluster.YarnClientSchedulerBackend: Requesting to kill executor(s) 2
17/08/01 14:50:34 INFO cluster.YarnClientSchedulerBackend: Actual list of executor(s) to be killed is 2
17/08/01 14:50:34 INFO spark.ExecutorAllocationManager: Removing executor 2 because it has been idle for 60 seconds (new desired total will be 1)
17/08/01 14:50:35 INFO scheduler.JobScheduler: Added jobs for time 1501599035000 ms
17/08/01 14:50:35 INFO scheduler.JobScheduler: Starting job streaming job 1501599035000 ms.0 from job set of time 1501599035000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:35 INFO scheduler.JobScheduler: Finished job streaming job 1501599035000 ms.0 from job set of time 1501599035000 ms
17/08/01 14:50:35 INFO python.PythonRDD: Removing RDD 127 from persistence list
17/08/01 14:50:35 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599035000 ms (execution: 0.013 s)
17/08/01 14:50:35 INFO storage.BlockManager: Removing RDD 127
17/08/01 14:50:35 INFO rdd.BlockRDD: Removing RDD 126 from persistence list
17/08/01 14:50:35 INFO storage.BlockManager: Removing RDD 126
17/08/01 14:50:35 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[126] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599035000 ms
17/08/01 14:50:35 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599033000 ms
17/08/01 14:50:35 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599033000 ms
17/08/01 14:50:36 INFO scheduler.JobScheduler: Added jobs for time 1501599036000 ms
17/08/01 14:50:36 INFO scheduler.JobScheduler: Starting job streaming job 1501599036000 ms.0 from job set of time 1501599036000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:36 INFO scheduler.JobScheduler: Finished job streaming job 1501599036000 ms.0 from job set of time 1501599036000 ms
17/08/01 14:50:36 INFO python.PythonRDD: Removing RDD 129 from persistence list
17/08/01 14:50:36 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599036000 ms (execution: 0.007 s)
17/08/01 14:50:36 INFO storage.BlockManager: Removing RDD 129
17/08/01 14:50:36 INFO rdd.BlockRDD: Removing RDD 128 from persistence list
17/08/01 14:50:36 INFO storage.BlockManager: Removing RDD 128
17/08/01 14:50:36 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[128] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599036000 ms
17/08/01 14:50:36 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599034000 ms
17/08/01 14:50:36 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599034000 ms
17/08/01 14:50:37 INFO scheduler.JobScheduler: Added jobs for time 1501599037000 ms
17/08/01 14:50:37 INFO scheduler.JobScheduler: Starting job streaming job 1501599037000 ms.0 from job set of time 1501599037000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:37 INFO scheduler.JobScheduler: Finished job streaming job 1501599037000 ms.0 from job set of time 1501599037000 ms
17/08/01 14:50:37 INFO python.PythonRDD: Removing RDD 131 from persistence list
17/08/01 14:50:37 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599037000 ms (execution: 0.006 s)
17/08/01 14:50:37 INFO storage.BlockManager: Removing RDD 131
17/08/01 14:50:37 INFO rdd.BlockRDD: Removing RDD 130 from persistence list
17/08/01 14:50:37 INFO storage.BlockManager: Removing RDD 130
17/08/01 14:50:37 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[130] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599037000 ms
17/08/01 14:50:37 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599035000 ms
17/08/01 14:50:37 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599035000 ms
17/08/01 14:50:38 INFO scheduler.JobScheduler: Added jobs for time 1501599038000 ms
17/08/01 14:50:38 INFO scheduler.JobScheduler: Starting job streaming job 1501599038000 ms.0 from job set of time 1501599038000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:38 INFO scheduler.JobScheduler: Finished job streaming job 1501599038000 ms.0 from job set of time 1501599038000 ms
17/08/01 14:50:38 INFO python.PythonRDD: Removing RDD 133 from persistence list
17/08/01 14:50:38 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599038000 ms (execution: 0.007 s)
17/08/01 14:50:38 INFO storage.BlockManager: Removing RDD 133
17/08/01 14:50:38 INFO rdd.BlockRDD: Removing RDD 132 from persistence list
17/08/01 14:50:38 INFO storage.BlockManager: Removing RDD 132
17/08/01 14:50:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[132] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599038000 ms
17/08/01 14:50:38 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599036000 ms
17/08/01 14:50:38 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599036000 ms
17/08/01 14:50:38 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Disabling executor 2.
17/08/01 14:50:38 INFO scheduler.DAGScheduler: Executor lost: 2 (epoch 1)
17/08/01 14:50:38 INFO storage.BlockManagerMasterEndpoint: Trying to remove executor 2 from BlockManagerMaster.
17/08/01 14:50:38 INFO storage.BlockManagerMasterEndpoint: Removing block manager BlockManagerId(2, ip-10-0-0-127.ec2.internal, 59267, None)
17/08/01 14:50:38 INFO storage.BlockManagerMaster: Removed 2 successfully in removeExecutor
17/08/01 14:50:38 INFO cluster.YarnScheduler: Executor 2 on ip-10-0-0-127.ec2.internal killed by driver.
17/08/01 14:50:38 INFO spark.ExecutorAllocationManager: Existing executor 2 has been removed (new total is 1)
17/08/01 14:50:39 INFO scheduler.JobScheduler: Added jobs for time 1501599039000 ms
17/08/01 14:50:39 INFO scheduler.JobScheduler: Starting job streaming job 1501599039000 ms.0 from job set of time 1501599039000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:39 INFO scheduler.JobScheduler: Finished job streaming job 1501599039000 ms.0 from job set of time 1501599039000 ms
17/08/01 14:50:39 INFO python.PythonRDD: Removing RDD 135 from persistence list
17/08/01 14:50:39 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599039000 ms (execution: 0.006 s)
17/08/01 14:50:39 INFO storage.BlockManager: Removing RDD 135
17/08/01 14:50:39 INFO rdd.BlockRDD: Removing RDD 134 from persistence list
17/08/01 14:50:39 INFO storage.BlockManager: Removing RDD 134
17/08/01 14:50:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[134] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599039000 ms
17/08/01 14:50:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599037000 ms
17/08/01 14:50:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599037000 ms
17/08/01 14:50:40 INFO scheduler.JobScheduler: Added jobs for time 1501599040000 ms
17/08/01 14:50:40 INFO scheduler.JobScheduler: Starting job streaming job 1501599040000 ms.0 from job set of time 1501599040000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:40 INFO scheduler.JobScheduler: Finished job streaming job 1501599040000 ms.0 from job set of time 1501599040000 ms
17/08/01 14:50:40 INFO python.PythonRDD: Removing RDD 137 from persistence list
17/08/01 14:50:40 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599040000 ms (execution: 0.006 s)
17/08/01 14:50:40 INFO storage.BlockManager: Removing RDD 137
17/08/01 14:50:40 INFO rdd.BlockRDD: Removing RDD 136 from persistence list
17/08/01 14:50:40 INFO storage.BlockManager: Removing RDD 136
17/08/01 14:50:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[136] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599040000 ms
17/08/01 14:50:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599038000 ms
17/08/01 14:50:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599038000 ms
17/08/01 14:50:41 INFO scheduler.JobScheduler: Added jobs for time 1501599041000 ms
17/08/01 14:50:41 INFO scheduler.JobScheduler: Starting job streaming job 1501599041000 ms.0 from job set of time 1501599041000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:41 INFO scheduler.JobScheduler: Finished job streaming job 1501599041000 ms.0 from job set of time 1501599041000 ms
17/08/01 14:50:41 INFO python.PythonRDD: Removing RDD 139 from persistence list
17/08/01 14:50:41 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599041000 ms (execution: 0.006 s)
17/08/01 14:50:41 INFO storage.BlockManager: Removing RDD 139
17/08/01 14:50:41 INFO rdd.BlockRDD: Removing RDD 138 from persistence list
17/08/01 14:50:41 INFO storage.BlockManager: Removing RDD 138
17/08/01 14:50:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[138] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599041000 ms
17/08/01 14:50:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599039000 ms
17/08/01 14:50:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599039000 ms
17/08/01 14:50:42 INFO scheduler.JobScheduler: Added jobs for time 1501599042000 ms
17/08/01 14:50:42 INFO scheduler.JobScheduler: Starting job streaming job 1501599042000 ms.0 from job set of time 1501599042000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:42 INFO scheduler.JobScheduler: Finished job streaming job 1501599042000 ms.0 from job set of time 1501599042000 ms
17/08/01 14:50:42 INFO python.PythonRDD: Removing RDD 141 from persistence list
17/08/01 14:50:42 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599042000 ms (execution: 0.014 s)
17/08/01 14:50:42 INFO storage.BlockManager: Removing RDD 141
17/08/01 14:50:42 INFO rdd.BlockRDD: Removing RDD 140 from persistence list
17/08/01 14:50:42 INFO storage.BlockManager: Removing RDD 140
17/08/01 14:50:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[140] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599042000 ms
17/08/01 14:50:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599040000 ms
17/08/01 14:50:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599040000 ms
17/08/01 14:50:42 INFO storage.BlockManagerInfo: Added input-0-1501599042400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1059.6 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599042800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1057.9 MB)
17/08/01 14:50:43 INFO scheduler.JobScheduler: Added jobs for time 1501599043000 ms
17/08/01 14:50:43 INFO scheduler.JobScheduler: Starting job streaming job 1501599043000 ms.0 from job set of time 1501599043000 ms
17/08/01 14:50:43 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Got job 2 (runJob at PythonRDD.scala:441) with 1 output partitions
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Final stage: ResultStage 3 (runJob at PythonRDD.scala:441)
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Missing parents: List()
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Submitting ResultStage 3 (PythonRDD[146] at RDD at PythonRDD.scala:48), which has no missing parents
17/08/01 14:50:43 INFO memory.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/08/01 14:50:43 INFO memory.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.0.0.199:53355 (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:43 INFO spark.SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:996
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (PythonRDD[146] at RDD at PythonRDD.scala:48)
17/08/01 14:50:43 INFO cluster.YarnScheduler: Adding task set 3.0 with 1 tasks
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.3 KB, free: 1057.0 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1056.2 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.4 KB, free: 1055.3 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 878.9 KB, free: 1054.5 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599043800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1052.7 MB)
17/08/01 14:50:44 INFO scheduler.JobScheduler: Added jobs for time 1501599044000 ms
17/08/01 14:50:44 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 2)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1051.9 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1051.0 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1050.2 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.5 KB, free: 1049.3 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599044800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 887.9 KB, free: 1048.4 MB)
17/08/01 14:50:45 INFO scheduler.JobScheduler: Added jobs for time 1501599045000 ms
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1047.6 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1046.7 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1045.9 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.0 KB, free: 1044.1 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599045800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1043.3 MB)
17/08/01 14:50:46 INFO scheduler.JobScheduler: Added jobs for time 1501599046000 ms
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.6 KB, free: 1042.4 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.0 KB, free: 1041.6 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1040.7 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1039.9 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599046800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1039.0 MB)
17/08/01 14:50:47 INFO scheduler.JobScheduler: Added jobs for time 1501599047000 ms
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1038.1 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 878.9 KB, free: 1037.3 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1035.6 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1034.7 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599047800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1033.8 MB)
17/08/01 14:50:48 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.127:44785) with ID 3
17/08/01 14:50:48 INFO spark.ExecutorAllocationManager: New executor 3 has registered (new total is 2)
17/08/01 14:50:48 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 3.0 (TID 71, ip-10-0-0-127.ec2.internal, executor 3, partition 0, RACK_LOCAL, 5800 bytes)
17/08/01 14:50:48 INFO scheduler.JobScheduler: Added jobs for time 1501599048000 ms
17/08/01 14:50:48 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-127.ec2.internal:51258 with 1060.5 MB RAM, BlockManagerId(3, ip-10-0-0-127.ec2.internal, 51258, None)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1033.0 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on ip-10-0-0-127.ec2.internal:51258 (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1032.1 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.5 KB, free: 1031.3 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1030.4 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599048800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1029.5 MB)
17/08/01 14:50:49 INFO scheduler.JobScheduler: Added jobs for time 1501599049000 ms
17/08/01 14:50:49 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 3.0 (TID 71) in 1023 ms on ip-10-0-0-127.ec2.internal (executor 3) (1/1)
17/08/01 14:50:49 INFO cluster.YarnScheduler: Removed TaskSet 3.0, whose tasks have all completed, from pool
17/08/01 14:50:49 INFO scheduler.DAGScheduler: ResultStage 3 (runJob at PythonRDD.scala:441) finished in 5.985 s
17/08/01 14:50:49 INFO scheduler.DAGScheduler: Job 2 finished: runJob at PythonRDD.scala:441, took 6.003911 s
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.0 KB, free: 1027.8 MB)
17/08/01 14:50:49 INFO spark.ContextCleaner: Cleaned shuffle 0
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 10.0.0.199:53355 in memory (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1988.0 B, free: 1027.8 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on 10.0.0.199:53355 in memory (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1956.0 B, free: 1027.8 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on 10.0.0.199:53355 in memory (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on ip-10-0-0-127.ec2.internal:51258 in memory (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1027.0 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1026.1 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1025.3 MB)
17/08/01 14:50:49 INFO execution.SparkSqlParser: Parsing command: spotdb.proxy
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599049800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1024.4 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Added jobs for time 1501599050000 ms
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1023.5 MB)
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1022.7 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599043000 ms.0 from job set of time 1501599043000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 143 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 7.504 s for time 1501599043000 ms (execution: 7.487 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599044000 ms.0 from job set of time 1501599044000 ms
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 142 from persistence list
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599043000 ms.0
org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 69, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[142] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599043000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599041000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599041000 ms
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 142
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 143
Traceback (most recent call last):
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 148, in <module>
main()
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 77, in main
bluecoat_parse(args.zk,args.topic,args.db,args.db_table,args.num_of_workers,args.batch_size)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 144, in bluecoat_parse
ssc.awaitTermination()
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/context.py", line 206, in awaitTermination
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o50.awaitTermination.
: org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 69, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599044000 ms.0 from job set of time 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 6.542 s for time 1501599044000 ms (execution: 0.037 s)
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 145 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599045000 ms.0 from job set of time 1501599045000 ms
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599044000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 145
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 144 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 144
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[144] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599042000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599042000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599042400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1023.5 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599045000 ms.0 from job set of time 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 5.547 s for time 1501599045000 ms (execution: 0.003 s)
17/08/01 14:50:50 INFO streaming.StreamingContext: Invoking stop(stopGracefully=false) from shutdown hook
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599046000 ms.0 from job set of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599046000 ms.0 from job set of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 4.559 s for time 1501599046000 ms (execution: 0.004 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599047000 ms.0 from job set of time 1501599047000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 148 from persistence list
17/08/01 14:50:50 INFO scheduler.ReceiverTracker: Sent stop signal to all 1 receivers
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 148
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599047000 ms.0 from job set of time 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 3.562 s for time 1501599047000 ms (execution: 0.002 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599048000 ms.0 from job set of time 1501599048000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599048000 ms.0 from job set of time 1501599048000 ms
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 147 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 2.571 s for time 1501599048000 ms (execution: 0.001 s)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 147
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[147] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599049000 ms.0 from job set of time 1501599049000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599049000 ms.0 from job set of time 1501599049000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 1.572 s for time 1501599049000 ms (execution: 0.001 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599050000 ms.0 from job set of time 1501599050000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599050000 ms.0 from job set of time 1501599050000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 0.574 s for time 1501599050000 ms (execution: 0.001 s)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599045000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599046000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599047000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599048000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599049000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599042800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1025.3 MB)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599050000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.3 KB, free: 1026.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1027.0 MB)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599043000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599043000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 150 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 878.9 KB, free: 1027.8 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.4 KB, free: 1028.7 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 150
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 149 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 149
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[149] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599044000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 152 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 152
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1030.4 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1031.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1032.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1033.0 MB)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 151 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 151
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[151] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599045000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 154 from persistence list
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 153 from persistence list
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[153] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599048000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.5 KB, free: 1033.8 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 153
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 154
17/08/01 14:50:50 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1034.7 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599046000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 156 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 887.9 KB, free: 1035.6 MB)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 155 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.4 KB, free: 1034.7 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 155
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 156
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[155] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599049000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1035.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1036.4 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599047000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 158 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 158
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 157 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 157
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[157] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599050000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.6 KB, free: 1037.3 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599048000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599048000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1038.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.0 KB, free: 1039.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.0 KB, free: 1040.7 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1041.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1042.4 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1043.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 878.9 KB, free: 1044.2 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1045.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1046.7 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1047.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1048.5 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.5 KB, free: 1049.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1050.2 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1051.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1051.9 MB)
17/08/01 14:50:51 ERROR scheduler.JobScheduler: Error generating jobs for time 1501599051000 ms
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 2.0 (TID 70) in 77942 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/1)
17/08/01 14:50:51 INFO cluster.YarnScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool
17/08/01 14:50:51 INFO scheduler.DAGScheduler: ResultStage 2 (start at NativeMethodAccessorImpl.java:0) finished in 77.943 s
17/08/01 14:50:51 INFO scheduler.ReceiverTracker: All of the receivers have deregistered successfully
17/08/01 14:50:51 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
17/08/01 14:50:51 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
17/08/01 14:50:51 INFO util.RecurringTimer: Stopped timer for JobGenerator after time 1501599051000
17/08/01 14:50:51 INFO scheduler.JobGenerator: Stopped JobGenerator
17/08/01 14:50:51 INFO scheduler.JobScheduler: Stopped JobScheduler
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@625d3c2d{/streaming,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@39edaf54{/streaming/batch,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@c773c0d{/static/streaming,null,UNAVAILABLE}
17/08/01 14:50:51 INFO streaming.StreamingContext: StreamingContext stopped successfully
17/08/01 14:50:51 INFO spark.SparkContext: Invoking stop() from shutdown hook
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 INFO server.ServerConnector: Stopped ServerConnector@37681deb{HTTP/1.1}{0.0.0.0:4040}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2aa3e9a6{/stages/stage/kill,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ee30613{/jobs/job/kill,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@15ec1723{/api,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14a7d27d{/,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@e4408f4{/static,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@8a515f3{/executors/threadDump/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@f241039{/executors/threadDump,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@79b9b526{/executors/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5fa63fa{/executors,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@77729e5a{/environment/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@242eb7e{/environment,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@49ddd2ed{/storage/rdd/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@699743b3{/storage/rdd,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@736cc380{/storage/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19cf381c{/storage,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4f734944{/stages/pool/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@b7d4da0{/stages/pool,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ab71bb9{/stages/stage/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5d37ce06{/stages/stage,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@27f55b38{/stages/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1159f15e{/stages,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ff970c2{/jobs/job/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@615b86f5{/jobs/job,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19bb7285{/jobs/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3b9882ec{/jobs,null,UNAVAILABLE}
17/08/01 14:50:51 INFO ui.SparkUI: Stopped Spark web UI at http://10.0.0.199:4040
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors
17/08/01 14:50:51 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
17/08/01 14:50:51 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
services=List(),
started=false)
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Stopped
17/08/01 14:50:51 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/08/01 14:50:51 INFO memory.MemoryStore: MemoryStore cleared
17/08/01 14:50:51 INFO storage.BlockManager: BlockManager stopped
17/08/01 14:50:51 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
17/08/01 14:50:51 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/08/01 14:50:51 INFO spark.SparkContext: Successfully stopped SparkContext
17/08/01 14:50:51 INFO util.ShutdownHookManager: Shutdown hook called
17/08/01 14:50:51 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25
17/08/01 14:50:51 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25/pyspark-c2380580-feb5-47ef-bc69-7b2c9ba38770
Closing worker...
From: Deon Griessel [mailto:dgriessel@searchtechnologies.com]
Sent: Monday, July 31, 2017 4:53 PM
To: user@spot.apache.org
Subject: Spot Ingest Proxy Error
I am trying to run the spark-ingest for the sample proxy log: Bluecoat_ProxySG_Sample.log
I run bash ./start_ingest_standalone.sh proxy 1
Then add the demo log to my collector_path. The worker dies, seems that it cannot find the hive metastore, probably a spark/hive configuration issue. Also, notice that the ip=unknown-ip-addr.
Would appreciate any help on this.
This is what I'm seeing in the worker screen output:
2017-07-31 18:54:15,855 - SPOT.INGEST.WORKER - INFO - Initializing kafka instance
2017-07-31 18:54:15,855 - SPOT.INGEST.WRK.PROXY - INFO - Creating Spark Job for topic: SPOT-INGEST-proxy-18_54_13
2017-07-31 18:54:15,856 - SPOT.INGEST.WRK.PROXY - INFO - SPOT.Utils: Executing: spark2-submit --master yarn --driver-memory 2325m --num-executors 1 --conf spark.executor.memory=2325m --conf spark.executor.cores=1 --jars /home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar /home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py -zk ip-10-0-0-199.ec2.internal:2181 -t SPOT-INGEST-proxy-18_54_13 -db spotdb -dt proxy -w 1 -bs 1
17/07/31 18:54:17 INFO spark.SparkContext: Running Spark version 2.1.0.cloudera1
17/07/31 18:54:17 WARN spark.SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0
17/07/31 18:54:17 INFO spark.SecurityManager: Changing view acls to: spot-user
17/07/31 18:54:17 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/07/31 18:54:17 INFO spark.SecurityManager: Changing view acls groups to:
17/07/31 18:54:17 INFO spark.SecurityManager: Changing modify acls groups to:
17/07/31 18:54:17 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/07/31 18:54:18 INFO util.Utils: Successfully started service 'sparkDriver' on port 54401.
17/07/31 18:54:18 INFO spark.SparkEnv: Registering MapOutputTracker
17/07/31 18:54:18 INFO spark.SparkEnv: Registering BlockManagerMaster
17/07/31 18:54:18 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
17/07/31 18:54:18 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
17/07/31 18:54:18 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-1db8ad37-9fcc-4799-b9e0-a206b6041d04
17/07/31 18:54:18 INFO memory.MemoryStore: MemoryStore started with capacity 1060.5 MB
17/07/31 18:54:18 INFO spark.SparkEnv: Registering OutputCommitCoordinator
17/07/31 18:54:18 INFO util.log: Logging initialized @2379ms
17/07/31 18:54:18 INFO server.Server: jetty-9.2.z-SNAPSHOT
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1a3b1e79{/jobs,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1a3b1e79%7b/jobs,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1f4da763{/jobs/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1f4da763%7b/jobs/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@232864a3{/jobs/job,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@232864a3%7b/jobs/job,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@30e71b5d{/jobs/job/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@30e71b5d%7b/jobs/job/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14b58fc0{/stages,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14b58fc0%7b/stages,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1bf090df{/stages/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1bf090df%7b/stages/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4eb72ecd{/stages/stage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4eb72ecd%7b/stages/stage,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5c61bd1a{/stages/stage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5c61bd1a%7b/stages/stage/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14c62558{/stages/pool,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14c62558%7b/stages/pool,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5cbdbf0f{/stages/pool/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5cbdbf0f%7b/stages/pool/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2d4aa15a{/storage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@2d4aa15a%7b/storage,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ea732f0{/storage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ea732f0%7b/storage/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@13016b86{/storage/rdd,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@13016b86%7b/storage/rdd,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@423a3429{/storage/rdd/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@423a3429%7b/storage/rdd/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7b3691b6{/environment,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7b3691b6%7b/environment,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3b9882ec{/environment/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3b9882ec%7b/environment/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19bb7285{/executors,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19bb7285%7b/executors,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@615b86f5{/executors/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@615b86f5%7b/executors/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ff970c2{/executors/threadDump,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3ff970c2%7b/executors/threadDump,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1159f15e{/executors/threadDump/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1159f15e%7b/executors/threadDump/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27f55b38{/static,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27f55b38%7b/static,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5d37ce06{/,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5d37ce06%7b/,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ab71bb9{/api,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ab71bb9%7b/api,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b7d4da0{/jobs/job/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@b7d4da0%7b/jobs/job/kill,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f734944{/stages/stage/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4f734944%7b/stages/stage/kill,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO server.ServerConnector: Started ServerConnector@188e6c7f{HTTP/1.1}{0.0.0.0:4040}<mailto:ServerConnector@188e6c7f%7bHTTP/1.1%7d%7b0.0.0.0:4040%7d>
17/07/31 18:54:18 INFO server.Server: Started @2532ms
17/07/31 18:54:18 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
17/07/31 18:54:18 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.0.0.199:4040
17/07/31 18:54:18 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/07/31 18:54:19 INFO client.RMProxy: Connecting to ResourceManager at ip-10-0-0-199.ec2.internal/10.0.0.199:8032
17/07/31 18:54:19 INFO yarn.Client: Requesting a new application from cluster with 3 NodeManagers
17/07/31 18:54:19 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (2725 MB per container)
17/07/31 18:54:19 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/07/31 18:54:19 INFO yarn.Client: Setting up container launch context for our AM
17/07/31 18:54:19 INFO yarn.Client: Setting up the launch environment for our AM container
17/07/31 18:54:19 INFO yarn.Client: Preparing resources for our AM container
17/07/31 18:54:21 INFO yarn.Client: Uploading resource file:/home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501523941584_0005/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar
17/07/31 18:54:21 INFO yarn.Client: Uploading resource file:/tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40/__spark_conf__5838957838871043110.zip -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501523941584_0005/__spark_conf__.zip
17/07/31 18:54:21 INFO spark.SecurityManager: Changing view acls to: spot-user
17/07/31 18:54:21 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/07/31 18:54:21 INFO spark.SecurityManager: Changing view acls groups to:
17/07/31 18:54:21 INFO spark.SecurityManager: Changing modify acls groups to:
17/07/31 18:54:21 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/07/31 18:54:21 INFO yarn.Client: Submitting application application_1501523941584_0005 to ResourceManager
17/07/31 18:54:21 INFO impl.YarnClientImpl: Submitted application application_1501523941584_0005
17/07/31 18:54:21 INFO cluster.SchedulerExtensionServices: Starting Yarn extension services with app application_1501523941584_0005 and attemptId None
17/07/31 18:54:22 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:22 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.users.spot-user
start time: 1501527261635
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005/
user: spot-user
17/07/31 18:54:23 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:24 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:25 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/07/31 18:54:25 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> ip-10-0-0-199.ec2.internal, PROXY_URI_BASES -> http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005), /proxy/application_1501523941584_0005
17/07/31 18:54:25 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/07/31 18:54:25 INFO yarn.Client: Application report for application_1501523941584_0005 (state: RUNNING)
17/07/31 18:54:25 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 10.0.0.127
ApplicationMaster RPC port: 0
queue: root.users.spot-user
start time: 1501527261635
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005/
user: spot-user
17/07/31 18:54:25 INFO cluster.YarnClientSchedulerBackend: Application application_1501523941584_0005 has started running.
17/07/31 18:54:25 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 59442.
17/07/31 18:54:25 INFO netty.NettyBlockTransferService: Server created on 10.0.0.199:59442
17/07/31 18:54:25 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/07/31 18:54:25 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManagerMasterEndpoint: Registering block manager 10.0.0.199:59442 with 1060.5 MB RAM, BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManager: external shuffle service port = 7337
17/07/31 18:54:25 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3e93262b{/metrics/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3e93262b%7b/metrics/json,null,AVAILABLE%7d>
17/07/31 18:54:25 INFO scheduler.EventLoggingListener: Logging events to hdfs://ip-10-0-0-199.ec2.internal:8020/user/spark/spark2ApplicationHistory/application_1501523941584_0005
17/07/31 18:54:25 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/07/31 18:54:28 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.229:51685) with ID 1
17/07/31 18:54:28 INFO spark.ExecutorAllocationManager: New executor 1 has registered (new total is 1)
17/07/31 18:54:28 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-229.ec2.internal:35659 with 1060.5 MB RAM, BlockManagerId(1, ip-10-0-0-229.ec2.internal, 35659, None)
17/07/31 18:54:28 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/07/31 18:54:29 INFO internal.SharedState: Warehouse path is 'file:/home/spot-user/spot-ingest/spark-warehouse'.
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7a9c6710{/SQL,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7a9c6710%7b/SQL,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@141768e4{/SQL/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@141768e4%7b/SQL/json,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1e367d45{/SQL/execution,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1e367d45%7b/SQL/execution,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5554f0a0{/SQL/execution/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5554f0a0%7b/SQL/execution/json,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@144776d3{/static/sql,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@144776d3%7b/static/sql,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO hive.HiveUtils: Initializing HiveMetastoreConnection version 1.1.0 using Spark classes.
17/07/31 18:54:30 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
17/07/31 18:54:30 INFO metastore.ObjectStore: ObjectStore, initialize called
17/07/31 18:54:30 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
17/07/31 18:54:30 INFO DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
17/07/31 18:54:30 WARN bonecp.BoneCPConfig: Max Connections < 1. Setting to 20
17/07/31 18:54:31 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
17/07/31 18:54:32 WARN bonecp.BoneCPConfig: Max Connections < 1. Setting to 20
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0<ma...@0>" since the connection used is closing
17/07/31 18:54:32 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
17/07/31 18:54:32 INFO metastore.ObjectStore: Initialized ObjectStore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: Added admin role in metastore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: Added public role in metastore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_all_functions from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_all_functions
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_all_functions
17/07/31 18:54:33 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_all_functions start=1501527273291 end=1501527273356 duration=65 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=0 error=false>
17/07/31 18:54:33 INFO session.SessionState: Created local directory: /tmp/aaecce13-256b-4b91-b661-9046c9c418c6_resources
17/07/31 18:54:33 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6
17/07/31 18:54:33 INFO session.SessionState: Created local directory: /tmp/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6
17/07/31 18:54:33 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6/_tmp_space.db
17/07/31 18:54:33 INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
17/07/31 18:54:33 INFO client.HiveClientImpl: Warehouse location for Hive client (version 1.1.0) is file:/home/spot-user/spot-ingest/spark-warehouse
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_database: default
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: default
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527273406 end=1501527273408 duration=2 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=0 error=false>
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_database: global_temp
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: global_temp
17/07/31 18:54:33 WARN metastore.ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527273409 end=1501527273412 duration=3 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=-1 error=true>
17/07/31 18:54:33 WARN streaming.StreamingContext: Dynamic Allocation is enabled for this application. Enabling Dynamic allocation for Spark Streaming applications can cause data loss if Write Ahead Log is not enabled for non-replayable sources like Flume. See the programming guide for details on how to enable the Write Ahead Log.
17/07/31 18:54:33 INFO spark.SparkContext: Starting job: start at NativeMethodAccessorImpl.java:0
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Registering RDD 1 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Got job 0 (start at NativeMethodAccessorImpl.java:0) with 20 output partitions
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/07/31 18:54:33 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 3.1 KB, free 1060.5 MB)
17/07/31 18:54:33 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 1988.0 B, free 1060.5 MB)
17/07/31 18:54:33 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.0.0.199:59442 (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:54:33 INFO spark.SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Submitting 50 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO cluster.YarnScheduler: Adding task set 0.0 with 50 tasks
17/07/31 18:54:33 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, ip-10-0-0-229.ec2.internal, executor 1, partition 1, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 669 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, ip-10-0-0-229.ec2.internal, executor 1, partition 2, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 77 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3, ip-10-0-0-229.ec2.internal, executor 1, partition 3, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 72 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 0.0 (TID 4, ip-10-0-0-229.ec2.internal, executor 1, partition 4, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 69 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 0.0 (TID 5, ip-10-0-0-229.ec2.internal, executor 1, partition 5, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 59 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 0.0 (TID 6, ip-10-0-0-229.ec2.internal, executor 1, partition 6, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 0.0 (TID 7, ip-10-0-0-229.ec2.internal, executor 1, partition 7, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 0.0 (TID 8, ip-10-0-0-229.ec2.internal, executor 1, partition 8, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 0.0 (TID 7) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/50)
17/07/31 18:54:34 INFO spark.ExecutorAllocationManager: Requesting 1 new executor because tasks are backlogged (new desired total will be 2)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 0.0 (TID 9, ip-10-0-0-229.ec2.internal, executor 1, partition 9, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 0.0 (TID 8) in 58 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 0.0 (TID 10, ip-10-0-0-229.ec2.internal, executor 1, partition 10, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 0.0 (TID 9) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 0.0 (TID 11, ip-10-0-0-229.ec2.internal, executor 1, partition 11, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 0.0 (TID 10) in 52 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 0.0 (TID 12, ip-10-0-0-229.ec2.internal, executor 1, partition 12, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 0.0 (TID 11) in 58 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 0.0 (TID 13, ip-10-0-0-229.ec2.internal, executor 1, partition 13, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 0.0 (TID 12) in 51 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 0.0 (TID 14, ip-10-0-0-229.ec2.internal, executor 1, partition 14, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 0.0 (TID 13) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 0.0 (TID 15, ip-10-0-0-229.ec2.internal, executor 1, partition 15, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 0.0 (TID 14) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 0.0 (TID 16, ip-10-0-0-229.ec2.internal, executor 1, partition 16, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 0.0 (TID 15) in 63 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 0.0 (TID 17, ip-10-0-0-229.ec2.internal, executor 1, partition 17, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 0.0 (TID 16) in 50 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 0.0 (TID 18, ip-10-0-0-229.ec2.internal, executor 1, partition 18, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 0.0 (TID 17) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 0.0 (TID 19, ip-10-0-0-229.ec2.internal, executor 1, partition 19, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 0.0 (TID 18) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 0.0 (TID 20, ip-10-0-0-229.ec2.internal, executor 1, partition 20, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 0.0 (TID 19) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 0.0 (TID 21, ip-10-0-0-229.ec2.internal, executor 1, partition 21, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 0.0 (TID 20) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (21/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 0.0 (TID 22, ip-10-0-0-229.ec2.internal, executor 1, partition 22, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 0.0 (TID 21) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (22/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 0.0 (TID 23, ip-10-0-0-229.ec2.internal, executor 1, partition 23, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 0.0 (TID 22) in 47 ms on ip-10-0-0-229.ec2.internal (executor 1) (23/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 0.0 (TID 24, ip-10-0-0-229.ec2.internal, executor 1, partition 24, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 0.0 (TID 23) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (24/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 0.0 (TID 25, ip-10-0-0-229.ec2.internal, executor 1, partition 25, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 0.0 (TID 24) in 46 ms on ip-10-0-0-229.ec2.internal (executor 1) (25/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 0.0 (TID 26, ip-10-0-0-229.ec2.internal, executor 1, partition 26, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 0.0 (TID 25) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (26/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 0.0 (TID 27, ip-10-0-0-229.ec2.internal, executor 1, partition 27, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 0.0 (TID 26) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (27/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 0.0 (TID 28, ip-10-0-0-229.ec2.internal, executor 1, partition 28, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 0.0 (TID 27) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (28/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 0.0 (TID 29, ip-10-0-0-229.ec2.internal, executor 1, partition 29, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 0.0 (TID 28) in 39 ms on ip-10-0-0-229.ec2.internal (executor 1) (29/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 0.0 (TID 30, ip-10-0-0-229.ec2.internal, executor 1, partition 30, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 0.0 (TID 29) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (30/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 0.0 (TID 31, ip-10-0-0-229.ec2.internal, executor 1, partition 31, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 0.0 (TID 30) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (31/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 0.0 (TID 32, ip-10-0-0-229.ec2.internal, executor 1, partition 32, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 0.0 (TID 31) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (32/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 0.0 (TID 33, ip-10-0-0-229.ec2.internal, executor 1, partition 33, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 0.0 (TID 32) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (33/50)
17/07/31 18:54:35 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 4)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 0.0 (TID 34, ip-10-0-0-229.ec2.internal, executor 1, partition 34, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 0.0 (TID 33) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (34/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 0.0 (TID 35, ip-10-0-0-229.ec2.internal, executor 1, partition 35, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 0.0 (TID 34) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (35/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 0.0 (TID 36, ip-10-0-0-229.ec2.internal, executor 1, partition 36, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 0.0 (TID 35) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (36/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 0.0 (TID 37, ip-10-0-0-229.ec2.internal, executor 1, partition 37, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 0.0 (TID 36) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (37/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 0.0 (TID 38, ip-10-0-0-229.ec2.internal, executor 1, partition 38, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 0.0 (TID 37) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (38/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 0.0 (TID 39, ip-10-0-0-229.ec2.internal, executor 1, partition 39, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 0.0 (TID 38) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (39/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 0.0 (TID 40, ip-10-0-0-229.ec2.internal, executor 1, partition 40, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 0.0 (TID 39) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (40/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 0.0 (TID 41, ip-10-0-0-229.ec2.internal, executor 1, partition 41, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 0.0 (TID 40) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (41/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 0.0 (TID 42, ip-10-0-0-229.ec2.internal, executor 1, partition 42, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 0.0 (TID 41) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (42/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 0.0 (TID 43, ip-10-0-0-229.ec2.internal, executor 1, partition 43, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 0.0 (TID 42) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (43/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 0.0 (TID 44, ip-10-0-0-229.ec2.internal, executor 1, partition 44, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 0.0 (TID 43) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (44/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 0.0 (TID 45, ip-10-0-0-229.ec2.internal, executor 1, partition 45, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 0.0 (TID 44) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (45/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 0.0 (TID 46, ip-10-0-0-229.ec2.internal, executor 1, partition 46, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 0.0 (TID 45) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (46/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 0.0 (TID 47, ip-10-0-0-229.ec2.internal, executor 1, partition 47, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 0.0 (TID 46) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (47/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 0.0 (TID 48, ip-10-0-0-229.ec2.internal, executor 1, partition 48, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 0.0 (TID 47) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (48/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 0.0 (TID 49, ip-10-0-0-229.ec2.internal, executor 1, partition 49, PROCESS_LOCAL, 6676 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 0.0 (TID 48) in 29 ms on ip-10-0-0-229.ec2.internal (executor 1) (49/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 0.0 (TID 49) in 27 ms on ip-10-0-0-229.ec2.internal (executor 1) (50/50)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (start at NativeMethodAccessorImpl.java:0) finished in 2.499 s
17/07/31 18:54:36 INFO cluster.YarnScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/07/31 18:54:36 INFO scheduler.DAGScheduler: looking for newly runnable stages
17/07/31 18:54:36 INFO scheduler.DAGScheduler: running: Set()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 1)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: failed: Set()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.2 KB, free 1060.5 MB)
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 1956.0 B, free 1060.5 MB)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.0.0.199:59442 (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting 20 missing tasks from ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Adding task set 1.0 with 20 tasks
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 50, ip-10-0-0-229.ec2.internal, executor 1, partition 0, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.0.0.229:51685
17/07/31 18:54:36 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 283 bytes
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.0 (TID 51, ip-10-0-0-229.ec2.internal, executor 1, partition 1, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 50) in 91 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 1.0 (TID 52, ip-10-0-0-229.ec2.internal, executor 1, partition 2, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 1.0 (TID 51) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 1.0 (TID 53, ip-10-0-0-229.ec2.internal, executor 1, partition 3, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 1.0 (TID 52) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 1.0 (TID 54, ip-10-0-0-229.ec2.internal, executor 1, partition 4, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 1.0 (TID 53) in 24 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 1.0 (TID 55, ip-10-0-0-229.ec2.internal, executor 1, partition 5, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 1.0 (TID 54) in 24 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 1.0 (TID 56, ip-10-0-0-229.ec2.internal, executor 1, partition 6, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 1.0 (TID 55) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 1.0 (TID 57, ip-10-0-0-229.ec2.internal, executor 1, partition 7, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 1.0 (TID 56) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 1.0 (TID 58, ip-10-0-0-229.ec2.internal, executor 1, partition 8, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 1.0 (TID 57) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 1.0 (TID 59, ip-10-0-0-229.ec2.internal, executor 1, partition 9, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 1.0 (TID 58) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 1.0 (TID 60, ip-10-0-0-229.ec2.internal, executor 1, partition 10, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 1.0 (TID 59) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 1.0 (TID 61, ip-10-0-0-229.ec2.internal, executor 1, partition 11, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 1.0 (TID 60) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 1.0 (TID 62, ip-10-0-0-229.ec2.internal, executor 1, partition 12, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 1.0 (TID 61) in 16 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 1.0 (TID 63, ip-10-0-0-229.ec2.internal, executor 1, partition 13, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 1.0 (TID 62) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 1.0 (TID 64, ip-10-0-0-229.ec2.internal, executor 1, partition 14, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 1.0 (TID 63) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 1.0 (TID 65, ip-10-0-0-229.ec2.internal, executor 1, partition 15, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 1.0 (TID 64) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 1.0 (TID 66, ip-10-0-0-229.ec2.internal, executor 1, partition 16, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 1.0 (TID 65) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 1.0 (TID 67, ip-10-0-0-229.ec2.internal, executor 1, partition 17, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 1.0 (TID 66) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 1.0 (TID 68, ip-10-0-0-229.ec2.internal, executor 1, partition 18, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 1.0 (TID 67) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 1.0 (TID 69, ip-10-0-0-229.ec2.internal, executor 1, partition 19, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 1.0 (TID 68) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 1.0 (TID 69) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/20)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool
17/07/31 18:54:36 INFO scheduler.DAGScheduler: ResultStage 1 (start at NativeMethodAccessorImpl.java:0) finished in 0.416 s
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Job 0 finished: start at NativeMethodAccessorImpl.java:0, took 3.184418 s
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: Starting 1 receivers
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: ReceiverTracker started
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Initialized and validated org.apache.spark.streaming.kafka.KafkaInputDStream@7b67308f<ma...@7b67308f>
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Initialized and validated org.apache.spark.streaming.api.python.PythonTransformedDStream@33a53b94<ma...@33a53b94>
17/07/31 18:54:36 INFO dstream.ForEachDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO dstream.ForEachDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO dstream.ForEachDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO dstream.ForEachDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO dstream.ForEachDStream: Initialized and validated org.apache.spark.streaming.dstream.ForEachDStream@24ea8c09<ma...@24ea8c09>
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Got job 1 (start at NativeMethodAccessorImpl.java:0) with 1 output partitions
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Final stage: ResultStage 2 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620), which has no missing parents
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: Receiver 0 started
17/07/31 18:54:36 INFO util.RecurringTimer: Started timer for JobGenerator at time 1501527277000
17/07/31 18:54:36 INFO scheduler.JobGenerator: Started JobGenerator at 1501527277000 ms
17/07/31 18:54:36 INFO scheduler.JobScheduler: Started JobScheduler
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19fd4723{/streaming,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19fd4723%7b/streaming,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@43b0e42{/streaming/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@43b0e42%7b/streaming/json,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@22116fc3{/streaming/batch,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@22116fc3%7b/streaming/batch,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27399762{/streaming/batch/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27399762%7b/streaming/batch/json,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@70d301d6{/static/streaming,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@70d301d6%7b/static/streaming,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO streaming.StreamingContext: StreamingContext started
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 74.4 KB, free 1060.4 MB)
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 27.3 KB, free 1060.4 MB)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.0.0.199:59442 (size: 27.3 KB, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Adding task set 2.0 with 1 tasks
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 2.0 (TID 70, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 7067 bytes)
17/07/31 18:54:37 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 27.3 KB, free: 1060.5 MB)
17/07/31 18:54:37 INFO scheduler.JobScheduler: Added jobs for time 1501527277000 ms
17/07/31 18:54:37 INFO scheduler.JobScheduler: Starting job streaming job 1501527277000 ms.0 from job set of time 1501527277000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:37 INFO scheduler.JobScheduler: Finished job streaming job 1501527277000 ms.0 from job set of time 1501527277000 ms
17/07/31 18:54:37 INFO scheduler.JobScheduler: Total delay: 0.071 s for time 1501527277000 ms (execution: 0.014 s)
17/07/31 18:54:37 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/07/31 18:54:37 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/07/31 18:54:37 INFO scheduler.ReceiverTracker: Registered receiver for stream 0 from 10.0.0.229:51685
17/07/31 18:54:38 INFO scheduler.JobScheduler: Added jobs for time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.JobScheduler: Starting job streaming job 1501527278000 ms.0 from job set of time 1501527278000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:38 INFO scheduler.JobScheduler: Finished job streaming job 1501527278000 ms.0 from job set of time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.JobScheduler: Total delay: 0.045 s for time 1501527278000 ms (execution: 0.010 s)
17/07/31 18:54:38 INFO python.PythonRDD: Removing RDD 5 from persistence list
17/07/31 18:54:38 INFO rdd.BlockRDD: Removing RDD 4 from persistence list
17/07/31 18:54:38 INFO storage.BlockManager: Removing RDD 5
17/07/31 18:54:38 INFO storage.BlockManager: Removing RDD 4
17/07/31 18:54:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[4] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/07/31 18:54:38 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/07/31 18:54:38 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.151:57862) with ID 2
17/07/31 18:54:38 INFO spark.ExecutorAllocationManager: New executor 2 has registered (new total is 2)
17/07/31 18:54:38 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-151.ec2.internal:54365 with 1060.5 MB RAM, BlockManagerId(2, ip-10-0-0-151.ec2.internal, 54365, None)
17/07/31 18:54:39 INFO scheduler.JobScheduler: Added jobs for time 1501527279000 ms
17/07/31 18:54:39 INFO scheduler.JobScheduler: Starting job streaming job 1501527279000 ms.0 from job set of time 1501527279000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:39 INFO scheduler.JobScheduler: Finished job streaming job 1501527279000 ms.0 from job set of time 1501527279000 ms
17/07/31 18:54:39 INFO python.PythonRDD: Removing RDD 7 from persistence list
17/07/31 18:54:39 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527279000 ms (execution: 0.006 s)
17/07/31 18:54:39 INFO storage.BlockManager: Removing RDD 7
17/07/31 18:54:39 INFO rdd.BlockRDD: Removing RDD 6 from persistence list
17/07/31 18:54:39 INFO storage.BlockManager: Removing RDD 6
17/07/31 18:54:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[6] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527279000 ms
17/07/31 18:54:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527277000 ms
17/07/31 18:54:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527277000 ms
17/07/31 18:54:40 INFO scheduler.JobScheduler: Added jobs for time 1501527280000 ms
17/07/31 18:54:40 INFO scheduler.JobScheduler: Starting job streaming job 1501527280000 ms.0 from job set of time 1501527280000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:40 INFO scheduler.JobScheduler: Finished job streaming job 1501527280000 ms.0 from job set of time 1501527280000 ms
17/07/31 18:54:40 INFO python.PythonRDD: Removing RDD 9 from persistence list
17/07/31 18:54:40 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501527280000 ms (execution: 0.009 s)
17/07/31 18:54:40 INFO storage.BlockManager: Removing RDD 9
17/07/31 18:54:40 INFO rdd.BlockRDD: Removing RDD 8 from persistence list
17/07/31 18:54:40 INFO storage.BlockManager: Removing RDD 8
17/07/31 18:54:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[8] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527280000 ms
17/07/31 18:54:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527278000 ms
17/07/31 18:54:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527278000 ms
17/07/31 18:54:41 INFO scheduler.JobScheduler: Added jobs for time 1501527281000 ms
17/07/31 18:54:41 INFO scheduler.JobScheduler: Starting job streaming job 1501527281000 ms.0 from job set of time 1501527281000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:41 INFO scheduler.JobScheduler: Finished job streaming job 1501527281000 ms.0 from job set of time 1501527281000 ms
17/07/31 18:54:41 INFO python.PythonRDD: Removing RDD 11 from persistence list
17/07/31 18:54:41 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527281000 ms (execution: 0.006 s)
17/07/31 18:54:41 INFO storage.BlockManager: Removing RDD 11
17/07/31 18:54:41 INFO rdd.BlockRDD: Removing RDD 10 from persistence list
17/07/31 18:54:41 INFO storage.BlockManager: Removing RDD 10
17/07/31 18:54:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[10] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527281000 ms
17/07/31 18:54:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527279000 ms
17/07/31 18:54:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527279000 ms
17/07/31 18:54:42 INFO scheduler.JobScheduler: Added jobs for time 1501527282000 ms
17/07/31 18:54:42 INFO scheduler.JobScheduler: Starting job streaming job 1501527282000 ms.0 from job set of time 1501527282000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:42 INFO scheduler.JobScheduler: Finished job streaming job 1501527282000 ms.0 from job set of time 1501527282000 ms
17/07/31 18:54:42 INFO python.PythonRDD: Removing RDD 13 from persistence list
17/07/31 18:54:42 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527282000 ms (execution: 0.006 s)
17/07/31 18:54:42 INFO storage.BlockManager: Removing RDD 13
17/07/31 18:54:42 INFO rdd.BlockRDD: Removing RDD 12 from persistence list
17/07/31 18:54:42 INFO storage.BlockManager: Removing RDD 12
17/07/31 18:54:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[12] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527282000 ms
17/07/31 18:54:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527280000 ms
17/07/31 18:54:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527280000 ms
17/07/31 18:54:43 INFO scheduler.JobScheduler: Added jobs for time 1501527283000 ms
17/07/31 18:54:43 INFO scheduler.JobScheduler: Starting job streaming job 1501527283000 ms.0 from job set of time 1501527283000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:43 INFO scheduler.JobScheduler: Finished job streaming job 1501527283000 ms.0 from job set of time 1501527283000 ms
17/07/31 18:54:43 INFO python.PythonRDD: Removing RDD 15 from persistence list
17/07/31 18:54:43 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527283000 ms (execution: 0.008 s)
17/07/31 18:54:43 INFO storage.BlockManager: Removing RDD 15
17/07/31 18:54:43 INFO rdd.BlockRDD: Removing RDD 14 from persistence list
17/07/31 18:54:43 INFO storage.BlockManager: Removing RDD 14
17/07/31 18:54:43 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[14] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527283000 ms
17/07/31 18:54:43 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527281000 ms
17/07/31 18:54:43 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527281000 ms
17/07/31 18:54:44 INFO scheduler.JobScheduler: Added jobs for time 1501527284000 ms
17/07/31 18:54:44 INFO scheduler.JobScheduler: Starting job streaming job 1501527284000 ms.0 from job set of time 1501527284000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:44 INFO scheduler.JobScheduler: Finished job streaming job 1501527284000 ms.0 from job set of time 1501527284000 ms
17/07/31 18:54:44 INFO python.PythonRDD: Removing RDD 17 from persistence list
17/07/31 18:54:44 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527284000 ms (execution: 0.016 s)
17/07/31 18:54:44 INFO storage.BlockManager: Removing RDD 17
17/07/31 18:54:44 INFO rdd.BlockRDD: Removing RDD 16 from persistence list
17/07/31 18:54:44 INFO storage.BlockManager: Removing RDD 16
17/07/31 18:54:44 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[16] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527284000 ms
17/07/31 18:54:44 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527282000 ms
17/07/31 18:54:44 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527282000 ms
17/07/31 18:54:45 INFO scheduler.JobScheduler: Added jobs for time 1501527285000 ms
17/07/31 18:54:45 INFO scheduler.JobScheduler: Starting job streaming job 1501527285000 ms.0 from job set of time 1501527285000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:45 INFO scheduler.JobScheduler: Finished job streaming job 1501527285000 ms.0 from job set of time 1501527285000 ms
17/07/31 18:54:45 INFO python.PythonRDD: Removing RDD 19 from persistence list
17/07/31 18:54:45 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527285000 ms (execution: 0.007 s)
17/07/31 18:54:45 INFO rdd.BlockRDD: Removing RDD 18 from persistence list
17/07/31 18:54:45 INFO storage.BlockManager: Removing RDD 19
17/07/31 18:54:45 INFO storage.BlockManager: Removing RDD 18
17/07/31 18:54:45 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[18] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527285000 ms
17/07/31 18:54:45 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527283000 ms
17/07/31 18:54:45 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527283000 ms
17/07/31 18:54:46 INFO scheduler.JobScheduler: Added jobs for time 1501527286000 ms
17/07/31 18:54:46 INFO scheduler.JobScheduler: Starting job streaming job 1501527286000 ms.0 from job set of time 1501527286000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:46 INFO scheduler.JobScheduler: Finished job streaming job 1501527286000 ms.0 from job set of time 1501527286000 ms
17/07/31 18:54:46 INFO python.PythonRDD: Removing RDD 21 from persistence list
17/07/31 18:54:46 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527286000 ms (execution: 0.008 s)
17/07/31 18:54:46 INFO storage.BlockManager: Removing RDD 21
17/07/31 18:54:46 INFO rdd.BlockRDD: Removing RDD 20 from persistence list
17/07/31 18:54:46 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[20] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527286000 ms
17/07/31 18:54:46 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527284000 ms
17/07/31 18:54:46 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527284000 ms
17/07/31 18:54:46 INFO storage.BlockManager: Removing RDD 20
17/07/31 18:54:47 INFO scheduler.JobScheduler: Added jobs for time 1501527287000 ms
17/07/31 18:54:47 INFO scheduler.JobScheduler: Starting job streaming job 1501527287000 ms.0 from job set of time 1501527287000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:47 INFO scheduler.JobScheduler: Finished job streaming job 1501527287000 ms.0 from job set of time 1501527287000 ms
17/07/31 18:54:47 INFO python.PythonRDD: Removing RDD 23 from persistence list
17/07/31 18:54:47 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527287000 ms (execution: 0.006 s)
17/07/31 18:54:47 INFO storage.BlockManager: Removing RDD 23
17/07/31 18:54:47 INFO rdd.BlockRDD: Removing RDD 22 from persistence list
17/07/31 18:54:47 INFO storage.BlockManager: Removing RDD 22
17/07/31 18:54:47 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[22] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527287000 ms
17/07/31 18:54:47 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527285000 ms
17/07/31 18:54:47 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527285000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Added jobs for time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Starting job streaming job 1501527288000 ms.0 from job set of time 1501527288000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:48 INFO scheduler.JobScheduler: Finished job streaming job 1501527288000 ms.0 from job set of time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527288000 ms (execution: 0.008 s)
17/07/31 18:54:48 INFO python.PythonRDD: Removing RDD 25 from persistence list
17/07/31 18:54:48 INFO storage.BlockManager: Removing RDD 25
17/07/31 18:54:48 INFO rdd.BlockRDD: Removing RDD 24 from persistence list
17/07/31 18:54:48 INFO storage.BlockManager: Removing RDD 24
17/07/31 18:54:48 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[24] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527286000 ms
17/07/31 18:54:48 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527286000 ms
17/07/31 18:54:49 INFO scheduler.JobScheduler: Added jobs for time 1501527289000 ms
17/07/31 18:54:49 INFO scheduler.JobScheduler: Starting job streaming job 1501527289000 ms.0 from job set of time 1501527289000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:49 INFO scheduler.JobScheduler: Finished job streaming job 1501527289000 ms.0 from job set of time 1501527289000 ms
17/07/31 18:54:49 INFO python.PythonRDD: Removing RDD 27 from persistence list
17/07/31 18:54:49 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501527289000 ms (execution: 0.008 s)
17/07/31 18:54:49 INFO storage.BlockManager: Removing RDD 27
17/07/31 18:54:49 INFO rdd.BlockRDD: Removing RDD 26 from persistence list
17/07/31 18:54:49 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[26] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527289000 ms
17/07/31 18:54:49 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527287000 ms
17/07/31 18:54:49 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527287000 ms
17/07/31 18:54:49 INFO storage.BlockManager: Removing RDD 26
17/07/31 18:54:50 INFO scheduler.JobScheduler: Added jobs for time 1501527290000 ms
17/07/31 18:54:50 INFO scheduler.JobScheduler: Starting job streaming job 1501527290000 ms.0 from job set of time 1501527290000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:50 INFO scheduler.JobScheduler: Finished job streaming job 1501527290000 ms.0 from job set of time 1501527290000 ms
17/07/31 18:54:50 INFO python.PythonRDD: Removing RDD 29 from persistence list
17/07/31 18:54:50 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527290000 ms (execution: 0.008 s)
17/07/31 18:54:50 INFO storage.BlockManager: Removing RDD 29
17/07/31 18:54:50 INFO rdd.BlockRDD: Removing RDD 28 from persistence list
17/07/31 18:54:50 INFO storage.BlockManager: Removing RDD 28
17/07/31 18:54:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[28] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527290000 ms
17/07/31 18:54:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527288000 ms
17/07/31 18:54:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527288000 ms
17/07/31 18:54:51 INFO scheduler.JobScheduler: Added jobs for time 1501527291000 ms
17/07/31 18:54:51 INFO scheduler.JobScheduler: Starting job streaming job 1501527291000 ms.0 from job set of time 1501527291000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:51 INFO scheduler.JobScheduler: Finished job streaming job 1501527291000 ms.0 from job set of time 1501527291000 ms
17/07/31 18:54:51 INFO python.PythonRDD: Removing RDD 31 from persistence list
17/07/31 18:54:51 INFO scheduler.JobScheduler: Total delay: 0.033 s for time 1501527291000 ms (execution: 0.018 s)
17/07/31 18:54:51 INFO storage.BlockManager: Removing RDD 31
17/07/31 18:54:51 INFO rdd.BlockRDD: Removing RDD 30 from persistence list
17/07/31 18:54:51 INFO storage.BlockManager: Removing RDD 30
17/07/31 18:54:51 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[30] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527291000 ms
17/07/31 18:54:51 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527289000 ms
17/07/31 18:54:51 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527289000 ms
17/07/31 18:54:52 INFO scheduler.JobScheduler: Added jobs for time 1501527292000 ms
17/07/31 18:54:52 INFO scheduler.JobScheduler: Starting job streaming job 1501527292000 ms.0 from job set of time 1501527292000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:52 INFO scheduler.JobScheduler: Finished job streaming job 1501527292000 ms.0 from job set of time 1501527292000 ms
17/07/31 18:54:52 INFO python.PythonRDD: Removing RDD 33 from persistence list
17/07/31 18:54:52 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501527292000 ms (execution: 0.008 s)
17/07/31 18:54:52 INFO storage.BlockManager: Removing RDD 33
17/07/31 18:54:52 INFO rdd.BlockRDD: Removing RDD 32 from persistence list
17/07/31 18:54:52 INFO storage.BlockManager: Removing RDD 32
17/07/31 18:54:52 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[32] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527292000 ms
17/07/31 18:54:52 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527290000 ms
17/07/31 18:54:52 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527290000 ms
17/07/31 18:54:53 INFO scheduler.JobScheduler: Added jobs for time 1501527293000 ms
17/07/31 18:54:53 INFO scheduler.JobScheduler: Starting job streaming job 1501527293000 ms.0 from job set of time 1501527293000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:53 INFO scheduler.JobScheduler: Finished job streaming job 1501527293000 ms.0 from job set of time 1501527293000 ms
17/07/31 18:54:53 INFO python.PythonRDD: Removing RDD 35 from persistence list
17/07/31 18:54:53 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501527293000 ms (execution: 0.008 s)
17/07/31 18:54:53 INFO storage.BlockManager: Removing RDD 35
17/07/31 18:54:53 INFO rdd.BlockRDD: Removing RDD 34 from persistence list
17/07/31 18:54:53 INFO storage.BlockManager: Removing RDD 34
17/07/31 18:54:53 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[34] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527293000 ms
17/07/31 18:54:53 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527291000 ms
17/07/31 18:54:53 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527291000 ms
17/07/31 18:54:54 INFO scheduler.JobScheduler: Added jobs for time 1501527294000 ms
17/07/31 18:54:54 INFO scheduler.JobScheduler: Starting job streaming job 1501527294000 ms.0 from job set of time 1501527294000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:54 INFO scheduler.JobScheduler: Finished job streaming job 1501527294000 ms.0 from job set of time 1501527294000 ms
17/07/31 18:54:54 INFO python.PythonRDD: Removing RDD 37 from persistence list
17/07/31 18:54:54 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527294000 ms (execution: 0.006 s)
17/07/31 18:54:54 INFO storage.BlockManager: Removing RDD 37
17/07/31 18:54:54 INFO rdd.BlockRDD: Removing RDD 36 from persistence list
17/07/31 18:54:54 INFO storage.BlockManager: Removing RDD 36
17/07/31 18:54:54 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[36] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527294000 ms
17/07/31 18:54:54 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527292000 ms
17/07/31 18:54:54 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527292000 ms
17/07/31 18:54:55 INFO scheduler.JobScheduler: Added jobs for time 1501527295000 ms
17/07/31 18:54:55 INFO scheduler.JobScheduler: Starting job streaming job 1501527295000 ms.0 from job set of time 1501527295000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:55 INFO scheduler.JobScheduler: Finished job streaming job 1501527295000 ms.0 from job set of time 1501527295000 ms
17/07/31 18:54:55 INFO python.PythonRDD: Removing RDD 39 from persistence list
17/07/31 18:54:55 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527295000 ms (execution: 0.007 s)
17/07/31 18:54:55 INFO storage.BlockManager: Removing RDD 39
17/07/31 18:54:55 INFO rdd.BlockRDD: Removing RDD 38 from persistence list
17/07/31 18:54:55 INFO storage.BlockManager: Removing RDD 38
17/07/31 18:54:55 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[38] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527295000 ms
17/07/31 18:54:55 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527293000 ms
17/07/31 18:54:55 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527293000 ms
17/07/31 18:54:56 INFO scheduler.JobScheduler: Added jobs for time 1501527296000 ms
17/07/31 18:54:56 INFO scheduler.JobScheduler: Starting job streaming job 1501527296000 ms.0 from job set of time 1501527296000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:56 INFO scheduler.JobScheduler: Finished job streaming job 1501527296000 ms.0 from job set of time 1501527296000 ms
17/07/31 18:54:56 INFO python.PythonRDD: Removing RDD 41 from persistence list
17/07/31 18:54:56 INFO scheduler.JobScheduler: Total delay: 0.031 s for time 1501527296000 ms (execution: 0.007 s)
17/07/31 18:54:56 INFO storage.BlockManager: Removing RDD 41
17/07/31 18:54:56 INFO rdd.BlockRDD: Removing RDD 40 from persistence list
17/07/31 18:54:56 INFO storage.BlockManager: Removing RDD 40
17/07/31 18:54:56 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[40] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527296000 ms
17/07/31 18:54:56 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527294000 ms
17/07/31 18:54:56 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527294000 ms
17/07/31 18:54:57 INFO scheduler.JobScheduler: Added jobs for time 1501527297000 ms
17/07/31 18:54:57 INFO scheduler.JobScheduler: Starting job streaming job 1501527297000 ms.0 from job set of time 1501527297000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:57 INFO scheduler.JobScheduler: Finished job streaming job 1501527297000 ms.0 from job set of time 1501527297000 ms
17/07/31 18:54:57 INFO python.PythonRDD: Removing RDD 43 from persistence list
17/07/31 18:54:57 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527297000 ms (execution: 0.008 s)
17/07/31 18:54:57 INFO storage.BlockManager: Removing RDD 43
17/07/31 18:54:57 INFO rdd.BlockRDD: Removing RDD 42 from persistence list
17/07/31 18:54:57 INFO storage.BlockManager: Removing RDD 42
17/07/31 18:54:57 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[42] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527297000 ms
17/07/31 18:54:57 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527295000 ms
17/07/31 18:54:57 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527295000 ms
17/07/31 18:54:58 INFO scheduler.JobScheduler: Added jobs for time 1501527298000 ms
17/07/31 18:54:58 INFO scheduler.JobScheduler: Starting job streaming job 1501527298000 ms.0 from job set of time 1501527298000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:58 INFO scheduler.JobScheduler: Finished job streaming job 1501527298000 ms.0 from job set of time 1501527298000 ms
17/07/31 18:54:58 INFO python.PythonRDD: Removing RDD 45 from persistence list
17/07/31 18:54:58 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527298000 ms (execution: 0.008 s)
17/07/31 18:54:58 INFO storage.BlockManager: Removing RDD 45
17/07/31 18:54:58 INFO rdd.BlockRDD: Removing RDD 44 from persistence list
17/07/31 18:54:58 INFO storage.BlockManager: Removing RDD 44
17/07/31 18:54:58 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[44] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527298000 ms
17/07/31 18:54:58 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527296000 ms
17/07/31 18:54:58 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527296000 ms
17/07/31 18:54:59 INFO scheduler.JobScheduler: Added jobs for time 1501527299000 ms
17/07/31 18:54:59 INFO scheduler.JobScheduler: Starting job streaming job 1501527299000 ms.0 from job set of time 1501527299000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:59 INFO scheduler.JobScheduler: Finished job streaming job 1501527299000 ms.0 from job set of time 1501527299000 ms
17/07/31 18:54:59 INFO python.PythonRDD: Removing RDD 47 from persistence list
17/07/31 18:54:59 INFO scheduler.JobScheduler: Total delay: 0.031 s for time 1501527299000 ms (execution: 0.009 s)
17/07/31 18:54:59 INFO storage.BlockManager: Removing RDD 47
17/07/31 18:54:59 INFO rdd.BlockRDD: Removing RDD 46 from persistence list
17/07/31 18:54:59 INFO storage.BlockManager: Removing RDD 46
17/07/31 18:54:59 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[46] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527299000 ms
17/07/31 18:54:59 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527297000 ms
17/07/31 18:54:59 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527297000 ms
17/07/31 18:55:00 INFO scheduler.JobScheduler: Added jobs for time 1501527300000 ms
17/07/31 18:55:00 INFO scheduler.JobScheduler: Starting job streaming job 1501527300000 ms.0 from job set of time 1501527300000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:00 INFO scheduler.JobScheduler: Finished job streaming job 1501527300000 ms.0 from job set of time 1501527300000 ms
17/07/31 18:55:00 INFO python.PythonRDD: Removing RDD 49 from persistence list
17/07/31 18:55:00 INFO scheduler.JobScheduler: Total delay: 0.035 s for time 1501527300000 ms (execution: 0.021 s)
17/07/31 18:55:00 INFO storage.BlockManager: Removing RDD 49
17/07/31 18:55:00 INFO rdd.BlockRDD: Removing RDD 48 from persistence list
17/07/31 18:55:00 INFO storage.BlockManager: Removing RDD 48
17/07/31 18:55:00 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[48] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527300000 ms
17/07/31 18:55:00 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527298000 ms
17/07/31 18:55:00 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527298000 ms
17/07/31 18:55:01 INFO scheduler.JobScheduler: Added jobs for time 1501527301000 ms
17/07/31 18:55:01 INFO scheduler.JobScheduler: Starting job streaming job 1501527301000 ms.0 from job set of time 1501527301000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:01 INFO scheduler.JobScheduler: Finished job streaming job 1501527301000 ms.0 from job set of time 1501527301000 ms
17/07/31 18:55:01 INFO python.PythonRDD: Removing RDD 51 from persistence list
17/07/31 18:55:01 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527301000 ms (execution: 0.008 s)
17/07/31 18:55:01 INFO storage.BlockManager: Removing RDD 51
17/07/31 18:55:01 INFO rdd.BlockRDD: Removing RDD 50 from persistence list
17/07/31 18:55:01 INFO storage.BlockManager: Removing RDD 50
17/07/31 18:55:01 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[50] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527301000 ms
17/07/31 18:55:01 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527299000 ms
17/07/31 18:55:01 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527299000 ms
17/07/31 18:55:02 INFO scheduler.JobScheduler: Added jobs for time 1501527302000 ms
17/07/31 18:55:02 INFO scheduler.JobScheduler: Starting job streaming job 1501527302000 ms.0 from job set of time 1501527302000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:02 INFO scheduler.JobScheduler: Finished job streaming job 1501527302000 ms.0 from job set of time 1501527302000 ms
17/07/31 18:55:02 INFO python.PythonRDD: Removing RDD 53 from persistence list
17/07/31 18:55:02 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501527302000 ms (execution: 0.009 s)
17/07/31 18:55:02 INFO storage.BlockManager: Removing RDD 53
17/07/31 18:55:02 INFO rdd.BlockRDD: Removing RDD 52 from persistence list
17/07/31 18:55:02 INFO storage.BlockManager: Removing RDD 52
17/07/31 18:55:02 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[52] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527302000 ms
17/07/31 18:55:02 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527300000 ms
17/07/31 18:55:02 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527300000 ms
17/07/31 18:55:03 INFO scheduler.JobScheduler: Added jobs for time 1501527303000 ms
17/07/31 18:55:03 INFO scheduler.JobScheduler: Starting job streaming job 1501527303000 ms.0 from job set of time 1501527303000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:03 INFO scheduler.JobScheduler: Finished job streaming job 1501527303000 ms.0 from job set of time 1501527303000 ms
17/07/31 18:55:03 INFO python.PythonRDD: Removing RDD 55 from persistence list
17/07/31 18:55:03 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527303000 ms (execution: 0.008 s)
17/07/31 18:55:03 INFO storage.BlockManager: Removing RDD 55
17/07/31 18:55:03 INFO rdd.BlockRDD: Removing RDD 54 from persistence list
17/07/31 18:55:03 INFO storage.BlockManager: Removing RDD 54
17/07/31 18:55:03 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[54] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527303000 ms
17/07/31 18:55:03 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527301000 ms
17/07/31 18:55:03 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527301000 ms
17/07/31 18:55:04 INFO scheduler.JobScheduler: Added jobs for time 1501527304000 ms
17/07/31 18:55:04 INFO scheduler.JobScheduler: Starting job streaming job 1501527304000 ms.0 from job set of time 1501527304000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:04 INFO scheduler.JobScheduler: Finished job streaming job 1501527304000 ms.0 from job set of time 1501527304000 ms
17/07/31 18:55:04 INFO python.PythonRDD: Removing RDD 57 from persistence list
17/07/31 18:55:04 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501527304000 ms (execution: 0.008 s)
17/07/31 18:55:04 INFO storage.BlockManager: Removing RDD 57
17/07/31 18:55:04 INFO rdd.BlockRDD: Removing RDD 56 from persistence list
17/07/31 18:55:04 INFO storage.BlockManager: Removing RDD 56
17/07/31 18:55:04 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[56] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527304000 ms
17/07/31 18:55:04 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527302000 ms
17/07/31 18:55:04 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527302000 ms
17/07/31 18:55:05 INFO scheduler.JobScheduler: Added jobs for time 1501527305000 ms
17/07/31 18:55:05 INFO scheduler.JobScheduler: Starting job streaming job 1501527305000 ms.0 from job set of time 1501527305000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:05 INFO scheduler.JobScheduler: Finished job streaming job 1501527305000 ms.0 from job set of time 1501527305000 ms
17/07/31 18:55:05 INFO python.PythonRDD: Removing RDD 59 from persistence list
17/07/31 18:55:05 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527305000 ms (execution: 0.008 s)
17/07/31 18:55:05 INFO storage.BlockManager: Removing RDD 59
17/07/31 18:55:05 INFO rdd.BlockRDD: Removing RDD 58 from persistence list
17/07/31 18:55:05 INFO storage.BlockManager: Removing RDD 58
17/07/31 18:55:05 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[58] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527305000 ms
17/07/31 18:55:05 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527303000 ms
17/07/31 18:55:05 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527303000 ms
17/07/31 18:55:06 INFO scheduler.JobScheduler: Added jobs for time 1501527306000 ms
17/07/31 18:55:06 INFO scheduler.JobScheduler: Starting job streaming job 1501527306000 ms.0 from job set of time 1501527306000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:06 INFO scheduler.JobScheduler: Finished job streaming job 1501527306000 ms.0 from job set of time 1501527306000 ms
17/07/31 18:55:06 INFO python.PythonRDD: Removing RDD 61 from persistence list
17/07/31 18:55:06 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527306000 ms (execution: 0.007 s)
17/07/31 18:55:06 INFO storage.BlockManager: Removing RDD 61
17/07/31 18:55:06 INFO rdd.BlockRDD: Removing RDD 60 from persistence list
17/07/31 18:55:06 INFO storage.BlockManager: Removing RDD 60
17/07/31 18:55:06 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[60] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527306000 ms
17/07/31 18:55:06 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527304000 ms
17/07/31 18:55:06 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527304000 ms
17/07/31 18:55:07 INFO scheduler.JobScheduler: Added jobs for time 1501527307000 ms
17/07/31 18:55:07 INFO scheduler.JobScheduler: Starting job streaming job 1501527307000 ms.0 from job set of time 1501527307000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:07 INFO scheduler.JobScheduler: Finished job streaming job 1501527307000 ms.0 from job set of time 1501527307000 ms
17/07/31 18:55:07 INFO python.PythonRDD: Removing RDD 63 from persistence list
17/07/31 18:55:07 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501527307000 ms (execution: 0.007 s)
17/07/31 18:55:07 INFO storage.BlockManager: Removing RDD 63
17/07/31 18:55:07 INFO rdd.BlockRDD: Removing RDD 62 from persistence list
17/07/31 18:55:07 INFO storage.BlockManager: Removing RDD 62
17/07/31 18:55:07 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[62] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527307000 ms
17/07/31 18:55:07 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527305000 ms
17/07/31 18:55:07 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527305000 ms
17/07/31 18:55:08 INFO scheduler.JobScheduler: Added jobs for time 1501527308000 ms
17/07/31 18:55:08 INFO scheduler.JobScheduler: Starting job streaming job 1501527308000 ms.0 from job set of time 1501527308000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:08 INFO scheduler.JobScheduler: Finished job streaming job 1501527308000 ms.0 from job set of time 1501527308000 ms
17/07/31 18:55:08 INFO python.PythonRDD: Removing RDD 65 from persistence list
17/07/31 18:55:08 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527308000 ms (execution: 0.008 s)
17/07/31 18:55:08 INFO storage.BlockManager: Removing RDD 65
17/07/31 18:55:08 INFO rdd.BlockRDD: Removing RDD 64 from persistence list
17/07/31 18:55:08 INFO storage.BlockManager: Removing RDD 64
17/07/31 18:55:08 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[64] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527308000 ms
17/07/31 18:55:08 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527306000 ms
17/07/31 18:55:08 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527306000 ms
17/07/31 18:55:09 INFO scheduler.JobScheduler: Added jobs for time 1501527309000 ms
17/07/31 18:55:09 INFO scheduler.JobScheduler: Starting job streaming job 1501527309000 ms.0 from job set of time 1501527309000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:09 INFO scheduler.JobScheduler: Finished job streaming job 1501527309000 ms.0 from job set of time 1501527309000 ms
17/07/31 18:55:09 INFO python.PythonRDD: Removing RDD 67 from persistence list
17/07/31 18:55:09 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527309000 ms (execution: 0.015 s)
17/07/31 18:55:09 INFO storage.BlockManager: Removing RDD 67
17/07/31 18:55:09 INFO rdd.BlockRDD: Removing RDD 66 from persistence list
17/07/31 18:55:09 INFO storage.BlockManager: Removing RDD 66
17/07/31 18:55:09 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[66] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527309000 ms
17/07/31 18:55:09 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527307000 ms
17/07/31 18:55:09 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527307000 ms
17/07/31 18:55:10 INFO scheduler.JobScheduler: Added jobs for time 1501527310000 ms
17/07/31 18:55:10 INFO scheduler.JobScheduler: Starting job streaming job 1501527310000 ms.0 from job set of time 1501527310000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:10 INFO scheduler.JobScheduler: Finished job streaming job 1501527310000 ms.0 from job set of time 1501527310000 ms
17/07/31 18:55:10 INFO python.PythonRDD: Removing RDD 69 from persistence list
17/07/31 18:55:10 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527310000 ms (execution: 0.007 s)
17/07/31 18:55:10 INFO storage.BlockManager: Removing RDD 69
17/07/31 18:55:10 INFO rdd.BlockRDD: Removing RDD 68 from persistence list
17/07/31 18:55:10 INFO storage.BlockManager: Removing RDD 68
17/07/31 18:55:10 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[68] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527310000 ms
17/07/31 18:55:10 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527308000 ms
17/07/31 18:55:10 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527308000 ms
17/07/31 18:55:11 INFO scheduler.JobScheduler: Added jobs for time 1501527311000 ms
17/07/31 18:55:11 INFO scheduler.JobScheduler: Starting job streaming job 1501527311000 ms.0 from job set of time 1501527311000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:11 INFO scheduler.JobScheduler: Finished job streaming job 1501527311000 ms.0 from job set of time 1501527311000 ms
17/07/31 18:55:11 INFO python.PythonRDD: Removing RDD 71 from persistence list
17/07/31 18:55:11 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527311000 ms (execution: 0.007 s)
17/07/31 18:55:11 INFO storage.BlockManager: Removing RDD 71
17/07/31 18:55:11 INFO rdd.BlockRDD: Removing RDD 70 from persistence list
17/07/31 18:55:11 INFO storage.BlockManager: Removing RDD 70
17/07/31 18:55:11 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[70] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527311000 ms
17/07/31 18:55:11 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527309000 ms
17/07/31 18:55:11 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527309000 ms
17/07/31 18:55:12 INFO scheduler.JobScheduler: Added jobs for time 1501527312000 ms
17/07/31 18:55:12 INFO scheduler.JobScheduler: Starting job streaming job 1501527312000 ms.0 from job set of time 1501527312000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:12 INFO scheduler.JobScheduler: Finished job streaming job 1501527312000 ms.0 from job set of time 1501527312000 ms
17/07/31 18:55:12 INFO python.PythonRDD: Removing RDD 73 from persistence list
17/07/31 18:55:12 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501527312000 ms (execution: 0.008 s)
17/07/31 18:55:12 INFO storage.BlockManager: Removing RDD 73
17/07/31 18:55:12 INFO rdd.BlockRDD: Removing RDD 72 from persistence list
17/07/31 18:55:12 INFO storage.BlockManager: Removing RDD 72
17/07/31 18:55:12 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[72] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527312000 ms
17/07/31 18:55:12 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527310000 ms
17/07/31 18:55:12 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527310000 ms
17/07/31 18:55:13 INFO scheduler.JobScheduler: Added jobs for time 1501527313000 ms
17/07/31 18:55:13 INFO scheduler.JobScheduler: Starting job streaming job 1501527313000 ms.0 from job set of time 1501527313000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:13 INFO scheduler.JobScheduler: Finished job streaming job 1501527313000 ms.0 from job set of time 1501527313000 ms
17/07/31 18:55:13 INFO python.PythonRDD: Removing RDD 75 from persistence list
17/07/31 18:55:13 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527313000 ms (execution: 0.007 s)
17/07/31 18:55:13 INFO storage.BlockManager: Removing RDD 75
17/07/31 18:55:13 INFO rdd.BlockRDD: Removing RDD 74 from persistence list
17/07/31 18:55:13 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[74] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527313000 ms
17/07/31 18:55:13 INFO storage.BlockManager: Removing RDD 74
17/07/31 18:55:13 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527311000 ms
17/07/31 18:55:13 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527311000 ms
17/07/31 18:55:14 INFO scheduler.JobScheduler: Added jobs for time 1501527314000 ms
17/07/31 18:55:14 INFO scheduler.JobScheduler: Starting job streaming job 1501527314000 ms.0 from job set of time 1501527314000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:14 INFO scheduler.JobScheduler: Finished job streaming job 1501527314000 ms.0 from job set of time 1501527314000 ms
17/07/31 18:55:14 INFO python.PythonRDD: Removing RDD 77 from persistence list
17/07/31 18:55:14 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527314000 ms (execution: 0.007 s)
17/07/31 18:55:14 INFO storage.BlockManager: Removing RDD 77
17/07/31 18:55:14 INFO rdd.BlockRDD: Removing RDD 76 from persistence list
17/07/31 18:55:14 INFO storage.BlockManager: Removing RDD 76
17/07/31 18:55:14 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[76] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527314000 ms
17/07/31 18:55:14 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527312000 ms
17/07/31 18:55:14 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527312000 ms
17/07/31 18:55:15 INFO scheduler.JobScheduler: Added jobs for time 1501527315000 ms
17/07/31 18:55:15 INFO scheduler.JobScheduler: Starting job streaming job 1501527315000 ms.0 from job set of time 1501527315000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:15 INFO scheduler.JobScheduler: Finished job streaming job 1501527315000 ms.0 from job set of time 1501527315000 ms
17/07/31 18:55:15 INFO python.PythonRDD: Removing RDD 79 from persistence list
17/07/31 18:55:15 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527315000 ms (execution: 0.007 s)
17/07/31 18:55:15 INFO storage.BlockManager: Removing RDD 79
17/07/31 18:55:15 INFO rdd.BlockRDD: Removing RDD 78 from persistence list
17/07/31 18:55:15 INFO storage.BlockManager: Removing RDD 78
17/07/31 18:55:15 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[78] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527315000 ms
17/07/31 18:55:15 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527313000 ms
17/07/31 18:55:15 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527313000 ms
17/07/31 18:55:16 INFO scheduler.JobScheduler: Added jobs for time 1501527316000 ms
17/07/31 18:55:16 INFO scheduler.JobScheduler: Starting job streaming job 1501527316000 ms.0 from job set of time 1501527316000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:16 INFO scheduler.JobScheduler: Finished job streaming job 1501527316000 ms.0 from job set of time 1501527316000 ms
17/07/31 18:55:16 INFO python.PythonRDD: Removing RDD 81 from persistence list
17/07/31 18:55:16 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501527316000 ms (execution: 0.014 s)
17/07/31 18:55:16 INFO storage.BlockManager: Removing RDD 81
17/07/31 18:55:16 INFO rdd.BlockRDD: Removing RDD 80 from persistence list
17/07/31 18:55:16 INFO storage.BlockManager: Removing RDD 80
17/07/31 18:55:16 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[80] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527316000 ms
17/07/31 18:55:16 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527314000 ms
17/07/31 18:55:16 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527314000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Added jobs for time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Starting job streaming job 1501527317000 ms.0 from job set of time 1501527317000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:17 INFO scheduler.JobScheduler: Finished job streaming job 1501527317000 ms.0 from job set of time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527317000 ms (execution: 0.011 s)
17/07/31 18:55:17 INFO python.PythonRDD: Removing RDD 83 from persistence list
17/07/31 18:55:17 INFO storage.BlockManager: Removing RDD 83
17/07/31 18:55:17 INFO rdd.BlockRDD: Removing RDD 82 from persistence list
17/07/31 18:55:17 INFO storage.BlockManager: Removing RDD 82
17/07/31 18:55:17 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[82] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527315000 ms
17/07/31 18:55:17 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527315000 ms
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1059.6 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1059.6 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.4 KB, free: 1057.9 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.4 KB, free: 1057.9 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 880.3 KB, free: 1057.0 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 880.3 KB, free: 1057.1 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.0 KB, free: 1056.2 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.0 KB, free: 1056.2 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527317800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.3 KB, free: 1054.5 MB)
17/07/31 18:55:18 INFO scheduler.JobScheduler: Added jobs for time 1501527318000 ms
17/07/31 18:55:18 INFO scheduler.JobScheduler: Starting job streaming job 1501527318000 ms.0 from job set of time 1501527318000 ms
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527317800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.3 KB, free: 1054.5 MB)
17/07/31 18:55:18 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Got job 2 (runJob at PythonRDD.scala:441) with 1 output partitions
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Final stage: ResultStage 3 (runJob at PythonRDD.scala:441)
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Submitting ResultStage 3 (PythonRDD[88] at RDD at PythonRDD.scala:48), which has no missing parents
17/07/31 18:55:18 INFO memory.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/07/31 18:55:18 INFO memory.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.0.0.199:59442 (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:18 INFO spark.SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:996
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (PythonRDD[88] at RDD at PythonRDD.scala:48)
17/07/31 18:55:18 INFO cluster.YarnScheduler: Adding task set 3.0 with 1 tasks
17/07/31 18:55:18 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 3.0 (TID 71, ip-10-0-0-151.ec2.internal, executor 2, partition 0, NODE_LOCAL, 5800 bytes)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 5.3 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.3 KB, free: 1052.7 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.3 KB, free: 1052.8 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.1 KB, free: 1050.2 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.1 KB, free: 1050.2 MB)
17/07/31 18:55:18 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 3.0 (TID 71) in 893 ms on ip-10-0-0-151.ec2.internal (executor 2) (1/1)
17/07/31 18:55:18 INFO cluster.YarnScheduler: Removed TaskSet 3.0, whose tasks have all completed, from pool
17/07/31 18:55:18 INFO scheduler.DAGScheduler: ResultStage 3 (runJob at PythonRDD.scala:441) finished in 0.896 s
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Job 2 finished: runJob at PythonRDD.scala:441, took 0.913501 s
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527318800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.5 KB, free: 1049.3 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527318800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.5 KB, free: 1049.3 MB)
17/07/31 18:55:19 INFO scheduler.JobScheduler: Added jobs for time 1501527319000 ms
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 887.9 KB, free: 1048.4 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 887.9 KB, free: 1048.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1047.6 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1047.6 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1046.7 MB)
17/07/31 18:55:19 INFO spark.ContextCleaner: Cleaned shuffle 0
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 10.0.0.199:59442 in memory (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1988.0 B, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on 10.0.0.199:59442 in memory (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1956.0 B, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on 10.0.0.199:59442 in memory (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 5.3 KB, free: 1046.8 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.2 KB, free: 1045.0 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.2 KB, free: 1045.0 MB)
17/07/31 18:55:19 INFO execution.SparkSqlParser: Parsing command: spotdb.proxy
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527319800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527319800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
17/07/31 18:55:20 INFO metastore.ObjectStore: ObjectStore, initialize called
17/07/31 18:55:20 INFO scheduler.JobScheduler: Added jobs for time 1501527320000 ms
17/07/31 18:55:20 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing
17/07/31 18:55:20 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
17/07/31 18:55:20 INFO metastore.ObjectStore: Initialized ObjectStore
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_table start=1501527320011 end=1501527320059 duration=48 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_table start=1501527320177 end=1501527320179 duration=2 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_database: spotdb
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: spotdb
17/07/31 18:55:20 WARN metastore.ObjectStore: Failed to get database spotdb, returning NoSuchObjectException
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527320239 end=1501527320242 duration=3 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527318000 ms.0 from job set of time 1501527318000 ms
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 2.266 s for time 1501527318000 ms (execution: 2.249 s)
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 85 from persistence list
17/07/31 18:55:20 INFO scheduler.JobScheduler: Starting job streaming job 1501527319000 ms.0 from job set of time 1501527319000 ms
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 84 from persistence list
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527318000 ms.0
org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 71, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u"Database 'spotdb' not found;"
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[84] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527318000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 85
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527316000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527316000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 84
17/07/31 18:55:20 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Got job 3 (runJob at PythonRDD.scala:441) with 1 output partitions
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Final stage: ResultStage 4 (runJob at PythonRDD.scala:441)
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Submitting ResultStage 4 (PythonRDD[97] at RDD at PythonRDD.scala:48), which has no missing parents
Traceback (most recent call last):
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 148, in <module>
main()
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 77, in main
bluecoat_parse(args.zk,args.topic,args.db,args.db_table,args.num_of_workers,args.batch_size)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 144, in bluecoat_parse
ssc.awaitTermination()
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/context.py", line 206, in awaitTermination
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o48.awaitTermination.
: org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 71, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u"Database 'spotdb' not found;"
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/07/31 18:55:20 INFO memory.MemoryStore: Block broadcast_4 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/07/31 18:55:20 INFO memory.MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on 10.0.0.199:59442 (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:20 INFO spark.SparkContext: Created broadcast 4 from broadcast at DAGScheduler.scala:996
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 4 (PythonRDD[97] at RDD at PythonRDD.scala:48)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Adding task set 4.0 with 1 tasks
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 4.0 (TID 72, ip-10-0-0-151.ec2.internal, executor 2, partition 0, NODE_LOCAL, 5800 bytes)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 5.3 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527319000 ms.0 from job set of time 1501527319000 ms
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 1.329 s for time 1501527319000 ms (execution: 0.051 s)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Starting job streaming job 1501527320000 ms.0 from job set of time 1501527320000 ms
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527319000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy36.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 87 from persistence list
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 86 from persistence list
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 87
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[86] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527319000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 86
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527317000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527317000 ms
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317000 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.2 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317400 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 880.3 KB, free: 1045.0 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317200 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.4 KB, free: 1046.7 MB)
17/07/31 18:55:20 INFO streaming.StreamingContext: Invoking stop(stopGracefully=false) from shutdown hook
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317600 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.0 KB, free: 1047.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317000 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.2 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317200 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.4 KB, free: 1045.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317400 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 880.3 KB, free: 1046.7 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317600 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.0 KB, free: 1047.6 MB)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527320000 ms.0 from job set of time 1501527320000 ms
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 90 from persistence list
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 0.333 s for time 1501527320000 ms (execution: 0.003 s)
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527320000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy36.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 89 from persistence list
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 90
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 89
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[89] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527320000 ms
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: Sent stop signal to all 1 receivers
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317800 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.3 KB, free: 1049.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318000 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.1 KB, free: 1050.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318200 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.3 KB, free: 1051.0 MB)
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527318000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527318000 ms
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318400 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318600 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.1 KB, free: 1053.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317800 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.3 KB, free: 1049.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318000 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.1 KB, free: 1050.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318200 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.3 KB, free: 1051.0 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318400 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318600 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.1 KB, free: 1053.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 880.6 KB, free: 1052.7 MB)
17/07/31 18:55:20 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 880.6 KB, free: 1052.8 MB)
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 4.0 (TID 72) in 289 ms on ip-10-0-0-151.ec2.internal (executor 2) (1/1)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Removed TaskSet 4.0, whose tasks have all completed, from pool
17/07/31 18:55:20 ERROR scheduler.DAGScheduler: Failed to update accumulators for task 0
org.apache.spark.SparkException: EOF reached before Python server acknowledged
at org.apache.spark.api.python.PythonAccumulatorV2.merge(PythonRDD.scala:919)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$updateAccumulators$1.apply(DAGScheduler.scala:1088)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$updateAccumulators$1.apply(DAGScheduler.scala:1080)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.DAGScheduler.updateAccumulators(DAGScheduler.scala:1080)
at org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:1156)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1647)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/07/31 18:55:20 INFO scheduler.DAGScheduler: ResultStage 4 (runJob at PythonRDD.scala:441) finished in 0.292 s
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Job 3 finished: runJob at PythonRDD.scala:441, took 0.309062 s
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 2.0 (TID 70) in 43848 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/1)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool
17/07/31 18:55:20 INFO scheduler.DAGScheduler: ResultStage 2 (start at NativeMethodAccessorImpl.java:0) finished in 43.849 s
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: All of the receivers have deregistered successfully
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
17/07/31 18:55:20 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
17/07/31 18:55:20 INFO util.RecurringTimer: Stopped timer for JobGenerator after time 1501527320000
17/07/31 18:55:20 INFO scheduler.JobGenerator: Stopped JobGenerator
17/07/31 18:55:20 INFO scheduler.JobScheduler: Stopped JobScheduler
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19fd4723{/streaming,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@22116fc3{/streaming/batch,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@70d301d6{/static/streaming,null,UNAVAILABLE}
17/07/31 18:55:20 INFO streaming.StreamingContext: StreamingContext stopped successfully
17/07/31 18:55:20 INFO spark.SparkContext: Invoking stop() from shutdown hook
17/07/31 18:55:20 INFO server.ServerConnector: Stopped ServerConnector@188e6c7f{HTTP/1.1}{0.0.0.0:4040}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4f734944{/stages/stage/kill,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@b7d4da0{/jobs/job/kill,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ab71bb9{/api,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5d37ce06{/,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@27f55b38{/static,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1159f15e{/executors/threadDump/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ff970c2{/executors/threadDump,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@615b86f5{/executors/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19bb7285{/executors,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3b9882ec{/environment/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7b3691b6{/environment,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@423a3429{/storage/rdd/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@13016b86{/storage/rdd,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ea732f0{/storage/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2d4aa15a{/storage,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5cbdbf0f{/stages/pool/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14c62558{/stages/pool,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5c61bd1a{/stages/stage/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4eb72ecd{/stages/stage,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1bf090df{/stages/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14b58fc0{/stages,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@30e71b5d{/jobs/job/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@232864a3{/jobs/job,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1f4da763{/jobs/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1a3b1e79{/jobs,null,UNAVAILABLE}
17/07/31 18:55:20 INFO ui.SparkUI: Stopped Spark web UI at http://10.0.0.199:4040
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors
17/07/31 18:55:20 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
17/07/31 18:55:20 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
services=List(),
started=false)
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Stopped
17/07/31 18:55:20 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/07/31 18:55:20 INFO memory.MemoryStore: MemoryStore cleared
17/07/31 18:55:20 INFO storage.BlockManager: BlockManager stopped
17/07/31 18:55:20 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
17/07/31 18:55:20 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/07/31 18:55:20 INFO spark.SparkContext: Successfully stopped SparkContext
17/07/31 18:55:20 INFO util.ShutdownHookManager: Shutdown hook called
17/07/31 18:55:20 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40
17/07/31 18:55:20 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40/pyspark-506939c9-a852-404b-adea-6ad40ad0d73f
Closing worker...
Thanks
Deon Griessel
RE: Spot Ingest Proxy Error
Posted by Deon Griessel <dg...@searchtechnologies.com>.
FYI – I also had to modify spot-oa/oa/proxy/proxy_oa.py …
Adding line >> fulluri = fulluri.replace("'","\\'") to below code in order to escape single quotes in URI which appear in sample data.
def _get_suspicious_details(self):
hash_list = []
iana_conf_file = "{0}/components/iana/iana_config.json".format(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
if os.path.isfile(iana_conf_file):
iana_config = json.loads(open(iana_conf_file).read())
proxy_iana = IanaTransform(iana_config["IANA"])
for conn in self._proxy_scores:
conn_hash = conn[self._conf["proxy_score_fields"]["hash"]]
if conn_hash not in hash_list:
hash_list.append(conn_hash)
clientip = conn[self._conf["proxy_score_fields"]["clientip"]]
fulluri = conn[self._conf["proxy_score_fields"]["fulluri"]]
fulluri = fulluri.replace("'","\\'")
self._logger.info("fulluri: {0}".format(fulluri))
date=conn[self._conf["proxy_score_fields"]["p_date"]].split('-')
if len(date) == 3:
year=date[0]
month=date[1].zfill(2)
day=date[2].zfill(2)
hh=(conn[self._conf["proxy_score_fields"]["p_time"]].split(":"))[0]
self._get_proxy_details(fulluri,clientip,conn_hash,year,month,day,hh,proxy_iana)
From: Barona, Ricardo [mailto:ricardo.barona@intel.com]
Sent: Tuesday, August 1, 2017 12:37 PM
To: user@spot.incubator.apache.org; user@spot.apache.org
Subject: Re: Spot Ingest Proxy Error
Great!
From: Deon Griessel <dg...@searchtechnologies.com>>
Reply-To: "user@spot.incubator.apache.org<ma...@spot.incubator.apache.org>" <us...@spot.incubator.apache.org>>
Date: Tuesday, August 1, 2017 at 11:15 AM
To: "user@spot.incubator.apache.org<ma...@spot.incubator.apache.org>" <us...@spot.incubator.apache.org>>, "user@spot.apache.org<ma...@spot.apache.org>" <us...@spot.apache.org>>
Subject: RE: Spot Ingest Proxy Error
I replaced the following line in bluecoat.py:
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
with
df.write.format("parquet").mode("append").insertInto(hive_table)
That seemed to do the trick, it then bombed on bad input data in the sample data provided in: Bluecoat_ProxySG_Sample.log
Seems like one of the records has a duration/time-taken which is larger than int and thus caused the python script to die.
Anyhow for now I removed the offending line and was able to get it to run through.
-Deon
From: Barona, Ricardo [mailto:ricardo.barona@intel.com]
Sent: Tuesday, August 1, 2017 11:14 AM
To: user@spot.incubator.apache.org<ma...@spot.incubator.apache.org>; user@spot.apache.org<ma...@spot.apache.org>
Subject: Re: Spot Ingest Proxy Error
Hi Dean,
Let me ask around.
From: Deon Griessel <dg...@searchtechnologies.com>>
Reply-To: "user@spot.incubator.apache.org<ma...@spot.incubator.apache.org>" <us...@spot.incubator.apache.org>>
Date: Tuesday, August 1, 2017 at 10:04 AM
To: "user@spot.incubator.apache.org<ma...@spot.incubator.apache.org>" <us...@spot.incubator.apache.org>>, "user@spot.apache.org<ma...@spot.apache.org>" <us...@spot.apache.org>>
Subject: RE: Spot Ingest Proxy Error
Ok, it seems like I got around the original error by enabling the hive service in Spark2 configuration in Cloudera Manager and restarting all services.
Now getting the following error:
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
Anybody seen this before? Full log below.
Thanks
---------------------------
2017-08-01 14:49:13,694 - SPOT.INGEST.WORKER - INFO - Initializing kafka instance
2017-08-01 14:49:13,694 - SPOT.INGEST.WRK.PROXY - INFO - Creating Spark Job for topic: SPOT-INGEST-proxy-14_49_11
2017-08-01 14:49:13,694 - SPOT.INGEST.WRK.PROXY - INFO - SPOT.Utils: Executing: spark2-submit --master yarn --driver-memory 2325m --num-executors 1 --conf spark.executor.memory=2325m --conf spark.executor.cores=1 --jars /home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar /home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py -zk ip-10-0-0-199.ec2.internal:2181 -t SPOT-INGEST-proxy-14_49_11 -db spotdb -dt proxy -w 1 -bs 1
17/08/01 14:49:14 INFO spark.SparkContext: Running Spark version 2.1.0.cloudera1
17/08/01 14:49:14 WARN spark.SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0
17/08/01 14:49:15 INFO spark.SecurityManager: Changing view acls to: spot-user
17/08/01 14:49:15 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/08/01 14:49:15 INFO spark.SecurityManager: Changing view acls groups to:
17/08/01 14:49:15 INFO spark.SecurityManager: Changing modify acls groups to:
17/08/01 14:49:15 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/08/01 14:49:15 INFO util.Utils: Successfully started service 'sparkDriver' on port 56925.
17/08/01 14:49:15 INFO spark.SparkEnv: Registering MapOutputTracker
17/08/01 14:49:15 INFO spark.SparkEnv: Registering BlockManagerMaster
17/08/01 14:49:15 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
17/08/01 14:49:15 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
17/08/01 14:49:15 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-23e4ad79-bd38-4e37-b688-a75f9565bda4
17/08/01 14:49:16 INFO memory.MemoryStore: MemoryStore started with capacity 1060.5 MB
17/08/01 14:49:16 INFO spark.SparkEnv: Registering OutputCommitCoordinator
17/08/01 14:49:16 INFO util.log: Logging initialized @2378ms
17/08/01 14:49:16 INFO server.Server: jetty-9.2.z-SNAPSHOT
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3b9882ec{/jobs,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3b9882ec%7b/jobs,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19bb7285{/jobs/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19bb7285%7b/jobs/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@615b86f5{/jobs/job,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@615b86f5%7b/jobs/job,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ff970c2{/jobs/job/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3ff970c2%7b/jobs/job/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1159f15e{/stages,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1159f15e%7b/stages,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27f55b38{/stages/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27f55b38%7b/stages/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5d37ce06{/stages/stage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5d37ce06%7b/stages/stage,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ab71bb9{/stages/stage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ab71bb9%7b/stages/stage/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b7d4da0{/stages/pool,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@b7d4da0%7b/stages/pool,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f734944{/stages/pool/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4f734944%7b/stages/pool/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19cf381c{/storage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19cf381c%7b/storage,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@736cc380{/storage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@736cc380%7b/storage/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@699743b3{/storage/rdd,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@699743b3%7b/storage/rdd,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@49ddd2ed{/storage/rdd/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@49ddd2ed%7b/storage/rdd/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@242eb7e{/environment,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@242eb7e%7b/environment,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@77729e5a{/environment/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@77729e5a%7b/environment/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5fa63fa{/executors,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5fa63fa%7b/executors,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@79b9b526{/executors/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@79b9b526%7b/executors/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@f241039{/executors/threadDump,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@f241039%7b/executors/threadDump,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@8a515f3{/executors/threadDump/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@8a515f3%7b/executors/threadDump/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@e4408f4{/static,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@e4408f4%7b/static,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14a7d27d{/,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14a7d27d%7b/,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@15ec1723{/api,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@15ec1723%7b/api,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ee30613{/jobs/job/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3ee30613%7b/jobs/job/kill,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2aa3e9a6{/stages/stage/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@2aa3e9a6%7b/stages/stage/kill,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO server.ServerConnector: Started ServerConnector@37681deb{HTTP/1.1}{0.0.0.0:4040}<mailto:ServerConnector@37681deb%7bHTTP/1.1%7d%7b0.0.0.0:4040%7d>
17/08/01 14:49:16 INFO server.Server: Started @2532ms
17/08/01 14:49:16 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
17/08/01 14:49:16 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.0.0.199:4040
17/08/01 14:49:16 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/08/01 14:49:17 INFO client.RMProxy: Connecting to ResourceManager at ip-10-0-0-199.ec2.internal/10.0.0.199:8032
17/08/01 14:49:17 INFO yarn.Client: Requesting a new application from cluster with 3 NodeManagers
17/08/01 14:49:17 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (2725 MB per container)
17/08/01 14:49:17 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/08/01 14:49:17 INFO yarn.Client: Setting up container launch context for our AM
17/08/01 14:49:17 INFO yarn.Client: Setting up the launch environment for our AM container
17/08/01 14:49:17 INFO yarn.Client: Preparing resources for our AM container
17/08/01 14:49:19 INFO yarn.Client: Uploading resource file:/home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501598591600_0003/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar
17/08/01 14:49:19 INFO yarn.Client: Uploading resource file:/tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25/__spark_conf__4374658135010889941.zip -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501598591600_0003/__spark_conf__.zip
17/08/01 14:49:19 INFO spark.SecurityManager: Changing view acls to: spot-user
17/08/01 14:49:19 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/08/01 14:49:19 INFO spark.SecurityManager: Changing view acls groups to:
17/08/01 14:49:19 INFO spark.SecurityManager: Changing modify acls groups to:
17/08/01 14:49:19 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/08/01 14:49:19 INFO yarn.Client: Submitting application application_1501598591600_0003 to ResourceManager
17/08/01 14:49:19 INFO impl.YarnClientImpl: Submitted application application_1501598591600_0003
17/08/01 14:49:19 INFO cluster.SchedulerExtensionServices: Starting Yarn extension services with app application_1501598591600_0003 and attemptId None
17/08/01 14:49:20 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:20 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.users.spot-user
start time: 1501598959525
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003/
user: spot-user
17/08/01 14:49:21 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:22 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:23 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/08/01 14:49:23 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> ip-10-0-0-199.ec2.internal, PROXY_URI_BASES -> http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003), /proxy/application_1501598591600_0003
17/08/01 14:49:23 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/08/01 14:49:23 INFO yarn.Client: Application report for application_1501598591600_0003 (state: RUNNING)
17/08/01 14:49:23 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 10.0.0.151
ApplicationMaster RPC port: 0
queue: root.users.spot-user
start time: 1501598959525
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003/
user: spot-user
17/08/01 14:49:23 INFO cluster.YarnClientSchedulerBackend: Application application_1501598591600_0003 has started running.
17/08/01 14:49:23 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 53355.
17/08/01 14:49:23 INFO netty.NettyBlockTransferService: Server created on 10.0.0.199:53355
17/08/01 14:49:23 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/08/01 14:49:23 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManagerMasterEndpoint: Registering block manager 10.0.0.199:53355 with 1060.5 MB RAM, BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManager: external shuffle service port = 7337
17/08/01 14:49:23 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@52c2dab5{/metrics/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@52c2dab5%7b/metrics/json,null,AVAILABLE%7d>
17/08/01 14:49:23 INFO scheduler.EventLoggingListener: Logging events to hdfs://ip-10-0-0-199.ec2.internal:8020/user/spark/spark2ApplicationHistory/application_1501598591600_0003
17/08/01 14:49:23 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/08/01 14:49:27 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.229:37096) with ID 1
17/08/01 14:49:27 INFO spark.ExecutorAllocationManager: New executor 1 has registered (new total is 1)
17/08/01 14:49:27 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-229.ec2.internal:51770 with 1060.5 MB RAM, BlockManagerId(1, ip-10-0-0-229.ec2.internal, 51770, None)
17/08/01 14:49:27 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/08/01 14:49:27 INFO internal.SharedState: spark.sql.warehouse.dir is not set, but hive.metastore.warehouse.dir is set. Setting spark.sql.warehouse.dir to the value of hive.metastore.warehouse.dir ('/user/hive/warehouse').
17/08/01 14:49:27 INFO internal.SharedState: Warehouse path is '/user/hive/warehouse'.
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5554f0a0{/SQL,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5554f0a0%7b/SQL,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@44743601{/SQL/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@44743601%7b/SQL/json,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@144776d3{/SQL/execution,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@144776d3%7b/SQL/execution,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f3f8a4e{/SQL/execution/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4f3f8a4e%7b/SQL/execution/json,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27946137{/static/sql,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27946137%7b/static/sql,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO hive.HiveUtils: Initializing HiveMetastoreConnection version 1.1.0 using file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-logging-1.1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-exec-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-exec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-1.1.0-cdh5.12.0-standalone.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-standalone.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-metastore-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-metastore.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-serde-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-serde.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-service-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-service.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/libfb303-0.9.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/libthrift-0.9.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/log4j-1.2.16.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-hadoop-compat.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-hadoop2-compat.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-protocol.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-server.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/htrace-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ST4-4.0.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-core-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-fate-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-start-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-trace-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/activation-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ant-1.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ant-launcher-1.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/antlr-2.7.7.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/antlr-runtime-3.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/apache-log4j-extras-1.2.17.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-3.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-commons-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-tree-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/avro.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/bonecp-0.8.0.RELEASE.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-avatica-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-core-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-linq4j-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-beanutils-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-beanutils-core-1.8.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-cli-1.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-codec-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-collections-3.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-compiler-2.7.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-compress-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-configuration-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-dbcp-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-digester-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-el-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-httpclient-3.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-io-2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-lang-2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-lang3-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-math-2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-pool-1.5.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-vfs2-2.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-client-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-framework-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-recipes-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-api-jdo-3.2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-core-3.2.10.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-rdbms-3.2.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/derby-10.11.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/eigenbase-properties-1.1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/findbugs-annotations-1.3.9-1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-annotation_1.0_spec-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-jaspic_1.0_spec-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-jta_1.1_spec-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/groovy-all-2.4.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/gson-2.2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/guava-14.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hamcrest-core-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/high-scale-lib-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-accumulo-handler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-accumulo-handler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-ant-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-ant.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-beeline-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-beeline.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-cli-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-cli.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-common-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-contrib-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-contrib.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hbase-handler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hbase-handler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hwi-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hwi.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-0.23-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-0.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-common-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-scheduler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-scheduler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-testutils-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-testutils.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/httpclient-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/httpcore-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-core-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ivy-2.0.0-rc2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/parquet-hadoop-bundle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/stringtemplate-3.2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/regexp-1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/xz-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/tempus-fugit-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/super-csv-2.2.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/stax-api-1.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/servlet-api-2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/opencsv-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-jvm-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-json-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-core-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-provider-svnexe-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-provider-svn-commons-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-api-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/mail-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/logredactor-1.0.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/junit-4.11.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jta-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jsr305-3.0.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jsp-api-2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jpam-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/joda-time-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jline-2.12.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jetty-all-server-7.6.0.v20120127.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jetty-all-7.6.0.v20120127.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jersey-servlet-1.14.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jersey-server-1.14.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jdo-api-3.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jcommander-1.32.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jasper-runtime-5.5.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jasper-compiler-5.5.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/janino-2.7.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jamon-runtime-2.3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-xc-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-jaxrs-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-databind-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-annotations-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/zookeeper.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/velocity-1.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/snappy-java-1.0.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/plexus-utils-1.5.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/paranamer-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/oro-2.0.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/activation-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/activation.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-i18n-2.0.0-M15.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-i18n.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-kerberos-codec-2.0.0-M15.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-kerberos-codec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-asn1-api-1.0.0-M20.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-asn1-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-util-1.0.0-M20.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-util.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/avro.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/aws-java-sdk-bundle-1.11.134.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/aws-java-sdk-bundle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/azure-data-lake-store-sdk-2.1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/azure-data-lake-store-sdk.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-core-1.8.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-cli-1.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-cli.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-codec-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-codec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-collections-3.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-collections.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-compress-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-compress.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-configuration-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-configuration.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-digester-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-digester.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-httpclient-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-httpclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-io-2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-io.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-lang-2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-lang.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-logging-1.1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-logging.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-math3-3.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-math3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-net-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-net.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-client-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-framework-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-framework.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-recipes-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-recipes.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/gson-2.2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/gson.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/guava-11.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/guava.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-annotations-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-auth-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-auth.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-aws-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-aws.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-azure-datalake-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-azure-datalake.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-hdfs-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-hdfs.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-app-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-app.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-core-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-jobclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-shuffle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-api-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-client-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-server-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-server-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/htrace-core4-4.0.1-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/htrace-core4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpclient-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpcore-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpcore.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-annotations-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-core-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-databind-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-databind.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-jaxrs-1.8.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-jaxrs.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-xc-1.8.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-xc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jaxb-api-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jaxb-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-client-1.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-core-1.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/zookeeper.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xz.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xz-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xmlenc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xmlenc-0.52.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xml-apis.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xml-apis-1.3.04.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xercesImpl.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xercesImpl-2.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/stax-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/stax-api-1.0-2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/snappy-java.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/snappy-java-1.0.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-log4j12.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-api-1.7.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/servlet-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/servlet-api-2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/protobuf-java.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/protobuf-java-2.5.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/paranamer.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/paranamer-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/netty.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/netty-3.10.5.Final.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/log4j.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/log4j-1.2.17.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/leveldbjni-all.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/leveldbjni-all-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jsr305.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jsr305-3.0.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jetty-util.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jetty-util-6.1.26.cloudera.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-core.jar
17/08/01 14:49:28 INFO session.SessionState: Created local directory: /tmp/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2_resources
17/08/01 14:49:28 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2
17/08/01 14:49:28 INFO session.SessionState: Created local directory: /tmp/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2
17/08/01 14:49:28 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2/_tmp_space.db
17/08/01 14:49:28 INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
17/08/01 14:49:28 INFO client.HiveClientImpl: Warehouse location for Hive client (version 1.1.0) is /user/hive/warehouse
17/08/01 14:49:29 INFO hive.metastore: Trying to connect to metastore with URI thrift://ip-10-0-0-8.ec2.internal:9083
17/08/01 14:49:29 INFO hive.metastore: Opened a connection to metastore, current connections: 1
17/08/01 14:49:29 INFO hive.metastore: Connected to metastore.
17/08/01 14:49:29 WARN streaming.StreamingContext: Dynamic Allocation is enabled for this application. Enabling Dynamic allocation for Spark Streaming applications can cause data loss if Write Ahead Log is not enabled for non-replayable sources like Flume. See the programming guide for details on how to enable the Write Ahead Log.
17/08/01 14:49:29 INFO spark.SparkContext: Starting job: start at NativeMethodAccessorImpl.java:0
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Registering RDD 1 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Got job 0 (start at NativeMethodAccessorImpl.java:0) with 20 output partitions
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/08/01 14:49:29 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 3.1 KB, free 1060.5 MB)
17/08/01 14:49:29 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 1988.0 B, free 1060.5 MB)
17/08/01 14:49:29 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.0.0.199:53355 (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:49:29 INFO spark.SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Submitting 50 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO cluster.YarnScheduler: Adding task set 0.0 with 50 tasks
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, ip-10-0-0-229.ec2.internal, executor 1, partition 1, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 664 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, ip-10-0-0-229.ec2.internal, executor 1, partition 2, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 78 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3, ip-10-0-0-229.ec2.internal, executor 1, partition 3, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 68 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 0.0 (TID 4, ip-10-0-0-229.ec2.internal, executor 1, partition 4, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 65 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 0.0 (TID 5, ip-10-0-0-229.ec2.internal, executor 1, partition 5, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 59 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/50)
17/08/01 14:49:30 INFO spark.ExecutorAllocationManager: Requesting 1 new executor because tasks are backlogged (new desired total will be 2)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 0.0 (TID 6, ip-10-0-0-229.ec2.internal, executor 1, partition 6, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 0.0 (TID 7, ip-10-0-0-229.ec2.internal, executor 1, partition 7, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 60 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 0.0 (TID 8, ip-10-0-0-229.ec2.internal, executor 1, partition 8, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 0.0 (TID 7) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 0.0 (TID 9, ip-10-0-0-229.ec2.internal, executor 1, partition 9, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 0.0 (TID 8) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 0.0 (TID 10, ip-10-0-0-229.ec2.internal, executor 1, partition 10, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 0.0 (TID 9) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 0.0 (TID 11, ip-10-0-0-229.ec2.internal, executor 1, partition 11, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 0.0 (TID 10) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 0.0 (TID 12, ip-10-0-0-229.ec2.internal, executor 1, partition 12, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 0.0 (TID 11) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 0.0 (TID 13, ip-10-0-0-229.ec2.internal, executor 1, partition 13, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 0.0 (TID 12) in 60 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 0.0 (TID 14, ip-10-0-0-229.ec2.internal, executor 1, partition 14, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 0.0 (TID 13) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 0.0 (TID 15, ip-10-0-0-229.ec2.internal, executor 1, partition 15, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 0.0 (TID 14) in 51 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 0.0 (TID 16, ip-10-0-0-229.ec2.internal, executor 1, partition 16, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 0.0 (TID 15) in 63 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 0.0 (TID 17, ip-10-0-0-229.ec2.internal, executor 1, partition 17, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 0.0 (TID 16) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 0.0 (TID 18, ip-10-0-0-229.ec2.internal, executor 1, partition 18, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 0.0 (TID 17) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 0.0 (TID 19, ip-10-0-0-229.ec2.internal, executor 1, partition 19, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 0.0 (TID 18) in 56 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 0.0 (TID 20, ip-10-0-0-229.ec2.internal, executor 1, partition 20, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 0.0 (TID 19) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 0.0 (TID 21, ip-10-0-0-229.ec2.internal, executor 1, partition 21, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 0.0 (TID 20) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (21/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 0.0 (TID 22, ip-10-0-0-229.ec2.internal, executor 1, partition 22, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 0.0 (TID 21) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (22/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 0.0 (TID 23, ip-10-0-0-229.ec2.internal, executor 1, partition 23, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 0.0 (TID 22) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (23/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 0.0 (TID 24, ip-10-0-0-229.ec2.internal, executor 1, partition 24, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 0.0 (TID 23) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (24/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 0.0 (TID 25, ip-10-0-0-229.ec2.internal, executor 1, partition 25, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 0.0 (TID 24) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (25/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 0.0 (TID 26, ip-10-0-0-229.ec2.internal, executor 1, partition 26, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 0.0 (TID 25) in 46 ms on ip-10-0-0-229.ec2.internal (executor 1) (26/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 0.0 (TID 27, ip-10-0-0-229.ec2.internal, executor 1, partition 27, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 0.0 (TID 26) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (27/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 0.0 (TID 28, ip-10-0-0-229.ec2.internal, executor 1, partition 28, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 0.0 (TID 27) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (28/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 0.0 (TID 29, ip-10-0-0-229.ec2.internal, executor 1, partition 29, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 0.0 (TID 28) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (29/50)
17/08/01 14:49:31 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 4)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 0.0 (TID 30, ip-10-0-0-229.ec2.internal, executor 1, partition 30, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 0.0 (TID 29) in 42 ms on ip-10-0-0-229.ec2.internal (executor 1) (30/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 0.0 (TID 31, ip-10-0-0-229.ec2.internal, executor 1, partition 31, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 0.0 (TID 30) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (31/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 0.0 (TID 32, ip-10-0-0-229.ec2.internal, executor 1, partition 32, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 0.0 (TID 31) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (32/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 0.0 (TID 33, ip-10-0-0-229.ec2.internal, executor 1, partition 33, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 0.0 (TID 32) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (33/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 0.0 (TID 34, ip-10-0-0-229.ec2.internal, executor 1, partition 34, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 0.0 (TID 33) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (34/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 0.0 (TID 35, ip-10-0-0-229.ec2.internal, executor 1, partition 35, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 0.0 (TID 34) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (35/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 0.0 (TID 36, ip-10-0-0-229.ec2.internal, executor 1, partition 36, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 0.0 (TID 35) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (36/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 0.0 (TID 37, ip-10-0-0-229.ec2.internal, executor 1, partition 37, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 0.0 (TID 36) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (37/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 0.0 (TID 38, ip-10-0-0-229.ec2.internal, executor 1, partition 38, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 0.0 (TID 37) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (38/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 0.0 (TID 39, ip-10-0-0-229.ec2.internal, executor 1, partition 39, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 0.0 (TID 38) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (39/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 0.0 (TID 40, ip-10-0-0-229.ec2.internal, executor 1, partition 40, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 0.0 (TID 39) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (40/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 0.0 (TID 41, ip-10-0-0-229.ec2.internal, executor 1, partition 41, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 0.0 (TID 40) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (41/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 0.0 (TID 42, ip-10-0-0-229.ec2.internal, executor 1, partition 42, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 0.0 (TID 41) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (42/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 0.0 (TID 43, ip-10-0-0-229.ec2.internal, executor 1, partition 43, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 0.0 (TID 42) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (43/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 0.0 (TID 44, ip-10-0-0-229.ec2.internal, executor 1, partition 44, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 0.0 (TID 43) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (44/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 0.0 (TID 45, ip-10-0-0-229.ec2.internal, executor 1, partition 45, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 0.0 (TID 44) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (45/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 0.0 (TID 46, ip-10-0-0-229.ec2.internal, executor 1, partition 46, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 0.0 (TID 45) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (46/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 0.0 (TID 47, ip-10-0-0-229.ec2.internal, executor 1, partition 47, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 0.0 (TID 46) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (47/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 0.0 (TID 48, ip-10-0-0-229.ec2.internal, executor 1, partition 48, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 0.0 (TID 47) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (48/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 0.0 (TID 49, ip-10-0-0-229.ec2.internal, executor 1, partition 49, PROCESS_LOCAL, 6676 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 0.0 (TID 48) in 28 ms on ip-10-0-0-229.ec2.internal (executor 1) (49/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 0.0 (TID 49) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (50/50)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/08/01 14:49:32 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (start at NativeMethodAccessorImpl.java:0) finished in 2.517 s
17/08/01 14:49:32 INFO scheduler.DAGScheduler: looking for newly runnable stages
17/08/01 14:49:32 INFO scheduler.DAGScheduler: running: Set()
17/08/01 14:49:32 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 1)
17/08/01 14:49:32 INFO scheduler.DAGScheduler: failed: Set()
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/08/01 14:49:32 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.2 KB, free 1060.5 MB)
17/08/01 14:49:32 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 1956.0 B, free 1060.5 MB)
17/08/01 14:49:32 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.0.0.199:53355 (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:49:32 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Submitting 20 missing tasks from ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Adding task set 1.0 with 20 tasks
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 50, ip-10-0-0-229.ec2.internal, executor 1, partition 0, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:49:32 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.0.0.229:37096
17/08/01 14:49:32 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 283 bytes
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.0 (TID 51, ip-10-0-0-229.ec2.internal, executor 1, partition 1, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 50) in 89 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 1.0 (TID 52, ip-10-0-0-229.ec2.internal, executor 1, partition 2, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 1.0 (TID 51) in 23 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 1.0 (TID 53, ip-10-0-0-229.ec2.internal, executor 1, partition 3, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 1.0 (TID 52) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 1.0 (TID 54, ip-10-0-0-229.ec2.internal, executor 1, partition 4, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 1.0 (TID 53) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 1.0 (TID 55, ip-10-0-0-229.ec2.internal, executor 1, partition 5, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 1.0 (TID 54) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 1.0 (TID 56, ip-10-0-0-229.ec2.internal, executor 1, partition 6, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 1.0 (TID 55) in 25 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 1.0 (TID 57, ip-10-0-0-229.ec2.internal, executor 1, partition 7, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 1.0 (TID 56) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 1.0 (TID 58, ip-10-0-0-229.ec2.internal, executor 1, partition 8, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 1.0 (TID 57) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 1.0 (TID 59, ip-10-0-0-229.ec2.internal, executor 1, partition 9, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 1.0 (TID 58) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 1.0 (TID 60, ip-10-0-0-229.ec2.internal, executor 1, partition 10, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 1.0 (TID 59) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 1.0 (TID 61, ip-10-0-0-229.ec2.internal, executor 1, partition 11, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 1.0 (TID 60) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 1.0 (TID 62, ip-10-0-0-229.ec2.internal, executor 1, partition 12, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 1.0 (TID 61) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 1.0 (TID 63, ip-10-0-0-229.ec2.internal, executor 1, partition 13, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 1.0 (TID 62) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 1.0 (TID 64, ip-10-0-0-229.ec2.internal, executor 1, partition 14, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 1.0 (TID 63) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 1.0 (TID 65, ip-10-0-0-229.ec2.internal, executor 1, partition 15, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 1.0 (TID 64) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 1.0 (TID 66, ip-10-0-0-229.ec2.internal, executor 1, partition 16, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 1.0 (TID 65) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 1.0 (TID 67, ip-10-0-0-229.ec2.internal, executor 1, partition 17, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 1.0 (TID 66) in 22 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 1.0 (TID 68, ip-10-0-0-229.ec2.internal, executor 1, partition 18, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 1.0 (TID 67) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 1.0 (TID 69, ip-10-0-0-229.ec2.internal, executor 1, partition 19, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 1.0 (TID 68) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 1.0 (TID 69) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/20)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool
17/08/01 14:49:32 INFO scheduler.DAGScheduler: ResultStage 1 (start at NativeMethodAccessorImpl.java:0) finished in 0.420 s
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Job 0 finished: start at NativeMethodAccessorImpl.java:0, took 3.286639 s
17/08/01 14:49:32 INFO scheduler.ReceiverTracker: Starting 1 receivers
17/08/01 14:49:32 INFO scheduler.ReceiverTracker: ReceiverTracker started
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Initialized and validated org.apache.spark.streaming.kafka.KafkaInputDStream@71871773
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Initialized and validated org.apache.spark.streaming.api.python.PythonTransformedDStream@6593d619
17/08/01 14:49:32 INFO dstream.ForEachDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO dstream.ForEachDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO dstream.ForEachDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO dstream.ForEachDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO dstream.ForEachDStream: Initialized and validated org.apache.spark.streaming.dstream.ForEachDStream@26c8ea4b
17/08/01 14:49:33 INFO util.RecurringTimer: Started timer for JobGenerator at time 1501598973000
17/08/01 14:49:33 INFO scheduler.JobGenerator: Started JobGenerator at 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Started JobScheduler
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@625d3c2d{/streaming,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@48a256f9{/streaming/json,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@39edaf54{/streaming/batch,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@648a544d{/streaming/batch/json,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@c773c0d{/static/streaming,null,AVAILABLE}
17/08/01 14:49:33 INFO streaming.StreamingContext: StreamingContext started
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Got job 1 (start at NativeMethodAccessorImpl.java:0) with 1 output partitions
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Final stage: ResultStage 2 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Missing parents: List()
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Submitting ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620), which has no missing parents
17/08/01 14:49:33 INFO scheduler.ReceiverTracker: Receiver 0 started
17/08/01 14:49:33 INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 80.8 KB, free 1060.4 MB)
17/08/01 14:49:33 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 29.1 KB, free 1060.4 MB)
17/08/01 14:49:33 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.0.0.199:53355 (size: 29.1 KB, free: 1060.5 MB)
17/08/01 14:49:33 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620)
17/08/01 14:49:33 INFO cluster.YarnScheduler: Adding task set 2.0 with 1 tasks
17/08/01 14:49:33 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 2.0 (TID 70, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 7067 bytes)
17/08/01 14:49:33 INFO scheduler.JobScheduler: Added jobs for time 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Starting job streaming job 1501598973000 ms.0 from job set of time 1501598973000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:33 INFO scheduler.JobScheduler: Finished job streaming job 1501598973000 ms.0 from job set of time 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Total delay: 0.102 s for time 1501598973000 ms (execution: 0.026 s)
17/08/01 14:49:33 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 29.1 KB, free: 1060.5 MB)
17/08/01 14:49:33 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/08/01 14:49:33 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/08/01 14:49:33 INFO scheduler.ReceiverTracker: Registered receiver for stream 0 from 10.0.0.229:37096
17/08/01 14:49:34 INFO scheduler.JobScheduler: Added jobs for time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.JobScheduler: Starting job streaming job 1501598974000 ms.0 from job set of time 1501598974000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:34 INFO scheduler.JobScheduler: Finished job streaming job 1501598974000 ms.0 from job set of time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598974000 ms (execution: 0.009 s)
17/08/01 14:49:34 INFO python.PythonRDD: Removing RDD 5 from persistence list
17/08/01 14:49:34 INFO storage.BlockManager: Removing RDD 5
17/08/01 14:49:34 INFO rdd.BlockRDD: Removing RDD 4 from persistence list
17/08/01 14:49:34 INFO storage.BlockManager: Removing RDD 4
17/08/01 14:49:34 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[4] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/08/01 14:49:34 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/08/01 14:49:34 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.127:44695) with ID 2
17/08/01 14:49:34 INFO spark.ExecutorAllocationManager: New executor 2 has registered (new total is 2)
17/08/01 14:49:34 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-127.ec2.internal:59267 with 1060.5 MB RAM, BlockManagerId(2, ip-10-0-0-127.ec2.internal, 59267, None)
17/08/01 14:49:35 INFO scheduler.JobScheduler: Added jobs for time 1501598975000 ms
17/08/01 14:49:35 INFO scheduler.JobScheduler: Starting job streaming job 1501598975000 ms.0 from job set of time 1501598975000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:35 INFO scheduler.JobScheduler: Finished job streaming job 1501598975000 ms.0 from job set of time 1501598975000 ms
17/08/01 14:49:35 INFO python.PythonRDD: Removing RDD 7 from persistence list
17/08/01 14:49:35 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501598975000 ms (execution: 0.009 s)
17/08/01 14:49:35 INFO rdd.BlockRDD: Removing RDD 6 from persistence list
17/08/01 14:49:35 INFO storage.BlockManager: Removing RDD 6
17/08/01 14:49:35 INFO storage.BlockManager: Removing RDD 7
17/08/01 14:49:35 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[6] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598975000 ms
17/08/01 14:49:35 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598973000 ms
17/08/01 14:49:35 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598973000 ms
17/08/01 14:49:36 INFO scheduler.JobScheduler: Added jobs for time 1501598976000 ms
17/08/01 14:49:36 INFO scheduler.JobScheduler: Starting job streaming job 1501598976000 ms.0 from job set of time 1501598976000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:36 INFO scheduler.JobScheduler: Finished job streaming job 1501598976000 ms.0 from job set of time 1501598976000 ms
17/08/01 14:49:36 INFO python.PythonRDD: Removing RDD 9 from persistence list
17/08/01 14:49:36 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501598976000 ms (execution: 0.008 s)
17/08/01 14:49:36 INFO storage.BlockManager: Removing RDD 9
17/08/01 14:49:36 INFO rdd.BlockRDD: Removing RDD 8 from persistence list
17/08/01 14:49:36 INFO storage.BlockManager: Removing RDD 8
17/08/01 14:49:36 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[8] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598976000 ms
17/08/01 14:49:36 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598974000 ms
17/08/01 14:49:36 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598974000 ms
17/08/01 14:49:37 INFO scheduler.JobScheduler: Added jobs for time 1501598977000 ms
17/08/01 14:49:37 INFO scheduler.JobScheduler: Starting job streaming job 1501598977000 ms.0 from job set of time 1501598977000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:37 INFO scheduler.JobScheduler: Finished job streaming job 1501598977000 ms.0 from job set of time 1501598977000 ms
17/08/01 14:49:37 INFO python.PythonRDD: Removing RDD 11 from persistence list
17/08/01 14:49:37 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598977000 ms (execution: 0.008 s)
17/08/01 14:49:37 INFO storage.BlockManager: Removing RDD 11
17/08/01 14:49:37 INFO rdd.BlockRDD: Removing RDD 10 from persistence list
17/08/01 14:49:37 INFO storage.BlockManager: Removing RDD 10
17/08/01 14:49:37 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[10] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598977000 ms
17/08/01 14:49:37 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598975000 ms
17/08/01 14:49:37 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598975000 ms
17/08/01 14:49:38 INFO scheduler.JobScheduler: Added jobs for time 1501598978000 ms
17/08/01 14:49:38 INFO scheduler.JobScheduler: Starting job streaming job 1501598978000 ms.0 from job set of time 1501598978000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:38 INFO scheduler.JobScheduler: Finished job streaming job 1501598978000 ms.0 from job set of time 1501598978000 ms
17/08/01 14:49:38 INFO python.PythonRDD: Removing RDD 13 from persistence list
17/08/01 14:49:38 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501598978000 ms (execution: 0.007 s)
17/08/01 14:49:38 INFO storage.BlockManager: Removing RDD 13
17/08/01 14:49:38 INFO rdd.BlockRDD: Removing RDD 12 from persistence list
17/08/01 14:49:38 INFO storage.BlockManager: Removing RDD 12
17/08/01 14:49:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[12] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598978000 ms
17/08/01 14:49:38 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598976000 ms
17/08/01 14:49:38 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598976000 ms
17/08/01 14:49:39 INFO scheduler.JobScheduler: Added jobs for time 1501598979000 ms
17/08/01 14:49:39 INFO scheduler.JobScheduler: Starting job streaming job 1501598979000 ms.0 from job set of time 1501598979000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:39 INFO scheduler.JobScheduler: Finished job streaming job 1501598979000 ms.0 from job set of time 1501598979000 ms
17/08/01 14:49:39 INFO python.PythonRDD: Removing RDD 15 from persistence list
17/08/01 14:49:39 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598979000 ms (execution: 0.007 s)
17/08/01 14:49:39 INFO storage.BlockManager: Removing RDD 15
17/08/01 14:49:39 INFO rdd.BlockRDD: Removing RDD 14 from persistence list
17/08/01 14:49:39 INFO storage.BlockManager: Removing RDD 14
17/08/01 14:49:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[14] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598979000 ms
17/08/01 14:49:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598977000 ms
17/08/01 14:49:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598977000 ms
17/08/01 14:49:40 INFO scheduler.JobScheduler: Added jobs for time 1501598980000 ms
17/08/01 14:49:40 INFO scheduler.JobScheduler: Starting job streaming job 1501598980000 ms.0 from job set of time 1501598980000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:40 INFO scheduler.JobScheduler: Finished job streaming job 1501598980000 ms.0 from job set of time 1501598980000 ms
17/08/01 14:49:40 INFO python.PythonRDD: Removing RDD 17 from persistence list
17/08/01 14:49:40 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598980000 ms (execution: 0.015 s)
17/08/01 14:49:40 INFO storage.BlockManager: Removing RDD 17
17/08/01 14:49:40 INFO rdd.BlockRDD: Removing RDD 16 from persistence list
17/08/01 14:49:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[16] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598980000 ms
17/08/01 14:49:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598978000 ms
17/08/01 14:49:40 INFO storage.BlockManager: Removing RDD 16
17/08/01 14:49:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598978000 ms
17/08/01 14:49:41 INFO scheduler.JobScheduler: Added jobs for time 1501598981000 ms
17/08/01 14:49:41 INFO scheduler.JobScheduler: Starting job streaming job 1501598981000 ms.0 from job set of time 1501598981000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:41 INFO scheduler.JobScheduler: Finished job streaming job 1501598981000 ms.0 from job set of time 1501598981000 ms
17/08/01 14:49:41 INFO python.PythonRDD: Removing RDD 19 from persistence list
17/08/01 14:49:41 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501598981000 ms (execution: 0.006 s)
17/08/01 14:49:41 INFO storage.BlockManager: Removing RDD 19
17/08/01 14:49:41 INFO rdd.BlockRDD: Removing RDD 18 from persistence list
17/08/01 14:49:41 INFO storage.BlockManager: Removing RDD 18
17/08/01 14:49:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[18] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598981000 ms
17/08/01 14:49:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598979000 ms
17/08/01 14:49:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598979000 ms
17/08/01 14:49:42 INFO scheduler.JobScheduler: Added jobs for time 1501598982000 ms
17/08/01 14:49:42 INFO scheduler.JobScheduler: Starting job streaming job 1501598982000 ms.0 from job set of time 1501598982000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:42 INFO scheduler.JobScheduler: Finished job streaming job 1501598982000 ms.0 from job set of time 1501598982000 ms
17/08/01 14:49:42 INFO python.PythonRDD: Removing RDD 21 from persistence list
17/08/01 14:49:42 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598982000 ms (execution: 0.008 s)
17/08/01 14:49:42 INFO storage.BlockManager: Removing RDD 21
17/08/01 14:49:42 INFO rdd.BlockRDD: Removing RDD 20 from persistence list
17/08/01 14:49:42 INFO storage.BlockManager: Removing RDD 20
17/08/01 14:49:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[20] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598982000 ms
17/08/01 14:49:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598980000 ms
17/08/01 14:49:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598980000 ms
17/08/01 14:49:43 INFO scheduler.JobScheduler: Added jobs for time 1501598983000 ms
17/08/01 14:49:43 INFO scheduler.JobScheduler: Starting job streaming job 1501598983000 ms.0 from job set of time 1501598983000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:43 INFO scheduler.JobScheduler: Finished job streaming job 1501598983000 ms.0 from job set of time 1501598983000 ms
17/08/01 14:49:43 INFO python.PythonRDD: Removing RDD 23 from persistence list
17/08/01 14:49:43 INFO scheduler.JobScheduler: Total delay: 0.036 s for time 1501598983000 ms (execution: 0.009 s)
17/08/01 14:49:43 INFO storage.BlockManager: Removing RDD 23
17/08/01 14:49:43 INFO rdd.BlockRDD: Removing RDD 22 from persistence list
17/08/01 14:49:43 INFO storage.BlockManager: Removing RDD 22
17/08/01 14:49:43 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[22] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598983000 ms
17/08/01 14:49:43 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598981000 ms
17/08/01 14:49:43 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598981000 ms
17/08/01 14:49:44 INFO scheduler.JobScheduler: Added jobs for time 1501598984000 ms
17/08/01 14:49:44 INFO scheduler.JobScheduler: Starting job streaming job 1501598984000 ms.0 from job set of time 1501598984000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:44 INFO scheduler.JobScheduler: Finished job streaming job 1501598984000 ms.0 from job set of time 1501598984000 ms
17/08/01 14:49:44 INFO python.PythonRDD: Removing RDD 25 from persistence list
17/08/01 14:49:44 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598984000 ms (execution: 0.007 s)
17/08/01 14:49:44 INFO storage.BlockManager: Removing RDD 25
17/08/01 14:49:44 INFO rdd.BlockRDD: Removing RDD 24 from persistence list
17/08/01 14:49:44 INFO storage.BlockManager: Removing RDD 24
17/08/01 14:49:44 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[24] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598984000 ms
17/08/01 14:49:44 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598982000 ms
17/08/01 14:49:44 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598982000 ms
17/08/01 14:49:45 INFO scheduler.JobScheduler: Added jobs for time 1501598985000 ms
17/08/01 14:49:45 INFO scheduler.JobScheduler: Starting job streaming job 1501598985000 ms.0 from job set of time 1501598985000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:45 INFO scheduler.JobScheduler: Finished job streaming job 1501598985000 ms.0 from job set of time 1501598985000 ms
17/08/01 14:49:45 INFO python.PythonRDD: Removing RDD 27 from persistence list
17/08/01 14:49:45 INFO scheduler.JobScheduler: Total delay: 0.033 s for time 1501598985000 ms (execution: 0.007 s)
17/08/01 14:49:45 INFO storage.BlockManager: Removing RDD 27
17/08/01 14:49:45 INFO rdd.BlockRDD: Removing RDD 26 from persistence list
17/08/01 14:49:45 INFO storage.BlockManager: Removing RDD 26
17/08/01 14:49:45 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[26] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598985000 ms
17/08/01 14:49:45 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598983000 ms
17/08/01 14:49:45 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598983000 ms
17/08/01 14:49:46 INFO scheduler.JobScheduler: Added jobs for time 1501598986000 ms
17/08/01 14:49:46 INFO scheduler.JobScheduler: Starting job streaming job 1501598986000 ms.0 from job set of time 1501598986000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:46 INFO scheduler.JobScheduler: Finished job streaming job 1501598986000 ms.0 from job set of time 1501598986000 ms
17/08/01 14:49:46 INFO python.PythonRDD: Removing RDD 29 from persistence list
17/08/01 14:49:46 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598986000 ms (execution: 0.008 s)
17/08/01 14:49:46 INFO storage.BlockManager: Removing RDD 29
17/08/01 14:49:46 INFO rdd.BlockRDD: Removing RDD 28 from persistence list
17/08/01 14:49:46 INFO storage.BlockManager: Removing RDD 28
17/08/01 14:49:46 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[28] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598986000 ms
17/08/01 14:49:46 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598984000 ms
17/08/01 14:49:46 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598984000 ms
17/08/01 14:49:47 INFO scheduler.JobScheduler: Added jobs for time 1501598987000 ms
17/08/01 14:49:47 INFO scheduler.JobScheduler: Starting job streaming job 1501598987000 ms.0 from job set of time 1501598987000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:47 INFO scheduler.JobScheduler: Finished job streaming job 1501598987000 ms.0 from job set of time 1501598987000 ms
17/08/01 14:49:47 INFO python.PythonRDD: Removing RDD 31 from persistence list
17/08/01 14:49:47 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598987000 ms (execution: 0.014 s)
17/08/01 14:49:47 INFO storage.BlockManager: Removing RDD 31
17/08/01 14:49:47 INFO rdd.BlockRDD: Removing RDD 30 from persistence list
17/08/01 14:49:47 INFO storage.BlockManager: Removing RDD 30
17/08/01 14:49:47 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[30] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598987000 ms
17/08/01 14:49:47 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598985000 ms
17/08/01 14:49:47 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598985000 ms
17/08/01 14:49:48 INFO scheduler.JobScheduler: Added jobs for time 1501598988000 ms
17/08/01 14:49:48 INFO scheduler.JobScheduler: Starting job streaming job 1501598988000 ms.0 from job set of time 1501598988000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:48 INFO scheduler.JobScheduler: Finished job streaming job 1501598988000 ms.0 from job set of time 1501598988000 ms
17/08/01 14:49:48 INFO python.PythonRDD: Removing RDD 33 from persistence list
17/08/01 14:49:48 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501598988000 ms (execution: 0.008 s)
17/08/01 14:49:48 INFO storage.BlockManager: Removing RDD 33
17/08/01 14:49:48 INFO rdd.BlockRDD: Removing RDD 32 from persistence list
17/08/01 14:49:48 INFO storage.BlockManager: Removing RDD 32
17/08/01 14:49:48 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[32] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598988000 ms
17/08/01 14:49:48 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598986000 ms
17/08/01 14:49:48 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598986000 ms
17/08/01 14:49:49 INFO scheduler.JobScheduler: Added jobs for time 1501598989000 ms
17/08/01 14:49:49 INFO scheduler.JobScheduler: Starting job streaming job 1501598989000 ms.0 from job set of time 1501598989000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:49 INFO scheduler.JobScheduler: Finished job streaming job 1501598989000 ms.0 from job set of time 1501598989000 ms
17/08/01 14:49:49 INFO python.PythonRDD: Removing RDD 35 from persistence list
17/08/01 14:49:49 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501598989000 ms (execution: 0.006 s)
17/08/01 14:49:49 INFO storage.BlockManager: Removing RDD 35
17/08/01 14:49:49 INFO rdd.BlockRDD: Removing RDD 34 from persistence list
17/08/01 14:49:49 INFO storage.BlockManager: Removing RDD 34
17/08/01 14:49:49 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[34] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598989000 ms
17/08/01 14:49:49 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598987000 ms
17/08/01 14:49:49 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598987000 ms
17/08/01 14:49:50 INFO scheduler.JobScheduler: Added jobs for time 1501598990000 ms
17/08/01 14:49:50 INFO scheduler.JobScheduler: Starting job streaming job 1501598990000 ms.0 from job set of time 1501598990000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:50 INFO scheduler.JobScheduler: Finished job streaming job 1501598990000 ms.0 from job set of time 1501598990000 ms
17/08/01 14:49:50 INFO python.PythonRDD: Removing RDD 37 from persistence list
17/08/01 14:49:50 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501598990000 ms (execution: 0.008 s)
17/08/01 14:49:50 INFO storage.BlockManager: Removing RDD 37
17/08/01 14:49:50 INFO rdd.BlockRDD: Removing RDD 36 from persistence list
17/08/01 14:49:50 INFO storage.BlockManager: Removing RDD 36
17/08/01 14:49:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[36] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598990000 ms
17/08/01 14:49:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598988000 ms
17/08/01 14:49:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598988000 ms
17/08/01 14:49:51 INFO scheduler.JobScheduler: Added jobs for time 1501598991000 ms
17/08/01 14:49:51 INFO scheduler.JobScheduler: Starting job streaming job 1501598991000 ms.0 from job set of time 1501598991000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:51 INFO scheduler.JobScheduler: Finished job streaming job 1501598991000 ms.0 from job set of time 1501598991000 ms
17/08/01 14:49:51 INFO python.PythonRDD: Removing RDD 39 from persistence list
17/08/01 14:49:51 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501598991000 ms (execution: 0.006 s)
17/08/01 14:49:51 INFO storage.BlockManager: Removing RDD 39
17/08/01 14:49:51 INFO rdd.BlockRDD: Removing RDD 38 from persistence list
17/08/01 14:49:51 INFO storage.BlockManager: Removing RDD 38
17/08/01 14:49:51 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[38] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598991000 ms
17/08/01 14:49:51 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598989000 ms
17/08/01 14:49:51 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598989000 ms
17/08/01 14:49:52 INFO scheduler.JobScheduler: Added jobs for time 1501598992000 ms
17/08/01 14:49:52 INFO scheduler.JobScheduler: Starting job streaming job 1501598992000 ms.0 from job set of time 1501598992000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:52 INFO scheduler.JobScheduler: Finished job streaming job 1501598992000 ms.0 from job set of time 1501598992000 ms
17/08/01 14:49:52 INFO python.PythonRDD: Removing RDD 41 from persistence list
17/08/01 14:49:52 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501598992000 ms (execution: 0.007 s)
17/08/01 14:49:52 INFO storage.BlockManager: Removing RDD 41
17/08/01 14:49:52 INFO rdd.BlockRDD: Removing RDD 40 from persistence list
17/08/01 14:49:52 INFO storage.BlockManager: Removing RDD 40
17/08/01 14:49:52 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[40] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598992000 ms
17/08/01 14:49:52 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598990000 ms
17/08/01 14:49:52 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598990000 ms
17/08/01 14:49:53 INFO scheduler.JobScheduler: Added jobs for time 1501598993000 ms
17/08/01 14:49:53 INFO scheduler.JobScheduler: Starting job streaming job 1501598993000 ms.0 from job set of time 1501598993000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:53 INFO scheduler.JobScheduler: Finished job streaming job 1501598993000 ms.0 from job set of time 1501598993000 ms
17/08/01 14:49:53 INFO python.PythonRDD: Removing RDD 43 from persistence list
17/08/01 14:49:53 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598993000 ms (execution: 0.008 s)
17/08/01 14:49:53 INFO storage.BlockManager: Removing RDD 43
17/08/01 14:49:53 INFO rdd.BlockRDD: Removing RDD 42 from persistence list
17/08/01 14:49:53 INFO storage.BlockManager: Removing RDD 42
17/08/01 14:49:53 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[42] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598993000 ms
17/08/01 14:49:53 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598991000 ms
17/08/01 14:49:53 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598991000 ms
17/08/01 14:49:54 INFO scheduler.JobScheduler: Added jobs for time 1501598994000 ms
17/08/01 14:49:54 INFO scheduler.JobScheduler: Starting job streaming job 1501598994000 ms.0 from job set of time 1501598994000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:54 INFO scheduler.JobScheduler: Finished job streaming job 1501598994000 ms.0 from job set of time 1501598994000 ms
17/08/01 14:49:54 INFO python.PythonRDD: Removing RDD 45 from persistence list
17/08/01 14:49:54 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501598994000 ms (execution: 0.006 s)
17/08/01 14:49:54 INFO storage.BlockManager: Removing RDD 45
17/08/01 14:49:54 INFO rdd.BlockRDD: Removing RDD 44 from persistence list
17/08/01 14:49:54 INFO storage.BlockManager: Removing RDD 44
17/08/01 14:49:54 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[44] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598994000 ms
17/08/01 14:49:54 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598992000 ms
17/08/01 14:49:54 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598992000 ms
17/08/01 14:49:55 INFO scheduler.JobScheduler: Added jobs for time 1501598995000 ms
17/08/01 14:49:55 INFO scheduler.JobScheduler: Starting job streaming job 1501598995000 ms.0 from job set of time 1501598995000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:55 INFO scheduler.JobScheduler: Finished job streaming job 1501598995000 ms.0 from job set of time 1501598995000 ms
17/08/01 14:49:55 INFO python.PythonRDD: Removing RDD 47 from persistence list
17/08/01 14:49:55 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598995000 ms (execution: 0.008 s)
17/08/01 14:49:55 INFO storage.BlockManager: Removing RDD 47
17/08/01 14:49:55 INFO rdd.BlockRDD: Removing RDD 46 from persistence list
17/08/01 14:49:55 INFO storage.BlockManager: Removing RDD 46
17/08/01 14:49:55 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[46] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598995000 ms
17/08/01 14:49:55 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598993000 ms
17/08/01 14:49:55 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598993000 ms
17/08/01 14:49:56 INFO scheduler.JobScheduler: Added jobs for time 1501598996000 ms
17/08/01 14:49:56 INFO scheduler.JobScheduler: Starting job streaming job 1501598996000 ms.0 from job set of time 1501598996000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:56 INFO scheduler.JobScheduler: Finished job streaming job 1501598996000 ms.0 from job set of time 1501598996000 ms
17/08/01 14:49:56 INFO python.PythonRDD: Removing RDD 49 from persistence list
17/08/01 14:49:56 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501598996000 ms (execution: 0.020 s)
17/08/01 14:49:56 INFO storage.BlockManager: Removing RDD 49
17/08/01 14:49:56 INFO rdd.BlockRDD: Removing RDD 48 from persistence list
17/08/01 14:49:56 INFO storage.BlockManager: Removing RDD 48
17/08/01 14:49:56 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[48] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598996000 ms
17/08/01 14:49:56 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598994000 ms
17/08/01 14:49:56 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598994000 ms
17/08/01 14:49:57 INFO scheduler.JobScheduler: Added jobs for time 1501598997000 ms
17/08/01 14:49:57 INFO scheduler.JobScheduler: Starting job streaming job 1501598997000 ms.0 from job set of time 1501598997000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:57 INFO scheduler.JobScheduler: Finished job streaming job 1501598997000 ms.0 from job set of time 1501598997000 ms
17/08/01 14:49:57 INFO python.PythonRDD: Removing RDD 51 from persistence list
17/08/01 14:49:57 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598997000 ms (execution: 0.007 s)
17/08/01 14:49:57 INFO storage.BlockManager: Removing RDD 51
17/08/01 14:49:57 INFO rdd.BlockRDD: Removing RDD 50 from persistence list
17/08/01 14:49:57 INFO storage.BlockManager: Removing RDD 50
17/08/01 14:49:57 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[50] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598997000 ms
17/08/01 14:49:57 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598995000 ms
17/08/01 14:49:57 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598995000 ms
17/08/01 14:49:58 INFO scheduler.JobScheduler: Added jobs for time 1501598998000 ms
17/08/01 14:49:58 INFO scheduler.JobScheduler: Starting job streaming job 1501598998000 ms.0 from job set of time 1501598998000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:58 INFO scheduler.JobScheduler: Finished job streaming job 1501598998000 ms.0 from job set of time 1501598998000 ms
17/08/01 14:49:58 INFO python.PythonRDD: Removing RDD 53 from persistence list
17/08/01 14:49:58 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501598998000 ms (execution: 0.008 s)
17/08/01 14:49:58 INFO storage.BlockManager: Removing RDD 53
17/08/01 14:49:58 INFO rdd.BlockRDD: Removing RDD 52 from persistence list
17/08/01 14:49:58 INFO storage.BlockManager: Removing RDD 52
17/08/01 14:49:58 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[52] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598998000 ms
17/08/01 14:49:58 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598996000 ms
17/08/01 14:49:58 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598996000 ms
17/08/01 14:49:59 INFO scheduler.JobScheduler: Added jobs for time 1501598999000 ms
17/08/01 14:49:59 INFO scheduler.JobScheduler: Starting job streaming job 1501598999000 ms.0 from job set of time 1501598999000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:59 INFO scheduler.JobScheduler: Finished job streaming job 1501598999000 ms.0 from job set of time 1501598999000 ms
17/08/01 14:49:59 INFO python.PythonRDD: Removing RDD 55 from persistence list
17/08/01 14:49:59 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598999000 ms (execution: 0.007 s)
17/08/01 14:49:59 INFO storage.BlockManager: Removing RDD 55
17/08/01 14:49:59 INFO rdd.BlockRDD: Removing RDD 54 from persistence list
17/08/01 14:49:59 INFO storage.BlockManager: Removing RDD 54
17/08/01 14:49:59 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[54] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598999000 ms
17/08/01 14:49:59 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598997000 ms
17/08/01 14:49:59 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598997000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Added jobs for time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Starting job streaming job 1501599000000 ms.0 from job set of time 1501599000000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:00 INFO scheduler.JobScheduler: Finished job streaming job 1501599000000 ms.0 from job set of time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599000000 ms (execution: 0.006 s)
17/08/01 14:50:00 INFO python.PythonRDD: Removing RDD 57 from persistence list
17/08/01 14:50:00 INFO storage.BlockManager: Removing RDD 57
17/08/01 14:50:00 INFO rdd.BlockRDD: Removing RDD 56 from persistence list
17/08/01 14:50:00 INFO storage.BlockManager: Removing RDD 56
17/08/01 14:50:00 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[56] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598998000 ms
17/08/01 14:50:00 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598998000 ms
17/08/01 14:50:01 INFO scheduler.JobScheduler: Added jobs for time 1501599001000 ms
17/08/01 14:50:01 INFO scheduler.JobScheduler: Starting job streaming job 1501599001000 ms.0 from job set of time 1501599001000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:01 INFO scheduler.JobScheduler: Finished job streaming job 1501599001000 ms.0 from job set of time 1501599001000 ms
17/08/01 14:50:01 INFO python.PythonRDD: Removing RDD 59 from persistence list
17/08/01 14:50:01 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599001000 ms (execution: 0.007 s)
17/08/01 14:50:01 INFO storage.BlockManager: Removing RDD 59
17/08/01 14:50:01 INFO rdd.BlockRDD: Removing RDD 58 from persistence list
17/08/01 14:50:01 INFO storage.BlockManager: Removing RDD 58
17/08/01 14:50:01 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[58] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599001000 ms
17/08/01 14:50:01 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598999000 ms
17/08/01 14:50:01 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598999000 ms
17/08/01 14:50:02 INFO scheduler.JobScheduler: Added jobs for time 1501599002000 ms
17/08/01 14:50:02 INFO scheduler.JobScheduler: Starting job streaming job 1501599002000 ms.0 from job set of time 1501599002000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:02 INFO scheduler.JobScheduler: Finished job streaming job 1501599002000 ms.0 from job set of time 1501599002000 ms
17/08/01 14:50:02 INFO python.PythonRDD: Removing RDD 61 from persistence list
17/08/01 14:50:02 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599002000 ms (execution: 0.006 s)
17/08/01 14:50:02 INFO storage.BlockManager: Removing RDD 61
17/08/01 14:50:02 INFO rdd.BlockRDD: Removing RDD 60 from persistence list
17/08/01 14:50:02 INFO storage.BlockManager: Removing RDD 60
17/08/01 14:50:02 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[60] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599002000 ms
17/08/01 14:50:02 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599000000 ms
17/08/01 14:50:02 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599000000 ms
17/08/01 14:50:03 INFO scheduler.JobScheduler: Added jobs for time 1501599003000 ms
17/08/01 14:50:03 INFO scheduler.JobScheduler: Starting job streaming job 1501599003000 ms.0 from job set of time 1501599003000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:03 INFO scheduler.JobScheduler: Finished job streaming job 1501599003000 ms.0 from job set of time 1501599003000 ms
17/08/01 14:50:03 INFO python.PythonRDD: Removing RDD 63 from persistence list
17/08/01 14:50:03 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599003000 ms (execution: 0.007 s)
17/08/01 14:50:03 INFO storage.BlockManager: Removing RDD 63
17/08/01 14:50:03 INFO rdd.BlockRDD: Removing RDD 62 from persistence list
17/08/01 14:50:03 INFO storage.BlockManager: Removing RDD 62
17/08/01 14:50:03 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[62] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599003000 ms
17/08/01 14:50:03 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599001000 ms
17/08/01 14:50:03 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599001000 ms
17/08/01 14:50:04 INFO scheduler.JobScheduler: Added jobs for time 1501599004000 ms
17/08/01 14:50:04 INFO scheduler.JobScheduler: Starting job streaming job 1501599004000 ms.0 from job set of time 1501599004000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:04 INFO scheduler.JobScheduler: Finished job streaming job 1501599004000 ms.0 from job set of time 1501599004000 ms
17/08/01 14:50:04 INFO python.PythonRDD: Removing RDD 65 from persistence list
17/08/01 14:50:04 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599004000 ms (execution: 0.006 s)
17/08/01 14:50:04 INFO storage.BlockManager: Removing RDD 65
17/08/01 14:50:04 INFO rdd.BlockRDD: Removing RDD 64 from persistence list
17/08/01 14:50:04 INFO storage.BlockManager: Removing RDD 64
17/08/01 14:50:04 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[64] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599004000 ms
17/08/01 14:50:04 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599002000 ms
17/08/01 14:50:04 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599002000 ms
17/08/01 14:50:05 INFO scheduler.JobScheduler: Added jobs for time 1501599005000 ms
17/08/01 14:50:05 INFO scheduler.JobScheduler: Starting job streaming job 1501599005000 ms.0 from job set of time 1501599005000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:05 INFO scheduler.JobScheduler: Finished job streaming job 1501599005000 ms.0 from job set of time 1501599005000 ms
17/08/01 14:50:05 INFO python.PythonRDD: Removing RDD 67 from persistence list
17/08/01 14:50:05 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501599005000 ms (execution: 0.013 s)
17/08/01 14:50:05 INFO storage.BlockManager: Removing RDD 67
17/08/01 14:50:05 INFO rdd.BlockRDD: Removing RDD 66 from persistence list
17/08/01 14:50:05 INFO storage.BlockManager: Removing RDD 66
17/08/01 14:50:05 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[66] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599005000 ms
17/08/01 14:50:05 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599003000 ms
17/08/01 14:50:05 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599003000 ms
17/08/01 14:50:06 INFO scheduler.JobScheduler: Added jobs for time 1501599006000 ms
17/08/01 14:50:06 INFO scheduler.JobScheduler: Starting job streaming job 1501599006000 ms.0 from job set of time 1501599006000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:06 INFO scheduler.JobScheduler: Finished job streaming job 1501599006000 ms.0 from job set of time 1501599006000 ms
17/08/01 14:50:06 INFO python.PythonRDD: Removing RDD 69 from persistence list
17/08/01 14:50:06 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599006000 ms (execution: 0.008 s)
17/08/01 14:50:06 INFO storage.BlockManager: Removing RDD 69
17/08/01 14:50:06 INFO rdd.BlockRDD: Removing RDD 68 from persistence list
17/08/01 14:50:06 INFO storage.BlockManager: Removing RDD 68
17/08/01 14:50:06 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[68] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599006000 ms
17/08/01 14:50:06 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599004000 ms
17/08/01 14:50:06 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599004000 ms
17/08/01 14:50:07 INFO scheduler.JobScheduler: Added jobs for time 1501599007000 ms
17/08/01 14:50:07 INFO scheduler.JobScheduler: Starting job streaming job 1501599007000 ms.0 from job set of time 1501599007000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:07 INFO scheduler.JobScheduler: Finished job streaming job 1501599007000 ms.0 from job set of time 1501599007000 ms
17/08/01 14:50:07 INFO python.PythonRDD: Removing RDD 71 from persistence list
17/08/01 14:50:07 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599007000 ms (execution: 0.009 s)
17/08/01 14:50:07 INFO rdd.BlockRDD: Removing RDD 70 from persistence list
17/08/01 14:50:07 INFO storage.BlockManager: Removing RDD 71
17/08/01 14:50:07 INFO storage.BlockManager: Removing RDD 70
17/08/01 14:50:07 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[70] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599007000 ms
17/08/01 14:50:07 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599005000 ms
17/08/01 14:50:07 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599005000 ms
17/08/01 14:50:08 INFO scheduler.JobScheduler: Added jobs for time 1501599008000 ms
17/08/01 14:50:08 INFO scheduler.JobScheduler: Starting job streaming job 1501599008000 ms.0 from job set of time 1501599008000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:08 INFO scheduler.JobScheduler: Finished job streaming job 1501599008000 ms.0 from job set of time 1501599008000 ms
17/08/01 14:50:08 INFO python.PythonRDD: Removing RDD 73 from persistence list
17/08/01 14:50:08 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599008000 ms (execution: 0.006 s)
17/08/01 14:50:08 INFO storage.BlockManager: Removing RDD 73
17/08/01 14:50:08 INFO rdd.BlockRDD: Removing RDD 72 from persistence list
17/08/01 14:50:08 INFO storage.BlockManager: Removing RDD 72
17/08/01 14:50:08 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[72] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599008000 ms
17/08/01 14:50:08 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599006000 ms
17/08/01 14:50:08 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599006000 ms
17/08/01 14:50:09 INFO scheduler.JobScheduler: Added jobs for time 1501599009000 ms
17/08/01 14:50:09 INFO scheduler.JobScheduler: Starting job streaming job 1501599009000 ms.0 from job set of time 1501599009000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:09 INFO scheduler.JobScheduler: Finished job streaming job 1501599009000 ms.0 from job set of time 1501599009000 ms
17/08/01 14:50:09 INFO python.PythonRDD: Removing RDD 75 from persistence list
17/08/01 14:50:09 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599009000 ms (execution: 0.005 s)
17/08/01 14:50:09 INFO storage.BlockManager: Removing RDD 75
17/08/01 14:50:09 INFO rdd.BlockRDD: Removing RDD 74 from persistence list
17/08/01 14:50:09 INFO storage.BlockManager: Removing RDD 74
17/08/01 14:50:09 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[74] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599009000 ms
17/08/01 14:50:09 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599007000 ms
17/08/01 14:50:09 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599007000 ms
17/08/01 14:50:10 INFO scheduler.JobScheduler: Added jobs for time 1501599010000 ms
17/08/01 14:50:10 INFO scheduler.JobScheduler: Starting job streaming job 1501599010000 ms.0 from job set of time 1501599010000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:10 INFO scheduler.JobScheduler: Finished job streaming job 1501599010000 ms.0 from job set of time 1501599010000 ms
17/08/01 14:50:10 INFO python.PythonRDD: Removing RDD 77 from persistence list
17/08/01 14:50:10 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599010000 ms (execution: 0.005 s)
17/08/01 14:50:10 INFO rdd.BlockRDD: Removing RDD 76 from persistence list
17/08/01 14:50:10 INFO storage.BlockManager: Removing RDD 76
17/08/01 14:50:10 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[76] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599010000 ms
17/08/01 14:50:10 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599008000 ms
17/08/01 14:50:10 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599008000 ms
17/08/01 14:50:10 INFO storage.BlockManager: Removing RDD 77
17/08/01 14:50:11 INFO scheduler.JobScheduler: Added jobs for time 1501599011000 ms
17/08/01 14:50:11 INFO scheduler.JobScheduler: Starting job streaming job 1501599011000 ms.0 from job set of time 1501599011000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:11 INFO scheduler.JobScheduler: Finished job streaming job 1501599011000 ms.0 from job set of time 1501599011000 ms
17/08/01 14:50:11 INFO python.PythonRDD: Removing RDD 79 from persistence list
17/08/01 14:50:11 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599011000 ms (execution: 0.007 s)
17/08/01 14:50:11 INFO storage.BlockManager: Removing RDD 79
17/08/01 14:50:11 INFO rdd.BlockRDD: Removing RDD 78 from persistence list
17/08/01 14:50:11 INFO storage.BlockManager: Removing RDD 78
17/08/01 14:50:11 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[78] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599011000 ms
17/08/01 14:50:11 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599009000 ms
17/08/01 14:50:11 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599009000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Added jobs for time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Starting job streaming job 1501599012000 ms.0 from job set of time 1501599012000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:12 INFO scheduler.JobScheduler: Finished job streaming job 1501599012000 ms.0 from job set of time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599012000 ms (execution: 0.015 s)
17/08/01 14:50:12 INFO python.PythonRDD: Removing RDD 81 from persistence list
17/08/01 14:50:12 INFO storage.BlockManager: Removing RDD 81
17/08/01 14:50:12 INFO rdd.BlockRDD: Removing RDD 80 from persistence list
17/08/01 14:50:12 INFO storage.BlockManager: Removing RDD 80
17/08/01 14:50:12 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[80] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599010000 ms
17/08/01 14:50:12 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599010000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Added jobs for time 1501599013000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Starting job streaming job 1501599013000 ms.0 from job set of time 1501599013000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:13 INFO scheduler.JobScheduler: Finished job streaming job 1501599013000 ms.0 from job set of time 1501599013000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501599013000 ms (execution: 0.008 s)
17/08/01 14:50:13 INFO python.PythonRDD: Removing RDD 83 from persistence list
17/08/01 14:50:13 INFO storage.BlockManager: Removing RDD 83
17/08/01 14:50:13 INFO rdd.BlockRDD: Removing RDD 82 from persistence list
17/08/01 14:50:13 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[82] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599013000 ms
17/08/01 14:50:13 INFO storage.BlockManager: Removing RDD 82
17/08/01 14:50:13 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599011000 ms
17/08/01 14:50:13 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599011000 ms
17/08/01 14:50:14 INFO scheduler.JobScheduler: Added jobs for time 1501599014000 ms
17/08/01 14:50:14 INFO scheduler.JobScheduler: Starting job streaming job 1501599014000 ms.0 from job set of time 1501599014000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:14 INFO scheduler.JobScheduler: Finished job streaming job 1501599014000 ms.0 from job set of time 1501599014000 ms
17/08/01 14:50:14 INFO python.PythonRDD: Removing RDD 85 from persistence list
17/08/01 14:50:14 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599014000 ms (execution: 0.007 s)
17/08/01 14:50:14 INFO rdd.BlockRDD: Removing RDD 84 from persistence list
17/08/01 14:50:14 INFO storage.BlockManager: Removing RDD 85
17/08/01 14:50:14 INFO storage.BlockManager: Removing RDD 84
17/08/01 14:50:14 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[84] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599014000 ms
17/08/01 14:50:14 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599012000 ms
17/08/01 14:50:14 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599012000 ms
17/08/01 14:50:15 INFO scheduler.JobScheduler: Added jobs for time 1501599015000 ms
17/08/01 14:50:15 INFO scheduler.JobScheduler: Starting job streaming job 1501599015000 ms.0 from job set of time 1501599015000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:15 INFO scheduler.JobScheduler: Finished job streaming job 1501599015000 ms.0 from job set of time 1501599015000 ms
17/08/01 14:50:15 INFO python.PythonRDD: Removing RDD 87 from persistence list
17/08/01 14:50:15 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599015000 ms (execution: 0.007 s)
17/08/01 14:50:15 INFO storage.BlockManager: Removing RDD 87
17/08/01 14:50:15 INFO rdd.BlockRDD: Removing RDD 86 from persistence list
17/08/01 14:50:15 INFO storage.BlockManager: Removing RDD 86
17/08/01 14:50:15 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[86] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599015000 ms
17/08/01 14:50:15 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599013000 ms
17/08/01 14:50:15 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599013000 ms
17/08/01 14:50:16 INFO scheduler.JobScheduler: Added jobs for time 1501599016000 ms
17/08/01 14:50:16 INFO scheduler.JobScheduler: Starting job streaming job 1501599016000 ms.0 from job set of time 1501599016000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:16 INFO scheduler.JobScheduler: Finished job streaming job 1501599016000 ms.0 from job set of time 1501599016000 ms
17/08/01 14:50:16 INFO python.PythonRDD: Removing RDD 89 from persistence list
17/08/01 14:50:16 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501599016000 ms (execution: 0.008 s)
17/08/01 14:50:16 INFO storage.BlockManager: Removing RDD 89
17/08/01 14:50:16 INFO rdd.BlockRDD: Removing RDD 88 from persistence list
17/08/01 14:50:16 INFO storage.BlockManager: Removing RDD 88
17/08/01 14:50:16 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[88] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599016000 ms
17/08/01 14:50:16 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599014000 ms
17/08/01 14:50:16 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599014000 ms
17/08/01 14:50:17 INFO scheduler.JobScheduler: Added jobs for time 1501599017000 ms
17/08/01 14:50:17 INFO scheduler.JobScheduler: Starting job streaming job 1501599017000 ms.0 from job set of time 1501599017000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:17 INFO scheduler.JobScheduler: Finished job streaming job 1501599017000 ms.0 from job set of time 1501599017000 ms
17/08/01 14:50:17 INFO python.PythonRDD: Removing RDD 91 from persistence list
17/08/01 14:50:17 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501599017000 ms (execution: 0.008 s)
17/08/01 14:50:17 INFO storage.BlockManager: Removing RDD 91
17/08/01 14:50:17 INFO rdd.BlockRDD: Removing RDD 90 from persistence list
17/08/01 14:50:17 INFO storage.BlockManager: Removing RDD 90
17/08/01 14:50:17 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[90] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599017000 ms
17/08/01 14:50:17 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599015000 ms
17/08/01 14:50:17 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599015000 ms
17/08/01 14:50:18 INFO scheduler.JobScheduler: Added jobs for time 1501599018000 ms
17/08/01 14:50:18 INFO scheduler.JobScheduler: Starting job streaming job 1501599018000 ms.0 from job set of time 1501599018000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:18 INFO scheduler.JobScheduler: Finished job streaming job 1501599018000 ms.0 from job set of time 1501599018000 ms
17/08/01 14:50:18 INFO python.PythonRDD: Removing RDD 93 from persistence list
17/08/01 14:50:18 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599018000 ms (execution: 0.006 s)
17/08/01 14:50:18 INFO storage.BlockManager: Removing RDD 93
17/08/01 14:50:18 INFO rdd.BlockRDD: Removing RDD 92 from persistence list
17/08/01 14:50:18 INFO storage.BlockManager: Removing RDD 92
17/08/01 14:50:18 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[92] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599018000 ms
17/08/01 14:50:18 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599016000 ms
17/08/01 14:50:18 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599016000 ms
17/08/01 14:50:19 INFO scheduler.JobScheduler: Added jobs for time 1501599019000 ms
17/08/01 14:50:19 INFO scheduler.JobScheduler: Starting job streaming job 1501599019000 ms.0 from job set of time 1501599019000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:19 INFO scheduler.JobScheduler: Finished job streaming job 1501599019000 ms.0 from job set of time 1501599019000 ms
17/08/01 14:50:19 INFO python.PythonRDD: Removing RDD 95 from persistence list
17/08/01 14:50:19 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599019000 ms (execution: 0.015 s)
17/08/01 14:50:19 INFO storage.BlockManager: Removing RDD 95
17/08/01 14:50:19 INFO rdd.BlockRDD: Removing RDD 94 from persistence list
17/08/01 14:50:19 INFO storage.BlockManager: Removing RDD 94
17/08/01 14:50:19 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[94] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599019000 ms
17/08/01 14:50:19 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599017000 ms
17/08/01 14:50:19 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599017000 ms
17/08/01 14:50:20 INFO scheduler.JobScheduler: Added jobs for time 1501599020000 ms
17/08/01 14:50:20 INFO scheduler.JobScheduler: Starting job streaming job 1501599020000 ms.0 from job set of time 1501599020000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:20 INFO scheduler.JobScheduler: Finished job streaming job 1501599020000 ms.0 from job set of time 1501599020000 ms
17/08/01 14:50:20 INFO python.PythonRDD: Removing RDD 97 from persistence list
17/08/01 14:50:20 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599020000 ms (execution: 0.006 s)
17/08/01 14:50:20 INFO storage.BlockManager: Removing RDD 97
17/08/01 14:50:20 INFO rdd.BlockRDD: Removing RDD 96 from persistence list
17/08/01 14:50:20 INFO storage.BlockManager: Removing RDD 96
17/08/01 14:50:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[96] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599020000 ms
17/08/01 14:50:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599018000 ms
17/08/01 14:50:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599018000 ms
17/08/01 14:50:21 INFO scheduler.JobScheduler: Added jobs for time 1501599021000 ms
17/08/01 14:50:21 INFO scheduler.JobScheduler: Starting job streaming job 1501599021000 ms.0 from job set of time 1501599021000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:21 INFO scheduler.JobScheduler: Finished job streaming job 1501599021000 ms.0 from job set of time 1501599021000 ms
17/08/01 14:50:21 INFO python.PythonRDD: Removing RDD 99 from persistence list
17/08/01 14:50:21 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599021000 ms (execution: 0.007 s)
17/08/01 14:50:21 INFO storage.BlockManager: Removing RDD 99
17/08/01 14:50:21 INFO rdd.BlockRDD: Removing RDD 98 from persistence list
17/08/01 14:50:21 INFO storage.BlockManager: Removing RDD 98
17/08/01 14:50:21 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[98] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599021000 ms
17/08/01 14:50:21 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599019000 ms
17/08/01 14:50:21 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599019000 ms
17/08/01 14:50:22 INFO scheduler.JobScheduler: Added jobs for time 1501599022000 ms
17/08/01 14:50:22 INFO scheduler.JobScheduler: Starting job streaming job 1501599022000 ms.0 from job set of time 1501599022000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:22 INFO scheduler.JobScheduler: Finished job streaming job 1501599022000 ms.0 from job set of time 1501599022000 ms
17/08/01 14:50:22 INFO python.PythonRDD: Removing RDD 101 from persistence list
17/08/01 14:50:22 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501599022000 ms (execution: 0.007 s)
17/08/01 14:50:22 INFO storage.BlockManager: Removing RDD 101
17/08/01 14:50:22 INFO rdd.BlockRDD: Removing RDD 100 from persistence list
17/08/01 14:50:22 INFO storage.BlockManager: Removing RDD 100
17/08/01 14:50:22 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[100] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599022000 ms
17/08/01 14:50:22 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599020000 ms
17/08/01 14:50:22 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599020000 ms
17/08/01 14:50:23 INFO scheduler.JobScheduler: Added jobs for time 1501599023000 ms
17/08/01 14:50:23 INFO scheduler.JobScheduler: Starting job streaming job 1501599023000 ms.0 from job set of time 1501599023000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:23 INFO scheduler.JobScheduler: Finished job streaming job 1501599023000 ms.0 from job set of time 1501599023000 ms
17/08/01 14:50:23 INFO python.PythonRDD: Removing RDD 103 from persistence list
17/08/01 14:50:23 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599023000 ms (execution: 0.007 s)
17/08/01 14:50:23 INFO storage.BlockManager: Removing RDD 103
17/08/01 14:50:23 INFO rdd.BlockRDD: Removing RDD 102 from persistence list
17/08/01 14:50:23 INFO storage.BlockManager: Removing RDD 102
17/08/01 14:50:23 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[102] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599023000 ms
17/08/01 14:50:23 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599021000 ms
17/08/01 14:50:23 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599021000 ms
17/08/01 14:50:24 INFO scheduler.JobScheduler: Added jobs for time 1501599024000 ms
17/08/01 14:50:24 INFO scheduler.JobScheduler: Starting job streaming job 1501599024000 ms.0 from job set of time 1501599024000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:24 INFO scheduler.JobScheduler: Finished job streaming job 1501599024000 ms.0 from job set of time 1501599024000 ms
17/08/01 14:50:24 INFO python.PythonRDD: Removing RDD 105 from persistence list
17/08/01 14:50:24 INFO scheduler.JobScheduler: Total delay: 0.035 s for time 1501599024000 ms (execution: 0.007 s)
17/08/01 14:50:24 INFO storage.BlockManager: Removing RDD 105
17/08/01 14:50:24 INFO rdd.BlockRDD: Removing RDD 104 from persistence list
17/08/01 14:50:24 INFO storage.BlockManager: Removing RDD 104
17/08/01 14:50:24 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[104] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599024000 ms
17/08/01 14:50:24 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599022000 ms
17/08/01 14:50:24 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599022000 ms
17/08/01 14:50:25 INFO scheduler.JobScheduler: Added jobs for time 1501599025000 ms
17/08/01 14:50:25 INFO scheduler.JobScheduler: Starting job streaming job 1501599025000 ms.0 from job set of time 1501599025000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:25 INFO scheduler.JobScheduler: Finished job streaming job 1501599025000 ms.0 from job set of time 1501599025000 ms
17/08/01 14:50:25 INFO python.PythonRDD: Removing RDD 107 from persistence list
17/08/01 14:50:25 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599025000 ms (execution: 0.007 s)
17/08/01 14:50:25 INFO storage.BlockManager: Removing RDD 107
17/08/01 14:50:25 INFO rdd.BlockRDD: Removing RDD 106 from persistence list
17/08/01 14:50:25 INFO storage.BlockManager: Removing RDD 106
17/08/01 14:50:25 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[106] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599025000 ms
17/08/01 14:50:25 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599023000 ms
17/08/01 14:50:25 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599023000 ms
17/08/01 14:50:26 INFO scheduler.JobScheduler: Added jobs for time 1501599026000 ms
17/08/01 14:50:26 INFO scheduler.JobScheduler: Starting job streaming job 1501599026000 ms.0 from job set of time 1501599026000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:26 INFO scheduler.JobScheduler: Finished job streaming job 1501599026000 ms.0 from job set of time 1501599026000 ms
17/08/01 14:50:26 INFO python.PythonRDD: Removing RDD 109 from persistence list
17/08/01 14:50:26 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599026000 ms (execution: 0.007 s)
17/08/01 14:50:26 INFO storage.BlockManager: Removing RDD 109
17/08/01 14:50:26 INFO rdd.BlockRDD: Removing RDD 108 from persistence list
17/08/01 14:50:26 INFO storage.BlockManager: Removing RDD 108
17/08/01 14:50:26 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[108] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599026000 ms
17/08/01 14:50:26 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599024000 ms
17/08/01 14:50:26 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599024000 ms
17/08/01 14:50:27 INFO scheduler.JobScheduler: Added jobs for time 1501599027000 ms
17/08/01 14:50:27 INFO scheduler.JobScheduler: Starting job streaming job 1501599027000 ms.0 from job set of time 1501599027000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:27 INFO scheduler.JobScheduler: Finished job streaming job 1501599027000 ms.0 from job set of time 1501599027000 ms
17/08/01 14:50:27 INFO python.PythonRDD: Removing RDD 111 from persistence list
17/08/01 14:50:27 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599027000 ms (execution: 0.007 s)
17/08/01 14:50:27 INFO storage.BlockManager: Removing RDD 111
17/08/01 14:50:27 INFO rdd.BlockRDD: Removing RDD 110 from persistence list
17/08/01 14:50:27 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[110] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599027000 ms
17/08/01 14:50:27 INFO storage.BlockManager: Removing RDD 110
17/08/01 14:50:27 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599025000 ms
17/08/01 14:50:27 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599025000 ms
17/08/01 14:50:28 INFO scheduler.JobScheduler: Added jobs for time 1501599028000 ms
17/08/01 14:50:28 INFO scheduler.JobScheduler: Starting job streaming job 1501599028000 ms.0 from job set of time 1501599028000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:28 INFO scheduler.JobScheduler: Finished job streaming job 1501599028000 ms.0 from job set of time 1501599028000 ms
17/08/01 14:50:28 INFO python.PythonRDD: Removing RDD 113 from persistence list
17/08/01 14:50:28 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599028000 ms (execution: 0.017 s)
17/08/01 14:50:28 INFO storage.BlockManager: Removing RDD 113
17/08/01 14:50:28 INFO rdd.BlockRDD: Removing RDD 112 from persistence list
17/08/01 14:50:28 INFO storage.BlockManager: Removing RDD 112
17/08/01 14:50:28 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[112] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599028000 ms
17/08/01 14:50:28 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599026000 ms
17/08/01 14:50:28 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599026000 ms
17/08/01 14:50:29 INFO scheduler.JobScheduler: Added jobs for time 1501599029000 ms
17/08/01 14:50:29 INFO scheduler.JobScheduler: Starting job streaming job 1501599029000 ms.0 from job set of time 1501599029000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:29 INFO scheduler.JobScheduler: Finished job streaming job 1501599029000 ms.0 from job set of time 1501599029000 ms
17/08/01 14:50:29 INFO python.PythonRDD: Removing RDD 115 from persistence list
17/08/01 14:50:29 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599029000 ms (execution: 0.007 s)
17/08/01 14:50:29 INFO storage.BlockManager: Removing RDD 115
17/08/01 14:50:29 INFO rdd.BlockRDD: Removing RDD 114 from persistence list
17/08/01 14:50:29 INFO storage.BlockManager: Removing RDD 114
17/08/01 14:50:29 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[114] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599029000 ms
17/08/01 14:50:29 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599027000 ms
17/08/01 14:50:29 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599027000 ms
17/08/01 14:50:30 INFO scheduler.JobScheduler: Added jobs for time 1501599030000 ms
17/08/01 14:50:30 INFO scheduler.JobScheduler: Starting job streaming job 1501599030000 ms.0 from job set of time 1501599030000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:30 INFO scheduler.JobScheduler: Finished job streaming job 1501599030000 ms.0 from job set of time 1501599030000 ms
17/08/01 14:50:30 INFO python.PythonRDD: Removing RDD 117 from persistence list
17/08/01 14:50:30 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599030000 ms (execution: 0.007 s)
17/08/01 14:50:30 INFO storage.BlockManager: Removing RDD 117
17/08/01 14:50:30 INFO rdd.BlockRDD: Removing RDD 116 from persistence list
17/08/01 14:50:30 INFO storage.BlockManager: Removing RDD 116
17/08/01 14:50:30 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[116] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599030000 ms
17/08/01 14:50:30 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599028000 ms
17/08/01 14:50:30 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599028000 ms
17/08/01 14:50:31 INFO scheduler.JobScheduler: Added jobs for time 1501599031000 ms
17/08/01 14:50:31 INFO scheduler.JobScheduler: Starting job streaming job 1501599031000 ms.0 from job set of time 1501599031000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:31 INFO scheduler.JobScheduler: Finished job streaming job 1501599031000 ms.0 from job set of time 1501599031000 ms
17/08/01 14:50:31 INFO python.PythonRDD: Removing RDD 119 from persistence list
17/08/01 14:50:31 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599031000 ms (execution: 0.006 s)
17/08/01 14:50:31 INFO rdd.BlockRDD: Removing RDD 118 from persistence list
17/08/01 14:50:31 INFO storage.BlockManager: Removing RDD 119
17/08/01 14:50:31 INFO storage.BlockManager: Removing RDD 118
17/08/01 14:50:31 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[118] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599031000 ms
17/08/01 14:50:31 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599029000 ms
17/08/01 14:50:31 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599029000 ms
17/08/01 14:50:32 INFO scheduler.JobScheduler: Added jobs for time 1501599032000 ms
17/08/01 14:50:32 INFO scheduler.JobScheduler: Starting job streaming job 1501599032000 ms.0 from job set of time 1501599032000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:32 INFO scheduler.JobScheduler: Finished job streaming job 1501599032000 ms.0 from job set of time 1501599032000 ms
17/08/01 14:50:32 INFO python.PythonRDD: Removing RDD 121 from persistence list
17/08/01 14:50:32 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599032000 ms (execution: 0.007 s)
17/08/01 14:50:32 INFO storage.BlockManager: Removing RDD 121
17/08/01 14:50:32 INFO rdd.BlockRDD: Removing RDD 120 from persistence list
17/08/01 14:50:32 INFO storage.BlockManager: Removing RDD 120
17/08/01 14:50:32 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[120] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599032000 ms
17/08/01 14:50:32 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599030000 ms
17/08/01 14:50:32 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599030000 ms
17/08/01 14:50:33 INFO scheduler.JobScheduler: Added jobs for time 1501599033000 ms
17/08/01 14:50:33 INFO scheduler.JobScheduler: Starting job streaming job 1501599033000 ms.0 from job set of time 1501599033000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:33 INFO scheduler.JobScheduler: Finished job streaming job 1501599033000 ms.0 from job set of time 1501599033000 ms
17/08/01 14:50:33 INFO python.PythonRDD: Removing RDD 123 from persistence list
17/08/01 14:50:33 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599033000 ms (execution: 0.005 s)
17/08/01 14:50:33 INFO storage.BlockManager: Removing RDD 123
17/08/01 14:50:33 INFO rdd.BlockRDD: Removing RDD 122 from persistence list
17/08/01 14:50:33 INFO storage.BlockManager: Removing RDD 122
17/08/01 14:50:33 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[122] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599033000 ms
17/08/01 14:50:33 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599031000 ms
17/08/01 14:50:33 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599031000 ms
17/08/01 14:50:34 INFO scheduler.JobScheduler: Added jobs for time 1501599034000 ms
17/08/01 14:50:34 INFO scheduler.JobScheduler: Starting job streaming job 1501599034000 ms.0 from job set of time 1501599034000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:34 INFO scheduler.JobScheduler: Finished job streaming job 1501599034000 ms.0 from job set of time 1501599034000 ms
17/08/01 14:50:34 INFO python.PythonRDD: Removing RDD 125 from persistence list
17/08/01 14:50:34 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599034000 ms (execution: 0.006 s)
17/08/01 14:50:34 INFO storage.BlockManager: Removing RDD 125
17/08/01 14:50:34 INFO rdd.BlockRDD: Removing RDD 124 from persistence list
17/08/01 14:50:34 INFO storage.BlockManager: Removing RDD 124
17/08/01 14:50:34 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[124] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599034000 ms
17/08/01 14:50:34 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599032000 ms
17/08/01 14:50:34 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599032000 ms
17/08/01 14:50:34 INFO spark.ExecutorAllocationManager: Request to remove executorIds: 2
17/08/01 14:50:34 INFO cluster.YarnClientSchedulerBackend: Requesting to kill executor(s) 2
17/08/01 14:50:34 INFO cluster.YarnClientSchedulerBackend: Actual list of executor(s) to be killed is 2
17/08/01 14:50:34 INFO spark.ExecutorAllocationManager: Removing executor 2 because it has been idle for 60 seconds (new desired total will be 1)
17/08/01 14:50:35 INFO scheduler.JobScheduler: Added jobs for time 1501599035000 ms
17/08/01 14:50:35 INFO scheduler.JobScheduler: Starting job streaming job 1501599035000 ms.0 from job set of time 1501599035000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:35 INFO scheduler.JobScheduler: Finished job streaming job 1501599035000 ms.0 from job set of time 1501599035000 ms
17/08/01 14:50:35 INFO python.PythonRDD: Removing RDD 127 from persistence list
17/08/01 14:50:35 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599035000 ms (execution: 0.013 s)
17/08/01 14:50:35 INFO storage.BlockManager: Removing RDD 127
17/08/01 14:50:35 INFO rdd.BlockRDD: Removing RDD 126 from persistence list
17/08/01 14:50:35 INFO storage.BlockManager: Removing RDD 126
17/08/01 14:50:35 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[126] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599035000 ms
17/08/01 14:50:35 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599033000 ms
17/08/01 14:50:35 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599033000 ms
17/08/01 14:50:36 INFO scheduler.JobScheduler: Added jobs for time 1501599036000 ms
17/08/01 14:50:36 INFO scheduler.JobScheduler: Starting job streaming job 1501599036000 ms.0 from job set of time 1501599036000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:36 INFO scheduler.JobScheduler: Finished job streaming job 1501599036000 ms.0 from job set of time 1501599036000 ms
17/08/01 14:50:36 INFO python.PythonRDD: Removing RDD 129 from persistence list
17/08/01 14:50:36 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599036000 ms (execution: 0.007 s)
17/08/01 14:50:36 INFO storage.BlockManager: Removing RDD 129
17/08/01 14:50:36 INFO rdd.BlockRDD: Removing RDD 128 from persistence list
17/08/01 14:50:36 INFO storage.BlockManager: Removing RDD 128
17/08/01 14:50:36 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[128] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599036000 ms
17/08/01 14:50:36 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599034000 ms
17/08/01 14:50:36 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599034000 ms
17/08/01 14:50:37 INFO scheduler.JobScheduler: Added jobs for time 1501599037000 ms
17/08/01 14:50:37 INFO scheduler.JobScheduler: Starting job streaming job 1501599037000 ms.0 from job set of time 1501599037000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:37 INFO scheduler.JobScheduler: Finished job streaming job 1501599037000 ms.0 from job set of time 1501599037000 ms
17/08/01 14:50:37 INFO python.PythonRDD: Removing RDD 131 from persistence list
17/08/01 14:50:37 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599037000 ms (execution: 0.006 s)
17/08/01 14:50:37 INFO storage.BlockManager: Removing RDD 131
17/08/01 14:50:37 INFO rdd.BlockRDD: Removing RDD 130 from persistence list
17/08/01 14:50:37 INFO storage.BlockManager: Removing RDD 130
17/08/01 14:50:37 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[130] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599037000 ms
17/08/01 14:50:37 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599035000 ms
17/08/01 14:50:37 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599035000 ms
17/08/01 14:50:38 INFO scheduler.JobScheduler: Added jobs for time 1501599038000 ms
17/08/01 14:50:38 INFO scheduler.JobScheduler: Starting job streaming job 1501599038000 ms.0 from job set of time 1501599038000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:38 INFO scheduler.JobScheduler: Finished job streaming job 1501599038000 ms.0 from job set of time 1501599038000 ms
17/08/01 14:50:38 INFO python.PythonRDD: Removing RDD 133 from persistence list
17/08/01 14:50:38 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599038000 ms (execution: 0.007 s)
17/08/01 14:50:38 INFO storage.BlockManager: Removing RDD 133
17/08/01 14:50:38 INFO rdd.BlockRDD: Removing RDD 132 from persistence list
17/08/01 14:50:38 INFO storage.BlockManager: Removing RDD 132
17/08/01 14:50:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[132] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599038000 ms
17/08/01 14:50:38 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599036000 ms
17/08/01 14:50:38 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599036000 ms
17/08/01 14:50:38 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Disabling executor 2.
17/08/01 14:50:38 INFO scheduler.DAGScheduler: Executor lost: 2 (epoch 1)
17/08/01 14:50:38 INFO storage.BlockManagerMasterEndpoint: Trying to remove executor 2 from BlockManagerMaster.
17/08/01 14:50:38 INFO storage.BlockManagerMasterEndpoint: Removing block manager BlockManagerId(2, ip-10-0-0-127.ec2.internal, 59267, None)
17/08/01 14:50:38 INFO storage.BlockManagerMaster: Removed 2 successfully in removeExecutor
17/08/01 14:50:38 INFO cluster.YarnScheduler: Executor 2 on ip-10-0-0-127.ec2.internal killed by driver.
17/08/01 14:50:38 INFO spark.ExecutorAllocationManager: Existing executor 2 has been removed (new total is 1)
17/08/01 14:50:39 INFO scheduler.JobScheduler: Added jobs for time 1501599039000 ms
17/08/01 14:50:39 INFO scheduler.JobScheduler: Starting job streaming job 1501599039000 ms.0 from job set of time 1501599039000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:39 INFO scheduler.JobScheduler: Finished job streaming job 1501599039000 ms.0 from job set of time 1501599039000 ms
17/08/01 14:50:39 INFO python.PythonRDD: Removing RDD 135 from persistence list
17/08/01 14:50:39 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599039000 ms (execution: 0.006 s)
17/08/01 14:50:39 INFO storage.BlockManager: Removing RDD 135
17/08/01 14:50:39 INFO rdd.BlockRDD: Removing RDD 134 from persistence list
17/08/01 14:50:39 INFO storage.BlockManager: Removing RDD 134
17/08/01 14:50:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[134] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599039000 ms
17/08/01 14:50:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599037000 ms
17/08/01 14:50:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599037000 ms
17/08/01 14:50:40 INFO scheduler.JobScheduler: Added jobs for time 1501599040000 ms
17/08/01 14:50:40 INFO scheduler.JobScheduler: Starting job streaming job 1501599040000 ms.0 from job set of time 1501599040000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:40 INFO scheduler.JobScheduler: Finished job streaming job 1501599040000 ms.0 from job set of time 1501599040000 ms
17/08/01 14:50:40 INFO python.PythonRDD: Removing RDD 137 from persistence list
17/08/01 14:50:40 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599040000 ms (execution: 0.006 s)
17/08/01 14:50:40 INFO storage.BlockManager: Removing RDD 137
17/08/01 14:50:40 INFO rdd.BlockRDD: Removing RDD 136 from persistence list
17/08/01 14:50:40 INFO storage.BlockManager: Removing RDD 136
17/08/01 14:50:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[136] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599040000 ms
17/08/01 14:50:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599038000 ms
17/08/01 14:50:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599038000 ms
17/08/01 14:50:41 INFO scheduler.JobScheduler: Added jobs for time 1501599041000 ms
17/08/01 14:50:41 INFO scheduler.JobScheduler: Starting job streaming job 1501599041000 ms.0 from job set of time 1501599041000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:41 INFO scheduler.JobScheduler: Finished job streaming job 1501599041000 ms.0 from job set of time 1501599041000 ms
17/08/01 14:50:41 INFO python.PythonRDD: Removing RDD 139 from persistence list
17/08/01 14:50:41 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599041000 ms (execution: 0.006 s)
17/08/01 14:50:41 INFO storage.BlockManager: Removing RDD 139
17/08/01 14:50:41 INFO rdd.BlockRDD: Removing RDD 138 from persistence list
17/08/01 14:50:41 INFO storage.BlockManager: Removing RDD 138
17/08/01 14:50:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[138] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599041000 ms
17/08/01 14:50:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599039000 ms
17/08/01 14:50:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599039000 ms
17/08/01 14:50:42 INFO scheduler.JobScheduler: Added jobs for time 1501599042000 ms
17/08/01 14:50:42 INFO scheduler.JobScheduler: Starting job streaming job 1501599042000 ms.0 from job set of time 1501599042000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:42 INFO scheduler.JobScheduler: Finished job streaming job 1501599042000 ms.0 from job set of time 1501599042000 ms
17/08/01 14:50:42 INFO python.PythonRDD: Removing RDD 141 from persistence list
17/08/01 14:50:42 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599042000 ms (execution: 0.014 s)
17/08/01 14:50:42 INFO storage.BlockManager: Removing RDD 141
17/08/01 14:50:42 INFO rdd.BlockRDD: Removing RDD 140 from persistence list
17/08/01 14:50:42 INFO storage.BlockManager: Removing RDD 140
17/08/01 14:50:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[140] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599042000 ms
17/08/01 14:50:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599040000 ms
17/08/01 14:50:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599040000 ms
17/08/01 14:50:42 INFO storage.BlockManagerInfo: Added input-0-1501599042400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1059.6 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599042800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1057.9 MB)
17/08/01 14:50:43 INFO scheduler.JobScheduler: Added jobs for time 1501599043000 ms
17/08/01 14:50:43 INFO scheduler.JobScheduler: Starting job streaming job 1501599043000 ms.0 from job set of time 1501599043000 ms
17/08/01 14:50:43 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Got job 2 (runJob at PythonRDD.scala:441) with 1 output partitions
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Final stage: ResultStage 3 (runJob at PythonRDD.scala:441)
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Missing parents: List()
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Submitting ResultStage 3 (PythonRDD[146] at RDD at PythonRDD.scala:48), which has no missing parents
17/08/01 14:50:43 INFO memory.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/08/01 14:50:43 INFO memory.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.0.0.199:53355 (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:43 INFO spark.SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:996
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (PythonRDD[146] at RDD at PythonRDD.scala:48)
17/08/01 14:50:43 INFO cluster.YarnScheduler: Adding task set 3.0 with 1 tasks
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.3 KB, free: 1057.0 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1056.2 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.4 KB, free: 1055.3 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 878.9 KB, free: 1054.5 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599043800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1052.7 MB)
17/08/01 14:50:44 INFO scheduler.JobScheduler: Added jobs for time 1501599044000 ms
17/08/01 14:50:44 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 2)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1051.9 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1051.0 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1050.2 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.5 KB, free: 1049.3 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599044800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 887.9 KB, free: 1048.4 MB)
17/08/01 14:50:45 INFO scheduler.JobScheduler: Added jobs for time 1501599045000 ms
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1047.6 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1046.7 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1045.9 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.0 KB, free: 1044.1 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599045800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1043.3 MB)
17/08/01 14:50:46 INFO scheduler.JobScheduler: Added jobs for time 1501599046000 ms
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.6 KB, free: 1042.4 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.0 KB, free: 1041.6 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1040.7 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1039.9 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599046800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1039.0 MB)
17/08/01 14:50:47 INFO scheduler.JobScheduler: Added jobs for time 1501599047000 ms
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1038.1 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 878.9 KB, free: 1037.3 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1035.6 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1034.7 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599047800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1033.8 MB)
17/08/01 14:50:48 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.127:44785) with ID 3
17/08/01 14:50:48 INFO spark.ExecutorAllocationManager: New executor 3 has registered (new total is 2)
17/08/01 14:50:48 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 3.0 (TID 71, ip-10-0-0-127.ec2.internal, executor 3, partition 0, RACK_LOCAL, 5800 bytes)
17/08/01 14:50:48 INFO scheduler.JobScheduler: Added jobs for time 1501599048000 ms
17/08/01 14:50:48 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-127.ec2.internal:51258 with 1060.5 MB RAM, BlockManagerId(3, ip-10-0-0-127.ec2.internal, 51258, None)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1033.0 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on ip-10-0-0-127.ec2.internal:51258 (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1032.1 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.5 KB, free: 1031.3 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1030.4 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599048800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1029.5 MB)
17/08/01 14:50:49 INFO scheduler.JobScheduler: Added jobs for time 1501599049000 ms
17/08/01 14:50:49 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 3.0 (TID 71) in 1023 ms on ip-10-0-0-127.ec2.internal (executor 3) (1/1)
17/08/01 14:50:49 INFO cluster.YarnScheduler: Removed TaskSet 3.0, whose tasks have all completed, from pool
17/08/01 14:50:49 INFO scheduler.DAGScheduler: ResultStage 3 (runJob at PythonRDD.scala:441) finished in 5.985 s
17/08/01 14:50:49 INFO scheduler.DAGScheduler: Job 2 finished: runJob at PythonRDD.scala:441, took 6.003911 s
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.0 KB, free: 1027.8 MB)
17/08/01 14:50:49 INFO spark.ContextCleaner: Cleaned shuffle 0
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 10.0.0.199:53355 in memory (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1988.0 B, free: 1027.8 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on 10.0.0.199:53355 in memory (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1956.0 B, free: 1027.8 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on 10.0.0.199:53355 in memory (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on ip-10-0-0-127.ec2.internal:51258 in memory (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1027.0 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1026.1 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1025.3 MB)
17/08/01 14:50:49 INFO execution.SparkSqlParser: Parsing command: spotdb.proxy
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599049800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1024.4 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Added jobs for time 1501599050000 ms
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1023.5 MB)
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1022.7 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599043000 ms.0 from job set of time 1501599043000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 143 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 7.504 s for time 1501599043000 ms (execution: 7.487 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599044000 ms.0 from job set of time 1501599044000 ms
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 142 from persistence list
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599043000 ms.0
org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 69, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[142] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599043000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599041000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599041000 ms
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 142
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 143
Traceback (most recent call last):
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 148, in <module>
main()
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 77, in main
bluecoat_parse(args.zk,args.topic,args.db,args.db_table,args.num_of_workers,args.batch_size)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 144, in bluecoat_parse
ssc.awaitTermination()
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/context.py", line 206, in awaitTermination
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o50.awaitTermination.
: org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 69, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599044000 ms.0 from job set of time 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 6.542 s for time 1501599044000 ms (execution: 0.037 s)
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 145 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599045000 ms.0 from job set of time 1501599045000 ms
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599044000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 145
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 144 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 144
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[144] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599042000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599042000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599042400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1023.5 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599045000 ms.0 from job set of time 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 5.547 s for time 1501599045000 ms (execution: 0.003 s)
17/08/01 14:50:50 INFO streaming.StreamingContext: Invoking stop(stopGracefully=false) from shutdown hook
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599046000 ms.0 from job set of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599046000 ms.0 from job set of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 4.559 s for time 1501599046000 ms (execution: 0.004 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599047000 ms.0 from job set of time 1501599047000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 148 from persistence list
17/08/01 14:50:50 INFO scheduler.ReceiverTracker: Sent stop signal to all 1 receivers
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 148
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599047000 ms.0 from job set of time 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 3.562 s for time 1501599047000 ms (execution: 0.002 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599048000 ms.0 from job set of time 1501599048000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599048000 ms.0 from job set of time 1501599048000 ms
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 147 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 2.571 s for time 1501599048000 ms (execution: 0.001 s)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 147
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[147] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599049000 ms.0 from job set of time 1501599049000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599049000 ms.0 from job set of time 1501599049000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 1.572 s for time 1501599049000 ms (execution: 0.001 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599050000 ms.0 from job set of time 1501599050000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599050000 ms.0 from job set of time 1501599050000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 0.574 s for time 1501599050000 ms (execution: 0.001 s)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599045000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599046000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599047000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599048000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599049000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599042800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1025.3 MB)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599050000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.3 KB, free: 1026.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1027.0 MB)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599043000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599043000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 150 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 878.9 KB, free: 1027.8 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.4 KB, free: 1028.7 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 150
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 149 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 149
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[149] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599044000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 152 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 152
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1030.4 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1031.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1032.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1033.0 MB)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 151 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 151
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[151] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599045000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 154 from persistence list
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 153 from persistence list
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[153] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599048000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.5 KB, free: 1033.8 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 153
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 154
17/08/01 14:50:50 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1034.7 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599046000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 156 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 887.9 KB, free: 1035.6 MB)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 155 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.4 KB, free: 1034.7 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 155
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 156
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[155] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599049000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1035.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1036.4 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599047000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 158 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 158
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 157 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 157
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[157] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599050000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.6 KB, free: 1037.3 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599048000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599048000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1038.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.0 KB, free: 1039.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.0 KB, free: 1040.7 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1041.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1042.4 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1043.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 878.9 KB, free: 1044.2 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1045.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1046.7 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1047.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1048.5 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.5 KB, free: 1049.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1050.2 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1051.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1051.9 MB)
17/08/01 14:50:51 ERROR scheduler.JobScheduler: Error generating jobs for time 1501599051000 ms
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 2.0 (TID 70) in 77942 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/1)
17/08/01 14:50:51 INFO cluster.YarnScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool
17/08/01 14:50:51 INFO scheduler.DAGScheduler: ResultStage 2 (start at NativeMethodAccessorImpl.java:0) finished in 77.943 s
17/08/01 14:50:51 INFO scheduler.ReceiverTracker: All of the receivers have deregistered successfully
17/08/01 14:50:51 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
17/08/01 14:50:51 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
17/08/01 14:50:51 INFO util.RecurringTimer: Stopped timer for JobGenerator after time 1501599051000
17/08/01 14:50:51 INFO scheduler.JobGenerator: Stopped JobGenerator
17/08/01 14:50:51 INFO scheduler.JobScheduler: Stopped JobScheduler
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@625d3c2d{/streaming,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@39edaf54{/streaming/batch,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@c773c0d{/static/streaming,null,UNAVAILABLE}
17/08/01 14:50:51 INFO streaming.StreamingContext: StreamingContext stopped successfully
17/08/01 14:50:51 INFO spark.SparkContext: Invoking stop() from shutdown hook
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 INFO server.ServerConnector: Stopped ServerConnector@37681deb{HTTP/1.1}{0.0.0.0:4040}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2aa3e9a6{/stages/stage/kill,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ee30613{/jobs/job/kill,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@15ec1723{/api,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14a7d27d{/,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@e4408f4{/static,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@8a515f3{/executors/threadDump/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@f241039{/executors/threadDump,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@79b9b526{/executors/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5fa63fa{/executors,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@77729e5a{/environment/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@242eb7e{/environment,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@49ddd2ed{/storage/rdd/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@699743b3{/storage/rdd,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@736cc380{/storage/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19cf381c{/storage,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4f734944{/stages/pool/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@b7d4da0{/stages/pool,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ab71bb9{/stages/stage/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5d37ce06{/stages/stage,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@27f55b38{/stages/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1159f15e{/stages,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ff970c2{/jobs/job/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@615b86f5{/jobs/job,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19bb7285{/jobs/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3b9882ec{/jobs,null,UNAVAILABLE}
17/08/01 14:50:51 INFO ui.SparkUI: Stopped Spark web UI at http://10.0.0.199:4040
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors
17/08/01 14:50:51 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
17/08/01 14:50:51 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
services=List(),
started=false)
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Stopped
17/08/01 14:50:51 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/08/01 14:50:51 INFO memory.MemoryStore: MemoryStore cleared
17/08/01 14:50:51 INFO storage.BlockManager: BlockManager stopped
17/08/01 14:50:51 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
17/08/01 14:50:51 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/08/01 14:50:51 INFO spark.SparkContext: Successfully stopped SparkContext
17/08/01 14:50:51 INFO util.ShutdownHookManager: Shutdown hook called
17/08/01 14:50:51 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25
17/08/01 14:50:51 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25/pyspark-c2380580-feb5-47ef-bc69-7b2c9ba38770
Closing worker...
From: Deon Griessel [mailto:dgriessel@searchtechnologies.com]
Sent: Monday, July 31, 2017 4:53 PM
To: user@spot.apache.org
Subject: Spot Ingest Proxy Error
I am trying to run the spark-ingest for the sample proxy log: Bluecoat_ProxySG_Sample.log
I run bash ./start_ingest_standalone.sh proxy 1
Then add the demo log to my collector_path. The worker dies, seems that it cannot find the hive metastore, probably a spark/hive configuration issue. Also, notice that the ip=unknown-ip-addr.
Would appreciate any help on this.
This is what I’m seeing in the worker screen output:
2017-07-31 18:54:15,855 - SPOT.INGEST.WORKER - INFO - Initializing kafka instance
2017-07-31 18:54:15,855 - SPOT.INGEST.WRK.PROXY - INFO - Creating Spark Job for topic: SPOT-INGEST-proxy-18_54_13
2017-07-31 18:54:15,856 - SPOT.INGEST.WRK.PROXY - INFO - SPOT.Utils: Executing: spark2-submit --master yarn --driver-memory 2325m --num-executors 1 --conf spark.executor.memory=2325m --conf spark.executor.cores=1 --jars /home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar /home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py -zk ip-10-0-0-199.ec2.internal:2181 -t SPOT-INGEST-proxy-18_54_13 -db spotdb -dt proxy -w 1 -bs 1
17/07/31 18:54:17 INFO spark.SparkContext: Running Spark version 2.1.0.cloudera1
17/07/31 18:54:17 WARN spark.SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0
17/07/31 18:54:17 INFO spark.SecurityManager: Changing view acls to: spot-user
17/07/31 18:54:17 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/07/31 18:54:17 INFO spark.SecurityManager: Changing view acls groups to:
17/07/31 18:54:17 INFO spark.SecurityManager: Changing modify acls groups to:
17/07/31 18:54:17 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/07/31 18:54:18 INFO util.Utils: Successfully started service 'sparkDriver' on port 54401.
17/07/31 18:54:18 INFO spark.SparkEnv: Registering MapOutputTracker
17/07/31 18:54:18 INFO spark.SparkEnv: Registering BlockManagerMaster
17/07/31 18:54:18 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
17/07/31 18:54:18 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
17/07/31 18:54:18 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-1db8ad37-9fcc-4799-b9e0-a206b6041d04
17/07/31 18:54:18 INFO memory.MemoryStore: MemoryStore started with capacity 1060.5 MB
17/07/31 18:54:18 INFO spark.SparkEnv: Registering OutputCommitCoordinator
17/07/31 18:54:18 INFO util.log: Logging initialized @2379ms
17/07/31 18:54:18 INFO server.Server: jetty-9.2.z-SNAPSHOT
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1a3b1e79{/jobs,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1a3b1e79%7b/jobs,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1f4da763{/jobs/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1f4da763%7b/jobs/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@232864a3{/jobs/job,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@232864a3%7b/jobs/job,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@30e71b5d{/jobs/job/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@30e71b5d%7b/jobs/job/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14b58fc0{/stages,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14b58fc0%7b/stages,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1bf090df{/stages/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1bf090df%7b/stages/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4eb72ecd{/stages/stage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4eb72ecd%7b/stages/stage,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5c61bd1a{/stages/stage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5c61bd1a%7b/stages/stage/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14c62558{/stages/pool,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14c62558%7b/stages/pool,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5cbdbf0f{/stages/pool/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5cbdbf0f%7b/stages/pool/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2d4aa15a{/storage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@2d4aa15a%7b/storage,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ea732f0{/storage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ea732f0%7b/storage/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@13016b86{/storage/rdd,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@13016b86%7b/storage/rdd,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@423a3429{/storage/rdd/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@423a3429%7b/storage/rdd/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7b3691b6{/environment,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7b3691b6%7b/environment,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3b9882ec{/environment/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3b9882ec%7b/environment/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19bb7285{/executors,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19bb7285%7b/executors,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@615b86f5{/executors/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@615b86f5%7b/executors/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ff970c2{/executors/threadDump,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3ff970c2%7b/executors/threadDump,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1159f15e{/executors/threadDump/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1159f15e%7b/executors/threadDump/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27f55b38{/static,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27f55b38%7b/static,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5d37ce06{/,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5d37ce06%7b/,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ab71bb9{/api,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ab71bb9%7b/api,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b7d4da0{/jobs/job/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@b7d4da0%7b/jobs/job/kill,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f734944{/stages/stage/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4f734944%7b/stages/stage/kill,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO server.ServerConnector: Started ServerConnector@188e6c7f{HTTP/1.1}{0.0.0.0:4040}<mailto:ServerConnector@188e6c7f%7bHTTP/1.1%7d%7b0.0.0.0:4040%7d>
17/07/31 18:54:18 INFO server.Server: Started @2532ms
17/07/31 18:54:18 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
17/07/31 18:54:18 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.0.0.199:4040
17/07/31 18:54:18 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/07/31 18:54:19 INFO client.RMProxy: Connecting to ResourceManager at ip-10-0-0-199.ec2.internal/10.0.0.199:8032
17/07/31 18:54:19 INFO yarn.Client: Requesting a new application from cluster with 3 NodeManagers
17/07/31 18:54:19 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (2725 MB per container)
17/07/31 18:54:19 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/07/31 18:54:19 INFO yarn.Client: Setting up container launch context for our AM
17/07/31 18:54:19 INFO yarn.Client: Setting up the launch environment for our AM container
17/07/31 18:54:19 INFO yarn.Client: Preparing resources for our AM container
17/07/31 18:54:21 INFO yarn.Client: Uploading resource file:/home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501523941584_0005/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar
17/07/31 18:54:21 INFO yarn.Client: Uploading resource file:/tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40/__spark_conf__5838957838871043110.zip -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501523941584_0005/__spark_conf__.zip
17/07/31 18:54:21 INFO spark.SecurityManager: Changing view acls to: spot-user
17/07/31 18:54:21 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/07/31 18:54:21 INFO spark.SecurityManager: Changing view acls groups to:
17/07/31 18:54:21 INFO spark.SecurityManager: Changing modify acls groups to:
17/07/31 18:54:21 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/07/31 18:54:21 INFO yarn.Client: Submitting application application_1501523941584_0005 to ResourceManager
17/07/31 18:54:21 INFO impl.YarnClientImpl: Submitted application application_1501523941584_0005
17/07/31 18:54:21 INFO cluster.SchedulerExtensionServices: Starting Yarn extension services with app application_1501523941584_0005 and attemptId None
17/07/31 18:54:22 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:22 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.users.spot-user
start time: 1501527261635
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005/
user: spot-user
17/07/31 18:54:23 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:24 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:25 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/07/31 18:54:25 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> ip-10-0-0-199.ec2.internal, PROXY_URI_BASES -> http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005), /proxy/application_1501523941584_0005
17/07/31 18:54:25 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/07/31 18:54:25 INFO yarn.Client: Application report for application_1501523941584_0005 (state: RUNNING)
17/07/31 18:54:25 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 10.0.0.127
ApplicationMaster RPC port: 0
queue: root.users.spot-user
start time: 1501527261635
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005/
user: spot-user
17/07/31 18:54:25 INFO cluster.YarnClientSchedulerBackend: Application application_1501523941584_0005 has started running.
17/07/31 18:54:25 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 59442.
17/07/31 18:54:25 INFO netty.NettyBlockTransferService: Server created on 10.0.0.199:59442
17/07/31 18:54:25 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/07/31 18:54:25 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManagerMasterEndpoint: Registering block manager 10.0.0.199:59442 with 1060.5 MB RAM, BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManager: external shuffle service port = 7337
17/07/31 18:54:25 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3e93262b{/metrics/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3e93262b%7b/metrics/json,null,AVAILABLE%7d>
17/07/31 18:54:25 INFO scheduler.EventLoggingListener: Logging events to hdfs://ip-10-0-0-199.ec2.internal:8020/user/spark/spark2ApplicationHistory/application_1501523941584_0005
17/07/31 18:54:25 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/07/31 18:54:28 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.229:51685) with ID 1
17/07/31 18:54:28 INFO spark.ExecutorAllocationManager: New executor 1 has registered (new total is 1)
17/07/31 18:54:28 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-229.ec2.internal:35659 with 1060.5 MB RAM, BlockManagerId(1, ip-10-0-0-229.ec2.internal, 35659, None)
17/07/31 18:54:28 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/07/31 18:54:29 INFO internal.SharedState: Warehouse path is 'file:/home/spot-user/spot-ingest/spark-warehouse'.
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7a9c6710{/SQL,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7a9c6710%7b/SQL,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@141768e4{/SQL/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@141768e4%7b/SQL/json,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1e367d45{/SQL/execution,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1e367d45%7b/SQL/execution,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5554f0a0{/SQL/execution/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5554f0a0%7b/SQL/execution/json,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@144776d3{/static/sql,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@144776d3%7b/static/sql,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO hive.HiveUtils: Initializing HiveMetastoreConnection version 1.1.0 using Spark classes.
17/07/31 18:54:30 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
17/07/31 18:54:30 INFO metastore.ObjectStore: ObjectStore, initialize called
17/07/31 18:54:30 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
17/07/31 18:54:30 INFO DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
17/07/31 18:54:30 WARN bonecp.BoneCPConfig: Max Connections < 1. Setting to 20
17/07/31 18:54:31 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
17/07/31 18:54:32 WARN bonecp.BoneCPConfig: Max Connections < 1. Setting to 20
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0<ma...@0>" since the connection used is closing
17/07/31 18:54:32 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
17/07/31 18:54:32 INFO metastore.ObjectStore: Initialized ObjectStore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: Added admin role in metastore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: Added public role in metastore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_all_functions from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_all_functions
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_all_functions
17/07/31 18:54:33 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_all_functions start=1501527273291 end=1501527273356 duration=65 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=0 error=false>
17/07/31 18:54:33 INFO session.SessionState: Created local directory: /tmp/aaecce13-256b-4b91-b661-9046c9c418c6_resources
17/07/31 18:54:33 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6
17/07/31 18:54:33 INFO session.SessionState: Created local directory: /tmp/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6
17/07/31 18:54:33 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6/_tmp_space.db
17/07/31 18:54:33 INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
17/07/31 18:54:33 INFO client.HiveClientImpl: Warehouse location for Hive client (version 1.1.0) is file:/home/spot-user/spot-ingest/spark-warehouse
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_database: default
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: default
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527273406 end=1501527273408 duration=2 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=0 error=false>
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_database: global_temp
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: global_temp
17/07/31 18:54:33 WARN metastore.ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527273409 end=1501527273412 duration=3 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=-1 error=true>
17/07/31 18:54:33 WARN streaming.StreamingContext: Dynamic Allocation is enabled for this application. Enabling Dynamic allocation for Spark Streaming applications can cause data loss if Write Ahead Log is not enabled for non-replayable sources like Flume. See the programming guide for details on how to enable the Write Ahead Log.
17/07/31 18:54:33 INFO spark.SparkContext: Starting job: start at NativeMethodAccessorImpl.java:0
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Registering RDD 1 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Got job 0 (start at NativeMethodAccessorImpl.java:0) with 20 output partitions
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/07/31 18:54:33 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 3.1 KB, free 1060.5 MB)
17/07/31 18:54:33 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 1988.0 B, free 1060.5 MB)
17/07/31 18:54:33 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.0.0.199:59442 (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:54:33 INFO spark.SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Submitting 50 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO cluster.YarnScheduler: Adding task set 0.0 with 50 tasks
17/07/31 18:54:33 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, ip-10-0-0-229.ec2.internal, executor 1, partition 1, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 669 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, ip-10-0-0-229.ec2.internal, executor 1, partition 2, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 77 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3, ip-10-0-0-229.ec2.internal, executor 1, partition 3, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 72 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 0.0 (TID 4, ip-10-0-0-229.ec2.internal, executor 1, partition 4, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 69 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 0.0 (TID 5, ip-10-0-0-229.ec2.internal, executor 1, partition 5, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 59 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 0.0 (TID 6, ip-10-0-0-229.ec2.internal, executor 1, partition 6, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 0.0 (TID 7, ip-10-0-0-229.ec2.internal, executor 1, partition 7, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 0.0 (TID 8, ip-10-0-0-229.ec2.internal, executor 1, partition 8, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 0.0 (TID 7) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/50)
17/07/31 18:54:34 INFO spark.ExecutorAllocationManager: Requesting 1 new executor because tasks are backlogged (new desired total will be 2)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 0.0 (TID 9, ip-10-0-0-229.ec2.internal, executor 1, partition 9, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 0.0 (TID 8) in 58 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 0.0 (TID 10, ip-10-0-0-229.ec2.internal, executor 1, partition 10, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 0.0 (TID 9) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 0.0 (TID 11, ip-10-0-0-229.ec2.internal, executor 1, partition 11, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 0.0 (TID 10) in 52 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 0.0 (TID 12, ip-10-0-0-229.ec2.internal, executor 1, partition 12, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 0.0 (TID 11) in 58 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 0.0 (TID 13, ip-10-0-0-229.ec2.internal, executor 1, partition 13, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 0.0 (TID 12) in 51 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 0.0 (TID 14, ip-10-0-0-229.ec2.internal, executor 1, partition 14, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 0.0 (TID 13) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 0.0 (TID 15, ip-10-0-0-229.ec2.internal, executor 1, partition 15, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 0.0 (TID 14) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 0.0 (TID 16, ip-10-0-0-229.ec2.internal, executor 1, partition 16, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 0.0 (TID 15) in 63 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 0.0 (TID 17, ip-10-0-0-229.ec2.internal, executor 1, partition 17, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 0.0 (TID 16) in 50 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 0.0 (TID 18, ip-10-0-0-229.ec2.internal, executor 1, partition 18, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 0.0 (TID 17) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 0.0 (TID 19, ip-10-0-0-229.ec2.internal, executor 1, partition 19, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 0.0 (TID 18) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 0.0 (TID 20, ip-10-0-0-229.ec2.internal, executor 1, partition 20, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 0.0 (TID 19) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 0.0 (TID 21, ip-10-0-0-229.ec2.internal, executor 1, partition 21, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 0.0 (TID 20) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (21/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 0.0 (TID 22, ip-10-0-0-229.ec2.internal, executor 1, partition 22, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 0.0 (TID 21) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (22/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 0.0 (TID 23, ip-10-0-0-229.ec2.internal, executor 1, partition 23, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 0.0 (TID 22) in 47 ms on ip-10-0-0-229.ec2.internal (executor 1) (23/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 0.0 (TID 24, ip-10-0-0-229.ec2.internal, executor 1, partition 24, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 0.0 (TID 23) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (24/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 0.0 (TID 25, ip-10-0-0-229.ec2.internal, executor 1, partition 25, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 0.0 (TID 24) in 46 ms on ip-10-0-0-229.ec2.internal (executor 1) (25/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 0.0 (TID 26, ip-10-0-0-229.ec2.internal, executor 1, partition 26, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 0.0 (TID 25) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (26/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 0.0 (TID 27, ip-10-0-0-229.ec2.internal, executor 1, partition 27, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 0.0 (TID 26) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (27/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 0.0 (TID 28, ip-10-0-0-229.ec2.internal, executor 1, partition 28, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 0.0 (TID 27) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (28/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 0.0 (TID 29, ip-10-0-0-229.ec2.internal, executor 1, partition 29, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 0.0 (TID 28) in 39 ms on ip-10-0-0-229.ec2.internal (executor 1) (29/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 0.0 (TID 30, ip-10-0-0-229.ec2.internal, executor 1, partition 30, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 0.0 (TID 29) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (30/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 0.0 (TID 31, ip-10-0-0-229.ec2.internal, executor 1, partition 31, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 0.0 (TID 30) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (31/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 0.0 (TID 32, ip-10-0-0-229.ec2.internal, executor 1, partition 32, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 0.0 (TID 31) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (32/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 0.0 (TID 33, ip-10-0-0-229.ec2.internal, executor 1, partition 33, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 0.0 (TID 32) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (33/50)
17/07/31 18:54:35 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 4)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 0.0 (TID 34, ip-10-0-0-229.ec2.internal, executor 1, partition 34, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 0.0 (TID 33) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (34/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 0.0 (TID 35, ip-10-0-0-229.ec2.internal, executor 1, partition 35, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 0.0 (TID 34) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (35/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 0.0 (TID 36, ip-10-0-0-229.ec2.internal, executor 1, partition 36, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 0.0 (TID 35) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (36/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 0.0 (TID 37, ip-10-0-0-229.ec2.internal, executor 1, partition 37, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 0.0 (TID 36) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (37/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 0.0 (TID 38, ip-10-0-0-229.ec2.internal, executor 1, partition 38, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 0.0 (TID 37) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (38/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 0.0 (TID 39, ip-10-0-0-229.ec2.internal, executor 1, partition 39, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 0.0 (TID 38) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (39/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 0.0 (TID 40, ip-10-0-0-229.ec2.internal, executor 1, partition 40, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 0.0 (TID 39) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (40/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 0.0 (TID 41, ip-10-0-0-229.ec2.internal, executor 1, partition 41, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 0.0 (TID 40) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (41/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 0.0 (TID 42, ip-10-0-0-229.ec2.internal, executor 1, partition 42, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 0.0 (TID 41) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (42/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 0.0 (TID 43, ip-10-0-0-229.ec2.internal, executor 1, partition 43, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 0.0 (TID 42) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (43/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 0.0 (TID 44, ip-10-0-0-229.ec2.internal, executor 1, partition 44, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 0.0 (TID 43) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (44/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 0.0 (TID 45, ip-10-0-0-229.ec2.internal, executor 1, partition 45, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 0.0 (TID 44) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (45/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 0.0 (TID 46, ip-10-0-0-229.ec2.internal, executor 1, partition 46, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 0.0 (TID 45) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (46/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 0.0 (TID 47, ip-10-0-0-229.ec2.internal, executor 1, partition 47, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 0.0 (TID 46) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (47/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 0.0 (TID 48, ip-10-0-0-229.ec2.internal, executor 1, partition 48, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 0.0 (TID 47) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (48/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 0.0 (TID 49, ip-10-0-0-229.ec2.internal, executor 1, partition 49, PROCESS_LOCAL, 6676 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 0.0 (TID 48) in 29 ms on ip-10-0-0-229.ec2.internal (executor 1) (49/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 0.0 (TID 49) in 27 ms on ip-10-0-0-229.ec2.internal (executor 1) (50/50)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (start at NativeMethodAccessorImpl.java:0) finished in 2.499 s
17/07/31 18:54:36 INFO cluster.YarnScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/07/31 18:54:36 INFO scheduler.DAGScheduler: looking for newly runnable stages
17/07/31 18:54:36 INFO scheduler.DAGScheduler: running: Set()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 1)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: failed: Set()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.2 KB, free 1060.5 MB)
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 1956.0 B, free 1060.5 MB)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.0.0.199:59442 (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting 20 missing tasks from ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Adding task set 1.0 with 20 tasks
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 50, ip-10-0-0-229.ec2.internal, executor 1, partition 0, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.0.0.229:51685
17/07/31 18:54:36 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 283 bytes
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.0 (TID 51, ip-10-0-0-229.ec2.internal, executor 1, partition 1, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 50) in 91 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 1.0 (TID 52, ip-10-0-0-229.ec2.internal, executor 1, partition 2, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 1.0 (TID 51) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 1.0 (TID 53, ip-10-0-0-229.ec2.internal, executor 1, partition 3, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 1.0 (TID 52) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 1.0 (TID 54, ip-10-0-0-229.ec2.internal, executor 1, partition 4, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 1.0 (TID 53) in 24 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 1.0 (TID 55, ip-10-0-0-229.ec2.internal, executor 1, partition 5, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 1.0 (TID 54) in 24 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 1.0 (TID 56, ip-10-0-0-229.ec2.internal, executor 1, partition 6, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 1.0 (TID 55) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 1.0 (TID 57, ip-10-0-0-229.ec2.internal, executor 1, partition 7, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 1.0 (TID 56) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 1.0 (TID 58, ip-10-0-0-229.ec2.internal, executor 1, partition 8, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 1.0 (TID 57) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 1.0 (TID 59, ip-10-0-0-229.ec2.internal, executor 1, partition 9, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 1.0 (TID 58) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 1.0 (TID 60, ip-10-0-0-229.ec2.internal, executor 1, partition 10, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 1.0 (TID 59) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 1.0 (TID 61, ip-10-0-0-229.ec2.internal, executor 1, partition 11, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 1.0 (TID 60) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 1.0 (TID 62, ip-10-0-0-229.ec2.internal, executor 1, partition 12, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 1.0 (TID 61) in 16 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 1.0 (TID 63, ip-10-0-0-229.ec2.internal, executor 1, partition 13, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 1.0 (TID 62) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 1.0 (TID 64, ip-10-0-0-229.ec2.internal, executor 1, partition 14, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 1.0 (TID 63) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 1.0 (TID 65, ip-10-0-0-229.ec2.internal, executor 1, partition 15, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 1.0 (TID 64) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 1.0 (TID 66, ip-10-0-0-229.ec2.internal, executor 1, partition 16, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 1.0 (TID 65) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 1.0 (TID 67, ip-10-0-0-229.ec2.internal, executor 1, partition 17, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 1.0 (TID 66) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 1.0 (TID 68, ip-10-0-0-229.ec2.internal, executor 1, partition 18, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 1.0 (TID 67) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 1.0 (TID 69, ip-10-0-0-229.ec2.internal, executor 1, partition 19, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 1.0 (TID 68) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 1.0 (TID 69) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/20)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool
17/07/31 18:54:36 INFO scheduler.DAGScheduler: ResultStage 1 (start at NativeMethodAccessorImpl.java:0) finished in 0.416 s
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Job 0 finished: start at NativeMethodAccessorImpl.java:0, took 3.184418 s
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: Starting 1 receivers
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: ReceiverTracker started
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Initialized and validated org.apache.spark.streaming.kafka.KafkaInputDStream@7b67308f<ma...@7b67308f>
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Initialized and validated org.apache.spark.streaming.api.python.PythonTransformedDStream@33a53b94<ma...@33a53b94>
17/07/31 18:54:36 INFO dstream.ForEachDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO dstream.ForEachDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO dstream.ForEachDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO dstream.ForEachDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO dstream.ForEachDStream: Initialized and validated org.apache.spark.streaming.dstream.ForEachDStream@24ea8c09<ma...@24ea8c09>
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Got job 1 (start at NativeMethodAccessorImpl.java:0) with 1 output partitions
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Final stage: ResultStage 2 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620), which has no missing parents
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: Receiver 0 started
17/07/31 18:54:36 INFO util.RecurringTimer: Started timer for JobGenerator at time 1501527277000
17/07/31 18:54:36 INFO scheduler.JobGenerator: Started JobGenerator at 1501527277000 ms
17/07/31 18:54:36 INFO scheduler.JobScheduler: Started JobScheduler
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19fd4723{/streaming,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19fd4723%7b/streaming,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@43b0e42{/streaming/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@43b0e42%7b/streaming/json,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@22116fc3{/streaming/batch,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@22116fc3%7b/streaming/batch,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27399762{/streaming/batch/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27399762%7b/streaming/batch/json,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@70d301d6{/static/streaming,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@70d301d6%7b/static/streaming,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO streaming.StreamingContext: StreamingContext started
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 74.4 KB, free 1060.4 MB)
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 27.3 KB, free 1060.4 MB)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.0.0.199:59442 (size: 27.3 KB, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Adding task set 2.0 with 1 tasks
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 2.0 (TID 70, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 7067 bytes)
17/07/31 18:54:37 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 27.3 KB, free: 1060.5 MB)
17/07/31 18:54:37 INFO scheduler.JobScheduler: Added jobs for time 1501527277000 ms
17/07/31 18:54:37 INFO scheduler.JobScheduler: Starting job streaming job 1501527277000 ms.0 from job set of time 1501527277000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:37 INFO scheduler.JobScheduler: Finished job streaming job 1501527277000 ms.0 from job set of time 1501527277000 ms
17/07/31 18:54:37 INFO scheduler.JobScheduler: Total delay: 0.071 s for time 1501527277000 ms (execution: 0.014 s)
17/07/31 18:54:37 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/07/31 18:54:37 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/07/31 18:54:37 INFO scheduler.ReceiverTracker: Registered receiver for stream 0 from 10.0.0.229:51685
17/07/31 18:54:38 INFO scheduler.JobScheduler: Added jobs for time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.JobScheduler: Starting job streaming job 1501527278000 ms.0 from job set of time 1501527278000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:38 INFO scheduler.JobScheduler: Finished job streaming job 1501527278000 ms.0 from job set of time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.JobScheduler: Total delay: 0.045 s for time 1501527278000 ms (execution: 0.010 s)
17/07/31 18:54:38 INFO python.PythonRDD: Removing RDD 5 from persistence list
17/07/31 18:54:38 INFO rdd.BlockRDD: Removing RDD 4 from persistence list
17/07/31 18:54:38 INFO storage.BlockManager: Removing RDD 5
17/07/31 18:54:38 INFO storage.BlockManager: Removing RDD 4
17/07/31 18:54:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[4] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/07/31 18:54:38 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/07/31 18:54:38 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.151:57862) with ID 2
17/07/31 18:54:38 INFO spark.ExecutorAllocationManager: New executor 2 has registered (new total is 2)
17/07/31 18:54:38 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-151.ec2.internal:54365 with 1060.5 MB RAM, BlockManagerId(2, ip-10-0-0-151.ec2.internal, 54365, None)
17/07/31 18:54:39 INFO scheduler.JobScheduler: Added jobs for time 1501527279000 ms
17/07/31 18:54:39 INFO scheduler.JobScheduler: Starting job streaming job 1501527279000 ms.0 from job set of time 1501527279000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:39 INFO scheduler.JobScheduler: Finished job streaming job 1501527279000 ms.0 from job set of time 1501527279000 ms
17/07/31 18:54:39 INFO python.PythonRDD: Removing RDD 7 from persistence list
17/07/31 18:54:39 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527279000 ms (execution: 0.006 s)
17/07/31 18:54:39 INFO storage.BlockManager: Removing RDD 7
17/07/31 18:54:39 INFO rdd.BlockRDD: Removing RDD 6 from persistence list
17/07/31 18:54:39 INFO storage.BlockManager: Removing RDD 6
17/07/31 18:54:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[6] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527279000 ms
17/07/31 18:54:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527277000 ms
17/07/31 18:54:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527277000 ms
17/07/31 18:54:40 INFO scheduler.JobScheduler: Added jobs for time 1501527280000 ms
17/07/31 18:54:40 INFO scheduler.JobScheduler: Starting job streaming job 1501527280000 ms.0 from job set of time 1501527280000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:40 INFO scheduler.JobScheduler: Finished job streaming job 1501527280000 ms.0 from job set of time 1501527280000 ms
17/07/31 18:54:40 INFO python.PythonRDD: Removing RDD 9 from persistence list
17/07/31 18:54:40 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501527280000 ms (execution: 0.009 s)
17/07/31 18:54:40 INFO storage.BlockManager: Removing RDD 9
17/07/31 18:54:40 INFO rdd.BlockRDD: Removing RDD 8 from persistence list
17/07/31 18:54:40 INFO storage.BlockManager: Removing RDD 8
17/07/31 18:54:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[8] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527280000 ms
17/07/31 18:54:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527278000 ms
17/07/31 18:54:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527278000 ms
17/07/31 18:54:41 INFO scheduler.JobScheduler: Added jobs for time 1501527281000 ms
17/07/31 18:54:41 INFO scheduler.JobScheduler: Starting job streaming job 1501527281000 ms.0 from job set of time 1501527281000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:41 INFO scheduler.JobScheduler: Finished job streaming job 1501527281000 ms.0 from job set of time 1501527281000 ms
17/07/31 18:54:41 INFO python.PythonRDD: Removing RDD 11 from persistence list
17/07/31 18:54:41 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527281000 ms (execution: 0.006 s)
17/07/31 18:54:41 INFO storage.BlockManager: Removing RDD 11
17/07/31 18:54:41 INFO rdd.BlockRDD: Removing RDD 10 from persistence list
17/07/31 18:54:41 INFO storage.BlockManager: Removing RDD 10
17/07/31 18:54:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[10] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527281000 ms
17/07/31 18:54:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527279000 ms
17/07/31 18:54:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527279000 ms
17/07/31 18:54:42 INFO scheduler.JobScheduler: Added jobs for time 1501527282000 ms
17/07/31 18:54:42 INFO scheduler.JobScheduler: Starting job streaming job 1501527282000 ms.0 from job set of time 1501527282000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:42 INFO scheduler.JobScheduler: Finished job streaming job 1501527282000 ms.0 from job set of time 1501527282000 ms
17/07/31 18:54:42 INFO python.PythonRDD: Removing RDD 13 from persistence list
17/07/31 18:54:42 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527282000 ms (execution: 0.006 s)
17/07/31 18:54:42 INFO storage.BlockManager: Removing RDD 13
17/07/31 18:54:42 INFO rdd.BlockRDD: Removing RDD 12 from persistence list
17/07/31 18:54:42 INFO storage.BlockManager: Removing RDD 12
17/07/31 18:54:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[12] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527282000 ms
17/07/31 18:54:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527280000 ms
17/07/31 18:54:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527280000 ms
17/07/31 18:54:43 INFO scheduler.JobScheduler: Added jobs for time 1501527283000 ms
17/07/31 18:54:43 INFO scheduler.JobScheduler: Starting job streaming job 1501527283000 ms.0 from job set of time 1501527283000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:43 INFO scheduler.JobScheduler: Finished job streaming job 1501527283000 ms.0 from job set of time 1501527283000 ms
17/07/31 18:54:43 INFO python.PythonRDD: Removing RDD 15 from persistence list
17/07/31 18:54:43 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527283000 ms (execution: 0.008 s)
17/07/31 18:54:43 INFO storage.BlockManager: Removing RDD 15
17/07/31 18:54:43 INFO rdd.BlockRDD: Removing RDD 14 from persistence list
17/07/31 18:54:43 INFO storage.BlockManager: Removing RDD 14
17/07/31 18:54:43 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[14] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527283000 ms
17/07/31 18:54:43 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527281000 ms
17/07/31 18:54:43 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527281000 ms
17/07/31 18:54:44 INFO scheduler.JobScheduler: Added jobs for time 1501527284000 ms
17/07/31 18:54:44 INFO scheduler.JobScheduler: Starting job streaming job 1501527284000 ms.0 from job set of time 1501527284000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:44 INFO scheduler.JobScheduler: Finished job streaming job 1501527284000 ms.0 from job set of time 1501527284000 ms
17/07/31 18:54:44 INFO python.PythonRDD: Removing RDD 17 from persistence list
17/07/31 18:54:44 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527284000 ms (execution: 0.016 s)
17/07/31 18:54:44 INFO storage.BlockManager: Removing RDD 17
17/07/31 18:54:44 INFO rdd.BlockRDD: Removing RDD 16 from persistence list
17/07/31 18:54:44 INFO storage.BlockManager: Removing RDD 16
17/07/31 18:54:44 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[16] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527284000 ms
17/07/31 18:54:44 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527282000 ms
17/07/31 18:54:44 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527282000 ms
17/07/31 18:54:45 INFO scheduler.JobScheduler: Added jobs for time 1501527285000 ms
17/07/31 18:54:45 INFO scheduler.JobScheduler: Starting job streaming job 1501527285000 ms.0 from job set of time 1501527285000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:45 INFO scheduler.JobScheduler: Finished job streaming job 1501527285000 ms.0 from job set of time 1501527285000 ms
17/07/31 18:54:45 INFO python.PythonRDD: Removing RDD 19 from persistence list
17/07/31 18:54:45 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527285000 ms (execution: 0.007 s)
17/07/31 18:54:45 INFO rdd.BlockRDD: Removing RDD 18 from persistence list
17/07/31 18:54:45 INFO storage.BlockManager: Removing RDD 19
17/07/31 18:54:45 INFO storage.BlockManager: Removing RDD 18
17/07/31 18:54:45 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[18] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527285000 ms
17/07/31 18:54:45 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527283000 ms
17/07/31 18:54:45 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527283000 ms
17/07/31 18:54:46 INFO scheduler.JobScheduler: Added jobs for time 1501527286000 ms
17/07/31 18:54:46 INFO scheduler.JobScheduler: Starting job streaming job 1501527286000 ms.0 from job set of time 1501527286000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:46 INFO scheduler.JobScheduler: Finished job streaming job 1501527286000 ms.0 from job set of time 1501527286000 ms
17/07/31 18:54:46 INFO python.PythonRDD: Removing RDD 21 from persistence list
17/07/31 18:54:46 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527286000 ms (execution: 0.008 s)
17/07/31 18:54:46 INFO storage.BlockManager: Removing RDD 21
17/07/31 18:54:46 INFO rdd.BlockRDD: Removing RDD 20 from persistence list
17/07/31 18:54:46 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[20] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527286000 ms
17/07/31 18:54:46 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527284000 ms
17/07/31 18:54:46 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527284000 ms
17/07/31 18:54:46 INFO storage.BlockManager: Removing RDD 20
17/07/31 18:54:47 INFO scheduler.JobScheduler: Added jobs for time 1501527287000 ms
17/07/31 18:54:47 INFO scheduler.JobScheduler: Starting job streaming job 1501527287000 ms.0 from job set of time 1501527287000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:47 INFO scheduler.JobScheduler: Finished job streaming job 1501527287000 ms.0 from job set of time 1501527287000 ms
17/07/31 18:54:47 INFO python.PythonRDD: Removing RDD 23 from persistence list
17/07/31 18:54:47 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527287000 ms (execution: 0.006 s)
17/07/31 18:54:47 INFO storage.BlockManager: Removing RDD 23
17/07/31 18:54:47 INFO rdd.BlockRDD: Removing RDD 22 from persistence list
17/07/31 18:54:47 INFO storage.BlockManager: Removing RDD 22
17/07/31 18:54:47 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[22] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527287000 ms
17/07/31 18:54:47 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527285000 ms
17/07/31 18:54:47 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527285000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Added jobs for time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Starting job streaming job 1501527288000 ms.0 from job set of time 1501527288000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:48 INFO scheduler.JobScheduler: Finished job streaming job 1501527288000 ms.0 from job set of time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527288000 ms (execution: 0.008 s)
17/07/31 18:54:48 INFO python.PythonRDD: Removing RDD 25 from persistence list
17/07/31 18:54:48 INFO storage.BlockManager: Removing RDD 25
17/07/31 18:54:48 INFO rdd.BlockRDD: Removing RDD 24 from persistence list
17/07/31 18:54:48 INFO storage.BlockManager: Removing RDD 24
17/07/31 18:54:48 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[24] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527286000 ms
17/07/31 18:54:48 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527286000 ms
17/07/31 18:54:49 INFO scheduler.JobScheduler: Added jobs for time 1501527289000 ms
17/07/31 18:54:49 INFO scheduler.JobScheduler: Starting job streaming job 1501527289000 ms.0 from job set of time 1501527289000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:49 INFO scheduler.JobScheduler: Finished job streaming job 1501527289000 ms.0 from job set of time 1501527289000 ms
17/07/31 18:54:49 INFO python.PythonRDD: Removing RDD 27 from persistence list
17/07/31 18:54:49 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501527289000 ms (execution: 0.008 s)
17/07/31 18:54:49 INFO storage.BlockManager: Removing RDD 27
17/07/31 18:54:49 INFO rdd.BlockRDD: Removing RDD 26 from persistence list
17/07/31 18:54:49 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[26] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527289000 ms
17/07/31 18:54:49 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527287000 ms
17/07/31 18:54:49 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527287000 ms
17/07/31 18:54:49 INFO storage.BlockManager: Removing RDD 26
17/07/31 18:54:50 INFO scheduler.JobScheduler: Added jobs for time 1501527290000 ms
17/07/31 18:54:50 INFO scheduler.JobScheduler: Starting job streaming job 1501527290000 ms.0 from job set of time 1501527290000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:50 INFO scheduler.JobScheduler: Finished job streaming job 1501527290000 ms.0 from job set of time 1501527290000 ms
17/07/31 18:54:50 INFO python.PythonRDD: Removing RDD 29 from persistence list
17/07/31 18:54:50 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527290000 ms (execution: 0.008 s)
17/07/31 18:54:50 INFO storage.BlockManager: Removing RDD 29
17/07/31 18:54:50 INFO rdd.BlockRDD: Removing RDD 28 from persistence list
17/07/31 18:54:50 INFO storage.BlockManager: Removing RDD 28
17/07/31 18:54:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[28] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527290000 ms
17/07/31 18:54:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527288000 ms
17/07/31 18:54:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527288000 ms
17/07/31 18:54:51 INFO scheduler.JobScheduler: Added jobs for time 1501527291000 ms
17/07/31 18:54:51 INFO scheduler.JobScheduler: Starting job streaming job 1501527291000 ms.0 from job set of time 1501527291000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:51 INFO scheduler.JobScheduler: Finished job streaming job 1501527291000 ms.0 from job set of time 1501527291000 ms
17/07/31 18:54:51 INFO python.PythonRDD: Removing RDD 31 from persistence list
17/07/31 18:54:51 INFO scheduler.JobScheduler: Total delay: 0.033 s for time 1501527291000 ms (execution: 0.018 s)
17/07/31 18:54:51 INFO storage.BlockManager: Removing RDD 31
17/07/31 18:54:51 INFO rdd.BlockRDD: Removing RDD 30 from persistence list
17/07/31 18:54:51 INFO storage.BlockManager: Removing RDD 30
17/07/31 18:54:51 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[30] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527291000 ms
17/07/31 18:54:51 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527289000 ms
17/07/31 18:54:51 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527289000 ms
17/07/31 18:54:52 INFO scheduler.JobScheduler: Added jobs for time 1501527292000 ms
17/07/31 18:54:52 INFO scheduler.JobScheduler: Starting job streaming job 1501527292000 ms.0 from job set of time 1501527292000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:52 INFO scheduler.JobScheduler: Finished job streaming job 1501527292000 ms.0 from job set of time 1501527292000 ms
17/07/31 18:54:52 INFO python.PythonRDD: Removing RDD 33 from persistence list
17/07/31 18:54:52 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501527292000 ms (execution: 0.008 s)
17/07/31 18:54:52 INFO storage.BlockManager: Removing RDD 33
17/07/31 18:54:52 INFO rdd.BlockRDD: Removing RDD 32 from persistence list
17/07/31 18:54:52 INFO storage.BlockManager: Removing RDD 32
17/07/31 18:54:52 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[32] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527292000 ms
17/07/31 18:54:52 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527290000 ms
17/07/31 18:54:52 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527290000 ms
17/07/31 18:54:53 INFO scheduler.JobScheduler: Added jobs for time 1501527293000 ms
17/07/31 18:54:53 INFO scheduler.JobScheduler: Starting job streaming job 1501527293000 ms.0 from job set of time 1501527293000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:53 INFO scheduler.JobScheduler: Finished job streaming job 1501527293000 ms.0 from job set of time 1501527293000 ms
17/07/31 18:54:53 INFO python.PythonRDD: Removing RDD 35 from persistence list
17/07/31 18:54:53 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501527293000 ms (execution: 0.008 s)
17/07/31 18:54:53 INFO storage.BlockManager: Removing RDD 35
17/07/31 18:54:53 INFO rdd.BlockRDD: Removing RDD 34 from persistence list
17/07/31 18:54:53 INFO storage.BlockManager: Removing RDD 34
17/07/31 18:54:53 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[34] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527293000 ms
17/07/31 18:54:53 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527291000 ms
17/07/31 18:54:53 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527291000 ms
17/07/31 18:54:54 INFO scheduler.JobScheduler: Added jobs for time 1501527294000 ms
17/07/31 18:54:54 INFO scheduler.JobScheduler: Starting job streaming job 1501527294000 ms.0 from job set of time 1501527294000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:54 INFO scheduler.JobScheduler: Finished job streaming job 1501527294000 ms.0 from job set of time 1501527294000 ms
17/07/31 18:54:54 INFO python.PythonRDD: Removing RDD 37 from persistence list
17/07/31 18:54:54 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527294000 ms (execution: 0.006 s)
17/07/31 18:54:54 INFO storage.BlockManager: Removing RDD 37
17/07/31 18:54:54 INFO rdd.BlockRDD: Removing RDD 36 from persistence list
17/07/31 18:54:54 INFO storage.BlockManager: Removing RDD 36
17/07/31 18:54:54 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[36] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527294000 ms
17/07/31 18:54:54 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527292000 ms
17/07/31 18:54:54 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527292000 ms
17/07/31 18:54:55 INFO scheduler.JobScheduler: Added jobs for time 1501527295000 ms
17/07/31 18:54:55 INFO scheduler.JobScheduler: Starting job streaming job 1501527295000 ms.0 from job set of time 1501527295000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:55 INFO scheduler.JobScheduler: Finished job streaming job 1501527295000 ms.0 from job set of time 1501527295000 ms
17/07/31 18:54:55 INFO python.PythonRDD: Removing RDD 39 from persistence list
17/07/31 18:54:55 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527295000 ms (execution: 0.007 s)
17/07/31 18:54:55 INFO storage.BlockManager: Removing RDD 39
17/07/31 18:54:55 INFO rdd.BlockRDD: Removing RDD 38 from persistence list
17/07/31 18:54:55 INFO storage.BlockManager: Removing RDD 38
17/07/31 18:54:55 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[38] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527295000 ms
17/07/31 18:54:55 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527293000 ms
17/07/31 18:54:55 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527293000 ms
17/07/31 18:54:56 INFO scheduler.JobScheduler: Added jobs for time 1501527296000 ms
17/07/31 18:54:56 INFO scheduler.JobScheduler: Starting job streaming job 1501527296000 ms.0 from job set of time 1501527296000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:56 INFO scheduler.JobScheduler: Finished job streaming job 1501527296000 ms.0 from job set of time 1501527296000 ms
17/07/31 18:54:56 INFO python.PythonRDD: Removing RDD 41 from persistence list
17/07/31 18:54:56 INFO scheduler.JobScheduler: Total delay: 0.031 s for time 1501527296000 ms (execution: 0.007 s)
17/07/31 18:54:56 INFO storage.BlockManager: Removing RDD 41
17/07/31 18:54:56 INFO rdd.BlockRDD: Removing RDD 40 from persistence list
17/07/31 18:54:56 INFO storage.BlockManager: Removing RDD 40
17/07/31 18:54:56 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[40] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527296000 ms
17/07/31 18:54:56 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527294000 ms
17/07/31 18:54:56 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527294000 ms
17/07/31 18:54:57 INFO scheduler.JobScheduler: Added jobs for time 1501527297000 ms
17/07/31 18:54:57 INFO scheduler.JobScheduler: Starting job streaming job 1501527297000 ms.0 from job set of time 1501527297000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:57 INFO scheduler.JobScheduler: Finished job streaming job 1501527297000 ms.0 from job set of time 1501527297000 ms
17/07/31 18:54:57 INFO python.PythonRDD: Removing RDD 43 from persistence list
17/07/31 18:54:57 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527297000 ms (execution: 0.008 s)
17/07/31 18:54:57 INFO storage.BlockManager: Removing RDD 43
17/07/31 18:54:57 INFO rdd.BlockRDD: Removing RDD 42 from persistence list
17/07/31 18:54:57 INFO storage.BlockManager: Removing RDD 42
17/07/31 18:54:57 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[42] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527297000 ms
17/07/31 18:54:57 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527295000 ms
17/07/31 18:54:57 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527295000 ms
17/07/31 18:54:58 INFO scheduler.JobScheduler: Added jobs for time 1501527298000 ms
17/07/31 18:54:58 INFO scheduler.JobScheduler: Starting job streaming job 1501527298000 ms.0 from job set of time 1501527298000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:58 INFO scheduler.JobScheduler: Finished job streaming job 1501527298000 ms.0 from job set of time 1501527298000 ms
17/07/31 18:54:58 INFO python.PythonRDD: Removing RDD 45 from persistence list
17/07/31 18:54:58 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527298000 ms (execution: 0.008 s)
17/07/31 18:54:58 INFO storage.BlockManager: Removing RDD 45
17/07/31 18:54:58 INFO rdd.BlockRDD: Removing RDD 44 from persistence list
17/07/31 18:54:58 INFO storage.BlockManager: Removing RDD 44
17/07/31 18:54:58 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[44] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527298000 ms
17/07/31 18:54:58 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527296000 ms
17/07/31 18:54:58 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527296000 ms
17/07/31 18:54:59 INFO scheduler.JobScheduler: Added jobs for time 1501527299000 ms
17/07/31 18:54:59 INFO scheduler.JobScheduler: Starting job streaming job 1501527299000 ms.0 from job set of time 1501527299000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:59 INFO scheduler.JobScheduler: Finished job streaming job 1501527299000 ms.0 from job set of time 1501527299000 ms
17/07/31 18:54:59 INFO python.PythonRDD: Removing RDD 47 from persistence list
17/07/31 18:54:59 INFO scheduler.JobScheduler: Total delay: 0.031 s for time 1501527299000 ms (execution: 0.009 s)
17/07/31 18:54:59 INFO storage.BlockManager: Removing RDD 47
17/07/31 18:54:59 INFO rdd.BlockRDD: Removing RDD 46 from persistence list
17/07/31 18:54:59 INFO storage.BlockManager: Removing RDD 46
17/07/31 18:54:59 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[46] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527299000 ms
17/07/31 18:54:59 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527297000 ms
17/07/31 18:54:59 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527297000 ms
17/07/31 18:55:00 INFO scheduler.JobScheduler: Added jobs for time 1501527300000 ms
17/07/31 18:55:00 INFO scheduler.JobScheduler: Starting job streaming job 1501527300000 ms.0 from job set of time 1501527300000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:00 INFO scheduler.JobScheduler: Finished job streaming job 1501527300000 ms.0 from job set of time 1501527300000 ms
17/07/31 18:55:00 INFO python.PythonRDD: Removing RDD 49 from persistence list
17/07/31 18:55:00 INFO scheduler.JobScheduler: Total delay: 0.035 s for time 1501527300000 ms (execution: 0.021 s)
17/07/31 18:55:00 INFO storage.BlockManager: Removing RDD 49
17/07/31 18:55:00 INFO rdd.BlockRDD: Removing RDD 48 from persistence list
17/07/31 18:55:00 INFO storage.BlockManager: Removing RDD 48
17/07/31 18:55:00 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[48] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527300000 ms
17/07/31 18:55:00 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527298000 ms
17/07/31 18:55:00 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527298000 ms
17/07/31 18:55:01 INFO scheduler.JobScheduler: Added jobs for time 1501527301000 ms
17/07/31 18:55:01 INFO scheduler.JobScheduler: Starting job streaming job 1501527301000 ms.0 from job set of time 1501527301000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:01 INFO scheduler.JobScheduler: Finished job streaming job 1501527301000 ms.0 from job set of time 1501527301000 ms
17/07/31 18:55:01 INFO python.PythonRDD: Removing RDD 51 from persistence list
17/07/31 18:55:01 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527301000 ms (execution: 0.008 s)
17/07/31 18:55:01 INFO storage.BlockManager: Removing RDD 51
17/07/31 18:55:01 INFO rdd.BlockRDD: Removing RDD 50 from persistence list
17/07/31 18:55:01 INFO storage.BlockManager: Removing RDD 50
17/07/31 18:55:01 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[50] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527301000 ms
17/07/31 18:55:01 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527299000 ms
17/07/31 18:55:01 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527299000 ms
17/07/31 18:55:02 INFO scheduler.JobScheduler: Added jobs for time 1501527302000 ms
17/07/31 18:55:02 INFO scheduler.JobScheduler: Starting job streaming job 1501527302000 ms.0 from job set of time 1501527302000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:02 INFO scheduler.JobScheduler: Finished job streaming job 1501527302000 ms.0 from job set of time 1501527302000 ms
17/07/31 18:55:02 INFO python.PythonRDD: Removing RDD 53 from persistence list
17/07/31 18:55:02 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501527302000 ms (execution: 0.009 s)
17/07/31 18:55:02 INFO storage.BlockManager: Removing RDD 53
17/07/31 18:55:02 INFO rdd.BlockRDD: Removing RDD 52 from persistence list
17/07/31 18:55:02 INFO storage.BlockManager: Removing RDD 52
17/07/31 18:55:02 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[52] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527302000 ms
17/07/31 18:55:02 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527300000 ms
17/07/31 18:55:02 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527300000 ms
17/07/31 18:55:03 INFO scheduler.JobScheduler: Added jobs for time 1501527303000 ms
17/07/31 18:55:03 INFO scheduler.JobScheduler: Starting job streaming job 1501527303000 ms.0 from job set of time 1501527303000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:03 INFO scheduler.JobScheduler: Finished job streaming job 1501527303000 ms.0 from job set of time 1501527303000 ms
17/07/31 18:55:03 INFO python.PythonRDD: Removing RDD 55 from persistence list
17/07/31 18:55:03 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527303000 ms (execution: 0.008 s)
17/07/31 18:55:03 INFO storage.BlockManager: Removing RDD 55
17/07/31 18:55:03 INFO rdd.BlockRDD: Removing RDD 54 from persistence list
17/07/31 18:55:03 INFO storage.BlockManager: Removing RDD 54
17/07/31 18:55:03 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[54] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527303000 ms
17/07/31 18:55:03 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527301000 ms
17/07/31 18:55:03 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527301000 ms
17/07/31 18:55:04 INFO scheduler.JobScheduler: Added jobs for time 1501527304000 ms
17/07/31 18:55:04 INFO scheduler.JobScheduler: Starting job streaming job 1501527304000 ms.0 from job set of time 1501527304000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:04 INFO scheduler.JobScheduler: Finished job streaming job 1501527304000 ms.0 from job set of time 1501527304000 ms
17/07/31 18:55:04 INFO python.PythonRDD: Removing RDD 57 from persistence list
17/07/31 18:55:04 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501527304000 ms (execution: 0.008 s)
17/07/31 18:55:04 INFO storage.BlockManager: Removing RDD 57
17/07/31 18:55:04 INFO rdd.BlockRDD: Removing RDD 56 from persistence list
17/07/31 18:55:04 INFO storage.BlockManager: Removing RDD 56
17/07/31 18:55:04 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[56] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527304000 ms
17/07/31 18:55:04 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527302000 ms
17/07/31 18:55:04 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527302000 ms
17/07/31 18:55:05 INFO scheduler.JobScheduler: Added jobs for time 1501527305000 ms
17/07/31 18:55:05 INFO scheduler.JobScheduler: Starting job streaming job 1501527305000 ms.0 from job set of time 1501527305000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:05 INFO scheduler.JobScheduler: Finished job streaming job 1501527305000 ms.0 from job set of time 1501527305000 ms
17/07/31 18:55:05 INFO python.PythonRDD: Removing RDD 59 from persistence list
17/07/31 18:55:05 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527305000 ms (execution: 0.008 s)
17/07/31 18:55:05 INFO storage.BlockManager: Removing RDD 59
17/07/31 18:55:05 INFO rdd.BlockRDD: Removing RDD 58 from persistence list
17/07/31 18:55:05 INFO storage.BlockManager: Removing RDD 58
17/07/31 18:55:05 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[58] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527305000 ms
17/07/31 18:55:05 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527303000 ms
17/07/31 18:55:05 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527303000 ms
17/07/31 18:55:06 INFO scheduler.JobScheduler: Added jobs for time 1501527306000 ms
17/07/31 18:55:06 INFO scheduler.JobScheduler: Starting job streaming job 1501527306000 ms.0 from job set of time 1501527306000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:06 INFO scheduler.JobScheduler: Finished job streaming job 1501527306000 ms.0 from job set of time 1501527306000 ms
17/07/31 18:55:06 INFO python.PythonRDD: Removing RDD 61 from persistence list
17/07/31 18:55:06 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527306000 ms (execution: 0.007 s)
17/07/31 18:55:06 INFO storage.BlockManager: Removing RDD 61
17/07/31 18:55:06 INFO rdd.BlockRDD: Removing RDD 60 from persistence list
17/07/31 18:55:06 INFO storage.BlockManager: Removing RDD 60
17/07/31 18:55:06 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[60] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527306000 ms
17/07/31 18:55:06 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527304000 ms
17/07/31 18:55:06 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527304000 ms
17/07/31 18:55:07 INFO scheduler.JobScheduler: Added jobs for time 1501527307000 ms
17/07/31 18:55:07 INFO scheduler.JobScheduler: Starting job streaming job 1501527307000 ms.0 from job set of time 1501527307000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:07 INFO scheduler.JobScheduler: Finished job streaming job 1501527307000 ms.0 from job set of time 1501527307000 ms
17/07/31 18:55:07 INFO python.PythonRDD: Removing RDD 63 from persistence list
17/07/31 18:55:07 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501527307000 ms (execution: 0.007 s)
17/07/31 18:55:07 INFO storage.BlockManager: Removing RDD 63
17/07/31 18:55:07 INFO rdd.BlockRDD: Removing RDD 62 from persistence list
17/07/31 18:55:07 INFO storage.BlockManager: Removing RDD 62
17/07/31 18:55:07 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[62] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527307000 ms
17/07/31 18:55:07 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527305000 ms
17/07/31 18:55:07 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527305000 ms
17/07/31 18:55:08 INFO scheduler.JobScheduler: Added jobs for time 1501527308000 ms
17/07/31 18:55:08 INFO scheduler.JobScheduler: Starting job streaming job 1501527308000 ms.0 from job set of time 1501527308000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:08 INFO scheduler.JobScheduler: Finished job streaming job 1501527308000 ms.0 from job set of time 1501527308000 ms
17/07/31 18:55:08 INFO python.PythonRDD: Removing RDD 65 from persistence list
17/07/31 18:55:08 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527308000 ms (execution: 0.008 s)
17/07/31 18:55:08 INFO storage.BlockManager: Removing RDD 65
17/07/31 18:55:08 INFO rdd.BlockRDD: Removing RDD 64 from persistence list
17/07/31 18:55:08 INFO storage.BlockManager: Removing RDD 64
17/07/31 18:55:08 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[64] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527308000 ms
17/07/31 18:55:08 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527306000 ms
17/07/31 18:55:08 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527306000 ms
17/07/31 18:55:09 INFO scheduler.JobScheduler: Added jobs for time 1501527309000 ms
17/07/31 18:55:09 INFO scheduler.JobScheduler: Starting job streaming job 1501527309000 ms.0 from job set of time 1501527309000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:09 INFO scheduler.JobScheduler: Finished job streaming job 1501527309000 ms.0 from job set of time 1501527309000 ms
17/07/31 18:55:09 INFO python.PythonRDD: Removing RDD 67 from persistence list
17/07/31 18:55:09 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527309000 ms (execution: 0.015 s)
17/07/31 18:55:09 INFO storage.BlockManager: Removing RDD 67
17/07/31 18:55:09 INFO rdd.BlockRDD: Removing RDD 66 from persistence list
17/07/31 18:55:09 INFO storage.BlockManager: Removing RDD 66
17/07/31 18:55:09 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[66] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527309000 ms
17/07/31 18:55:09 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527307000 ms
17/07/31 18:55:09 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527307000 ms
17/07/31 18:55:10 INFO scheduler.JobScheduler: Added jobs for time 1501527310000 ms
17/07/31 18:55:10 INFO scheduler.JobScheduler: Starting job streaming job 1501527310000 ms.0 from job set of time 1501527310000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:10 INFO scheduler.JobScheduler: Finished job streaming job 1501527310000 ms.0 from job set of time 1501527310000 ms
17/07/31 18:55:10 INFO python.PythonRDD: Removing RDD 69 from persistence list
17/07/31 18:55:10 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527310000 ms (execution: 0.007 s)
17/07/31 18:55:10 INFO storage.BlockManager: Removing RDD 69
17/07/31 18:55:10 INFO rdd.BlockRDD: Removing RDD 68 from persistence list
17/07/31 18:55:10 INFO storage.BlockManager: Removing RDD 68
17/07/31 18:55:10 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[68] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527310000 ms
17/07/31 18:55:10 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527308000 ms
17/07/31 18:55:10 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527308000 ms
17/07/31 18:55:11 INFO scheduler.JobScheduler: Added jobs for time 1501527311000 ms
17/07/31 18:55:11 INFO scheduler.JobScheduler: Starting job streaming job 1501527311000 ms.0 from job set of time 1501527311000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:11 INFO scheduler.JobScheduler: Finished job streaming job 1501527311000 ms.0 from job set of time 1501527311000 ms
17/07/31 18:55:11 INFO python.PythonRDD: Removing RDD 71 from persistence list
17/07/31 18:55:11 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527311000 ms (execution: 0.007 s)
17/07/31 18:55:11 INFO storage.BlockManager: Removing RDD 71
17/07/31 18:55:11 INFO rdd.BlockRDD: Removing RDD 70 from persistence list
17/07/31 18:55:11 INFO storage.BlockManager: Removing RDD 70
17/07/31 18:55:11 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[70] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527311000 ms
17/07/31 18:55:11 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527309000 ms
17/07/31 18:55:11 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527309000 ms
17/07/31 18:55:12 INFO scheduler.JobScheduler: Added jobs for time 1501527312000 ms
17/07/31 18:55:12 INFO scheduler.JobScheduler: Starting job streaming job 1501527312000 ms.0 from job set of time 1501527312000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:12 INFO scheduler.JobScheduler: Finished job streaming job 1501527312000 ms.0 from job set of time 1501527312000 ms
17/07/31 18:55:12 INFO python.PythonRDD: Removing RDD 73 from persistence list
17/07/31 18:55:12 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501527312000 ms (execution: 0.008 s)
17/07/31 18:55:12 INFO storage.BlockManager: Removing RDD 73
17/07/31 18:55:12 INFO rdd.BlockRDD: Removing RDD 72 from persistence list
17/07/31 18:55:12 INFO storage.BlockManager: Removing RDD 72
17/07/31 18:55:12 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[72] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527312000 ms
17/07/31 18:55:12 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527310000 ms
17/07/31 18:55:12 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527310000 ms
17/07/31 18:55:13 INFO scheduler.JobScheduler: Added jobs for time 1501527313000 ms
17/07/31 18:55:13 INFO scheduler.JobScheduler: Starting job streaming job 1501527313000 ms.0 from job set of time 1501527313000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:13 INFO scheduler.JobScheduler: Finished job streaming job 1501527313000 ms.0 from job set of time 1501527313000 ms
17/07/31 18:55:13 INFO python.PythonRDD: Removing RDD 75 from persistence list
17/07/31 18:55:13 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527313000 ms (execution: 0.007 s)
17/07/31 18:55:13 INFO storage.BlockManager: Removing RDD 75
17/07/31 18:55:13 INFO rdd.BlockRDD: Removing RDD 74 from persistence list
17/07/31 18:55:13 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[74] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527313000 ms
17/07/31 18:55:13 INFO storage.BlockManager: Removing RDD 74
17/07/31 18:55:13 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527311000 ms
17/07/31 18:55:13 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527311000 ms
17/07/31 18:55:14 INFO scheduler.JobScheduler: Added jobs for time 1501527314000 ms
17/07/31 18:55:14 INFO scheduler.JobScheduler: Starting job streaming job 1501527314000 ms.0 from job set of time 1501527314000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:14 INFO scheduler.JobScheduler: Finished job streaming job 1501527314000 ms.0 from job set of time 1501527314000 ms
17/07/31 18:55:14 INFO python.PythonRDD: Removing RDD 77 from persistence list
17/07/31 18:55:14 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527314000 ms (execution: 0.007 s)
17/07/31 18:55:14 INFO storage.BlockManager: Removing RDD 77
17/07/31 18:55:14 INFO rdd.BlockRDD: Removing RDD 76 from persistence list
17/07/31 18:55:14 INFO storage.BlockManager: Removing RDD 76
17/07/31 18:55:14 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[76] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527314000 ms
17/07/31 18:55:14 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527312000 ms
17/07/31 18:55:14 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527312000 ms
17/07/31 18:55:15 INFO scheduler.JobScheduler: Added jobs for time 1501527315000 ms
17/07/31 18:55:15 INFO scheduler.JobScheduler: Starting job streaming job 1501527315000 ms.0 from job set of time 1501527315000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:15 INFO scheduler.JobScheduler: Finished job streaming job 1501527315000 ms.0 from job set of time 1501527315000 ms
17/07/31 18:55:15 INFO python.PythonRDD: Removing RDD 79 from persistence list
17/07/31 18:55:15 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527315000 ms (execution: 0.007 s)
17/07/31 18:55:15 INFO storage.BlockManager: Removing RDD 79
17/07/31 18:55:15 INFO rdd.BlockRDD: Removing RDD 78 from persistence list
17/07/31 18:55:15 INFO storage.BlockManager: Removing RDD 78
17/07/31 18:55:15 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[78] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527315000 ms
17/07/31 18:55:15 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527313000 ms
17/07/31 18:55:15 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527313000 ms
17/07/31 18:55:16 INFO scheduler.JobScheduler: Added jobs for time 1501527316000 ms
17/07/31 18:55:16 INFO scheduler.JobScheduler: Starting job streaming job 1501527316000 ms.0 from job set of time 1501527316000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:16 INFO scheduler.JobScheduler: Finished job streaming job 1501527316000 ms.0 from job set of time 1501527316000 ms
17/07/31 18:55:16 INFO python.PythonRDD: Removing RDD 81 from persistence list
17/07/31 18:55:16 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501527316000 ms (execution: 0.014 s)
17/07/31 18:55:16 INFO storage.BlockManager: Removing RDD 81
17/07/31 18:55:16 INFO rdd.BlockRDD: Removing RDD 80 from persistence list
17/07/31 18:55:16 INFO storage.BlockManager: Removing RDD 80
17/07/31 18:55:16 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[80] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527316000 ms
17/07/31 18:55:16 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527314000 ms
17/07/31 18:55:16 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527314000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Added jobs for time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Starting job streaming job 1501527317000 ms.0 from job set of time 1501527317000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:17 INFO scheduler.JobScheduler: Finished job streaming job 1501527317000 ms.0 from job set of time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527317000 ms (execution: 0.011 s)
17/07/31 18:55:17 INFO python.PythonRDD: Removing RDD 83 from persistence list
17/07/31 18:55:17 INFO storage.BlockManager: Removing RDD 83
17/07/31 18:55:17 INFO rdd.BlockRDD: Removing RDD 82 from persistence list
17/07/31 18:55:17 INFO storage.BlockManager: Removing RDD 82
17/07/31 18:55:17 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[82] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527315000 ms
17/07/31 18:55:17 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527315000 ms
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1059.6 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1059.6 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.4 KB, free: 1057.9 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.4 KB, free: 1057.9 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 880.3 KB, free: 1057.0 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 880.3 KB, free: 1057.1 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.0 KB, free: 1056.2 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.0 KB, free: 1056.2 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527317800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.3 KB, free: 1054.5 MB)
17/07/31 18:55:18 INFO scheduler.JobScheduler: Added jobs for time 1501527318000 ms
17/07/31 18:55:18 INFO scheduler.JobScheduler: Starting job streaming job 1501527318000 ms.0 from job set of time 1501527318000 ms
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527317800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.3 KB, free: 1054.5 MB)
17/07/31 18:55:18 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Got job 2 (runJob at PythonRDD.scala:441) with 1 output partitions
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Final stage: ResultStage 3 (runJob at PythonRDD.scala:441)
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Submitting ResultStage 3 (PythonRDD[88] at RDD at PythonRDD.scala:48), which has no missing parents
17/07/31 18:55:18 INFO memory.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/07/31 18:55:18 INFO memory.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.0.0.199:59442 (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:18 INFO spark.SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:996
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (PythonRDD[88] at RDD at PythonRDD.scala:48)
17/07/31 18:55:18 INFO cluster.YarnScheduler: Adding task set 3.0 with 1 tasks
17/07/31 18:55:18 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 3.0 (TID 71, ip-10-0-0-151.ec2.internal, executor 2, partition 0, NODE_LOCAL, 5800 bytes)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 5.3 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.3 KB, free: 1052.7 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.3 KB, free: 1052.8 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.1 KB, free: 1050.2 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.1 KB, free: 1050.2 MB)
17/07/31 18:55:18 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 3.0 (TID 71) in 893 ms on ip-10-0-0-151.ec2.internal (executor 2) (1/1)
17/07/31 18:55:18 INFO cluster.YarnScheduler: Removed TaskSet 3.0, whose tasks have all completed, from pool
17/07/31 18:55:18 INFO scheduler.DAGScheduler: ResultStage 3 (runJob at PythonRDD.scala:441) finished in 0.896 s
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Job 2 finished: runJob at PythonRDD.scala:441, took 0.913501 s
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527318800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.5 KB, free: 1049.3 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527318800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.5 KB, free: 1049.3 MB)
17/07/31 18:55:19 INFO scheduler.JobScheduler: Added jobs for time 1501527319000 ms
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 887.9 KB, free: 1048.4 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 887.9 KB, free: 1048.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1047.6 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1047.6 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1046.7 MB)
17/07/31 18:55:19 INFO spark.ContextCleaner: Cleaned shuffle 0
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 10.0.0.199:59442 in memory (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1988.0 B, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on 10.0.0.199:59442 in memory (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1956.0 B, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on 10.0.0.199:59442 in memory (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 5.3 KB, free: 1046.8 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.2 KB, free: 1045.0 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.2 KB, free: 1045.0 MB)
17/07/31 18:55:19 INFO execution.SparkSqlParser: Parsing command: spotdb.proxy
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527319800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527319800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
17/07/31 18:55:20 INFO metastore.ObjectStore: ObjectStore, initialize called
17/07/31 18:55:20 INFO scheduler.JobScheduler: Added jobs for time 1501527320000 ms
17/07/31 18:55:20 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing
17/07/31 18:55:20 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
17/07/31 18:55:20 INFO metastore.ObjectStore: Initialized ObjectStore
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_table start=1501527320011 end=1501527320059 duration=48 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_table start=1501527320177 end=1501527320179 duration=2 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_database: spotdb
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: spotdb
17/07/31 18:55:20 WARN metastore.ObjectStore: Failed to get database spotdb, returning NoSuchObjectException
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527320239 end=1501527320242 duration=3 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527318000 ms.0 from job set of time 1501527318000 ms
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 2.266 s for time 1501527318000 ms (execution: 2.249 s)
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 85 from persistence list
17/07/31 18:55:20 INFO scheduler.JobScheduler: Starting job streaming job 1501527319000 ms.0 from job set of time 1501527319000 ms
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 84 from persistence list
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527318000 ms.0
org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 71, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u"Database 'spotdb' not found;"
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[84] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527318000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 85
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527316000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527316000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 84
17/07/31 18:55:20 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Got job 3 (runJob at PythonRDD.scala:441) with 1 output partitions
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Final stage: ResultStage 4 (runJob at PythonRDD.scala:441)
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Submitting ResultStage 4 (PythonRDD[97] at RDD at PythonRDD.scala:48), which has no missing parents
Traceback (most recent call last):
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 148, in <module>
main()
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 77, in main
bluecoat_parse(args.zk,args.topic,args.db,args.db_table,args.num_of_workers,args.batch_size)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 144, in bluecoat_parse
ssc.awaitTermination()
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/context.py", line 206, in awaitTermination
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o48.awaitTermination.
: org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 71, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u"Database 'spotdb' not found;"
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/07/31 18:55:20 INFO memory.MemoryStore: Block broadcast_4 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/07/31 18:55:20 INFO memory.MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on 10.0.0.199:59442 (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:20 INFO spark.SparkContext: Created broadcast 4 from broadcast at DAGScheduler.scala:996
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 4 (PythonRDD[97] at RDD at PythonRDD.scala:48)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Adding task set 4.0 with 1 tasks
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 4.0 (TID 72, ip-10-0-0-151.ec2.internal, executor 2, partition 0, NODE_LOCAL, 5800 bytes)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 5.3 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527319000 ms.0 from job set of time 1501527319000 ms
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 1.329 s for time 1501527319000 ms (execution: 0.051 s)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Starting job streaming job 1501527320000 ms.0 from job set of time 1501527320000 ms
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527319000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy36.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 87 from persistence list
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 86 from persistence list
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 87
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[86] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527319000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 86
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527317000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527317000 ms
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317000 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.2 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317400 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 880.3 KB, free: 1045.0 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317200 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.4 KB, free: 1046.7 MB)
17/07/31 18:55:20 INFO streaming.StreamingContext: Invoking stop(stopGracefully=false) from shutdown hook
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317600 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.0 KB, free: 1047.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317000 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.2 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317200 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.4 KB, free: 1045.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317400 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 880.3 KB, free: 1046.7 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317600 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.0 KB, free: 1047.6 MB)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527320000 ms.0 from job set of time 1501527320000 ms
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 90 from persistence list
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 0.333 s for time 1501527320000 ms (execution: 0.003 s)
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527320000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy36.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 89 from persistence list
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 90
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 89
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[89] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527320000 ms
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: Sent stop signal to all 1 receivers
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317800 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.3 KB, free: 1049.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318000 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.1 KB, free: 1050.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318200 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.3 KB, free: 1051.0 MB)
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527318000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527318000 ms
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318400 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318600 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.1 KB, free: 1053.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317800 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.3 KB, free: 1049.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318000 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.1 KB, free: 1050.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318200 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.3 KB, free: 1051.0 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318400 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318600 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.1 KB, free: 1053.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 880.6 KB, free: 1052.7 MB)
17/07/31 18:55:20 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 880.6 KB, free: 1052.8 MB)
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 4.0 (TID 72) in 289 ms on ip-10-0-0-151.ec2.internal (executor 2) (1/1)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Removed TaskSet 4.0, whose tasks have all completed, from pool
17/07/31 18:55:20 ERROR scheduler.DAGScheduler: Failed to update accumulators for task 0
org.apache.spark.SparkException: EOF reached before Python server acknowledged
at org.apache.spark.api.python.PythonAccumulatorV2.merge(PythonRDD.scala:919)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$updateAccumulators$1.apply(DAGScheduler.scala:1088)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$updateAccumulators$1.apply(DAGScheduler.scala:1080)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.DAGScheduler.updateAccumulators(DAGScheduler.scala:1080)
at org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:1156)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1647)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/07/31 18:55:20 INFO scheduler.DAGScheduler: ResultStage 4 (runJob at PythonRDD.scala:441) finished in 0.292 s
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Job 3 finished: runJob at PythonRDD.scala:441, took 0.309062 s
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 2.0 (TID 70) in 43848 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/1)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool
17/07/31 18:55:20 INFO scheduler.DAGScheduler: ResultStage 2 (start at NativeMethodAccessorImpl.java:0) finished in 43.849 s
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: All of the receivers have deregistered successfully
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
17/07/31 18:55:20 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
17/07/31 18:55:20 INFO util.RecurringTimer: Stopped timer for JobGenerator after time 1501527320000
17/07/31 18:55:20 INFO scheduler.JobGenerator: Stopped JobGenerator
17/07/31 18:55:20 INFO scheduler.JobScheduler: Stopped JobScheduler
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19fd4723{/streaming,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@22116fc3{/streaming/batch,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@70d301d6{/static/streaming,null,UNAVAILABLE}
17/07/31 18:55:20 INFO streaming.StreamingContext: StreamingContext stopped successfully
17/07/31 18:55:20 INFO spark.SparkContext: Invoking stop() from shutdown hook
17/07/31 18:55:20 INFO server.ServerConnector: Stopped ServerConnector@188e6c7f{HTTP/1.1}{0.0.0.0:4040}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4f734944{/stages/stage/kill,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@b7d4da0{/jobs/job/kill,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ab71bb9{/api,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5d37ce06{/,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@27f55b38{/static,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1159f15e{/executors/threadDump/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ff970c2{/executors/threadDump,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@615b86f5{/executors/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19bb7285{/executors,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3b9882ec{/environment/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7b3691b6{/environment,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@423a3429{/storage/rdd/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@13016b86{/storage/rdd,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ea732f0{/storage/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2d4aa15a{/storage,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5cbdbf0f{/stages/pool/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14c62558{/stages/pool,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5c61bd1a{/stages/stage/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4eb72ecd{/stages/stage,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1bf090df{/stages/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14b58fc0{/stages,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@30e71b5d{/jobs/job/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@232864a3{/jobs/job,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1f4da763{/jobs/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1a3b1e79{/jobs,null,UNAVAILABLE}
17/07/31 18:55:20 INFO ui.SparkUI: Stopped Spark web UI at http://10.0.0.199:4040
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors
17/07/31 18:55:20 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
17/07/31 18:55:20 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
services=List(),
started=false)
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Stopped
17/07/31 18:55:20 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/07/31 18:55:20 INFO memory.MemoryStore: MemoryStore cleared
17/07/31 18:55:20 INFO storage.BlockManager: BlockManager stopped
17/07/31 18:55:20 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
17/07/31 18:55:20 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/07/31 18:55:20 INFO spark.SparkContext: Successfully stopped SparkContext
17/07/31 18:55:20 INFO util.ShutdownHookManager: Shutdown hook called
17/07/31 18:55:20 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40
17/07/31 18:55:20 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40/pyspark-506939c9-a852-404b-adea-6ad40ad0d73f
Closing worker...
Thanks
Deon Griessel
Re: Spot Ingest Proxy Error
Posted by "Barona, Ricardo" <ri...@intel.com>.
Great!
From: Deon Griessel <dg...@searchtechnologies.com>
Reply-To: "user@spot.incubator.apache.org" <us...@spot.incubator.apache.org>
Date: Tuesday, August 1, 2017 at 11:15 AM
To: "user@spot.incubator.apache.org" <us...@spot.incubator.apache.org>, "user@spot.apache.org" <us...@spot.apache.org>
Subject: RE: Spot Ingest Proxy Error
I replaced the following line in bluecoat.py:
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
with
df.write.format("parquet").mode("append").insertInto(hive_table)
That seemed to do the trick, it then bombed on bad input data in the sample data provided in: Bluecoat_ProxySG_Sample.log
Seems like one of the records has a duration/time-taken which is larger than int and thus caused the python script to die.
Anyhow for now I removed the offending line and was able to get it to run through.
-Deon
From: Barona, Ricardo [mailto:ricardo.barona@intel.com]
Sent: Tuesday, August 1, 2017 11:14 AM
To: user@spot.incubator.apache.org; user@spot.apache.org
Subject: Re: Spot Ingest Proxy Error
Hi Dean,
Let me ask around.
From: Deon Griessel <dg...@searchtechnologies.com>>
Reply-To: "user@spot.incubator.apache.org<ma...@spot.incubator.apache.org>" <us...@spot.incubator.apache.org>>
Date: Tuesday, August 1, 2017 at 10:04 AM
To: "user@spot.incubator.apache.org<ma...@spot.incubator.apache.org>" <us...@spot.incubator.apache.org>>, "user@spot.apache.org<ma...@spot.apache.org>" <us...@spot.apache.org>>
Subject: RE: Spot Ingest Proxy Error
Ok, it seems like I got around the original error by enabling the hive service in Spark2 configuration in Cloudera Manager and restarting all services.
Now getting the following error:
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
Anybody seen this before? Full log below.
Thanks
---------------------------
2017-08-01 14:49:13,694 - SPOT.INGEST.WORKER - INFO - Initializing kafka instance
2017-08-01 14:49:13,694 - SPOT.INGEST.WRK.PROXY - INFO - Creating Spark Job for topic: SPOT-INGEST-proxy-14_49_11
2017-08-01 14:49:13,694 - SPOT.INGEST.WRK.PROXY - INFO - SPOT.Utils: Executing: spark2-submit --master yarn --driver-memory 2325m --num-executors 1 --conf spark.executor.memory=2325m --conf spark.executor.cores=1 --jars /home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar /home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py -zk ip-10-0-0-199.ec2.internal:2181 -t SPOT-INGEST-proxy-14_49_11 -db spotdb -dt proxy -w 1 -bs 1
17/08/01 14:49:14 INFO spark.SparkContext: Running Spark version 2.1.0.cloudera1
17/08/01 14:49:14 WARN spark.SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0
17/08/01 14:49:15 INFO spark.SecurityManager: Changing view acls to: spot-user
17/08/01 14:49:15 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/08/01 14:49:15 INFO spark.SecurityManager: Changing view acls groups to:
17/08/01 14:49:15 INFO spark.SecurityManager: Changing modify acls groups to:
17/08/01 14:49:15 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/08/01 14:49:15 INFO util.Utils: Successfully started service 'sparkDriver' on port 56925.
17/08/01 14:49:15 INFO spark.SparkEnv: Registering MapOutputTracker
17/08/01 14:49:15 INFO spark.SparkEnv: Registering BlockManagerMaster
17/08/01 14:49:15 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
17/08/01 14:49:15 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
17/08/01 14:49:15 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-23e4ad79-bd38-4e37-b688-a75f9565bda4
17/08/01 14:49:16 INFO memory.MemoryStore: MemoryStore started with capacity 1060.5 MB
17/08/01 14:49:16 INFO spark.SparkEnv: Registering OutputCommitCoordinator
17/08/01 14:49:16 INFO util.log: Logging initialized @2378ms
17/08/01 14:49:16 INFO server.Server: jetty-9.2.z-SNAPSHOT
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3b9882ec{/jobs,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3b9882ec%7b/jobs,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19bb7285{/jobs/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19bb7285%7b/jobs/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@615b86f5{/jobs/job,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@615b86f5%7b/jobs/job,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ff970c2{/jobs/job/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3ff970c2%7b/jobs/job/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1159f15e{/stages,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1159f15e%7b/stages,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27f55b38{/stages/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27f55b38%7b/stages/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5d37ce06{/stages/stage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5d37ce06%7b/stages/stage,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ab71bb9{/stages/stage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ab71bb9%7b/stages/stage/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b7d4da0{/stages/pool,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@b7d4da0%7b/stages/pool,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f734944{/stages/pool/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4f734944%7b/stages/pool/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19cf381c{/storage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19cf381c%7b/storage,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@736cc380{/storage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@736cc380%7b/storage/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@699743b3{/storage/rdd,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@699743b3%7b/storage/rdd,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@49ddd2ed{/storage/rdd/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@49ddd2ed%7b/storage/rdd/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@242eb7e{/environment,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@242eb7e%7b/environment,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@77729e5a{/environment/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@77729e5a%7b/environment/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5fa63fa{/executors,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5fa63fa%7b/executors,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@79b9b526{/executors/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@79b9b526%7b/executors/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@f241039{/executors/threadDump,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@f241039%7b/executors/threadDump,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@8a515f3{/executors/threadDump/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@8a515f3%7b/executors/threadDump/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@e4408f4{/static,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@e4408f4%7b/static,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14a7d27d{/,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14a7d27d%7b/,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@15ec1723{/api,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@15ec1723%7b/api,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ee30613{/jobs/job/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3ee30613%7b/jobs/job/kill,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2aa3e9a6{/stages/stage/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@2aa3e9a6%7b/stages/stage/kill,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO server.ServerConnector: Started ServerConnector@37681deb{HTTP/1.1}{0.0.0.0:4040}<mailto:ServerConnector@37681deb%7bHTTP/1.1%7d%7b0.0.0.0:4040%7d>
17/08/01 14:49:16 INFO server.Server: Started @2532ms
17/08/01 14:49:16 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
17/08/01 14:49:16 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.0.0.199:4040
17/08/01 14:49:16 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/08/01 14:49:17 INFO client.RMProxy: Connecting to ResourceManager at ip-10-0-0-199.ec2.internal/10.0.0.199:8032
17/08/01 14:49:17 INFO yarn.Client: Requesting a new application from cluster with 3 NodeManagers
17/08/01 14:49:17 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (2725 MB per container)
17/08/01 14:49:17 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/08/01 14:49:17 INFO yarn.Client: Setting up container launch context for our AM
17/08/01 14:49:17 INFO yarn.Client: Setting up the launch environment for our AM container
17/08/01 14:49:17 INFO yarn.Client: Preparing resources for our AM container
17/08/01 14:49:19 INFO yarn.Client: Uploading resource file:/home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501598591600_0003/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar
17/08/01 14:49:19 INFO yarn.Client: Uploading resource file:/tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25/__spark_conf__4374658135010889941.zip -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501598591600_0003/__spark_conf__.zip
17/08/01 14:49:19 INFO spark.SecurityManager: Changing view acls to: spot-user
17/08/01 14:49:19 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/08/01 14:49:19 INFO spark.SecurityManager: Changing view acls groups to:
17/08/01 14:49:19 INFO spark.SecurityManager: Changing modify acls groups to:
17/08/01 14:49:19 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/08/01 14:49:19 INFO yarn.Client: Submitting application application_1501598591600_0003 to ResourceManager
17/08/01 14:49:19 INFO impl.YarnClientImpl: Submitted application application_1501598591600_0003
17/08/01 14:49:19 INFO cluster.SchedulerExtensionServices: Starting Yarn extension services with app application_1501598591600_0003 and attemptId None
17/08/01 14:49:20 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:20 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.users.spot-user
start time: 1501598959525
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003/
user: spot-user
17/08/01 14:49:21 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:22 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:23 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/08/01 14:49:23 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> ip-10-0-0-199.ec2.internal, PROXY_URI_BASES -> http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003), /proxy/application_1501598591600_0003
17/08/01 14:49:23 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/08/01 14:49:23 INFO yarn.Client: Application report for application_1501598591600_0003 (state: RUNNING)
17/08/01 14:49:23 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 10.0.0.151
ApplicationMaster RPC port: 0
queue: root.users.spot-user
start time: 1501598959525
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003/
user: spot-user
17/08/01 14:49:23 INFO cluster.YarnClientSchedulerBackend: Application application_1501598591600_0003 has started running.
17/08/01 14:49:23 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 53355.
17/08/01 14:49:23 INFO netty.NettyBlockTransferService: Server created on 10.0.0.199:53355
17/08/01 14:49:23 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/08/01 14:49:23 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManagerMasterEndpoint: Registering block manager 10.0.0.199:53355 with 1060.5 MB RAM, BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManager: external shuffle service port = 7337
17/08/01 14:49:23 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@52c2dab5{/metrics/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@52c2dab5%7b/metrics/json,null,AVAILABLE%7d>
17/08/01 14:49:23 INFO scheduler.EventLoggingListener: Logging events to hdfs://ip-10-0-0-199.ec2.internal:8020/user/spark/spark2ApplicationHistory/application_1501598591600_0003
17/08/01 14:49:23 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/08/01 14:49:27 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.229:37096) with ID 1
17/08/01 14:49:27 INFO spark.ExecutorAllocationManager: New executor 1 has registered (new total is 1)
17/08/01 14:49:27 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-229.ec2.internal:51770 with 1060.5 MB RAM, BlockManagerId(1, ip-10-0-0-229.ec2.internal, 51770, None)
17/08/01 14:49:27 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/08/01 14:49:27 INFO internal.SharedState: spark.sql.warehouse.dir is not set, but hive.metastore.warehouse.dir is set. Setting spark.sql.warehouse.dir to the value of hive.metastore.warehouse.dir ('/user/hive/warehouse').
17/08/01 14:49:27 INFO internal.SharedState: Warehouse path is '/user/hive/warehouse'.
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5554f0a0{/SQL,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5554f0a0%7b/SQL,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@44743601{/SQL/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@44743601%7b/SQL/json,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@144776d3{/SQL/execution,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@144776d3%7b/SQL/execution,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f3f8a4e{/SQL/execution/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4f3f8a4e%7b/SQL/execution/json,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27946137{/static/sql,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27946137%7b/static/sql,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO hive.HiveUtils: Initializing HiveMetastoreConnection version 1.1.0 using file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-logging-1.1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-exec-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-exec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-1.1.0-cdh5.12.0-standalone.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-standalone.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-metastore-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-metastore.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-serde-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-serde.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-service-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-service.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/libfb303-0.9.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/libthrift-0.9.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/log4j-1.2.16.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-hadoop-compat.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-hadoop2-compat.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-protocol.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-server.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/htrace-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ST4-4.0.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-core-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-fate-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-start-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-trace-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/activation-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ant-1.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ant-launcher-1.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/antlr-2.7.7.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/antlr-runtime-3.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/apache-log4j-extras-1.2.17.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-3.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-commons-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-tree-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/avro.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/bonecp-0.8.0.RELEASE.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-avatica-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-core-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-linq4j-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-beanutils-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-beanutils-core-1.8.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-cli-1.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-codec-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-collections-3.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-compiler-2.7.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-compress-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-configuration-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-dbcp-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-digester-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-el-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-httpclient-3.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-io-2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-lang-2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-lang3-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-math-2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-pool-1.5.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-vfs2-2.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-client-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-framework-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-recipes-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-api-jdo-3.2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-core-3.2.10.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-rdbms-3.2.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/derby-10.11.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/eigenbase-properties-1.1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/findbugs-annotations-1.3.9-1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-annotation_1.0_spec-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-jaspic_1.0_spec-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-jta_1.1_spec-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/groovy-all-2.4.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/gson-2.2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/guava-14.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hamcrest-core-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/high-scale-lib-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-accumulo-handler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-accumulo-handler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-ant-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-ant.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-beeline-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-beeline.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-cli-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-cli.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-common-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-contrib-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-contrib.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hbase-handler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hbase-handler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hwi-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hwi.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-0.23-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-0.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-common-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-scheduler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-scheduler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-testutils-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-testutils.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/httpclient-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/httpcore-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-core-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ivy-2.0.0-rc2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/parquet-hadoop-bundle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/stringtemplate-3.2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/regexp-1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/xz-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/tempus-fugit-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/super-csv-2.2.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/stax-api-1.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/servlet-api-2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/opencsv-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-jvm-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-json-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-core-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-provider-svnexe-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-provider-svn-commons-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-api-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/mail-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/logredactor-1.0.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/junit-4.11.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jta-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jsr305-3.0.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jsp-api-2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jpam-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/joda-time-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jline-2.12.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jetty-all-server-7.6.0.v20120127.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jetty-all-7.6.0.v20120127.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jersey-servlet-1.14.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jersey-server-1.14.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jdo-api-3.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jcommander-1.32.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jasper-runtime-5.5.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jasper-compiler-5.5.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/janino-2.7.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jamon-runtime-2.3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-xc-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-jaxrs-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-databind-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-annotations-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/zookeeper.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/velocity-1.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/snappy-java-1.0.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/plexus-utils-1.5.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/paranamer-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/oro-2.0.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/activation-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/activation.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-i18n-2.0.0-M15.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-i18n.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-kerberos-codec-2.0.0-M15.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-kerberos-codec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-asn1-api-1.0.0-M20.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-asn1-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-util-1.0.0-M20.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-util.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/avro.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/aws-java-sdk-bundle-1.11.134.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/aws-java-sdk-bundle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/azure-data-lake-store-sdk-2.1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/azure-data-lake-store-sdk.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-core-1.8.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-cli-1.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-cli.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-codec-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-codec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-collections-3.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-collections.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-compress-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-compress.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-configuration-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-configuration.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-digester-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-digester.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-httpclient-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-httpclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-io-2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-io.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-lang-2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-lang.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-logging-1.1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-logging.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-math3-3.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-math3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-net-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-net.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-client-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-framework-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-framework.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-recipes-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-recipes.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/gson-2.2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/gson.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/guava-11.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/guava.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-annotations-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-auth-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-auth.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-aws-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-aws.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-azure-datalake-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-azure-datalake.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-hdfs-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-hdfs.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-app-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-app.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-core-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-jobclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-shuffle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-api-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-client-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-server-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-server-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/htrace-core4-4.0.1-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/htrace-core4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpclient-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpcore-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpcore.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-annotations-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-core-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-databind-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-databind.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-jaxrs-1.8.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-jaxrs.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-xc-1.8.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-xc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jaxb-api-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jaxb-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-client-1.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-core-1.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/zookeeper.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xz.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xz-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xmlenc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xmlenc-0.52.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xml-apis.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xml-apis-1.3.04.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xercesImpl.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xercesImpl-2.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/stax-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/stax-api-1.0-2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/snappy-java.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/snappy-java-1.0.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-log4j12.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-api-1.7.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/servlet-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/servlet-api-2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/protobuf-java.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/protobuf-java-2.5.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/paranamer.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/paranamer-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/netty.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/netty-3.10.5.Final.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/log4j.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/log4j-1.2.17.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/leveldbjni-all.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/leveldbjni-all-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jsr305.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jsr305-3.0.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jetty-util.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jetty-util-6.1.26.cloudera.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-core.jar
17/08/01 14:49:28 INFO session.SessionState: Created local directory: /tmp/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2_resources
17/08/01 14:49:28 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2
17/08/01 14:49:28 INFO session.SessionState: Created local directory: /tmp/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2
17/08/01 14:49:28 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2/_tmp_space.db
17/08/01 14:49:28 INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
17/08/01 14:49:28 INFO client.HiveClientImpl: Warehouse location for Hive client (version 1.1.0) is /user/hive/warehouse
17/08/01 14:49:29 INFO hive.metastore: Trying to connect to metastore with URI thrift://ip-10-0-0-8.ec2.internal:9083
17/08/01 14:49:29 INFO hive.metastore: Opened a connection to metastore, current connections: 1
17/08/01 14:49:29 INFO hive.metastore: Connected to metastore.
17/08/01 14:49:29 WARN streaming.StreamingContext: Dynamic Allocation is enabled for this application. Enabling Dynamic allocation for Spark Streaming applications can cause data loss if Write Ahead Log is not enabled for non-replayable sources like Flume. See the programming guide for details on how to enable the Write Ahead Log.
17/08/01 14:49:29 INFO spark.SparkContext: Starting job: start at NativeMethodAccessorImpl.java:0
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Registering RDD 1 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Got job 0 (start at NativeMethodAccessorImpl.java:0) with 20 output partitions
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/08/01 14:49:29 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 3.1 KB, free 1060.5 MB)
17/08/01 14:49:29 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 1988.0 B, free 1060.5 MB)
17/08/01 14:49:29 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.0.0.199:53355 (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:49:29 INFO spark.SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Submitting 50 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO cluster.YarnScheduler: Adding task set 0.0 with 50 tasks
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, ip-10-0-0-229.ec2.internal, executor 1, partition 1, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 664 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, ip-10-0-0-229.ec2.internal, executor 1, partition 2, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 78 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3, ip-10-0-0-229.ec2.internal, executor 1, partition 3, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 68 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 0.0 (TID 4, ip-10-0-0-229.ec2.internal, executor 1, partition 4, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 65 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 0.0 (TID 5, ip-10-0-0-229.ec2.internal, executor 1, partition 5, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 59 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/50)
17/08/01 14:49:30 INFO spark.ExecutorAllocationManager: Requesting 1 new executor because tasks are backlogged (new desired total will be 2)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 0.0 (TID 6, ip-10-0-0-229.ec2.internal, executor 1, partition 6, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 0.0 (TID 7, ip-10-0-0-229.ec2.internal, executor 1, partition 7, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 60 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 0.0 (TID 8, ip-10-0-0-229.ec2.internal, executor 1, partition 8, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 0.0 (TID 7) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 0.0 (TID 9, ip-10-0-0-229.ec2.internal, executor 1, partition 9, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 0.0 (TID 8) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 0.0 (TID 10, ip-10-0-0-229.ec2.internal, executor 1, partition 10, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 0.0 (TID 9) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 0.0 (TID 11, ip-10-0-0-229.ec2.internal, executor 1, partition 11, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 0.0 (TID 10) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 0.0 (TID 12, ip-10-0-0-229.ec2.internal, executor 1, partition 12, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 0.0 (TID 11) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 0.0 (TID 13, ip-10-0-0-229.ec2.internal, executor 1, partition 13, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 0.0 (TID 12) in 60 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 0.0 (TID 14, ip-10-0-0-229.ec2.internal, executor 1, partition 14, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 0.0 (TID 13) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 0.0 (TID 15, ip-10-0-0-229.ec2.internal, executor 1, partition 15, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 0.0 (TID 14) in 51 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 0.0 (TID 16, ip-10-0-0-229.ec2.internal, executor 1, partition 16, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 0.0 (TID 15) in 63 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 0.0 (TID 17, ip-10-0-0-229.ec2.internal, executor 1, partition 17, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 0.0 (TID 16) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 0.0 (TID 18, ip-10-0-0-229.ec2.internal, executor 1, partition 18, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 0.0 (TID 17) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 0.0 (TID 19, ip-10-0-0-229.ec2.internal, executor 1, partition 19, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 0.0 (TID 18) in 56 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 0.0 (TID 20, ip-10-0-0-229.ec2.internal, executor 1, partition 20, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 0.0 (TID 19) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 0.0 (TID 21, ip-10-0-0-229.ec2.internal, executor 1, partition 21, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 0.0 (TID 20) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (21/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 0.0 (TID 22, ip-10-0-0-229.ec2.internal, executor 1, partition 22, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 0.0 (TID 21) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (22/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 0.0 (TID 23, ip-10-0-0-229.ec2.internal, executor 1, partition 23, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 0.0 (TID 22) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (23/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 0.0 (TID 24, ip-10-0-0-229.ec2.internal, executor 1, partition 24, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 0.0 (TID 23) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (24/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 0.0 (TID 25, ip-10-0-0-229.ec2.internal, executor 1, partition 25, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 0.0 (TID 24) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (25/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 0.0 (TID 26, ip-10-0-0-229.ec2.internal, executor 1, partition 26, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 0.0 (TID 25) in 46 ms on ip-10-0-0-229.ec2.internal (executor 1) (26/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 0.0 (TID 27, ip-10-0-0-229.ec2.internal, executor 1, partition 27, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 0.0 (TID 26) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (27/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 0.0 (TID 28, ip-10-0-0-229.ec2.internal, executor 1, partition 28, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 0.0 (TID 27) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (28/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 0.0 (TID 29, ip-10-0-0-229.ec2.internal, executor 1, partition 29, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 0.0 (TID 28) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (29/50)
17/08/01 14:49:31 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 4)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 0.0 (TID 30, ip-10-0-0-229.ec2.internal, executor 1, partition 30, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 0.0 (TID 29) in 42 ms on ip-10-0-0-229.ec2.internal (executor 1) (30/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 0.0 (TID 31, ip-10-0-0-229.ec2.internal, executor 1, partition 31, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 0.0 (TID 30) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (31/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 0.0 (TID 32, ip-10-0-0-229.ec2.internal, executor 1, partition 32, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 0.0 (TID 31) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (32/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 0.0 (TID 33, ip-10-0-0-229.ec2.internal, executor 1, partition 33, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 0.0 (TID 32) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (33/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 0.0 (TID 34, ip-10-0-0-229.ec2.internal, executor 1, partition 34, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 0.0 (TID 33) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (34/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 0.0 (TID 35, ip-10-0-0-229.ec2.internal, executor 1, partition 35, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 0.0 (TID 34) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (35/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 0.0 (TID 36, ip-10-0-0-229.ec2.internal, executor 1, partition 36, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 0.0 (TID 35) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (36/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 0.0 (TID 37, ip-10-0-0-229.ec2.internal, executor 1, partition 37, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 0.0 (TID 36) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (37/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 0.0 (TID 38, ip-10-0-0-229.ec2.internal, executor 1, partition 38, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 0.0 (TID 37) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (38/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 0.0 (TID 39, ip-10-0-0-229.ec2.internal, executor 1, partition 39, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 0.0 (TID 38) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (39/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 0.0 (TID 40, ip-10-0-0-229.ec2.internal, executor 1, partition 40, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 0.0 (TID 39) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (40/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 0.0 (TID 41, ip-10-0-0-229.ec2.internal, executor 1, partition 41, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 0.0 (TID 40) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (41/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 0.0 (TID 42, ip-10-0-0-229.ec2.internal, executor 1, partition 42, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 0.0 (TID 41) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (42/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 0.0 (TID 43, ip-10-0-0-229.ec2.internal, executor 1, partition 43, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 0.0 (TID 42) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (43/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 0.0 (TID 44, ip-10-0-0-229.ec2.internal, executor 1, partition 44, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 0.0 (TID 43) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (44/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 0.0 (TID 45, ip-10-0-0-229.ec2.internal, executor 1, partition 45, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 0.0 (TID 44) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (45/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 0.0 (TID 46, ip-10-0-0-229.ec2.internal, executor 1, partition 46, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 0.0 (TID 45) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (46/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 0.0 (TID 47, ip-10-0-0-229.ec2.internal, executor 1, partition 47, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 0.0 (TID 46) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (47/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 0.0 (TID 48, ip-10-0-0-229.ec2.internal, executor 1, partition 48, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 0.0 (TID 47) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (48/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 0.0 (TID 49, ip-10-0-0-229.ec2.internal, executor 1, partition 49, PROCESS_LOCAL, 6676 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 0.0 (TID 48) in 28 ms on ip-10-0-0-229.ec2.internal (executor 1) (49/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 0.0 (TID 49) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (50/50)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/08/01 14:49:32 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (start at NativeMethodAccessorImpl.java:0) finished in 2.517 s
17/08/01 14:49:32 INFO scheduler.DAGScheduler: looking for newly runnable stages
17/08/01 14:49:32 INFO scheduler.DAGScheduler: running: Set()
17/08/01 14:49:32 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 1)
17/08/01 14:49:32 INFO scheduler.DAGScheduler: failed: Set()
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/08/01 14:49:32 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.2 KB, free 1060.5 MB)
17/08/01 14:49:32 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 1956.0 B, free 1060.5 MB)
17/08/01 14:49:32 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.0.0.199:53355 (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:49:32 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Submitting 20 missing tasks from ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Adding task set 1.0 with 20 tasks
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 50, ip-10-0-0-229.ec2.internal, executor 1, partition 0, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:49:32 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.0.0.229:37096
17/08/01 14:49:32 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 283 bytes
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.0 (TID 51, ip-10-0-0-229.ec2.internal, executor 1, partition 1, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 50) in 89 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 1.0 (TID 52, ip-10-0-0-229.ec2.internal, executor 1, partition 2, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 1.0 (TID 51) in 23 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 1.0 (TID 53, ip-10-0-0-229.ec2.internal, executor 1, partition 3, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 1.0 (TID 52) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 1.0 (TID 54, ip-10-0-0-229.ec2.internal, executor 1, partition 4, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 1.0 (TID 53) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 1.0 (TID 55, ip-10-0-0-229.ec2.internal, executor 1, partition 5, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 1.0 (TID 54) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 1.0 (TID 56, ip-10-0-0-229.ec2.internal, executor 1, partition 6, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 1.0 (TID 55) in 25 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 1.0 (TID 57, ip-10-0-0-229.ec2.internal, executor 1, partition 7, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 1.0 (TID 56) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 1.0 (TID 58, ip-10-0-0-229.ec2.internal, executor 1, partition 8, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 1.0 (TID 57) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 1.0 (TID 59, ip-10-0-0-229.ec2.internal, executor 1, partition 9, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 1.0 (TID 58) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 1.0 (TID 60, ip-10-0-0-229.ec2.internal, executor 1, partition 10, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 1.0 (TID 59) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 1.0 (TID 61, ip-10-0-0-229.ec2.internal, executor 1, partition 11, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 1.0 (TID 60) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 1.0 (TID 62, ip-10-0-0-229.ec2.internal, executor 1, partition 12, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 1.0 (TID 61) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 1.0 (TID 63, ip-10-0-0-229.ec2.internal, executor 1, partition 13, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 1.0 (TID 62) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 1.0 (TID 64, ip-10-0-0-229.ec2.internal, executor 1, partition 14, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 1.0 (TID 63) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 1.0 (TID 65, ip-10-0-0-229.ec2.internal, executor 1, partition 15, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 1.0 (TID 64) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 1.0 (TID 66, ip-10-0-0-229.ec2.internal, executor 1, partition 16, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 1.0 (TID 65) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 1.0 (TID 67, ip-10-0-0-229.ec2.internal, executor 1, partition 17, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 1.0 (TID 66) in 22 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 1.0 (TID 68, ip-10-0-0-229.ec2.internal, executor 1, partition 18, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 1.0 (TID 67) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 1.0 (TID 69, ip-10-0-0-229.ec2.internal, executor 1, partition 19, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 1.0 (TID 68) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 1.0 (TID 69) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/20)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool
17/08/01 14:49:32 INFO scheduler.DAGScheduler: ResultStage 1 (start at NativeMethodAccessorImpl.java:0) finished in 0.420 s
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Job 0 finished: start at NativeMethodAccessorImpl.java:0, took 3.286639 s
17/08/01 14:49:32 INFO scheduler.ReceiverTracker: Starting 1 receivers
17/08/01 14:49:32 INFO scheduler.ReceiverTracker: ReceiverTracker started
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Initialized and validated org.apache.spark.streaming.kafka.KafkaInputDStream@71871773
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Initialized and validated org.apache.spark.streaming.api.python.PythonTransformedDStream@6593d619
17/08/01 14:49:32 INFO dstream.ForEachDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO dstream.ForEachDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO dstream.ForEachDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO dstream.ForEachDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO dstream.ForEachDStream: Initialized and validated org.apache.spark.streaming.dstream.ForEachDStream@26c8ea4b
17/08/01 14:49:33 INFO util.RecurringTimer: Started timer for JobGenerator at time 1501598973000
17/08/01 14:49:33 INFO scheduler.JobGenerator: Started JobGenerator at 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Started JobScheduler
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@625d3c2d{/streaming,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@48a256f9{/streaming/json,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@39edaf54{/streaming/batch,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@648a544d{/streaming/batch/json,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@c773c0d{/static/streaming,null,AVAILABLE}
17/08/01 14:49:33 INFO streaming.StreamingContext: StreamingContext started
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Got job 1 (start at NativeMethodAccessorImpl.java:0) with 1 output partitions
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Final stage: ResultStage 2 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Missing parents: List()
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Submitting ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620), which has no missing parents
17/08/01 14:49:33 INFO scheduler.ReceiverTracker: Receiver 0 started
17/08/01 14:49:33 INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 80.8 KB, free 1060.4 MB)
17/08/01 14:49:33 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 29.1 KB, free 1060.4 MB)
17/08/01 14:49:33 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.0.0.199:53355 (size: 29.1 KB, free: 1060.5 MB)
17/08/01 14:49:33 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620)
17/08/01 14:49:33 INFO cluster.YarnScheduler: Adding task set 2.0 with 1 tasks
17/08/01 14:49:33 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 2.0 (TID 70, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 7067 bytes)
17/08/01 14:49:33 INFO scheduler.JobScheduler: Added jobs for time 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Starting job streaming job 1501598973000 ms.0 from job set of time 1501598973000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:33 INFO scheduler.JobScheduler: Finished job streaming job 1501598973000 ms.0 from job set of time 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Total delay: 0.102 s for time 1501598973000 ms (execution: 0.026 s)
17/08/01 14:49:33 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 29.1 KB, free: 1060.5 MB)
17/08/01 14:49:33 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/08/01 14:49:33 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/08/01 14:49:33 INFO scheduler.ReceiverTracker: Registered receiver for stream 0 from 10.0.0.229:37096
17/08/01 14:49:34 INFO scheduler.JobScheduler: Added jobs for time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.JobScheduler: Starting job streaming job 1501598974000 ms.0 from job set of time 1501598974000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:34 INFO scheduler.JobScheduler: Finished job streaming job 1501598974000 ms.0 from job set of time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598974000 ms (execution: 0.009 s)
17/08/01 14:49:34 INFO python.PythonRDD: Removing RDD 5 from persistence list
17/08/01 14:49:34 INFO storage.BlockManager: Removing RDD 5
17/08/01 14:49:34 INFO rdd.BlockRDD: Removing RDD 4 from persistence list
17/08/01 14:49:34 INFO storage.BlockManager: Removing RDD 4
17/08/01 14:49:34 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[4] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/08/01 14:49:34 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/08/01 14:49:34 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.127:44695) with ID 2
17/08/01 14:49:34 INFO spark.ExecutorAllocationManager: New executor 2 has registered (new total is 2)
17/08/01 14:49:34 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-127.ec2.internal:59267 with 1060.5 MB RAM, BlockManagerId(2, ip-10-0-0-127.ec2.internal, 59267, None)
17/08/01 14:49:35 INFO scheduler.JobScheduler: Added jobs for time 1501598975000 ms
17/08/01 14:49:35 INFO scheduler.JobScheduler: Starting job streaming job 1501598975000 ms.0 from job set of time 1501598975000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:35 INFO scheduler.JobScheduler: Finished job streaming job 1501598975000 ms.0 from job set of time 1501598975000 ms
17/08/01 14:49:35 INFO python.PythonRDD: Removing RDD 7 from persistence list
17/08/01 14:49:35 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501598975000 ms (execution: 0.009 s)
17/08/01 14:49:35 INFO rdd.BlockRDD: Removing RDD 6 from persistence list
17/08/01 14:49:35 INFO storage.BlockManager: Removing RDD 6
17/08/01 14:49:35 INFO storage.BlockManager: Removing RDD 7
17/08/01 14:49:35 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[6] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598975000 ms
17/08/01 14:49:35 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598973000 ms
17/08/01 14:49:35 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598973000 ms
17/08/01 14:49:36 INFO scheduler.JobScheduler: Added jobs for time 1501598976000 ms
17/08/01 14:49:36 INFO scheduler.JobScheduler: Starting job streaming job 1501598976000 ms.0 from job set of time 1501598976000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:36 INFO scheduler.JobScheduler: Finished job streaming job 1501598976000 ms.0 from job set of time 1501598976000 ms
17/08/01 14:49:36 INFO python.PythonRDD: Removing RDD 9 from persistence list
17/08/01 14:49:36 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501598976000 ms (execution: 0.008 s)
17/08/01 14:49:36 INFO storage.BlockManager: Removing RDD 9
17/08/01 14:49:36 INFO rdd.BlockRDD: Removing RDD 8 from persistence list
17/08/01 14:49:36 INFO storage.BlockManager: Removing RDD 8
17/08/01 14:49:36 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[8] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598976000 ms
17/08/01 14:49:36 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598974000 ms
17/08/01 14:49:36 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598974000 ms
17/08/01 14:49:37 INFO scheduler.JobScheduler: Added jobs for time 1501598977000 ms
17/08/01 14:49:37 INFO scheduler.JobScheduler: Starting job streaming job 1501598977000 ms.0 from job set of time 1501598977000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:37 INFO scheduler.JobScheduler: Finished job streaming job 1501598977000 ms.0 from job set of time 1501598977000 ms
17/08/01 14:49:37 INFO python.PythonRDD: Removing RDD 11 from persistence list
17/08/01 14:49:37 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598977000 ms (execution: 0.008 s)
17/08/01 14:49:37 INFO storage.BlockManager: Removing RDD 11
17/08/01 14:49:37 INFO rdd.BlockRDD: Removing RDD 10 from persistence list
17/08/01 14:49:37 INFO storage.BlockManager: Removing RDD 10
17/08/01 14:49:37 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[10] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598977000 ms
17/08/01 14:49:37 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598975000 ms
17/08/01 14:49:37 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598975000 ms
17/08/01 14:49:38 INFO scheduler.JobScheduler: Added jobs for time 1501598978000 ms
17/08/01 14:49:38 INFO scheduler.JobScheduler: Starting job streaming job 1501598978000 ms.0 from job set of time 1501598978000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:38 INFO scheduler.JobScheduler: Finished job streaming job 1501598978000 ms.0 from job set of time 1501598978000 ms
17/08/01 14:49:38 INFO python.PythonRDD: Removing RDD 13 from persistence list
17/08/01 14:49:38 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501598978000 ms (execution: 0.007 s)
17/08/01 14:49:38 INFO storage.BlockManager: Removing RDD 13
17/08/01 14:49:38 INFO rdd.BlockRDD: Removing RDD 12 from persistence list
17/08/01 14:49:38 INFO storage.BlockManager: Removing RDD 12
17/08/01 14:49:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[12] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598978000 ms
17/08/01 14:49:38 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598976000 ms
17/08/01 14:49:38 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598976000 ms
17/08/01 14:49:39 INFO scheduler.JobScheduler: Added jobs for time 1501598979000 ms
17/08/01 14:49:39 INFO scheduler.JobScheduler: Starting job streaming job 1501598979000 ms.0 from job set of time 1501598979000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:39 INFO scheduler.JobScheduler: Finished job streaming job 1501598979000 ms.0 from job set of time 1501598979000 ms
17/08/01 14:49:39 INFO python.PythonRDD: Removing RDD 15 from persistence list
17/08/01 14:49:39 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598979000 ms (execution: 0.007 s)
17/08/01 14:49:39 INFO storage.BlockManager: Removing RDD 15
17/08/01 14:49:39 INFO rdd.BlockRDD: Removing RDD 14 from persistence list
17/08/01 14:49:39 INFO storage.BlockManager: Removing RDD 14
17/08/01 14:49:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[14] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598979000 ms
17/08/01 14:49:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598977000 ms
17/08/01 14:49:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598977000 ms
17/08/01 14:49:40 INFO scheduler.JobScheduler: Added jobs for time 1501598980000 ms
17/08/01 14:49:40 INFO scheduler.JobScheduler: Starting job streaming job 1501598980000 ms.0 from job set of time 1501598980000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:40 INFO scheduler.JobScheduler: Finished job streaming job 1501598980000 ms.0 from job set of time 1501598980000 ms
17/08/01 14:49:40 INFO python.PythonRDD: Removing RDD 17 from persistence list
17/08/01 14:49:40 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598980000 ms (execution: 0.015 s)
17/08/01 14:49:40 INFO storage.BlockManager: Removing RDD 17
17/08/01 14:49:40 INFO rdd.BlockRDD: Removing RDD 16 from persistence list
17/08/01 14:49:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[16] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598980000 ms
17/08/01 14:49:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598978000 ms
17/08/01 14:49:40 INFO storage.BlockManager: Removing RDD 16
17/08/01 14:49:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598978000 ms
17/08/01 14:49:41 INFO scheduler.JobScheduler: Added jobs for time 1501598981000 ms
17/08/01 14:49:41 INFO scheduler.JobScheduler: Starting job streaming job 1501598981000 ms.0 from job set of time 1501598981000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:41 INFO scheduler.JobScheduler: Finished job streaming job 1501598981000 ms.0 from job set of time 1501598981000 ms
17/08/01 14:49:41 INFO python.PythonRDD: Removing RDD 19 from persistence list
17/08/01 14:49:41 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501598981000 ms (execution: 0.006 s)
17/08/01 14:49:41 INFO storage.BlockManager: Removing RDD 19
17/08/01 14:49:41 INFO rdd.BlockRDD: Removing RDD 18 from persistence list
17/08/01 14:49:41 INFO storage.BlockManager: Removing RDD 18
17/08/01 14:49:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[18] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598981000 ms
17/08/01 14:49:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598979000 ms
17/08/01 14:49:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598979000 ms
17/08/01 14:49:42 INFO scheduler.JobScheduler: Added jobs for time 1501598982000 ms
17/08/01 14:49:42 INFO scheduler.JobScheduler: Starting job streaming job 1501598982000 ms.0 from job set of time 1501598982000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:42 INFO scheduler.JobScheduler: Finished job streaming job 1501598982000 ms.0 from job set of time 1501598982000 ms
17/08/01 14:49:42 INFO python.PythonRDD: Removing RDD 21 from persistence list
17/08/01 14:49:42 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598982000 ms (execution: 0.008 s)
17/08/01 14:49:42 INFO storage.BlockManager: Removing RDD 21
17/08/01 14:49:42 INFO rdd.BlockRDD: Removing RDD 20 from persistence list
17/08/01 14:49:42 INFO storage.BlockManager: Removing RDD 20
17/08/01 14:49:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[20] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598982000 ms
17/08/01 14:49:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598980000 ms
17/08/01 14:49:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598980000 ms
17/08/01 14:49:43 INFO scheduler.JobScheduler: Added jobs for time 1501598983000 ms
17/08/01 14:49:43 INFO scheduler.JobScheduler: Starting job streaming job 1501598983000 ms.0 from job set of time 1501598983000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:43 INFO scheduler.JobScheduler: Finished job streaming job 1501598983000 ms.0 from job set of time 1501598983000 ms
17/08/01 14:49:43 INFO python.PythonRDD: Removing RDD 23 from persistence list
17/08/01 14:49:43 INFO scheduler.JobScheduler: Total delay: 0.036 s for time 1501598983000 ms (execution: 0.009 s)
17/08/01 14:49:43 INFO storage.BlockManager: Removing RDD 23
17/08/01 14:49:43 INFO rdd.BlockRDD: Removing RDD 22 from persistence list
17/08/01 14:49:43 INFO storage.BlockManager: Removing RDD 22
17/08/01 14:49:43 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[22] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598983000 ms
17/08/01 14:49:43 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598981000 ms
17/08/01 14:49:43 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598981000 ms
17/08/01 14:49:44 INFO scheduler.JobScheduler: Added jobs for time 1501598984000 ms
17/08/01 14:49:44 INFO scheduler.JobScheduler: Starting job streaming job 1501598984000 ms.0 from job set of time 1501598984000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:44 INFO scheduler.JobScheduler: Finished job streaming job 1501598984000 ms.0 from job set of time 1501598984000 ms
17/08/01 14:49:44 INFO python.PythonRDD: Removing RDD 25 from persistence list
17/08/01 14:49:44 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598984000 ms (execution: 0.007 s)
17/08/01 14:49:44 INFO storage.BlockManager: Removing RDD 25
17/08/01 14:49:44 INFO rdd.BlockRDD: Removing RDD 24 from persistence list
17/08/01 14:49:44 INFO storage.BlockManager: Removing RDD 24
17/08/01 14:49:44 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[24] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598984000 ms
17/08/01 14:49:44 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598982000 ms
17/08/01 14:49:44 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598982000 ms
17/08/01 14:49:45 INFO scheduler.JobScheduler: Added jobs for time 1501598985000 ms
17/08/01 14:49:45 INFO scheduler.JobScheduler: Starting job streaming job 1501598985000 ms.0 from job set of time 1501598985000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:45 INFO scheduler.JobScheduler: Finished job streaming job 1501598985000 ms.0 from job set of time 1501598985000 ms
17/08/01 14:49:45 INFO python.PythonRDD: Removing RDD 27 from persistence list
17/08/01 14:49:45 INFO scheduler.JobScheduler: Total delay: 0.033 s for time 1501598985000 ms (execution: 0.007 s)
17/08/01 14:49:45 INFO storage.BlockManager: Removing RDD 27
17/08/01 14:49:45 INFO rdd.BlockRDD: Removing RDD 26 from persistence list
17/08/01 14:49:45 INFO storage.BlockManager: Removing RDD 26
17/08/01 14:49:45 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[26] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598985000 ms
17/08/01 14:49:45 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598983000 ms
17/08/01 14:49:45 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598983000 ms
17/08/01 14:49:46 INFO scheduler.JobScheduler: Added jobs for time 1501598986000 ms
17/08/01 14:49:46 INFO scheduler.JobScheduler: Starting job streaming job 1501598986000 ms.0 from job set of time 1501598986000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:46 INFO scheduler.JobScheduler: Finished job streaming job 1501598986000 ms.0 from job set of time 1501598986000 ms
17/08/01 14:49:46 INFO python.PythonRDD: Removing RDD 29 from persistence list
17/08/01 14:49:46 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598986000 ms (execution: 0.008 s)
17/08/01 14:49:46 INFO storage.BlockManager: Removing RDD 29
17/08/01 14:49:46 INFO rdd.BlockRDD: Removing RDD 28 from persistence list
17/08/01 14:49:46 INFO storage.BlockManager: Removing RDD 28
17/08/01 14:49:46 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[28] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598986000 ms
17/08/01 14:49:46 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598984000 ms
17/08/01 14:49:46 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598984000 ms
17/08/01 14:49:47 INFO scheduler.JobScheduler: Added jobs for time 1501598987000 ms
17/08/01 14:49:47 INFO scheduler.JobScheduler: Starting job streaming job 1501598987000 ms.0 from job set of time 1501598987000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:47 INFO scheduler.JobScheduler: Finished job streaming job 1501598987000 ms.0 from job set of time 1501598987000 ms
17/08/01 14:49:47 INFO python.PythonRDD: Removing RDD 31 from persistence list
17/08/01 14:49:47 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598987000 ms (execution: 0.014 s)
17/08/01 14:49:47 INFO storage.BlockManager: Removing RDD 31
17/08/01 14:49:47 INFO rdd.BlockRDD: Removing RDD 30 from persistence list
17/08/01 14:49:47 INFO storage.BlockManager: Removing RDD 30
17/08/01 14:49:47 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[30] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598987000 ms
17/08/01 14:49:47 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598985000 ms
17/08/01 14:49:47 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598985000 ms
17/08/01 14:49:48 INFO scheduler.JobScheduler: Added jobs for time 1501598988000 ms
17/08/01 14:49:48 INFO scheduler.JobScheduler: Starting job streaming job 1501598988000 ms.0 from job set of time 1501598988000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:48 INFO scheduler.JobScheduler: Finished job streaming job 1501598988000 ms.0 from job set of time 1501598988000 ms
17/08/01 14:49:48 INFO python.PythonRDD: Removing RDD 33 from persistence list
17/08/01 14:49:48 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501598988000 ms (execution: 0.008 s)
17/08/01 14:49:48 INFO storage.BlockManager: Removing RDD 33
17/08/01 14:49:48 INFO rdd.BlockRDD: Removing RDD 32 from persistence list
17/08/01 14:49:48 INFO storage.BlockManager: Removing RDD 32
17/08/01 14:49:48 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[32] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598988000 ms
17/08/01 14:49:48 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598986000 ms
17/08/01 14:49:48 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598986000 ms
17/08/01 14:49:49 INFO scheduler.JobScheduler: Added jobs for time 1501598989000 ms
17/08/01 14:49:49 INFO scheduler.JobScheduler: Starting job streaming job 1501598989000 ms.0 from job set of time 1501598989000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:49 INFO scheduler.JobScheduler: Finished job streaming job 1501598989000 ms.0 from job set of time 1501598989000 ms
17/08/01 14:49:49 INFO python.PythonRDD: Removing RDD 35 from persistence list
17/08/01 14:49:49 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501598989000 ms (execution: 0.006 s)
17/08/01 14:49:49 INFO storage.BlockManager: Removing RDD 35
17/08/01 14:49:49 INFO rdd.BlockRDD: Removing RDD 34 from persistence list
17/08/01 14:49:49 INFO storage.BlockManager: Removing RDD 34
17/08/01 14:49:49 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[34] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598989000 ms
17/08/01 14:49:49 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598987000 ms
17/08/01 14:49:49 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598987000 ms
17/08/01 14:49:50 INFO scheduler.JobScheduler: Added jobs for time 1501598990000 ms
17/08/01 14:49:50 INFO scheduler.JobScheduler: Starting job streaming job 1501598990000 ms.0 from job set of time 1501598990000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:50 INFO scheduler.JobScheduler: Finished job streaming job 1501598990000 ms.0 from job set of time 1501598990000 ms
17/08/01 14:49:50 INFO python.PythonRDD: Removing RDD 37 from persistence list
17/08/01 14:49:50 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501598990000 ms (execution: 0.008 s)
17/08/01 14:49:50 INFO storage.BlockManager: Removing RDD 37
17/08/01 14:49:50 INFO rdd.BlockRDD: Removing RDD 36 from persistence list
17/08/01 14:49:50 INFO storage.BlockManager: Removing RDD 36
17/08/01 14:49:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[36] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598990000 ms
17/08/01 14:49:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598988000 ms
17/08/01 14:49:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598988000 ms
17/08/01 14:49:51 INFO scheduler.JobScheduler: Added jobs for time 1501598991000 ms
17/08/01 14:49:51 INFO scheduler.JobScheduler: Starting job streaming job 1501598991000 ms.0 from job set of time 1501598991000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:51 INFO scheduler.JobScheduler: Finished job streaming job 1501598991000 ms.0 from job set of time 1501598991000 ms
17/08/01 14:49:51 INFO python.PythonRDD: Removing RDD 39 from persistence list
17/08/01 14:49:51 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501598991000 ms (execution: 0.006 s)
17/08/01 14:49:51 INFO storage.BlockManager: Removing RDD 39
17/08/01 14:49:51 INFO rdd.BlockRDD: Removing RDD 38 from persistence list
17/08/01 14:49:51 INFO storage.BlockManager: Removing RDD 38
17/08/01 14:49:51 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[38] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598991000 ms
17/08/01 14:49:51 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598989000 ms
17/08/01 14:49:51 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598989000 ms
17/08/01 14:49:52 INFO scheduler.JobScheduler: Added jobs for time 1501598992000 ms
17/08/01 14:49:52 INFO scheduler.JobScheduler: Starting job streaming job 1501598992000 ms.0 from job set of time 1501598992000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:52 INFO scheduler.JobScheduler: Finished job streaming job 1501598992000 ms.0 from job set of time 1501598992000 ms
17/08/01 14:49:52 INFO python.PythonRDD: Removing RDD 41 from persistence list
17/08/01 14:49:52 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501598992000 ms (execution: 0.007 s)
17/08/01 14:49:52 INFO storage.BlockManager: Removing RDD 41
17/08/01 14:49:52 INFO rdd.BlockRDD: Removing RDD 40 from persistence list
17/08/01 14:49:52 INFO storage.BlockManager: Removing RDD 40
17/08/01 14:49:52 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[40] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598992000 ms
17/08/01 14:49:52 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598990000 ms
17/08/01 14:49:52 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598990000 ms
17/08/01 14:49:53 INFO scheduler.JobScheduler: Added jobs for time 1501598993000 ms
17/08/01 14:49:53 INFO scheduler.JobScheduler: Starting job streaming job 1501598993000 ms.0 from job set of time 1501598993000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:53 INFO scheduler.JobScheduler: Finished job streaming job 1501598993000 ms.0 from job set of time 1501598993000 ms
17/08/01 14:49:53 INFO python.PythonRDD: Removing RDD 43 from persistence list
17/08/01 14:49:53 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598993000 ms (execution: 0.008 s)
17/08/01 14:49:53 INFO storage.BlockManager: Removing RDD 43
17/08/01 14:49:53 INFO rdd.BlockRDD: Removing RDD 42 from persistence list
17/08/01 14:49:53 INFO storage.BlockManager: Removing RDD 42
17/08/01 14:49:53 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[42] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598993000 ms
17/08/01 14:49:53 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598991000 ms
17/08/01 14:49:53 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598991000 ms
17/08/01 14:49:54 INFO scheduler.JobScheduler: Added jobs for time 1501598994000 ms
17/08/01 14:49:54 INFO scheduler.JobScheduler: Starting job streaming job 1501598994000 ms.0 from job set of time 1501598994000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:54 INFO scheduler.JobScheduler: Finished job streaming job 1501598994000 ms.0 from job set of time 1501598994000 ms
17/08/01 14:49:54 INFO python.PythonRDD: Removing RDD 45 from persistence list
17/08/01 14:49:54 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501598994000 ms (execution: 0.006 s)
17/08/01 14:49:54 INFO storage.BlockManager: Removing RDD 45
17/08/01 14:49:54 INFO rdd.BlockRDD: Removing RDD 44 from persistence list
17/08/01 14:49:54 INFO storage.BlockManager: Removing RDD 44
17/08/01 14:49:54 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[44] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598994000 ms
17/08/01 14:49:54 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598992000 ms
17/08/01 14:49:54 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598992000 ms
17/08/01 14:49:55 INFO scheduler.JobScheduler: Added jobs for time 1501598995000 ms
17/08/01 14:49:55 INFO scheduler.JobScheduler: Starting job streaming job 1501598995000 ms.0 from job set of time 1501598995000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:55 INFO scheduler.JobScheduler: Finished job streaming job 1501598995000 ms.0 from job set of time 1501598995000 ms
17/08/01 14:49:55 INFO python.PythonRDD: Removing RDD 47 from persistence list
17/08/01 14:49:55 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598995000 ms (execution: 0.008 s)
17/08/01 14:49:55 INFO storage.BlockManager: Removing RDD 47
17/08/01 14:49:55 INFO rdd.BlockRDD: Removing RDD 46 from persistence list
17/08/01 14:49:55 INFO storage.BlockManager: Removing RDD 46
17/08/01 14:49:55 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[46] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598995000 ms
17/08/01 14:49:55 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598993000 ms
17/08/01 14:49:55 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598993000 ms
17/08/01 14:49:56 INFO scheduler.JobScheduler: Added jobs for time 1501598996000 ms
17/08/01 14:49:56 INFO scheduler.JobScheduler: Starting job streaming job 1501598996000 ms.0 from job set of time 1501598996000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:56 INFO scheduler.JobScheduler: Finished job streaming job 1501598996000 ms.0 from job set of time 1501598996000 ms
17/08/01 14:49:56 INFO python.PythonRDD: Removing RDD 49 from persistence list
17/08/01 14:49:56 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501598996000 ms (execution: 0.020 s)
17/08/01 14:49:56 INFO storage.BlockManager: Removing RDD 49
17/08/01 14:49:56 INFO rdd.BlockRDD: Removing RDD 48 from persistence list
17/08/01 14:49:56 INFO storage.BlockManager: Removing RDD 48
17/08/01 14:49:56 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[48] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598996000 ms
17/08/01 14:49:56 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598994000 ms
17/08/01 14:49:56 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598994000 ms
17/08/01 14:49:57 INFO scheduler.JobScheduler: Added jobs for time 1501598997000 ms
17/08/01 14:49:57 INFO scheduler.JobScheduler: Starting job streaming job 1501598997000 ms.0 from job set of time 1501598997000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:57 INFO scheduler.JobScheduler: Finished job streaming job 1501598997000 ms.0 from job set of time 1501598997000 ms
17/08/01 14:49:57 INFO python.PythonRDD: Removing RDD 51 from persistence list
17/08/01 14:49:57 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598997000 ms (execution: 0.007 s)
17/08/01 14:49:57 INFO storage.BlockManager: Removing RDD 51
17/08/01 14:49:57 INFO rdd.BlockRDD: Removing RDD 50 from persistence list
17/08/01 14:49:57 INFO storage.BlockManager: Removing RDD 50
17/08/01 14:49:57 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[50] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598997000 ms
17/08/01 14:49:57 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598995000 ms
17/08/01 14:49:57 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598995000 ms
17/08/01 14:49:58 INFO scheduler.JobScheduler: Added jobs for time 1501598998000 ms
17/08/01 14:49:58 INFO scheduler.JobScheduler: Starting job streaming job 1501598998000 ms.0 from job set of time 1501598998000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:58 INFO scheduler.JobScheduler: Finished job streaming job 1501598998000 ms.0 from job set of time 1501598998000 ms
17/08/01 14:49:58 INFO python.PythonRDD: Removing RDD 53 from persistence list
17/08/01 14:49:58 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501598998000 ms (execution: 0.008 s)
17/08/01 14:49:58 INFO storage.BlockManager: Removing RDD 53
17/08/01 14:49:58 INFO rdd.BlockRDD: Removing RDD 52 from persistence list
17/08/01 14:49:58 INFO storage.BlockManager: Removing RDD 52
17/08/01 14:49:58 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[52] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598998000 ms
17/08/01 14:49:58 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598996000 ms
17/08/01 14:49:58 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598996000 ms
17/08/01 14:49:59 INFO scheduler.JobScheduler: Added jobs for time 1501598999000 ms
17/08/01 14:49:59 INFO scheduler.JobScheduler: Starting job streaming job 1501598999000 ms.0 from job set of time 1501598999000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:59 INFO scheduler.JobScheduler: Finished job streaming job 1501598999000 ms.0 from job set of time 1501598999000 ms
17/08/01 14:49:59 INFO python.PythonRDD: Removing RDD 55 from persistence list
17/08/01 14:49:59 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598999000 ms (execution: 0.007 s)
17/08/01 14:49:59 INFO storage.BlockManager: Removing RDD 55
17/08/01 14:49:59 INFO rdd.BlockRDD: Removing RDD 54 from persistence list
17/08/01 14:49:59 INFO storage.BlockManager: Removing RDD 54
17/08/01 14:49:59 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[54] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598999000 ms
17/08/01 14:49:59 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598997000 ms
17/08/01 14:49:59 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598997000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Added jobs for time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Starting job streaming job 1501599000000 ms.0 from job set of time 1501599000000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:00 INFO scheduler.JobScheduler: Finished job streaming job 1501599000000 ms.0 from job set of time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599000000 ms (execution: 0.006 s)
17/08/01 14:50:00 INFO python.PythonRDD: Removing RDD 57 from persistence list
17/08/01 14:50:00 INFO storage.BlockManager: Removing RDD 57
17/08/01 14:50:00 INFO rdd.BlockRDD: Removing RDD 56 from persistence list
17/08/01 14:50:00 INFO storage.BlockManager: Removing RDD 56
17/08/01 14:50:00 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[56] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598998000 ms
17/08/01 14:50:00 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598998000 ms
17/08/01 14:50:01 INFO scheduler.JobScheduler: Added jobs for time 1501599001000 ms
17/08/01 14:50:01 INFO scheduler.JobScheduler: Starting job streaming job 1501599001000 ms.0 from job set of time 1501599001000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:01 INFO scheduler.JobScheduler: Finished job streaming job 1501599001000 ms.0 from job set of time 1501599001000 ms
17/08/01 14:50:01 INFO python.PythonRDD: Removing RDD 59 from persistence list
17/08/01 14:50:01 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599001000 ms (execution: 0.007 s)
17/08/01 14:50:01 INFO storage.BlockManager: Removing RDD 59
17/08/01 14:50:01 INFO rdd.BlockRDD: Removing RDD 58 from persistence list
17/08/01 14:50:01 INFO storage.BlockManager: Removing RDD 58
17/08/01 14:50:01 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[58] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599001000 ms
17/08/01 14:50:01 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598999000 ms
17/08/01 14:50:01 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598999000 ms
17/08/01 14:50:02 INFO scheduler.JobScheduler: Added jobs for time 1501599002000 ms
17/08/01 14:50:02 INFO scheduler.JobScheduler: Starting job streaming job 1501599002000 ms.0 from job set of time 1501599002000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:02 INFO scheduler.JobScheduler: Finished job streaming job 1501599002000 ms.0 from job set of time 1501599002000 ms
17/08/01 14:50:02 INFO python.PythonRDD: Removing RDD 61 from persistence list
17/08/01 14:50:02 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599002000 ms (execution: 0.006 s)
17/08/01 14:50:02 INFO storage.BlockManager: Removing RDD 61
17/08/01 14:50:02 INFO rdd.BlockRDD: Removing RDD 60 from persistence list
17/08/01 14:50:02 INFO storage.BlockManager: Removing RDD 60
17/08/01 14:50:02 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[60] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599002000 ms
17/08/01 14:50:02 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599000000 ms
17/08/01 14:50:02 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599000000 ms
17/08/01 14:50:03 INFO scheduler.JobScheduler: Added jobs for time 1501599003000 ms
17/08/01 14:50:03 INFO scheduler.JobScheduler: Starting job streaming job 1501599003000 ms.0 from job set of time 1501599003000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:03 INFO scheduler.JobScheduler: Finished job streaming job 1501599003000 ms.0 from job set of time 1501599003000 ms
17/08/01 14:50:03 INFO python.PythonRDD: Removing RDD 63 from persistence list
17/08/01 14:50:03 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599003000 ms (execution: 0.007 s)
17/08/01 14:50:03 INFO storage.BlockManager: Removing RDD 63
17/08/01 14:50:03 INFO rdd.BlockRDD: Removing RDD 62 from persistence list
17/08/01 14:50:03 INFO storage.BlockManager: Removing RDD 62
17/08/01 14:50:03 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[62] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599003000 ms
17/08/01 14:50:03 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599001000 ms
17/08/01 14:50:03 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599001000 ms
17/08/01 14:50:04 INFO scheduler.JobScheduler: Added jobs for time 1501599004000 ms
17/08/01 14:50:04 INFO scheduler.JobScheduler: Starting job streaming job 1501599004000 ms.0 from job set of time 1501599004000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:04 INFO scheduler.JobScheduler: Finished job streaming job 1501599004000 ms.0 from job set of time 1501599004000 ms
17/08/01 14:50:04 INFO python.PythonRDD: Removing RDD 65 from persistence list
17/08/01 14:50:04 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599004000 ms (execution: 0.006 s)
17/08/01 14:50:04 INFO storage.BlockManager: Removing RDD 65
17/08/01 14:50:04 INFO rdd.BlockRDD: Removing RDD 64 from persistence list
17/08/01 14:50:04 INFO storage.BlockManager: Removing RDD 64
17/08/01 14:50:04 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[64] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599004000 ms
17/08/01 14:50:04 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599002000 ms
17/08/01 14:50:04 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599002000 ms
17/08/01 14:50:05 INFO scheduler.JobScheduler: Added jobs for time 1501599005000 ms
17/08/01 14:50:05 INFO scheduler.JobScheduler: Starting job streaming job 1501599005000 ms.0 from job set of time 1501599005000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:05 INFO scheduler.JobScheduler: Finished job streaming job 1501599005000 ms.0 from job set of time 1501599005000 ms
17/08/01 14:50:05 INFO python.PythonRDD: Removing RDD 67 from persistence list
17/08/01 14:50:05 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501599005000 ms (execution: 0.013 s)
17/08/01 14:50:05 INFO storage.BlockManager: Removing RDD 67
17/08/01 14:50:05 INFO rdd.BlockRDD: Removing RDD 66 from persistence list
17/08/01 14:50:05 INFO storage.BlockManager: Removing RDD 66
17/08/01 14:50:05 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[66] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599005000 ms
17/08/01 14:50:05 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599003000 ms
17/08/01 14:50:05 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599003000 ms
17/08/01 14:50:06 INFO scheduler.JobScheduler: Added jobs for time 1501599006000 ms
17/08/01 14:50:06 INFO scheduler.JobScheduler: Starting job streaming job 1501599006000 ms.0 from job set of time 1501599006000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:06 INFO scheduler.JobScheduler: Finished job streaming job 1501599006000 ms.0 from job set of time 1501599006000 ms
17/08/01 14:50:06 INFO python.PythonRDD: Removing RDD 69 from persistence list
17/08/01 14:50:06 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599006000 ms (execution: 0.008 s)
17/08/01 14:50:06 INFO storage.BlockManager: Removing RDD 69
17/08/01 14:50:06 INFO rdd.BlockRDD: Removing RDD 68 from persistence list
17/08/01 14:50:06 INFO storage.BlockManager: Removing RDD 68
17/08/01 14:50:06 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[68] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599006000 ms
17/08/01 14:50:06 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599004000 ms
17/08/01 14:50:06 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599004000 ms
17/08/01 14:50:07 INFO scheduler.JobScheduler: Added jobs for time 1501599007000 ms
17/08/01 14:50:07 INFO scheduler.JobScheduler: Starting job streaming job 1501599007000 ms.0 from job set of time 1501599007000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:07 INFO scheduler.JobScheduler: Finished job streaming job 1501599007000 ms.0 from job set of time 1501599007000 ms
17/08/01 14:50:07 INFO python.PythonRDD: Removing RDD 71 from persistence list
17/08/01 14:50:07 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599007000 ms (execution: 0.009 s)
17/08/01 14:50:07 INFO rdd.BlockRDD: Removing RDD 70 from persistence list
17/08/01 14:50:07 INFO storage.BlockManager: Removing RDD 71
17/08/01 14:50:07 INFO storage.BlockManager: Removing RDD 70
17/08/01 14:50:07 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[70] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599007000 ms
17/08/01 14:50:07 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599005000 ms
17/08/01 14:50:07 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599005000 ms
17/08/01 14:50:08 INFO scheduler.JobScheduler: Added jobs for time 1501599008000 ms
17/08/01 14:50:08 INFO scheduler.JobScheduler: Starting job streaming job 1501599008000 ms.0 from job set of time 1501599008000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:08 INFO scheduler.JobScheduler: Finished job streaming job 1501599008000 ms.0 from job set of time 1501599008000 ms
17/08/01 14:50:08 INFO python.PythonRDD: Removing RDD 73 from persistence list
17/08/01 14:50:08 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599008000 ms (execution: 0.006 s)
17/08/01 14:50:08 INFO storage.BlockManager: Removing RDD 73
17/08/01 14:50:08 INFO rdd.BlockRDD: Removing RDD 72 from persistence list
17/08/01 14:50:08 INFO storage.BlockManager: Removing RDD 72
17/08/01 14:50:08 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[72] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599008000 ms
17/08/01 14:50:08 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599006000 ms
17/08/01 14:50:08 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599006000 ms
17/08/01 14:50:09 INFO scheduler.JobScheduler: Added jobs for time 1501599009000 ms
17/08/01 14:50:09 INFO scheduler.JobScheduler: Starting job streaming job 1501599009000 ms.0 from job set of time 1501599009000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:09 INFO scheduler.JobScheduler: Finished job streaming job 1501599009000 ms.0 from job set of time 1501599009000 ms
17/08/01 14:50:09 INFO python.PythonRDD: Removing RDD 75 from persistence list
17/08/01 14:50:09 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599009000 ms (execution: 0.005 s)
17/08/01 14:50:09 INFO storage.BlockManager: Removing RDD 75
17/08/01 14:50:09 INFO rdd.BlockRDD: Removing RDD 74 from persistence list
17/08/01 14:50:09 INFO storage.BlockManager: Removing RDD 74
17/08/01 14:50:09 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[74] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599009000 ms
17/08/01 14:50:09 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599007000 ms
17/08/01 14:50:09 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599007000 ms
17/08/01 14:50:10 INFO scheduler.JobScheduler: Added jobs for time 1501599010000 ms
17/08/01 14:50:10 INFO scheduler.JobScheduler: Starting job streaming job 1501599010000 ms.0 from job set of time 1501599010000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:10 INFO scheduler.JobScheduler: Finished job streaming job 1501599010000 ms.0 from job set of time 1501599010000 ms
17/08/01 14:50:10 INFO python.PythonRDD: Removing RDD 77 from persistence list
17/08/01 14:50:10 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599010000 ms (execution: 0.005 s)
17/08/01 14:50:10 INFO rdd.BlockRDD: Removing RDD 76 from persistence list
17/08/01 14:50:10 INFO storage.BlockManager: Removing RDD 76
17/08/01 14:50:10 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[76] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599010000 ms
17/08/01 14:50:10 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599008000 ms
17/08/01 14:50:10 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599008000 ms
17/08/01 14:50:10 INFO storage.BlockManager: Removing RDD 77
17/08/01 14:50:11 INFO scheduler.JobScheduler: Added jobs for time 1501599011000 ms
17/08/01 14:50:11 INFO scheduler.JobScheduler: Starting job streaming job 1501599011000 ms.0 from job set of time 1501599011000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:11 INFO scheduler.JobScheduler: Finished job streaming job 1501599011000 ms.0 from job set of time 1501599011000 ms
17/08/01 14:50:11 INFO python.PythonRDD: Removing RDD 79 from persistence list
17/08/01 14:50:11 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599011000 ms (execution: 0.007 s)
17/08/01 14:50:11 INFO storage.BlockManager: Removing RDD 79
17/08/01 14:50:11 INFO rdd.BlockRDD: Removing RDD 78 from persistence list
17/08/01 14:50:11 INFO storage.BlockManager: Removing RDD 78
17/08/01 14:50:11 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[78] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599011000 ms
17/08/01 14:50:11 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599009000 ms
17/08/01 14:50:11 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599009000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Added jobs for time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Starting job streaming job 1501599012000 ms.0 from job set of time 1501599012000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:12 INFO scheduler.JobScheduler: Finished job streaming job 1501599012000 ms.0 from job set of time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599012000 ms (execution: 0.015 s)
17/08/01 14:50:12 INFO python.PythonRDD: Removing RDD 81 from persistence list
17/08/01 14:50:12 INFO storage.BlockManager: Removing RDD 81
17/08/01 14:50:12 INFO rdd.BlockRDD: Removing RDD 80 from persistence list
17/08/01 14:50:12 INFO storage.BlockManager: Removing RDD 80
17/08/01 14:50:12 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[80] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599010000 ms
17/08/01 14:50:12 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599010000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Added jobs for time 1501599013000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Starting job streaming job 1501599013000 ms.0 from job set of time 1501599013000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:13 INFO scheduler.JobScheduler: Finished job streaming job 1501599013000 ms.0 from job set of time 1501599013000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501599013000 ms (execution: 0.008 s)
17/08/01 14:50:13 INFO python.PythonRDD: Removing RDD 83 from persistence list
17/08/01 14:50:13 INFO storage.BlockManager: Removing RDD 83
17/08/01 14:50:13 INFO rdd.BlockRDD: Removing RDD 82 from persistence list
17/08/01 14:50:13 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[82] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599013000 ms
17/08/01 14:50:13 INFO storage.BlockManager: Removing RDD 82
17/08/01 14:50:13 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599011000 ms
17/08/01 14:50:13 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599011000 ms
17/08/01 14:50:14 INFO scheduler.JobScheduler: Added jobs for time 1501599014000 ms
17/08/01 14:50:14 INFO scheduler.JobScheduler: Starting job streaming job 1501599014000 ms.0 from job set of time 1501599014000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:14 INFO scheduler.JobScheduler: Finished job streaming job 1501599014000 ms.0 from job set of time 1501599014000 ms
17/08/01 14:50:14 INFO python.PythonRDD: Removing RDD 85 from persistence list
17/08/01 14:50:14 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599014000 ms (execution: 0.007 s)
17/08/01 14:50:14 INFO rdd.BlockRDD: Removing RDD 84 from persistence list
17/08/01 14:50:14 INFO storage.BlockManager: Removing RDD 85
17/08/01 14:50:14 INFO storage.BlockManager: Removing RDD 84
17/08/01 14:50:14 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[84] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599014000 ms
17/08/01 14:50:14 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599012000 ms
17/08/01 14:50:14 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599012000 ms
17/08/01 14:50:15 INFO scheduler.JobScheduler: Added jobs for time 1501599015000 ms
17/08/01 14:50:15 INFO scheduler.JobScheduler: Starting job streaming job 1501599015000 ms.0 from job set of time 1501599015000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:15 INFO scheduler.JobScheduler: Finished job streaming job 1501599015000 ms.0 from job set of time 1501599015000 ms
17/08/01 14:50:15 INFO python.PythonRDD: Removing RDD 87 from persistence list
17/08/01 14:50:15 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599015000 ms (execution: 0.007 s)
17/08/01 14:50:15 INFO storage.BlockManager: Removing RDD 87
17/08/01 14:50:15 INFO rdd.BlockRDD: Removing RDD 86 from persistence list
17/08/01 14:50:15 INFO storage.BlockManager: Removing RDD 86
17/08/01 14:50:15 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[86] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599015000 ms
17/08/01 14:50:15 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599013000 ms
17/08/01 14:50:15 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599013000 ms
17/08/01 14:50:16 INFO scheduler.JobScheduler: Added jobs for time 1501599016000 ms
17/08/01 14:50:16 INFO scheduler.JobScheduler: Starting job streaming job 1501599016000 ms.0 from job set of time 1501599016000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:16 INFO scheduler.JobScheduler: Finished job streaming job 1501599016000 ms.0 from job set of time 1501599016000 ms
17/08/01 14:50:16 INFO python.PythonRDD: Removing RDD 89 from persistence list
17/08/01 14:50:16 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501599016000 ms (execution: 0.008 s)
17/08/01 14:50:16 INFO storage.BlockManager: Removing RDD 89
17/08/01 14:50:16 INFO rdd.BlockRDD: Removing RDD 88 from persistence list
17/08/01 14:50:16 INFO storage.BlockManager: Removing RDD 88
17/08/01 14:50:16 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[88] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599016000 ms
17/08/01 14:50:16 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599014000 ms
17/08/01 14:50:16 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599014000 ms
17/08/01 14:50:17 INFO scheduler.JobScheduler: Added jobs for time 1501599017000 ms
17/08/01 14:50:17 INFO scheduler.JobScheduler: Starting job streaming job 1501599017000 ms.0 from job set of time 1501599017000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:17 INFO scheduler.JobScheduler: Finished job streaming job 1501599017000 ms.0 from job set of time 1501599017000 ms
17/08/01 14:50:17 INFO python.PythonRDD: Removing RDD 91 from persistence list
17/08/01 14:50:17 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501599017000 ms (execution: 0.008 s)
17/08/01 14:50:17 INFO storage.BlockManager: Removing RDD 91
17/08/01 14:50:17 INFO rdd.BlockRDD: Removing RDD 90 from persistence list
17/08/01 14:50:17 INFO storage.BlockManager: Removing RDD 90
17/08/01 14:50:17 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[90] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599017000 ms
17/08/01 14:50:17 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599015000 ms
17/08/01 14:50:17 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599015000 ms
17/08/01 14:50:18 INFO scheduler.JobScheduler: Added jobs for time 1501599018000 ms
17/08/01 14:50:18 INFO scheduler.JobScheduler: Starting job streaming job 1501599018000 ms.0 from job set of time 1501599018000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:18 INFO scheduler.JobScheduler: Finished job streaming job 1501599018000 ms.0 from job set of time 1501599018000 ms
17/08/01 14:50:18 INFO python.PythonRDD: Removing RDD 93 from persistence list
17/08/01 14:50:18 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599018000 ms (execution: 0.006 s)
17/08/01 14:50:18 INFO storage.BlockManager: Removing RDD 93
17/08/01 14:50:18 INFO rdd.BlockRDD: Removing RDD 92 from persistence list
17/08/01 14:50:18 INFO storage.BlockManager: Removing RDD 92
17/08/01 14:50:18 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[92] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599018000 ms
17/08/01 14:50:18 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599016000 ms
17/08/01 14:50:18 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599016000 ms
17/08/01 14:50:19 INFO scheduler.JobScheduler: Added jobs for time 1501599019000 ms
17/08/01 14:50:19 INFO scheduler.JobScheduler: Starting job streaming job 1501599019000 ms.0 from job set of time 1501599019000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:19 INFO scheduler.JobScheduler: Finished job streaming job 1501599019000 ms.0 from job set of time 1501599019000 ms
17/08/01 14:50:19 INFO python.PythonRDD: Removing RDD 95 from persistence list
17/08/01 14:50:19 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599019000 ms (execution: 0.015 s)
17/08/01 14:50:19 INFO storage.BlockManager: Removing RDD 95
17/08/01 14:50:19 INFO rdd.BlockRDD: Removing RDD 94 from persistence list
17/08/01 14:50:19 INFO storage.BlockManager: Removing RDD 94
17/08/01 14:50:19 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[94] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599019000 ms
17/08/01 14:50:19 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599017000 ms
17/08/01 14:50:19 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599017000 ms
17/08/01 14:50:20 INFO scheduler.JobScheduler: Added jobs for time 1501599020000 ms
17/08/01 14:50:20 INFO scheduler.JobScheduler: Starting job streaming job 1501599020000 ms.0 from job set of time 1501599020000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:20 INFO scheduler.JobScheduler: Finished job streaming job 1501599020000 ms.0 from job set of time 1501599020000 ms
17/08/01 14:50:20 INFO python.PythonRDD: Removing RDD 97 from persistence list
17/08/01 14:50:20 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599020000 ms (execution: 0.006 s)
17/08/01 14:50:20 INFO storage.BlockManager: Removing RDD 97
17/08/01 14:50:20 INFO rdd.BlockRDD: Removing RDD 96 from persistence list
17/08/01 14:50:20 INFO storage.BlockManager: Removing RDD 96
17/08/01 14:50:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[96] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599020000 ms
17/08/01 14:50:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599018000 ms
17/08/01 14:50:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599018000 ms
17/08/01 14:50:21 INFO scheduler.JobScheduler: Added jobs for time 1501599021000 ms
17/08/01 14:50:21 INFO scheduler.JobScheduler: Starting job streaming job 1501599021000 ms.0 from job set of time 1501599021000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:21 INFO scheduler.JobScheduler: Finished job streaming job 1501599021000 ms.0 from job set of time 1501599021000 ms
17/08/01 14:50:21 INFO python.PythonRDD: Removing RDD 99 from persistence list
17/08/01 14:50:21 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599021000 ms (execution: 0.007 s)
17/08/01 14:50:21 INFO storage.BlockManager: Removing RDD 99
17/08/01 14:50:21 INFO rdd.BlockRDD: Removing RDD 98 from persistence list
17/08/01 14:50:21 INFO storage.BlockManager: Removing RDD 98
17/08/01 14:50:21 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[98] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599021000 ms
17/08/01 14:50:21 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599019000 ms
17/08/01 14:50:21 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599019000 ms
17/08/01 14:50:22 INFO scheduler.JobScheduler: Added jobs for time 1501599022000 ms
17/08/01 14:50:22 INFO scheduler.JobScheduler: Starting job streaming job 1501599022000 ms.0 from job set of time 1501599022000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:22 INFO scheduler.JobScheduler: Finished job streaming job 1501599022000 ms.0 from job set of time 1501599022000 ms
17/08/01 14:50:22 INFO python.PythonRDD: Removing RDD 101 from persistence list
17/08/01 14:50:22 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501599022000 ms (execution: 0.007 s)
17/08/01 14:50:22 INFO storage.BlockManager: Removing RDD 101
17/08/01 14:50:22 INFO rdd.BlockRDD: Removing RDD 100 from persistence list
17/08/01 14:50:22 INFO storage.BlockManager: Removing RDD 100
17/08/01 14:50:22 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[100] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599022000 ms
17/08/01 14:50:22 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599020000 ms
17/08/01 14:50:22 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599020000 ms
17/08/01 14:50:23 INFO scheduler.JobScheduler: Added jobs for time 1501599023000 ms
17/08/01 14:50:23 INFO scheduler.JobScheduler: Starting job streaming job 1501599023000 ms.0 from job set of time 1501599023000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:23 INFO scheduler.JobScheduler: Finished job streaming job 1501599023000 ms.0 from job set of time 1501599023000 ms
17/08/01 14:50:23 INFO python.PythonRDD: Removing RDD 103 from persistence list
17/08/01 14:50:23 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599023000 ms (execution: 0.007 s)
17/08/01 14:50:23 INFO storage.BlockManager: Removing RDD 103
17/08/01 14:50:23 INFO rdd.BlockRDD: Removing RDD 102 from persistence list
17/08/01 14:50:23 INFO storage.BlockManager: Removing RDD 102
17/08/01 14:50:23 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[102] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599023000 ms
17/08/01 14:50:23 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599021000 ms
17/08/01 14:50:23 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599021000 ms
17/08/01 14:50:24 INFO scheduler.JobScheduler: Added jobs for time 1501599024000 ms
17/08/01 14:50:24 INFO scheduler.JobScheduler: Starting job streaming job 1501599024000 ms.0 from job set of time 1501599024000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:24 INFO scheduler.JobScheduler: Finished job streaming job 1501599024000 ms.0 from job set of time 1501599024000 ms
17/08/01 14:50:24 INFO python.PythonRDD: Removing RDD 105 from persistence list
17/08/01 14:50:24 INFO scheduler.JobScheduler: Total delay: 0.035 s for time 1501599024000 ms (execution: 0.007 s)
17/08/01 14:50:24 INFO storage.BlockManager: Removing RDD 105
17/08/01 14:50:24 INFO rdd.BlockRDD: Removing RDD 104 from persistence list
17/08/01 14:50:24 INFO storage.BlockManager: Removing RDD 104
17/08/01 14:50:24 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[104] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599024000 ms
17/08/01 14:50:24 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599022000 ms
17/08/01 14:50:24 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599022000 ms
17/08/01 14:50:25 INFO scheduler.JobScheduler: Added jobs for time 1501599025000 ms
17/08/01 14:50:25 INFO scheduler.JobScheduler: Starting job streaming job 1501599025000 ms.0 from job set of time 1501599025000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:25 INFO scheduler.JobScheduler: Finished job streaming job 1501599025000 ms.0 from job set of time 1501599025000 ms
17/08/01 14:50:25 INFO python.PythonRDD: Removing RDD 107 from persistence list
17/08/01 14:50:25 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599025000 ms (execution: 0.007 s)
17/08/01 14:50:25 INFO storage.BlockManager: Removing RDD 107
17/08/01 14:50:25 INFO rdd.BlockRDD: Removing RDD 106 from persistence list
17/08/01 14:50:25 INFO storage.BlockManager: Removing RDD 106
17/08/01 14:50:25 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[106] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599025000 ms
17/08/01 14:50:25 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599023000 ms
17/08/01 14:50:25 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599023000 ms
17/08/01 14:50:26 INFO scheduler.JobScheduler: Added jobs for time 1501599026000 ms
17/08/01 14:50:26 INFO scheduler.JobScheduler: Starting job streaming job 1501599026000 ms.0 from job set of time 1501599026000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:26 INFO scheduler.JobScheduler: Finished job streaming job 1501599026000 ms.0 from job set of time 1501599026000 ms
17/08/01 14:50:26 INFO python.PythonRDD: Removing RDD 109 from persistence list
17/08/01 14:50:26 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599026000 ms (execution: 0.007 s)
17/08/01 14:50:26 INFO storage.BlockManager: Removing RDD 109
17/08/01 14:50:26 INFO rdd.BlockRDD: Removing RDD 108 from persistence list
17/08/01 14:50:26 INFO storage.BlockManager: Removing RDD 108
17/08/01 14:50:26 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[108] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599026000 ms
17/08/01 14:50:26 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599024000 ms
17/08/01 14:50:26 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599024000 ms
17/08/01 14:50:27 INFO scheduler.JobScheduler: Added jobs for time 1501599027000 ms
17/08/01 14:50:27 INFO scheduler.JobScheduler: Starting job streaming job 1501599027000 ms.0 from job set of time 1501599027000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:27 INFO scheduler.JobScheduler: Finished job streaming job 1501599027000 ms.0 from job set of time 1501599027000 ms
17/08/01 14:50:27 INFO python.PythonRDD: Removing RDD 111 from persistence list
17/08/01 14:50:27 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599027000 ms (execution: 0.007 s)
17/08/01 14:50:27 INFO storage.BlockManager: Removing RDD 111
17/08/01 14:50:27 INFO rdd.BlockRDD: Removing RDD 110 from persistence list
17/08/01 14:50:27 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[110] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599027000 ms
17/08/01 14:50:27 INFO storage.BlockManager: Removing RDD 110
17/08/01 14:50:27 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599025000 ms
17/08/01 14:50:27 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599025000 ms
17/08/01 14:50:28 INFO scheduler.JobScheduler: Added jobs for time 1501599028000 ms
17/08/01 14:50:28 INFO scheduler.JobScheduler: Starting job streaming job 1501599028000 ms.0 from job set of time 1501599028000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:28 INFO scheduler.JobScheduler: Finished job streaming job 1501599028000 ms.0 from job set of time 1501599028000 ms
17/08/01 14:50:28 INFO python.PythonRDD: Removing RDD 113 from persistence list
17/08/01 14:50:28 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599028000 ms (execution: 0.017 s)
17/08/01 14:50:28 INFO storage.BlockManager: Removing RDD 113
17/08/01 14:50:28 INFO rdd.BlockRDD: Removing RDD 112 from persistence list
17/08/01 14:50:28 INFO storage.BlockManager: Removing RDD 112
17/08/01 14:50:28 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[112] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599028000 ms
17/08/01 14:50:28 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599026000 ms
17/08/01 14:50:28 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599026000 ms
17/08/01 14:50:29 INFO scheduler.JobScheduler: Added jobs for time 1501599029000 ms
17/08/01 14:50:29 INFO scheduler.JobScheduler: Starting job streaming job 1501599029000 ms.0 from job set of time 1501599029000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:29 INFO scheduler.JobScheduler: Finished job streaming job 1501599029000 ms.0 from job set of time 1501599029000 ms
17/08/01 14:50:29 INFO python.PythonRDD: Removing RDD 115 from persistence list
17/08/01 14:50:29 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599029000 ms (execution: 0.007 s)
17/08/01 14:50:29 INFO storage.BlockManager: Removing RDD 115
17/08/01 14:50:29 INFO rdd.BlockRDD: Removing RDD 114 from persistence list
17/08/01 14:50:29 INFO storage.BlockManager: Removing RDD 114
17/08/01 14:50:29 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[114] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599029000 ms
17/08/01 14:50:29 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599027000 ms
17/08/01 14:50:29 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599027000 ms
17/08/01 14:50:30 INFO scheduler.JobScheduler: Added jobs for time 1501599030000 ms
17/08/01 14:50:30 INFO scheduler.JobScheduler: Starting job streaming job 1501599030000 ms.0 from job set of time 1501599030000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:30 INFO scheduler.JobScheduler: Finished job streaming job 1501599030000 ms.0 from job set of time 1501599030000 ms
17/08/01 14:50:30 INFO python.PythonRDD: Removing RDD 117 from persistence list
17/08/01 14:50:30 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599030000 ms (execution: 0.007 s)
17/08/01 14:50:30 INFO storage.BlockManager: Removing RDD 117
17/08/01 14:50:30 INFO rdd.BlockRDD: Removing RDD 116 from persistence list
17/08/01 14:50:30 INFO storage.BlockManager: Removing RDD 116
17/08/01 14:50:30 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[116] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599030000 ms
17/08/01 14:50:30 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599028000 ms
17/08/01 14:50:30 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599028000 ms
17/08/01 14:50:31 INFO scheduler.JobScheduler: Added jobs for time 1501599031000 ms
17/08/01 14:50:31 INFO scheduler.JobScheduler: Starting job streaming job 1501599031000 ms.0 from job set of time 1501599031000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:31 INFO scheduler.JobScheduler: Finished job streaming job 1501599031000 ms.0 from job set of time 1501599031000 ms
17/08/01 14:50:31 INFO python.PythonRDD: Removing RDD 119 from persistence list
17/08/01 14:50:31 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599031000 ms (execution: 0.006 s)
17/08/01 14:50:31 INFO rdd.BlockRDD: Removing RDD 118 from persistence list
17/08/01 14:50:31 INFO storage.BlockManager: Removing RDD 119
17/08/01 14:50:31 INFO storage.BlockManager: Removing RDD 118
17/08/01 14:50:31 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[118] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599031000 ms
17/08/01 14:50:31 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599029000 ms
17/08/01 14:50:31 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599029000 ms
17/08/01 14:50:32 INFO scheduler.JobScheduler: Added jobs for time 1501599032000 ms
17/08/01 14:50:32 INFO scheduler.JobScheduler: Starting job streaming job 1501599032000 ms.0 from job set of time 1501599032000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:32 INFO scheduler.JobScheduler: Finished job streaming job 1501599032000 ms.0 from job set of time 1501599032000 ms
17/08/01 14:50:32 INFO python.PythonRDD: Removing RDD 121 from persistence list
17/08/01 14:50:32 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599032000 ms (execution: 0.007 s)
17/08/01 14:50:32 INFO storage.BlockManager: Removing RDD 121
17/08/01 14:50:32 INFO rdd.BlockRDD: Removing RDD 120 from persistence list
17/08/01 14:50:32 INFO storage.BlockManager: Removing RDD 120
17/08/01 14:50:32 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[120] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599032000 ms
17/08/01 14:50:32 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599030000 ms
17/08/01 14:50:32 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599030000 ms
17/08/01 14:50:33 INFO scheduler.JobScheduler: Added jobs for time 1501599033000 ms
17/08/01 14:50:33 INFO scheduler.JobScheduler: Starting job streaming job 1501599033000 ms.0 from job set of time 1501599033000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:33 INFO scheduler.JobScheduler: Finished job streaming job 1501599033000 ms.0 from job set of time 1501599033000 ms
17/08/01 14:50:33 INFO python.PythonRDD: Removing RDD 123 from persistence list
17/08/01 14:50:33 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599033000 ms (execution: 0.005 s)
17/08/01 14:50:33 INFO storage.BlockManager: Removing RDD 123
17/08/01 14:50:33 INFO rdd.BlockRDD: Removing RDD 122 from persistence list
17/08/01 14:50:33 INFO storage.BlockManager: Removing RDD 122
17/08/01 14:50:33 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[122] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599033000 ms
17/08/01 14:50:33 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599031000 ms
17/08/01 14:50:33 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599031000 ms
17/08/01 14:50:34 INFO scheduler.JobScheduler: Added jobs for time 1501599034000 ms
17/08/01 14:50:34 INFO scheduler.JobScheduler: Starting job streaming job 1501599034000 ms.0 from job set of time 1501599034000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:34 INFO scheduler.JobScheduler: Finished job streaming job 1501599034000 ms.0 from job set of time 1501599034000 ms
17/08/01 14:50:34 INFO python.PythonRDD: Removing RDD 125 from persistence list
17/08/01 14:50:34 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599034000 ms (execution: 0.006 s)
17/08/01 14:50:34 INFO storage.BlockManager: Removing RDD 125
17/08/01 14:50:34 INFO rdd.BlockRDD: Removing RDD 124 from persistence list
17/08/01 14:50:34 INFO storage.BlockManager: Removing RDD 124
17/08/01 14:50:34 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[124] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599034000 ms
17/08/01 14:50:34 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599032000 ms
17/08/01 14:50:34 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599032000 ms
17/08/01 14:50:34 INFO spark.ExecutorAllocationManager: Request to remove executorIds: 2
17/08/01 14:50:34 INFO cluster.YarnClientSchedulerBackend: Requesting to kill executor(s) 2
17/08/01 14:50:34 INFO cluster.YarnClientSchedulerBackend: Actual list of executor(s) to be killed is 2
17/08/01 14:50:34 INFO spark.ExecutorAllocationManager: Removing executor 2 because it has been idle for 60 seconds (new desired total will be 1)
17/08/01 14:50:35 INFO scheduler.JobScheduler: Added jobs for time 1501599035000 ms
17/08/01 14:50:35 INFO scheduler.JobScheduler: Starting job streaming job 1501599035000 ms.0 from job set of time 1501599035000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:35 INFO scheduler.JobScheduler: Finished job streaming job 1501599035000 ms.0 from job set of time 1501599035000 ms
17/08/01 14:50:35 INFO python.PythonRDD: Removing RDD 127 from persistence list
17/08/01 14:50:35 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599035000 ms (execution: 0.013 s)
17/08/01 14:50:35 INFO storage.BlockManager: Removing RDD 127
17/08/01 14:50:35 INFO rdd.BlockRDD: Removing RDD 126 from persistence list
17/08/01 14:50:35 INFO storage.BlockManager: Removing RDD 126
17/08/01 14:50:35 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[126] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599035000 ms
17/08/01 14:50:35 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599033000 ms
17/08/01 14:50:35 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599033000 ms
17/08/01 14:50:36 INFO scheduler.JobScheduler: Added jobs for time 1501599036000 ms
17/08/01 14:50:36 INFO scheduler.JobScheduler: Starting job streaming job 1501599036000 ms.0 from job set of time 1501599036000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:36 INFO scheduler.JobScheduler: Finished job streaming job 1501599036000 ms.0 from job set of time 1501599036000 ms
17/08/01 14:50:36 INFO python.PythonRDD: Removing RDD 129 from persistence list
17/08/01 14:50:36 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599036000 ms (execution: 0.007 s)
17/08/01 14:50:36 INFO storage.BlockManager: Removing RDD 129
17/08/01 14:50:36 INFO rdd.BlockRDD: Removing RDD 128 from persistence list
17/08/01 14:50:36 INFO storage.BlockManager: Removing RDD 128
17/08/01 14:50:36 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[128] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599036000 ms
17/08/01 14:50:36 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599034000 ms
17/08/01 14:50:36 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599034000 ms
17/08/01 14:50:37 INFO scheduler.JobScheduler: Added jobs for time 1501599037000 ms
17/08/01 14:50:37 INFO scheduler.JobScheduler: Starting job streaming job 1501599037000 ms.0 from job set of time 1501599037000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:37 INFO scheduler.JobScheduler: Finished job streaming job 1501599037000 ms.0 from job set of time 1501599037000 ms
17/08/01 14:50:37 INFO python.PythonRDD: Removing RDD 131 from persistence list
17/08/01 14:50:37 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599037000 ms (execution: 0.006 s)
17/08/01 14:50:37 INFO storage.BlockManager: Removing RDD 131
17/08/01 14:50:37 INFO rdd.BlockRDD: Removing RDD 130 from persistence list
17/08/01 14:50:37 INFO storage.BlockManager: Removing RDD 130
17/08/01 14:50:37 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[130] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599037000 ms
17/08/01 14:50:37 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599035000 ms
17/08/01 14:50:37 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599035000 ms
17/08/01 14:50:38 INFO scheduler.JobScheduler: Added jobs for time 1501599038000 ms
17/08/01 14:50:38 INFO scheduler.JobScheduler: Starting job streaming job 1501599038000 ms.0 from job set of time 1501599038000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:38 INFO scheduler.JobScheduler: Finished job streaming job 1501599038000 ms.0 from job set of time 1501599038000 ms
17/08/01 14:50:38 INFO python.PythonRDD: Removing RDD 133 from persistence list
17/08/01 14:50:38 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599038000 ms (execution: 0.007 s)
17/08/01 14:50:38 INFO storage.BlockManager: Removing RDD 133
17/08/01 14:50:38 INFO rdd.BlockRDD: Removing RDD 132 from persistence list
17/08/01 14:50:38 INFO storage.BlockManager: Removing RDD 132
17/08/01 14:50:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[132] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599038000 ms
17/08/01 14:50:38 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599036000 ms
17/08/01 14:50:38 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599036000 ms
17/08/01 14:50:38 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Disabling executor 2.
17/08/01 14:50:38 INFO scheduler.DAGScheduler: Executor lost: 2 (epoch 1)
17/08/01 14:50:38 INFO storage.BlockManagerMasterEndpoint: Trying to remove executor 2 from BlockManagerMaster.
17/08/01 14:50:38 INFO storage.BlockManagerMasterEndpoint: Removing block manager BlockManagerId(2, ip-10-0-0-127.ec2.internal, 59267, None)
17/08/01 14:50:38 INFO storage.BlockManagerMaster: Removed 2 successfully in removeExecutor
17/08/01 14:50:38 INFO cluster.YarnScheduler: Executor 2 on ip-10-0-0-127.ec2.internal killed by driver.
17/08/01 14:50:38 INFO spark.ExecutorAllocationManager: Existing executor 2 has been removed (new total is 1)
17/08/01 14:50:39 INFO scheduler.JobScheduler: Added jobs for time 1501599039000 ms
17/08/01 14:50:39 INFO scheduler.JobScheduler: Starting job streaming job 1501599039000 ms.0 from job set of time 1501599039000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:39 INFO scheduler.JobScheduler: Finished job streaming job 1501599039000 ms.0 from job set of time 1501599039000 ms
17/08/01 14:50:39 INFO python.PythonRDD: Removing RDD 135 from persistence list
17/08/01 14:50:39 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599039000 ms (execution: 0.006 s)
17/08/01 14:50:39 INFO storage.BlockManager: Removing RDD 135
17/08/01 14:50:39 INFO rdd.BlockRDD: Removing RDD 134 from persistence list
17/08/01 14:50:39 INFO storage.BlockManager: Removing RDD 134
17/08/01 14:50:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[134] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599039000 ms
17/08/01 14:50:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599037000 ms
17/08/01 14:50:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599037000 ms
17/08/01 14:50:40 INFO scheduler.JobScheduler: Added jobs for time 1501599040000 ms
17/08/01 14:50:40 INFO scheduler.JobScheduler: Starting job streaming job 1501599040000 ms.0 from job set of time 1501599040000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:40 INFO scheduler.JobScheduler: Finished job streaming job 1501599040000 ms.0 from job set of time 1501599040000 ms
17/08/01 14:50:40 INFO python.PythonRDD: Removing RDD 137 from persistence list
17/08/01 14:50:40 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599040000 ms (execution: 0.006 s)
17/08/01 14:50:40 INFO storage.BlockManager: Removing RDD 137
17/08/01 14:50:40 INFO rdd.BlockRDD: Removing RDD 136 from persistence list
17/08/01 14:50:40 INFO storage.BlockManager: Removing RDD 136
17/08/01 14:50:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[136] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599040000 ms
17/08/01 14:50:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599038000 ms
17/08/01 14:50:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599038000 ms
17/08/01 14:50:41 INFO scheduler.JobScheduler: Added jobs for time 1501599041000 ms
17/08/01 14:50:41 INFO scheduler.JobScheduler: Starting job streaming job 1501599041000 ms.0 from job set of time 1501599041000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:41 INFO scheduler.JobScheduler: Finished job streaming job 1501599041000 ms.0 from job set of time 1501599041000 ms
17/08/01 14:50:41 INFO python.PythonRDD: Removing RDD 139 from persistence list
17/08/01 14:50:41 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599041000 ms (execution: 0.006 s)
17/08/01 14:50:41 INFO storage.BlockManager: Removing RDD 139
17/08/01 14:50:41 INFO rdd.BlockRDD: Removing RDD 138 from persistence list
17/08/01 14:50:41 INFO storage.BlockManager: Removing RDD 138
17/08/01 14:50:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[138] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599041000 ms
17/08/01 14:50:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599039000 ms
17/08/01 14:50:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599039000 ms
17/08/01 14:50:42 INFO scheduler.JobScheduler: Added jobs for time 1501599042000 ms
17/08/01 14:50:42 INFO scheduler.JobScheduler: Starting job streaming job 1501599042000 ms.0 from job set of time 1501599042000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:42 INFO scheduler.JobScheduler: Finished job streaming job 1501599042000 ms.0 from job set of time 1501599042000 ms
17/08/01 14:50:42 INFO python.PythonRDD: Removing RDD 141 from persistence list
17/08/01 14:50:42 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599042000 ms (execution: 0.014 s)
17/08/01 14:50:42 INFO storage.BlockManager: Removing RDD 141
17/08/01 14:50:42 INFO rdd.BlockRDD: Removing RDD 140 from persistence list
17/08/01 14:50:42 INFO storage.BlockManager: Removing RDD 140
17/08/01 14:50:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[140] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599042000 ms
17/08/01 14:50:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599040000 ms
17/08/01 14:50:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599040000 ms
17/08/01 14:50:42 INFO storage.BlockManagerInfo: Added input-0-1501599042400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1059.6 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599042800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1057.9 MB)
17/08/01 14:50:43 INFO scheduler.JobScheduler: Added jobs for time 1501599043000 ms
17/08/01 14:50:43 INFO scheduler.JobScheduler: Starting job streaming job 1501599043000 ms.0 from job set of time 1501599043000 ms
17/08/01 14:50:43 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Got job 2 (runJob at PythonRDD.scala:441) with 1 output partitions
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Final stage: ResultStage 3 (runJob at PythonRDD.scala:441)
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Missing parents: List()
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Submitting ResultStage 3 (PythonRDD[146] at RDD at PythonRDD.scala:48), which has no missing parents
17/08/01 14:50:43 INFO memory.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/08/01 14:50:43 INFO memory.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.0.0.199:53355 (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:43 INFO spark.SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:996
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (PythonRDD[146] at RDD at PythonRDD.scala:48)
17/08/01 14:50:43 INFO cluster.YarnScheduler: Adding task set 3.0 with 1 tasks
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.3 KB, free: 1057.0 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1056.2 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.4 KB, free: 1055.3 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 878.9 KB, free: 1054.5 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599043800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1052.7 MB)
17/08/01 14:50:44 INFO scheduler.JobScheduler: Added jobs for time 1501599044000 ms
17/08/01 14:50:44 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 2)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1051.9 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1051.0 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1050.2 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.5 KB, free: 1049.3 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599044800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 887.9 KB, free: 1048.4 MB)
17/08/01 14:50:45 INFO scheduler.JobScheduler: Added jobs for time 1501599045000 ms
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1047.6 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1046.7 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1045.9 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.0 KB, free: 1044.1 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599045800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1043.3 MB)
17/08/01 14:50:46 INFO scheduler.JobScheduler: Added jobs for time 1501599046000 ms
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.6 KB, free: 1042.4 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.0 KB, free: 1041.6 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1040.7 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1039.9 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599046800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1039.0 MB)
17/08/01 14:50:47 INFO scheduler.JobScheduler: Added jobs for time 1501599047000 ms
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1038.1 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 878.9 KB, free: 1037.3 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1035.6 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1034.7 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599047800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1033.8 MB)
17/08/01 14:50:48 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.127:44785) with ID 3
17/08/01 14:50:48 INFO spark.ExecutorAllocationManager: New executor 3 has registered (new total is 2)
17/08/01 14:50:48 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 3.0 (TID 71, ip-10-0-0-127.ec2.internal, executor 3, partition 0, RACK_LOCAL, 5800 bytes)
17/08/01 14:50:48 INFO scheduler.JobScheduler: Added jobs for time 1501599048000 ms
17/08/01 14:50:48 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-127.ec2.internal:51258 with 1060.5 MB RAM, BlockManagerId(3, ip-10-0-0-127.ec2.internal, 51258, None)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1033.0 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on ip-10-0-0-127.ec2.internal:51258 (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1032.1 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.5 KB, free: 1031.3 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1030.4 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599048800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1029.5 MB)
17/08/01 14:50:49 INFO scheduler.JobScheduler: Added jobs for time 1501599049000 ms
17/08/01 14:50:49 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 3.0 (TID 71) in 1023 ms on ip-10-0-0-127.ec2.internal (executor 3) (1/1)
17/08/01 14:50:49 INFO cluster.YarnScheduler: Removed TaskSet 3.0, whose tasks have all completed, from pool
17/08/01 14:50:49 INFO scheduler.DAGScheduler: ResultStage 3 (runJob at PythonRDD.scala:441) finished in 5.985 s
17/08/01 14:50:49 INFO scheduler.DAGScheduler: Job 2 finished: runJob at PythonRDD.scala:441, took 6.003911 s
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.0 KB, free: 1027.8 MB)
17/08/01 14:50:49 INFO spark.ContextCleaner: Cleaned shuffle 0
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 10.0.0.199:53355 in memory (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1988.0 B, free: 1027.8 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on 10.0.0.199:53355 in memory (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1956.0 B, free: 1027.8 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on 10.0.0.199:53355 in memory (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on ip-10-0-0-127.ec2.internal:51258 in memory (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1027.0 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1026.1 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1025.3 MB)
17/08/01 14:50:49 INFO execution.SparkSqlParser: Parsing command: spotdb.proxy
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599049800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1024.4 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Added jobs for time 1501599050000 ms
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1023.5 MB)
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1022.7 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599043000 ms.0 from job set of time 1501599043000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 143 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 7.504 s for time 1501599043000 ms (execution: 7.487 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599044000 ms.0 from job set of time 1501599044000 ms
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 142 from persistence list
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599043000 ms.0
org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 69, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[142] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599043000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599041000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599041000 ms
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 142
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 143
Traceback (most recent call last):
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 148, in <module>
main()
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 77, in main
bluecoat_parse(args.zk,args.topic,args.db,args.db_table,args.num_of_workers,args.batch_size)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 144, in bluecoat_parse
ssc.awaitTermination()
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/context.py", line 206, in awaitTermination
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o50.awaitTermination.
: org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 69, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599044000 ms.0 from job set of time 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 6.542 s for time 1501599044000 ms (execution: 0.037 s)
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 145 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599045000 ms.0 from job set of time 1501599045000 ms
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599044000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 145
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 144 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 144
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[144] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599042000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599042000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599042400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1023.5 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599045000 ms.0 from job set of time 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 5.547 s for time 1501599045000 ms (execution: 0.003 s)
17/08/01 14:50:50 INFO streaming.StreamingContext: Invoking stop(stopGracefully=false) from shutdown hook
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599046000 ms.0 from job set of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599046000 ms.0 from job set of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 4.559 s for time 1501599046000 ms (execution: 0.004 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599047000 ms.0 from job set of time 1501599047000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 148 from persistence list
17/08/01 14:50:50 INFO scheduler.ReceiverTracker: Sent stop signal to all 1 receivers
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 148
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599047000 ms.0 from job set of time 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 3.562 s for time 1501599047000 ms (execution: 0.002 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599048000 ms.0 from job set of time 1501599048000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599048000 ms.0 from job set of time 1501599048000 ms
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 147 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 2.571 s for time 1501599048000 ms (execution: 0.001 s)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 147
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[147] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599049000 ms.0 from job set of time 1501599049000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599049000 ms.0 from job set of time 1501599049000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 1.572 s for time 1501599049000 ms (execution: 0.001 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599050000 ms.0 from job set of time 1501599050000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599050000 ms.0 from job set of time 1501599050000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 0.574 s for time 1501599050000 ms (execution: 0.001 s)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599045000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599046000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599047000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599048000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599049000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599042800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1025.3 MB)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599050000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.3 KB, free: 1026.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1027.0 MB)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599043000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599043000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 150 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 878.9 KB, free: 1027.8 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.4 KB, free: 1028.7 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 150
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 149 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 149
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[149] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599044000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 152 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 152
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1030.4 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1031.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1032.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1033.0 MB)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 151 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 151
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[151] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599045000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 154 from persistence list
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 153 from persistence list
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[153] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599048000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.5 KB, free: 1033.8 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 153
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 154
17/08/01 14:50:50 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1034.7 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599046000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 156 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 887.9 KB, free: 1035.6 MB)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 155 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.4 KB, free: 1034.7 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 155
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 156
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[155] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599049000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1035.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1036.4 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599047000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 158 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 158
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 157 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 157
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[157] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599050000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.6 KB, free: 1037.3 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599048000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599048000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1038.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.0 KB, free: 1039.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.0 KB, free: 1040.7 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1041.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1042.4 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1043.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 878.9 KB, free: 1044.2 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1045.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1046.7 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1047.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1048.5 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.5 KB, free: 1049.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1050.2 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1051.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1051.9 MB)
17/08/01 14:50:51 ERROR scheduler.JobScheduler: Error generating jobs for time 1501599051000 ms
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 2.0 (TID 70) in 77942 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/1)
17/08/01 14:50:51 INFO cluster.YarnScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool
17/08/01 14:50:51 INFO scheduler.DAGScheduler: ResultStage 2 (start at NativeMethodAccessorImpl.java:0) finished in 77.943 s
17/08/01 14:50:51 INFO scheduler.ReceiverTracker: All of the receivers have deregistered successfully
17/08/01 14:50:51 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
17/08/01 14:50:51 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
17/08/01 14:50:51 INFO util.RecurringTimer: Stopped timer for JobGenerator after time 1501599051000
17/08/01 14:50:51 INFO scheduler.JobGenerator: Stopped JobGenerator
17/08/01 14:50:51 INFO scheduler.JobScheduler: Stopped JobScheduler
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@625d3c2d{/streaming,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@39edaf54{/streaming/batch,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@c773c0d{/static/streaming,null,UNAVAILABLE}
17/08/01 14:50:51 INFO streaming.StreamingContext: StreamingContext stopped successfully
17/08/01 14:50:51 INFO spark.SparkContext: Invoking stop() from shutdown hook
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 INFO server.ServerConnector: Stopped ServerConnector@37681deb{HTTP/1.1}{0.0.0.0:4040}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2aa3e9a6{/stages/stage/kill,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ee30613{/jobs/job/kill,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@15ec1723{/api,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14a7d27d{/,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@e4408f4{/static,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@8a515f3{/executors/threadDump/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@f241039{/executors/threadDump,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@79b9b526{/executors/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5fa63fa{/executors,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@77729e5a{/environment/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@242eb7e{/environment,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@49ddd2ed{/storage/rdd/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@699743b3{/storage/rdd,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@736cc380{/storage/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19cf381c{/storage,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4f734944{/stages/pool/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@b7d4da0{/stages/pool,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ab71bb9{/stages/stage/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5d37ce06{/stages/stage,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@27f55b38{/stages/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1159f15e{/stages,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ff970c2{/jobs/job/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@615b86f5{/jobs/job,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19bb7285{/jobs/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3b9882ec{/jobs,null,UNAVAILABLE}
17/08/01 14:50:51 INFO ui.SparkUI: Stopped Spark web UI at http://10.0.0.199:4040
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors
17/08/01 14:50:51 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
17/08/01 14:50:51 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
services=List(),
started=false)
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Stopped
17/08/01 14:50:51 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/08/01 14:50:51 INFO memory.MemoryStore: MemoryStore cleared
17/08/01 14:50:51 INFO storage.BlockManager: BlockManager stopped
17/08/01 14:50:51 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
17/08/01 14:50:51 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/08/01 14:50:51 INFO spark.SparkContext: Successfully stopped SparkContext
17/08/01 14:50:51 INFO util.ShutdownHookManager: Shutdown hook called
17/08/01 14:50:51 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25
17/08/01 14:50:51 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25/pyspark-c2380580-feb5-47ef-bc69-7b2c9ba38770
Closing worker...
From: Deon Griessel [mailto:dgriessel@searchtechnologies.com]
Sent: Monday, July 31, 2017 4:53 PM
To: user@spot.apache.org
Subject: Spot Ingest Proxy Error
I am trying to run the spark-ingest for the sample proxy log: Bluecoat_ProxySG_Sample.log
I run bash ./start_ingest_standalone.sh proxy 1
Then add the demo log to my collector_path. The worker dies, seems that it cannot find the hive metastore, probably a spark/hive configuration issue. Also, notice that the ip=unknown-ip-addr.
Would appreciate any help on this.
This is what I’m seeing in the worker screen output:
2017-07-31 18:54:15,855 - SPOT.INGEST.WORKER - INFO - Initializing kafka instance
2017-07-31 18:54:15,855 - SPOT.INGEST.WRK.PROXY - INFO - Creating Spark Job for topic: SPOT-INGEST-proxy-18_54_13
2017-07-31 18:54:15,856 - SPOT.INGEST.WRK.PROXY - INFO - SPOT.Utils: Executing: spark2-submit --master yarn --driver-memory 2325m --num-executors 1 --conf spark.executor.memory=2325m --conf spark.executor.cores=1 --jars /home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar /home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py -zk ip-10-0-0-199.ec2.internal:2181 -t SPOT-INGEST-proxy-18_54_13 -db spotdb -dt proxy -w 1 -bs 1
17/07/31 18:54:17 INFO spark.SparkContext: Running Spark version 2.1.0.cloudera1
17/07/31 18:54:17 WARN spark.SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0
17/07/31 18:54:17 INFO spark.SecurityManager: Changing view acls to: spot-user
17/07/31 18:54:17 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/07/31 18:54:17 INFO spark.SecurityManager: Changing view acls groups to:
17/07/31 18:54:17 INFO spark.SecurityManager: Changing modify acls groups to:
17/07/31 18:54:17 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/07/31 18:54:18 INFO util.Utils: Successfully started service 'sparkDriver' on port 54401.
17/07/31 18:54:18 INFO spark.SparkEnv: Registering MapOutputTracker
17/07/31 18:54:18 INFO spark.SparkEnv: Registering BlockManagerMaster
17/07/31 18:54:18 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
17/07/31 18:54:18 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
17/07/31 18:54:18 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-1db8ad37-9fcc-4799-b9e0-a206b6041d04
17/07/31 18:54:18 INFO memory.MemoryStore: MemoryStore started with capacity 1060.5 MB
17/07/31 18:54:18 INFO spark.SparkEnv: Registering OutputCommitCoordinator
17/07/31 18:54:18 INFO util.log: Logging initialized @2379ms
17/07/31 18:54:18 INFO server.Server: jetty-9.2.z-SNAPSHOT
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1a3b1e79{/jobs,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1a3b1e79%7b/jobs,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1f4da763{/jobs/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1f4da763%7b/jobs/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@232864a3{/jobs/job,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@232864a3%7b/jobs/job,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@30e71b5d{/jobs/job/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@30e71b5d%7b/jobs/job/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14b58fc0{/stages,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14b58fc0%7b/stages,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1bf090df{/stages/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1bf090df%7b/stages/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4eb72ecd{/stages/stage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4eb72ecd%7b/stages/stage,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5c61bd1a{/stages/stage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5c61bd1a%7b/stages/stage/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14c62558{/stages/pool,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14c62558%7b/stages/pool,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5cbdbf0f{/stages/pool/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5cbdbf0f%7b/stages/pool/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2d4aa15a{/storage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@2d4aa15a%7b/storage,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ea732f0{/storage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ea732f0%7b/storage/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@13016b86{/storage/rdd,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@13016b86%7b/storage/rdd,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@423a3429{/storage/rdd/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@423a3429%7b/storage/rdd/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7b3691b6{/environment,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7b3691b6%7b/environment,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3b9882ec{/environment/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3b9882ec%7b/environment/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19bb7285{/executors,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19bb7285%7b/executors,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@615b86f5{/executors/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@615b86f5%7b/executors/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ff970c2{/executors/threadDump,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3ff970c2%7b/executors/threadDump,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1159f15e{/executors/threadDump/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1159f15e%7b/executors/threadDump/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27f55b38{/static,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27f55b38%7b/static,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5d37ce06{/,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5d37ce06%7b/,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ab71bb9{/api,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ab71bb9%7b/api,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b7d4da0{/jobs/job/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@b7d4da0%7b/jobs/job/kill,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f734944{/stages/stage/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4f734944%7b/stages/stage/kill,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO server.ServerConnector: Started ServerConnector@188e6c7f{HTTP/1.1}{0.0.0.0:4040}<mailto:ServerConnector@188e6c7f%7bHTTP/1.1%7d%7b0.0.0.0:4040%7d>
17/07/31 18:54:18 INFO server.Server: Started @2532ms
17/07/31 18:54:18 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
17/07/31 18:54:18 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.0.0.199:4040
17/07/31 18:54:18 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/07/31 18:54:19 INFO client.RMProxy: Connecting to ResourceManager at ip-10-0-0-199.ec2.internal/10.0.0.199:8032
17/07/31 18:54:19 INFO yarn.Client: Requesting a new application from cluster with 3 NodeManagers
17/07/31 18:54:19 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (2725 MB per container)
17/07/31 18:54:19 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/07/31 18:54:19 INFO yarn.Client: Setting up container launch context for our AM
17/07/31 18:54:19 INFO yarn.Client: Setting up the launch environment for our AM container
17/07/31 18:54:19 INFO yarn.Client: Preparing resources for our AM container
17/07/31 18:54:21 INFO yarn.Client: Uploading resource file:/home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501523941584_0005/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar
17/07/31 18:54:21 INFO yarn.Client: Uploading resource file:/tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40/__spark_conf__5838957838871043110.zip -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501523941584_0005/__spark_conf__.zip
17/07/31 18:54:21 INFO spark.SecurityManager: Changing view acls to: spot-user
17/07/31 18:54:21 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/07/31 18:54:21 INFO spark.SecurityManager: Changing view acls groups to:
17/07/31 18:54:21 INFO spark.SecurityManager: Changing modify acls groups to:
17/07/31 18:54:21 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/07/31 18:54:21 INFO yarn.Client: Submitting application application_1501523941584_0005 to ResourceManager
17/07/31 18:54:21 INFO impl.YarnClientImpl: Submitted application application_1501523941584_0005
17/07/31 18:54:21 INFO cluster.SchedulerExtensionServices: Starting Yarn extension services with app application_1501523941584_0005 and attemptId None
17/07/31 18:54:22 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:22 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.users.spot-user
start time: 1501527261635
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005/
user: spot-user
17/07/31 18:54:23 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:24 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:25 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/07/31 18:54:25 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> ip-10-0-0-199.ec2.internal, PROXY_URI_BASES -> http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005), /proxy/application_1501523941584_0005
17/07/31 18:54:25 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/07/31 18:54:25 INFO yarn.Client: Application report for application_1501523941584_0005 (state: RUNNING)
17/07/31 18:54:25 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 10.0.0.127
ApplicationMaster RPC port: 0
queue: root.users.spot-user
start time: 1501527261635
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005/
user: spot-user
17/07/31 18:54:25 INFO cluster.YarnClientSchedulerBackend: Application application_1501523941584_0005 has started running.
17/07/31 18:54:25 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 59442.
17/07/31 18:54:25 INFO netty.NettyBlockTransferService: Server created on 10.0.0.199:59442
17/07/31 18:54:25 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/07/31 18:54:25 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManagerMasterEndpoint: Registering block manager 10.0.0.199:59442 with 1060.5 MB RAM, BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManager: external shuffle service port = 7337
17/07/31 18:54:25 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3e93262b{/metrics/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3e93262b%7b/metrics/json,null,AVAILABLE%7d>
17/07/31 18:54:25 INFO scheduler.EventLoggingListener: Logging events to hdfs://ip-10-0-0-199.ec2.internal:8020/user/spark/spark2ApplicationHistory/application_1501523941584_0005
17/07/31 18:54:25 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/07/31 18:54:28 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.229:51685) with ID 1
17/07/31 18:54:28 INFO spark.ExecutorAllocationManager: New executor 1 has registered (new total is 1)
17/07/31 18:54:28 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-229.ec2.internal:35659 with 1060.5 MB RAM, BlockManagerId(1, ip-10-0-0-229.ec2.internal, 35659, None)
17/07/31 18:54:28 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/07/31 18:54:29 INFO internal.SharedState: Warehouse path is 'file:/home/spot-user/spot-ingest/spark-warehouse'.
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7a9c6710{/SQL,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7a9c6710%7b/SQL,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@141768e4{/SQL/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@141768e4%7b/SQL/json,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1e367d45{/SQL/execution,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1e367d45%7b/SQL/execution,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5554f0a0{/SQL/execution/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5554f0a0%7b/SQL/execution/json,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@144776d3{/static/sql,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@144776d3%7b/static/sql,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO hive.HiveUtils: Initializing HiveMetastoreConnection version 1.1.0 using Spark classes.
17/07/31 18:54:30 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
17/07/31 18:54:30 INFO metastore.ObjectStore: ObjectStore, initialize called
17/07/31 18:54:30 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
17/07/31 18:54:30 INFO DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
17/07/31 18:54:30 WARN bonecp.BoneCPConfig: Max Connections < 1. Setting to 20
17/07/31 18:54:31 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
17/07/31 18:54:32 WARN bonecp.BoneCPConfig: Max Connections < 1. Setting to 20
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0<ma...@0>" since the connection used is closing
17/07/31 18:54:32 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
17/07/31 18:54:32 INFO metastore.ObjectStore: Initialized ObjectStore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: Added admin role in metastore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: Added public role in metastore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_all_functions from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_all_functions
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_all_functions
17/07/31 18:54:33 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_all_functions start=1501527273291 end=1501527273356 duration=65 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=0 error=false>
17/07/31 18:54:33 INFO session.SessionState: Created local directory: /tmp/aaecce13-256b-4b91-b661-9046c9c418c6_resources
17/07/31 18:54:33 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6
17/07/31 18:54:33 INFO session.SessionState: Created local directory: /tmp/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6
17/07/31 18:54:33 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6/_tmp_space.db
17/07/31 18:54:33 INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
17/07/31 18:54:33 INFO client.HiveClientImpl: Warehouse location for Hive client (version 1.1.0) is file:/home/spot-user/spot-ingest/spark-warehouse
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_database: default
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: default
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527273406 end=1501527273408 duration=2 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=0 error=false>
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_database: global_temp
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: global_temp
17/07/31 18:54:33 WARN metastore.ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527273409 end=1501527273412 duration=3 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=-1 error=true>
17/07/31 18:54:33 WARN streaming.StreamingContext: Dynamic Allocation is enabled for this application. Enabling Dynamic allocation for Spark Streaming applications can cause data loss if Write Ahead Log is not enabled for non-replayable sources like Flume. See the programming guide for details on how to enable the Write Ahead Log.
17/07/31 18:54:33 INFO spark.SparkContext: Starting job: start at NativeMethodAccessorImpl.java:0
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Registering RDD 1 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Got job 0 (start at NativeMethodAccessorImpl.java:0) with 20 output partitions
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/07/31 18:54:33 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 3.1 KB, free 1060.5 MB)
17/07/31 18:54:33 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 1988.0 B, free 1060.5 MB)
17/07/31 18:54:33 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.0.0.199:59442 (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:54:33 INFO spark.SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Submitting 50 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO cluster.YarnScheduler: Adding task set 0.0 with 50 tasks
17/07/31 18:54:33 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, ip-10-0-0-229.ec2.internal, executor 1, partition 1, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 669 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, ip-10-0-0-229.ec2.internal, executor 1, partition 2, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 77 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3, ip-10-0-0-229.ec2.internal, executor 1, partition 3, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 72 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 0.0 (TID 4, ip-10-0-0-229.ec2.internal, executor 1, partition 4, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 69 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 0.0 (TID 5, ip-10-0-0-229.ec2.internal, executor 1, partition 5, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 59 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 0.0 (TID 6, ip-10-0-0-229.ec2.internal, executor 1, partition 6, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 0.0 (TID 7, ip-10-0-0-229.ec2.internal, executor 1, partition 7, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 0.0 (TID 8, ip-10-0-0-229.ec2.internal, executor 1, partition 8, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 0.0 (TID 7) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/50)
17/07/31 18:54:34 INFO spark.ExecutorAllocationManager: Requesting 1 new executor because tasks are backlogged (new desired total will be 2)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 0.0 (TID 9, ip-10-0-0-229.ec2.internal, executor 1, partition 9, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 0.0 (TID 8) in 58 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 0.0 (TID 10, ip-10-0-0-229.ec2.internal, executor 1, partition 10, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 0.0 (TID 9) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 0.0 (TID 11, ip-10-0-0-229.ec2.internal, executor 1, partition 11, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 0.0 (TID 10) in 52 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 0.0 (TID 12, ip-10-0-0-229.ec2.internal, executor 1, partition 12, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 0.0 (TID 11) in 58 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 0.0 (TID 13, ip-10-0-0-229.ec2.internal, executor 1, partition 13, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 0.0 (TID 12) in 51 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 0.0 (TID 14, ip-10-0-0-229.ec2.internal, executor 1, partition 14, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 0.0 (TID 13) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 0.0 (TID 15, ip-10-0-0-229.ec2.internal, executor 1, partition 15, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 0.0 (TID 14) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 0.0 (TID 16, ip-10-0-0-229.ec2.internal, executor 1, partition 16, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 0.0 (TID 15) in 63 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 0.0 (TID 17, ip-10-0-0-229.ec2.internal, executor 1, partition 17, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 0.0 (TID 16) in 50 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 0.0 (TID 18, ip-10-0-0-229.ec2.internal, executor 1, partition 18, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 0.0 (TID 17) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 0.0 (TID 19, ip-10-0-0-229.ec2.internal, executor 1, partition 19, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 0.0 (TID 18) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 0.0 (TID 20, ip-10-0-0-229.ec2.internal, executor 1, partition 20, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 0.0 (TID 19) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 0.0 (TID 21, ip-10-0-0-229.ec2.internal, executor 1, partition 21, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 0.0 (TID 20) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (21/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 0.0 (TID 22, ip-10-0-0-229.ec2.internal, executor 1, partition 22, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 0.0 (TID 21) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (22/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 0.0 (TID 23, ip-10-0-0-229.ec2.internal, executor 1, partition 23, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 0.0 (TID 22) in 47 ms on ip-10-0-0-229.ec2.internal (executor 1) (23/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 0.0 (TID 24, ip-10-0-0-229.ec2.internal, executor 1, partition 24, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 0.0 (TID 23) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (24/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 0.0 (TID 25, ip-10-0-0-229.ec2.internal, executor 1, partition 25, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 0.0 (TID 24) in 46 ms on ip-10-0-0-229.ec2.internal (executor 1) (25/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 0.0 (TID 26, ip-10-0-0-229.ec2.internal, executor 1, partition 26, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 0.0 (TID 25) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (26/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 0.0 (TID 27, ip-10-0-0-229.ec2.internal, executor 1, partition 27, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 0.0 (TID 26) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (27/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 0.0 (TID 28, ip-10-0-0-229.ec2.internal, executor 1, partition 28, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 0.0 (TID 27) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (28/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 0.0 (TID 29, ip-10-0-0-229.ec2.internal, executor 1, partition 29, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 0.0 (TID 28) in 39 ms on ip-10-0-0-229.ec2.internal (executor 1) (29/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 0.0 (TID 30, ip-10-0-0-229.ec2.internal, executor 1, partition 30, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 0.0 (TID 29) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (30/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 0.0 (TID 31, ip-10-0-0-229.ec2.internal, executor 1, partition 31, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 0.0 (TID 30) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (31/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 0.0 (TID 32, ip-10-0-0-229.ec2.internal, executor 1, partition 32, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 0.0 (TID 31) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (32/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 0.0 (TID 33, ip-10-0-0-229.ec2.internal, executor 1, partition 33, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 0.0 (TID 32) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (33/50)
17/07/31 18:54:35 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 4)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 0.0 (TID 34, ip-10-0-0-229.ec2.internal, executor 1, partition 34, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 0.0 (TID 33) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (34/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 0.0 (TID 35, ip-10-0-0-229.ec2.internal, executor 1, partition 35, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 0.0 (TID 34) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (35/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 0.0 (TID 36, ip-10-0-0-229.ec2.internal, executor 1, partition 36, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 0.0 (TID 35) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (36/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 0.0 (TID 37, ip-10-0-0-229.ec2.internal, executor 1, partition 37, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 0.0 (TID 36) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (37/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 0.0 (TID 38, ip-10-0-0-229.ec2.internal, executor 1, partition 38, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 0.0 (TID 37) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (38/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 0.0 (TID 39, ip-10-0-0-229.ec2.internal, executor 1, partition 39, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 0.0 (TID 38) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (39/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 0.0 (TID 40, ip-10-0-0-229.ec2.internal, executor 1, partition 40, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 0.0 (TID 39) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (40/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 0.0 (TID 41, ip-10-0-0-229.ec2.internal, executor 1, partition 41, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 0.0 (TID 40) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (41/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 0.0 (TID 42, ip-10-0-0-229.ec2.internal, executor 1, partition 42, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 0.0 (TID 41) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (42/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 0.0 (TID 43, ip-10-0-0-229.ec2.internal, executor 1, partition 43, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 0.0 (TID 42) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (43/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 0.0 (TID 44, ip-10-0-0-229.ec2.internal, executor 1, partition 44, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 0.0 (TID 43) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (44/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 0.0 (TID 45, ip-10-0-0-229.ec2.internal, executor 1, partition 45, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 0.0 (TID 44) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (45/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 0.0 (TID 46, ip-10-0-0-229.ec2.internal, executor 1, partition 46, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 0.0 (TID 45) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (46/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 0.0 (TID 47, ip-10-0-0-229.ec2.internal, executor 1, partition 47, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 0.0 (TID 46) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (47/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 0.0 (TID 48, ip-10-0-0-229.ec2.internal, executor 1, partition 48, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 0.0 (TID 47) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (48/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 0.0 (TID 49, ip-10-0-0-229.ec2.internal, executor 1, partition 49, PROCESS_LOCAL, 6676 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 0.0 (TID 48) in 29 ms on ip-10-0-0-229.ec2.internal (executor 1) (49/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 0.0 (TID 49) in 27 ms on ip-10-0-0-229.ec2.internal (executor 1) (50/50)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (start at NativeMethodAccessorImpl.java:0) finished in 2.499 s
17/07/31 18:54:36 INFO cluster.YarnScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/07/31 18:54:36 INFO scheduler.DAGScheduler: looking for newly runnable stages
17/07/31 18:54:36 INFO scheduler.DAGScheduler: running: Set()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 1)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: failed: Set()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.2 KB, free 1060.5 MB)
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 1956.0 B, free 1060.5 MB)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.0.0.199:59442 (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting 20 missing tasks from ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Adding task set 1.0 with 20 tasks
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 50, ip-10-0-0-229.ec2.internal, executor 1, partition 0, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.0.0.229:51685
17/07/31 18:54:36 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 283 bytes
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.0 (TID 51, ip-10-0-0-229.ec2.internal, executor 1, partition 1, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 50) in 91 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 1.0 (TID 52, ip-10-0-0-229.ec2.internal, executor 1, partition 2, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 1.0 (TID 51) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 1.0 (TID 53, ip-10-0-0-229.ec2.internal, executor 1, partition 3, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 1.0 (TID 52) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 1.0 (TID 54, ip-10-0-0-229.ec2.internal, executor 1, partition 4, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 1.0 (TID 53) in 24 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 1.0 (TID 55, ip-10-0-0-229.ec2.internal, executor 1, partition 5, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 1.0 (TID 54) in 24 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 1.0 (TID 56, ip-10-0-0-229.ec2.internal, executor 1, partition 6, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 1.0 (TID 55) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 1.0 (TID 57, ip-10-0-0-229.ec2.internal, executor 1, partition 7, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 1.0 (TID 56) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 1.0 (TID 58, ip-10-0-0-229.ec2.internal, executor 1, partition 8, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 1.0 (TID 57) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 1.0 (TID 59, ip-10-0-0-229.ec2.internal, executor 1, partition 9, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 1.0 (TID 58) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 1.0 (TID 60, ip-10-0-0-229.ec2.internal, executor 1, partition 10, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 1.0 (TID 59) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 1.0 (TID 61, ip-10-0-0-229.ec2.internal, executor 1, partition 11, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 1.0 (TID 60) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 1.0 (TID 62, ip-10-0-0-229.ec2.internal, executor 1, partition 12, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 1.0 (TID 61) in 16 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 1.0 (TID 63, ip-10-0-0-229.ec2.internal, executor 1, partition 13, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 1.0 (TID 62) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 1.0 (TID 64, ip-10-0-0-229.ec2.internal, executor 1, partition 14, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 1.0 (TID 63) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 1.0 (TID 65, ip-10-0-0-229.ec2.internal, executor 1, partition 15, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 1.0 (TID 64) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 1.0 (TID 66, ip-10-0-0-229.ec2.internal, executor 1, partition 16, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 1.0 (TID 65) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 1.0 (TID 67, ip-10-0-0-229.ec2.internal, executor 1, partition 17, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 1.0 (TID 66) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 1.0 (TID 68, ip-10-0-0-229.ec2.internal, executor 1, partition 18, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 1.0 (TID 67) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 1.0 (TID 69, ip-10-0-0-229.ec2.internal, executor 1, partition 19, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 1.0 (TID 68) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 1.0 (TID 69) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/20)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool
17/07/31 18:54:36 INFO scheduler.DAGScheduler: ResultStage 1 (start at NativeMethodAccessorImpl.java:0) finished in 0.416 s
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Job 0 finished: start at NativeMethodAccessorImpl.java:0, took 3.184418 s
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: Starting 1 receivers
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: ReceiverTracker started
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Initialized and validated org.apache.spark.streaming.kafka.KafkaInputDStream@7b67308f<ma...@7b67308f>
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Initialized and validated org.apache.spark.streaming.api.python.PythonTransformedDStream@33a53b94<ma...@33a53b94>
17/07/31 18:54:36 INFO dstream.ForEachDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO dstream.ForEachDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO dstream.ForEachDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO dstream.ForEachDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO dstream.ForEachDStream: Initialized and validated org.apache.spark.streaming.dstream.ForEachDStream@24ea8c09<ma...@24ea8c09>
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Got job 1 (start at NativeMethodAccessorImpl.java:0) with 1 output partitions
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Final stage: ResultStage 2 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620), which has no missing parents
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: Receiver 0 started
17/07/31 18:54:36 INFO util.RecurringTimer: Started timer for JobGenerator at time 1501527277000
17/07/31 18:54:36 INFO scheduler.JobGenerator: Started JobGenerator at 1501527277000 ms
17/07/31 18:54:36 INFO scheduler.JobScheduler: Started JobScheduler
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19fd4723{/streaming,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19fd4723%7b/streaming,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@43b0e42{/streaming/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@43b0e42%7b/streaming/json,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@22116fc3{/streaming/batch,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@22116fc3%7b/streaming/batch,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27399762{/streaming/batch/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27399762%7b/streaming/batch/json,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@70d301d6{/static/streaming,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@70d301d6%7b/static/streaming,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO streaming.StreamingContext: StreamingContext started
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 74.4 KB, free 1060.4 MB)
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 27.3 KB, free 1060.4 MB)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.0.0.199:59442 (size: 27.3 KB, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Adding task set 2.0 with 1 tasks
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 2.0 (TID 70, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 7067 bytes)
17/07/31 18:54:37 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 27.3 KB, free: 1060.5 MB)
17/07/31 18:54:37 INFO scheduler.JobScheduler: Added jobs for time 1501527277000 ms
17/07/31 18:54:37 INFO scheduler.JobScheduler: Starting job streaming job 1501527277000 ms.0 from job set of time 1501527277000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:37 INFO scheduler.JobScheduler: Finished job streaming job 1501527277000 ms.0 from job set of time 1501527277000 ms
17/07/31 18:54:37 INFO scheduler.JobScheduler: Total delay: 0.071 s for time 1501527277000 ms (execution: 0.014 s)
17/07/31 18:54:37 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/07/31 18:54:37 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/07/31 18:54:37 INFO scheduler.ReceiverTracker: Registered receiver for stream 0 from 10.0.0.229:51685
17/07/31 18:54:38 INFO scheduler.JobScheduler: Added jobs for time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.JobScheduler: Starting job streaming job 1501527278000 ms.0 from job set of time 1501527278000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:38 INFO scheduler.JobScheduler: Finished job streaming job 1501527278000 ms.0 from job set of time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.JobScheduler: Total delay: 0.045 s for time 1501527278000 ms (execution: 0.010 s)
17/07/31 18:54:38 INFO python.PythonRDD: Removing RDD 5 from persistence list
17/07/31 18:54:38 INFO rdd.BlockRDD: Removing RDD 4 from persistence list
17/07/31 18:54:38 INFO storage.BlockManager: Removing RDD 5
17/07/31 18:54:38 INFO storage.BlockManager: Removing RDD 4
17/07/31 18:54:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[4] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/07/31 18:54:38 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/07/31 18:54:38 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.151:57862) with ID 2
17/07/31 18:54:38 INFO spark.ExecutorAllocationManager: New executor 2 has registered (new total is 2)
17/07/31 18:54:38 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-151.ec2.internal:54365 with 1060.5 MB RAM, BlockManagerId(2, ip-10-0-0-151.ec2.internal, 54365, None)
17/07/31 18:54:39 INFO scheduler.JobScheduler: Added jobs for time 1501527279000 ms
17/07/31 18:54:39 INFO scheduler.JobScheduler: Starting job streaming job 1501527279000 ms.0 from job set of time 1501527279000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:39 INFO scheduler.JobScheduler: Finished job streaming job 1501527279000 ms.0 from job set of time 1501527279000 ms
17/07/31 18:54:39 INFO python.PythonRDD: Removing RDD 7 from persistence list
17/07/31 18:54:39 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527279000 ms (execution: 0.006 s)
17/07/31 18:54:39 INFO storage.BlockManager: Removing RDD 7
17/07/31 18:54:39 INFO rdd.BlockRDD: Removing RDD 6 from persistence list
17/07/31 18:54:39 INFO storage.BlockManager: Removing RDD 6
17/07/31 18:54:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[6] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527279000 ms
17/07/31 18:54:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527277000 ms
17/07/31 18:54:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527277000 ms
17/07/31 18:54:40 INFO scheduler.JobScheduler: Added jobs for time 1501527280000 ms
17/07/31 18:54:40 INFO scheduler.JobScheduler: Starting job streaming job 1501527280000 ms.0 from job set of time 1501527280000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:40 INFO scheduler.JobScheduler: Finished job streaming job 1501527280000 ms.0 from job set of time 1501527280000 ms
17/07/31 18:54:40 INFO python.PythonRDD: Removing RDD 9 from persistence list
17/07/31 18:54:40 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501527280000 ms (execution: 0.009 s)
17/07/31 18:54:40 INFO storage.BlockManager: Removing RDD 9
17/07/31 18:54:40 INFO rdd.BlockRDD: Removing RDD 8 from persistence list
17/07/31 18:54:40 INFO storage.BlockManager: Removing RDD 8
17/07/31 18:54:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[8] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527280000 ms
17/07/31 18:54:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527278000 ms
17/07/31 18:54:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527278000 ms
17/07/31 18:54:41 INFO scheduler.JobScheduler: Added jobs for time 1501527281000 ms
17/07/31 18:54:41 INFO scheduler.JobScheduler: Starting job streaming job 1501527281000 ms.0 from job set of time 1501527281000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:41 INFO scheduler.JobScheduler: Finished job streaming job 1501527281000 ms.0 from job set of time 1501527281000 ms
17/07/31 18:54:41 INFO python.PythonRDD: Removing RDD 11 from persistence list
17/07/31 18:54:41 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527281000 ms (execution: 0.006 s)
17/07/31 18:54:41 INFO storage.BlockManager: Removing RDD 11
17/07/31 18:54:41 INFO rdd.BlockRDD: Removing RDD 10 from persistence list
17/07/31 18:54:41 INFO storage.BlockManager: Removing RDD 10
17/07/31 18:54:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[10] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527281000 ms
17/07/31 18:54:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527279000 ms
17/07/31 18:54:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527279000 ms
17/07/31 18:54:42 INFO scheduler.JobScheduler: Added jobs for time 1501527282000 ms
17/07/31 18:54:42 INFO scheduler.JobScheduler: Starting job streaming job 1501527282000 ms.0 from job set of time 1501527282000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:42 INFO scheduler.JobScheduler: Finished job streaming job 1501527282000 ms.0 from job set of time 1501527282000 ms
17/07/31 18:54:42 INFO python.PythonRDD: Removing RDD 13 from persistence list
17/07/31 18:54:42 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527282000 ms (execution: 0.006 s)
17/07/31 18:54:42 INFO storage.BlockManager: Removing RDD 13
17/07/31 18:54:42 INFO rdd.BlockRDD: Removing RDD 12 from persistence list
17/07/31 18:54:42 INFO storage.BlockManager: Removing RDD 12
17/07/31 18:54:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[12] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527282000 ms
17/07/31 18:54:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527280000 ms
17/07/31 18:54:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527280000 ms
17/07/31 18:54:43 INFO scheduler.JobScheduler: Added jobs for time 1501527283000 ms
17/07/31 18:54:43 INFO scheduler.JobScheduler: Starting job streaming job 1501527283000 ms.0 from job set of time 1501527283000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:43 INFO scheduler.JobScheduler: Finished job streaming job 1501527283000 ms.0 from job set of time 1501527283000 ms
17/07/31 18:54:43 INFO python.PythonRDD: Removing RDD 15 from persistence list
17/07/31 18:54:43 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527283000 ms (execution: 0.008 s)
17/07/31 18:54:43 INFO storage.BlockManager: Removing RDD 15
17/07/31 18:54:43 INFO rdd.BlockRDD: Removing RDD 14 from persistence list
17/07/31 18:54:43 INFO storage.BlockManager: Removing RDD 14
17/07/31 18:54:43 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[14] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527283000 ms
17/07/31 18:54:43 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527281000 ms
17/07/31 18:54:43 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527281000 ms
17/07/31 18:54:44 INFO scheduler.JobScheduler: Added jobs for time 1501527284000 ms
17/07/31 18:54:44 INFO scheduler.JobScheduler: Starting job streaming job 1501527284000 ms.0 from job set of time 1501527284000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:44 INFO scheduler.JobScheduler: Finished job streaming job 1501527284000 ms.0 from job set of time 1501527284000 ms
17/07/31 18:54:44 INFO python.PythonRDD: Removing RDD 17 from persistence list
17/07/31 18:54:44 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527284000 ms (execution: 0.016 s)
17/07/31 18:54:44 INFO storage.BlockManager: Removing RDD 17
17/07/31 18:54:44 INFO rdd.BlockRDD: Removing RDD 16 from persistence list
17/07/31 18:54:44 INFO storage.BlockManager: Removing RDD 16
17/07/31 18:54:44 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[16] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527284000 ms
17/07/31 18:54:44 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527282000 ms
17/07/31 18:54:44 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527282000 ms
17/07/31 18:54:45 INFO scheduler.JobScheduler: Added jobs for time 1501527285000 ms
17/07/31 18:54:45 INFO scheduler.JobScheduler: Starting job streaming job 1501527285000 ms.0 from job set of time 1501527285000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:45 INFO scheduler.JobScheduler: Finished job streaming job 1501527285000 ms.0 from job set of time 1501527285000 ms
17/07/31 18:54:45 INFO python.PythonRDD: Removing RDD 19 from persistence list
17/07/31 18:54:45 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527285000 ms (execution: 0.007 s)
17/07/31 18:54:45 INFO rdd.BlockRDD: Removing RDD 18 from persistence list
17/07/31 18:54:45 INFO storage.BlockManager: Removing RDD 19
17/07/31 18:54:45 INFO storage.BlockManager: Removing RDD 18
17/07/31 18:54:45 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[18] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527285000 ms
17/07/31 18:54:45 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527283000 ms
17/07/31 18:54:45 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527283000 ms
17/07/31 18:54:46 INFO scheduler.JobScheduler: Added jobs for time 1501527286000 ms
17/07/31 18:54:46 INFO scheduler.JobScheduler: Starting job streaming job 1501527286000 ms.0 from job set of time 1501527286000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:46 INFO scheduler.JobScheduler: Finished job streaming job 1501527286000 ms.0 from job set of time 1501527286000 ms
17/07/31 18:54:46 INFO python.PythonRDD: Removing RDD 21 from persistence list
17/07/31 18:54:46 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527286000 ms (execution: 0.008 s)
17/07/31 18:54:46 INFO storage.BlockManager: Removing RDD 21
17/07/31 18:54:46 INFO rdd.BlockRDD: Removing RDD 20 from persistence list
17/07/31 18:54:46 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[20] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527286000 ms
17/07/31 18:54:46 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527284000 ms
17/07/31 18:54:46 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527284000 ms
17/07/31 18:54:46 INFO storage.BlockManager: Removing RDD 20
17/07/31 18:54:47 INFO scheduler.JobScheduler: Added jobs for time 1501527287000 ms
17/07/31 18:54:47 INFO scheduler.JobScheduler: Starting job streaming job 1501527287000 ms.0 from job set of time 1501527287000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:47 INFO scheduler.JobScheduler: Finished job streaming job 1501527287000 ms.0 from job set of time 1501527287000 ms
17/07/31 18:54:47 INFO python.PythonRDD: Removing RDD 23 from persistence list
17/07/31 18:54:47 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527287000 ms (execution: 0.006 s)
17/07/31 18:54:47 INFO storage.BlockManager: Removing RDD 23
17/07/31 18:54:47 INFO rdd.BlockRDD: Removing RDD 22 from persistence list
17/07/31 18:54:47 INFO storage.BlockManager: Removing RDD 22
17/07/31 18:54:47 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[22] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527287000 ms
17/07/31 18:54:47 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527285000 ms
17/07/31 18:54:47 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527285000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Added jobs for time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Starting job streaming job 1501527288000 ms.0 from job set of time 1501527288000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:48 INFO scheduler.JobScheduler: Finished job streaming job 1501527288000 ms.0 from job set of time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527288000 ms (execution: 0.008 s)
17/07/31 18:54:48 INFO python.PythonRDD: Removing RDD 25 from persistence list
17/07/31 18:54:48 INFO storage.BlockManager: Removing RDD 25
17/07/31 18:54:48 INFO rdd.BlockRDD: Removing RDD 24 from persistence list
17/07/31 18:54:48 INFO storage.BlockManager: Removing RDD 24
17/07/31 18:54:48 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[24] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527286000 ms
17/07/31 18:54:48 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527286000 ms
17/07/31 18:54:49 INFO scheduler.JobScheduler: Added jobs for time 1501527289000 ms
17/07/31 18:54:49 INFO scheduler.JobScheduler: Starting job streaming job 1501527289000 ms.0 from job set of time 1501527289000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:49 INFO scheduler.JobScheduler: Finished job streaming job 1501527289000 ms.0 from job set of time 1501527289000 ms
17/07/31 18:54:49 INFO python.PythonRDD: Removing RDD 27 from persistence list
17/07/31 18:54:49 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501527289000 ms (execution: 0.008 s)
17/07/31 18:54:49 INFO storage.BlockManager: Removing RDD 27
17/07/31 18:54:49 INFO rdd.BlockRDD: Removing RDD 26 from persistence list
17/07/31 18:54:49 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[26] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527289000 ms
17/07/31 18:54:49 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527287000 ms
17/07/31 18:54:49 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527287000 ms
17/07/31 18:54:49 INFO storage.BlockManager: Removing RDD 26
17/07/31 18:54:50 INFO scheduler.JobScheduler: Added jobs for time 1501527290000 ms
17/07/31 18:54:50 INFO scheduler.JobScheduler: Starting job streaming job 1501527290000 ms.0 from job set of time 1501527290000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:50 INFO scheduler.JobScheduler: Finished job streaming job 1501527290000 ms.0 from job set of time 1501527290000 ms
17/07/31 18:54:50 INFO python.PythonRDD: Removing RDD 29 from persistence list
17/07/31 18:54:50 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527290000 ms (execution: 0.008 s)
17/07/31 18:54:50 INFO storage.BlockManager: Removing RDD 29
17/07/31 18:54:50 INFO rdd.BlockRDD: Removing RDD 28 from persistence list
17/07/31 18:54:50 INFO storage.BlockManager: Removing RDD 28
17/07/31 18:54:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[28] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527290000 ms
17/07/31 18:54:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527288000 ms
17/07/31 18:54:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527288000 ms
17/07/31 18:54:51 INFO scheduler.JobScheduler: Added jobs for time 1501527291000 ms
17/07/31 18:54:51 INFO scheduler.JobScheduler: Starting job streaming job 1501527291000 ms.0 from job set of time 1501527291000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:51 INFO scheduler.JobScheduler: Finished job streaming job 1501527291000 ms.0 from job set of time 1501527291000 ms
17/07/31 18:54:51 INFO python.PythonRDD: Removing RDD 31 from persistence list
17/07/31 18:54:51 INFO scheduler.JobScheduler: Total delay: 0.033 s for time 1501527291000 ms (execution: 0.018 s)
17/07/31 18:54:51 INFO storage.BlockManager: Removing RDD 31
17/07/31 18:54:51 INFO rdd.BlockRDD: Removing RDD 30 from persistence list
17/07/31 18:54:51 INFO storage.BlockManager: Removing RDD 30
17/07/31 18:54:51 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[30] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527291000 ms
17/07/31 18:54:51 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527289000 ms
17/07/31 18:54:51 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527289000 ms
17/07/31 18:54:52 INFO scheduler.JobScheduler: Added jobs for time 1501527292000 ms
17/07/31 18:54:52 INFO scheduler.JobScheduler: Starting job streaming job 1501527292000 ms.0 from job set of time 1501527292000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:52 INFO scheduler.JobScheduler: Finished job streaming job 1501527292000 ms.0 from job set of time 1501527292000 ms
17/07/31 18:54:52 INFO python.PythonRDD: Removing RDD 33 from persistence list
17/07/31 18:54:52 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501527292000 ms (execution: 0.008 s)
17/07/31 18:54:52 INFO storage.BlockManager: Removing RDD 33
17/07/31 18:54:52 INFO rdd.BlockRDD: Removing RDD 32 from persistence list
17/07/31 18:54:52 INFO storage.BlockManager: Removing RDD 32
17/07/31 18:54:52 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[32] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527292000 ms
17/07/31 18:54:52 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527290000 ms
17/07/31 18:54:52 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527290000 ms
17/07/31 18:54:53 INFO scheduler.JobScheduler: Added jobs for time 1501527293000 ms
17/07/31 18:54:53 INFO scheduler.JobScheduler: Starting job streaming job 1501527293000 ms.0 from job set of time 1501527293000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:53 INFO scheduler.JobScheduler: Finished job streaming job 1501527293000 ms.0 from job set of time 1501527293000 ms
17/07/31 18:54:53 INFO python.PythonRDD: Removing RDD 35 from persistence list
17/07/31 18:54:53 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501527293000 ms (execution: 0.008 s)
17/07/31 18:54:53 INFO storage.BlockManager: Removing RDD 35
17/07/31 18:54:53 INFO rdd.BlockRDD: Removing RDD 34 from persistence list
17/07/31 18:54:53 INFO storage.BlockManager: Removing RDD 34
17/07/31 18:54:53 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[34] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527293000 ms
17/07/31 18:54:53 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527291000 ms
17/07/31 18:54:53 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527291000 ms
17/07/31 18:54:54 INFO scheduler.JobScheduler: Added jobs for time 1501527294000 ms
17/07/31 18:54:54 INFO scheduler.JobScheduler: Starting job streaming job 1501527294000 ms.0 from job set of time 1501527294000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:54 INFO scheduler.JobScheduler: Finished job streaming job 1501527294000 ms.0 from job set of time 1501527294000 ms
17/07/31 18:54:54 INFO python.PythonRDD: Removing RDD 37 from persistence list
17/07/31 18:54:54 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527294000 ms (execution: 0.006 s)
17/07/31 18:54:54 INFO storage.BlockManager: Removing RDD 37
17/07/31 18:54:54 INFO rdd.BlockRDD: Removing RDD 36 from persistence list
17/07/31 18:54:54 INFO storage.BlockManager: Removing RDD 36
17/07/31 18:54:54 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[36] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527294000 ms
17/07/31 18:54:54 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527292000 ms
17/07/31 18:54:54 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527292000 ms
17/07/31 18:54:55 INFO scheduler.JobScheduler: Added jobs for time 1501527295000 ms
17/07/31 18:54:55 INFO scheduler.JobScheduler: Starting job streaming job 1501527295000 ms.0 from job set of time 1501527295000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:55 INFO scheduler.JobScheduler: Finished job streaming job 1501527295000 ms.0 from job set of time 1501527295000 ms
17/07/31 18:54:55 INFO python.PythonRDD: Removing RDD 39 from persistence list
17/07/31 18:54:55 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527295000 ms (execution: 0.007 s)
17/07/31 18:54:55 INFO storage.BlockManager: Removing RDD 39
17/07/31 18:54:55 INFO rdd.BlockRDD: Removing RDD 38 from persistence list
17/07/31 18:54:55 INFO storage.BlockManager: Removing RDD 38
17/07/31 18:54:55 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[38] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527295000 ms
17/07/31 18:54:55 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527293000 ms
17/07/31 18:54:55 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527293000 ms
17/07/31 18:54:56 INFO scheduler.JobScheduler: Added jobs for time 1501527296000 ms
17/07/31 18:54:56 INFO scheduler.JobScheduler: Starting job streaming job 1501527296000 ms.0 from job set of time 1501527296000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:56 INFO scheduler.JobScheduler: Finished job streaming job 1501527296000 ms.0 from job set of time 1501527296000 ms
17/07/31 18:54:56 INFO python.PythonRDD: Removing RDD 41 from persistence list
17/07/31 18:54:56 INFO scheduler.JobScheduler: Total delay: 0.031 s for time 1501527296000 ms (execution: 0.007 s)
17/07/31 18:54:56 INFO storage.BlockManager: Removing RDD 41
17/07/31 18:54:56 INFO rdd.BlockRDD: Removing RDD 40 from persistence list
17/07/31 18:54:56 INFO storage.BlockManager: Removing RDD 40
17/07/31 18:54:56 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[40] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527296000 ms
17/07/31 18:54:56 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527294000 ms
17/07/31 18:54:56 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527294000 ms
17/07/31 18:54:57 INFO scheduler.JobScheduler: Added jobs for time 1501527297000 ms
17/07/31 18:54:57 INFO scheduler.JobScheduler: Starting job streaming job 1501527297000 ms.0 from job set of time 1501527297000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:57 INFO scheduler.JobScheduler: Finished job streaming job 1501527297000 ms.0 from job set of time 1501527297000 ms
17/07/31 18:54:57 INFO python.PythonRDD: Removing RDD 43 from persistence list
17/07/31 18:54:57 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527297000 ms (execution: 0.008 s)
17/07/31 18:54:57 INFO storage.BlockManager: Removing RDD 43
17/07/31 18:54:57 INFO rdd.BlockRDD: Removing RDD 42 from persistence list
17/07/31 18:54:57 INFO storage.BlockManager: Removing RDD 42
17/07/31 18:54:57 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[42] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527297000 ms
17/07/31 18:54:57 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527295000 ms
17/07/31 18:54:57 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527295000 ms
17/07/31 18:54:58 INFO scheduler.JobScheduler: Added jobs for time 1501527298000 ms
17/07/31 18:54:58 INFO scheduler.JobScheduler: Starting job streaming job 1501527298000 ms.0 from job set of time 1501527298000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:58 INFO scheduler.JobScheduler: Finished job streaming job 1501527298000 ms.0 from job set of time 1501527298000 ms
17/07/31 18:54:58 INFO python.PythonRDD: Removing RDD 45 from persistence list
17/07/31 18:54:58 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527298000 ms (execution: 0.008 s)
17/07/31 18:54:58 INFO storage.BlockManager: Removing RDD 45
17/07/31 18:54:58 INFO rdd.BlockRDD: Removing RDD 44 from persistence list
17/07/31 18:54:58 INFO storage.BlockManager: Removing RDD 44
17/07/31 18:54:58 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[44] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527298000 ms
17/07/31 18:54:58 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527296000 ms
17/07/31 18:54:58 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527296000 ms
17/07/31 18:54:59 INFO scheduler.JobScheduler: Added jobs for time 1501527299000 ms
17/07/31 18:54:59 INFO scheduler.JobScheduler: Starting job streaming job 1501527299000 ms.0 from job set of time 1501527299000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:59 INFO scheduler.JobScheduler: Finished job streaming job 1501527299000 ms.0 from job set of time 1501527299000 ms
17/07/31 18:54:59 INFO python.PythonRDD: Removing RDD 47 from persistence list
17/07/31 18:54:59 INFO scheduler.JobScheduler: Total delay: 0.031 s for time 1501527299000 ms (execution: 0.009 s)
17/07/31 18:54:59 INFO storage.BlockManager: Removing RDD 47
17/07/31 18:54:59 INFO rdd.BlockRDD: Removing RDD 46 from persistence list
17/07/31 18:54:59 INFO storage.BlockManager: Removing RDD 46
17/07/31 18:54:59 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[46] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527299000 ms
17/07/31 18:54:59 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527297000 ms
17/07/31 18:54:59 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527297000 ms
17/07/31 18:55:00 INFO scheduler.JobScheduler: Added jobs for time 1501527300000 ms
17/07/31 18:55:00 INFO scheduler.JobScheduler: Starting job streaming job 1501527300000 ms.0 from job set of time 1501527300000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:00 INFO scheduler.JobScheduler: Finished job streaming job 1501527300000 ms.0 from job set of time 1501527300000 ms
17/07/31 18:55:00 INFO python.PythonRDD: Removing RDD 49 from persistence list
17/07/31 18:55:00 INFO scheduler.JobScheduler: Total delay: 0.035 s for time 1501527300000 ms (execution: 0.021 s)
17/07/31 18:55:00 INFO storage.BlockManager: Removing RDD 49
17/07/31 18:55:00 INFO rdd.BlockRDD: Removing RDD 48 from persistence list
17/07/31 18:55:00 INFO storage.BlockManager: Removing RDD 48
17/07/31 18:55:00 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[48] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527300000 ms
17/07/31 18:55:00 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527298000 ms
17/07/31 18:55:00 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527298000 ms
17/07/31 18:55:01 INFO scheduler.JobScheduler: Added jobs for time 1501527301000 ms
17/07/31 18:55:01 INFO scheduler.JobScheduler: Starting job streaming job 1501527301000 ms.0 from job set of time 1501527301000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:01 INFO scheduler.JobScheduler: Finished job streaming job 1501527301000 ms.0 from job set of time 1501527301000 ms
17/07/31 18:55:01 INFO python.PythonRDD: Removing RDD 51 from persistence list
17/07/31 18:55:01 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527301000 ms (execution: 0.008 s)
17/07/31 18:55:01 INFO storage.BlockManager: Removing RDD 51
17/07/31 18:55:01 INFO rdd.BlockRDD: Removing RDD 50 from persistence list
17/07/31 18:55:01 INFO storage.BlockManager: Removing RDD 50
17/07/31 18:55:01 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[50] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527301000 ms
17/07/31 18:55:01 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527299000 ms
17/07/31 18:55:01 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527299000 ms
17/07/31 18:55:02 INFO scheduler.JobScheduler: Added jobs for time 1501527302000 ms
17/07/31 18:55:02 INFO scheduler.JobScheduler: Starting job streaming job 1501527302000 ms.0 from job set of time 1501527302000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:02 INFO scheduler.JobScheduler: Finished job streaming job 1501527302000 ms.0 from job set of time 1501527302000 ms
17/07/31 18:55:02 INFO python.PythonRDD: Removing RDD 53 from persistence list
17/07/31 18:55:02 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501527302000 ms (execution: 0.009 s)
17/07/31 18:55:02 INFO storage.BlockManager: Removing RDD 53
17/07/31 18:55:02 INFO rdd.BlockRDD: Removing RDD 52 from persistence list
17/07/31 18:55:02 INFO storage.BlockManager: Removing RDD 52
17/07/31 18:55:02 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[52] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527302000 ms
17/07/31 18:55:02 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527300000 ms
17/07/31 18:55:02 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527300000 ms
17/07/31 18:55:03 INFO scheduler.JobScheduler: Added jobs for time 1501527303000 ms
17/07/31 18:55:03 INFO scheduler.JobScheduler: Starting job streaming job 1501527303000 ms.0 from job set of time 1501527303000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:03 INFO scheduler.JobScheduler: Finished job streaming job 1501527303000 ms.0 from job set of time 1501527303000 ms
17/07/31 18:55:03 INFO python.PythonRDD: Removing RDD 55 from persistence list
17/07/31 18:55:03 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527303000 ms (execution: 0.008 s)
17/07/31 18:55:03 INFO storage.BlockManager: Removing RDD 55
17/07/31 18:55:03 INFO rdd.BlockRDD: Removing RDD 54 from persistence list
17/07/31 18:55:03 INFO storage.BlockManager: Removing RDD 54
17/07/31 18:55:03 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[54] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527303000 ms
17/07/31 18:55:03 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527301000 ms
17/07/31 18:55:03 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527301000 ms
17/07/31 18:55:04 INFO scheduler.JobScheduler: Added jobs for time 1501527304000 ms
17/07/31 18:55:04 INFO scheduler.JobScheduler: Starting job streaming job 1501527304000 ms.0 from job set of time 1501527304000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:04 INFO scheduler.JobScheduler: Finished job streaming job 1501527304000 ms.0 from job set of time 1501527304000 ms
17/07/31 18:55:04 INFO python.PythonRDD: Removing RDD 57 from persistence list
17/07/31 18:55:04 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501527304000 ms (execution: 0.008 s)
17/07/31 18:55:04 INFO storage.BlockManager: Removing RDD 57
17/07/31 18:55:04 INFO rdd.BlockRDD: Removing RDD 56 from persistence list
17/07/31 18:55:04 INFO storage.BlockManager: Removing RDD 56
17/07/31 18:55:04 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[56] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527304000 ms
17/07/31 18:55:04 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527302000 ms
17/07/31 18:55:04 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527302000 ms
17/07/31 18:55:05 INFO scheduler.JobScheduler: Added jobs for time 1501527305000 ms
17/07/31 18:55:05 INFO scheduler.JobScheduler: Starting job streaming job 1501527305000 ms.0 from job set of time 1501527305000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:05 INFO scheduler.JobScheduler: Finished job streaming job 1501527305000 ms.0 from job set of time 1501527305000 ms
17/07/31 18:55:05 INFO python.PythonRDD: Removing RDD 59 from persistence list
17/07/31 18:55:05 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527305000 ms (execution: 0.008 s)
17/07/31 18:55:05 INFO storage.BlockManager: Removing RDD 59
17/07/31 18:55:05 INFO rdd.BlockRDD: Removing RDD 58 from persistence list
17/07/31 18:55:05 INFO storage.BlockManager: Removing RDD 58
17/07/31 18:55:05 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[58] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527305000 ms
17/07/31 18:55:05 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527303000 ms
17/07/31 18:55:05 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527303000 ms
17/07/31 18:55:06 INFO scheduler.JobScheduler: Added jobs for time 1501527306000 ms
17/07/31 18:55:06 INFO scheduler.JobScheduler: Starting job streaming job 1501527306000 ms.0 from job set of time 1501527306000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:06 INFO scheduler.JobScheduler: Finished job streaming job 1501527306000 ms.0 from job set of time 1501527306000 ms
17/07/31 18:55:06 INFO python.PythonRDD: Removing RDD 61 from persistence list
17/07/31 18:55:06 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527306000 ms (execution: 0.007 s)
17/07/31 18:55:06 INFO storage.BlockManager: Removing RDD 61
17/07/31 18:55:06 INFO rdd.BlockRDD: Removing RDD 60 from persistence list
17/07/31 18:55:06 INFO storage.BlockManager: Removing RDD 60
17/07/31 18:55:06 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[60] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527306000 ms
17/07/31 18:55:06 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527304000 ms
17/07/31 18:55:06 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527304000 ms
17/07/31 18:55:07 INFO scheduler.JobScheduler: Added jobs for time 1501527307000 ms
17/07/31 18:55:07 INFO scheduler.JobScheduler: Starting job streaming job 1501527307000 ms.0 from job set of time 1501527307000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:07 INFO scheduler.JobScheduler: Finished job streaming job 1501527307000 ms.0 from job set of time 1501527307000 ms
17/07/31 18:55:07 INFO python.PythonRDD: Removing RDD 63 from persistence list
17/07/31 18:55:07 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501527307000 ms (execution: 0.007 s)
17/07/31 18:55:07 INFO storage.BlockManager: Removing RDD 63
17/07/31 18:55:07 INFO rdd.BlockRDD: Removing RDD 62 from persistence list
17/07/31 18:55:07 INFO storage.BlockManager: Removing RDD 62
17/07/31 18:55:07 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[62] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527307000 ms
17/07/31 18:55:07 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527305000 ms
17/07/31 18:55:07 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527305000 ms
17/07/31 18:55:08 INFO scheduler.JobScheduler: Added jobs for time 1501527308000 ms
17/07/31 18:55:08 INFO scheduler.JobScheduler: Starting job streaming job 1501527308000 ms.0 from job set of time 1501527308000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:08 INFO scheduler.JobScheduler: Finished job streaming job 1501527308000 ms.0 from job set of time 1501527308000 ms
17/07/31 18:55:08 INFO python.PythonRDD: Removing RDD 65 from persistence list
17/07/31 18:55:08 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527308000 ms (execution: 0.008 s)
17/07/31 18:55:08 INFO storage.BlockManager: Removing RDD 65
17/07/31 18:55:08 INFO rdd.BlockRDD: Removing RDD 64 from persistence list
17/07/31 18:55:08 INFO storage.BlockManager: Removing RDD 64
17/07/31 18:55:08 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[64] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527308000 ms
17/07/31 18:55:08 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527306000 ms
17/07/31 18:55:08 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527306000 ms
17/07/31 18:55:09 INFO scheduler.JobScheduler: Added jobs for time 1501527309000 ms
17/07/31 18:55:09 INFO scheduler.JobScheduler: Starting job streaming job 1501527309000 ms.0 from job set of time 1501527309000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:09 INFO scheduler.JobScheduler: Finished job streaming job 1501527309000 ms.0 from job set of time 1501527309000 ms
17/07/31 18:55:09 INFO python.PythonRDD: Removing RDD 67 from persistence list
17/07/31 18:55:09 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527309000 ms (execution: 0.015 s)
17/07/31 18:55:09 INFO storage.BlockManager: Removing RDD 67
17/07/31 18:55:09 INFO rdd.BlockRDD: Removing RDD 66 from persistence list
17/07/31 18:55:09 INFO storage.BlockManager: Removing RDD 66
17/07/31 18:55:09 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[66] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527309000 ms
17/07/31 18:55:09 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527307000 ms
17/07/31 18:55:09 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527307000 ms
17/07/31 18:55:10 INFO scheduler.JobScheduler: Added jobs for time 1501527310000 ms
17/07/31 18:55:10 INFO scheduler.JobScheduler: Starting job streaming job 1501527310000 ms.0 from job set of time 1501527310000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:10 INFO scheduler.JobScheduler: Finished job streaming job 1501527310000 ms.0 from job set of time 1501527310000 ms
17/07/31 18:55:10 INFO python.PythonRDD: Removing RDD 69 from persistence list
17/07/31 18:55:10 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527310000 ms (execution: 0.007 s)
17/07/31 18:55:10 INFO storage.BlockManager: Removing RDD 69
17/07/31 18:55:10 INFO rdd.BlockRDD: Removing RDD 68 from persistence list
17/07/31 18:55:10 INFO storage.BlockManager: Removing RDD 68
17/07/31 18:55:10 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[68] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527310000 ms
17/07/31 18:55:10 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527308000 ms
17/07/31 18:55:10 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527308000 ms
17/07/31 18:55:11 INFO scheduler.JobScheduler: Added jobs for time 1501527311000 ms
17/07/31 18:55:11 INFO scheduler.JobScheduler: Starting job streaming job 1501527311000 ms.0 from job set of time 1501527311000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:11 INFO scheduler.JobScheduler: Finished job streaming job 1501527311000 ms.0 from job set of time 1501527311000 ms
17/07/31 18:55:11 INFO python.PythonRDD: Removing RDD 71 from persistence list
17/07/31 18:55:11 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527311000 ms (execution: 0.007 s)
17/07/31 18:55:11 INFO storage.BlockManager: Removing RDD 71
17/07/31 18:55:11 INFO rdd.BlockRDD: Removing RDD 70 from persistence list
17/07/31 18:55:11 INFO storage.BlockManager: Removing RDD 70
17/07/31 18:55:11 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[70] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527311000 ms
17/07/31 18:55:11 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527309000 ms
17/07/31 18:55:11 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527309000 ms
17/07/31 18:55:12 INFO scheduler.JobScheduler: Added jobs for time 1501527312000 ms
17/07/31 18:55:12 INFO scheduler.JobScheduler: Starting job streaming job 1501527312000 ms.0 from job set of time 1501527312000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:12 INFO scheduler.JobScheduler: Finished job streaming job 1501527312000 ms.0 from job set of time 1501527312000 ms
17/07/31 18:55:12 INFO python.PythonRDD: Removing RDD 73 from persistence list
17/07/31 18:55:12 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501527312000 ms (execution: 0.008 s)
17/07/31 18:55:12 INFO storage.BlockManager: Removing RDD 73
17/07/31 18:55:12 INFO rdd.BlockRDD: Removing RDD 72 from persistence list
17/07/31 18:55:12 INFO storage.BlockManager: Removing RDD 72
17/07/31 18:55:12 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[72] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527312000 ms
17/07/31 18:55:12 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527310000 ms
17/07/31 18:55:12 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527310000 ms
17/07/31 18:55:13 INFO scheduler.JobScheduler: Added jobs for time 1501527313000 ms
17/07/31 18:55:13 INFO scheduler.JobScheduler: Starting job streaming job 1501527313000 ms.0 from job set of time 1501527313000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:13 INFO scheduler.JobScheduler: Finished job streaming job 1501527313000 ms.0 from job set of time 1501527313000 ms
17/07/31 18:55:13 INFO python.PythonRDD: Removing RDD 75 from persistence list
17/07/31 18:55:13 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527313000 ms (execution: 0.007 s)
17/07/31 18:55:13 INFO storage.BlockManager: Removing RDD 75
17/07/31 18:55:13 INFO rdd.BlockRDD: Removing RDD 74 from persistence list
17/07/31 18:55:13 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[74] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527313000 ms
17/07/31 18:55:13 INFO storage.BlockManager: Removing RDD 74
17/07/31 18:55:13 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527311000 ms
17/07/31 18:55:13 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527311000 ms
17/07/31 18:55:14 INFO scheduler.JobScheduler: Added jobs for time 1501527314000 ms
17/07/31 18:55:14 INFO scheduler.JobScheduler: Starting job streaming job 1501527314000 ms.0 from job set of time 1501527314000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:14 INFO scheduler.JobScheduler: Finished job streaming job 1501527314000 ms.0 from job set of time 1501527314000 ms
17/07/31 18:55:14 INFO python.PythonRDD: Removing RDD 77 from persistence list
17/07/31 18:55:14 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527314000 ms (execution: 0.007 s)
17/07/31 18:55:14 INFO storage.BlockManager: Removing RDD 77
17/07/31 18:55:14 INFO rdd.BlockRDD: Removing RDD 76 from persistence list
17/07/31 18:55:14 INFO storage.BlockManager: Removing RDD 76
17/07/31 18:55:14 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[76] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527314000 ms
17/07/31 18:55:14 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527312000 ms
17/07/31 18:55:14 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527312000 ms
17/07/31 18:55:15 INFO scheduler.JobScheduler: Added jobs for time 1501527315000 ms
17/07/31 18:55:15 INFO scheduler.JobScheduler: Starting job streaming job 1501527315000 ms.0 from job set of time 1501527315000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:15 INFO scheduler.JobScheduler: Finished job streaming job 1501527315000 ms.0 from job set of time 1501527315000 ms
17/07/31 18:55:15 INFO python.PythonRDD: Removing RDD 79 from persistence list
17/07/31 18:55:15 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527315000 ms (execution: 0.007 s)
17/07/31 18:55:15 INFO storage.BlockManager: Removing RDD 79
17/07/31 18:55:15 INFO rdd.BlockRDD: Removing RDD 78 from persistence list
17/07/31 18:55:15 INFO storage.BlockManager: Removing RDD 78
17/07/31 18:55:15 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[78] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527315000 ms
17/07/31 18:55:15 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527313000 ms
17/07/31 18:55:15 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527313000 ms
17/07/31 18:55:16 INFO scheduler.JobScheduler: Added jobs for time 1501527316000 ms
17/07/31 18:55:16 INFO scheduler.JobScheduler: Starting job streaming job 1501527316000 ms.0 from job set of time 1501527316000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:16 INFO scheduler.JobScheduler: Finished job streaming job 1501527316000 ms.0 from job set of time 1501527316000 ms
17/07/31 18:55:16 INFO python.PythonRDD: Removing RDD 81 from persistence list
17/07/31 18:55:16 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501527316000 ms (execution: 0.014 s)
17/07/31 18:55:16 INFO storage.BlockManager: Removing RDD 81
17/07/31 18:55:16 INFO rdd.BlockRDD: Removing RDD 80 from persistence list
17/07/31 18:55:16 INFO storage.BlockManager: Removing RDD 80
17/07/31 18:55:16 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[80] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527316000 ms
17/07/31 18:55:16 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527314000 ms
17/07/31 18:55:16 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527314000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Added jobs for time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Starting job streaming job 1501527317000 ms.0 from job set of time 1501527317000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:17 INFO scheduler.JobScheduler: Finished job streaming job 1501527317000 ms.0 from job set of time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527317000 ms (execution: 0.011 s)
17/07/31 18:55:17 INFO python.PythonRDD: Removing RDD 83 from persistence list
17/07/31 18:55:17 INFO storage.BlockManager: Removing RDD 83
17/07/31 18:55:17 INFO rdd.BlockRDD: Removing RDD 82 from persistence list
17/07/31 18:55:17 INFO storage.BlockManager: Removing RDD 82
17/07/31 18:55:17 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[82] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527315000 ms
17/07/31 18:55:17 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527315000 ms
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1059.6 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1059.6 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.4 KB, free: 1057.9 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.4 KB, free: 1057.9 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 880.3 KB, free: 1057.0 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 880.3 KB, free: 1057.1 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.0 KB, free: 1056.2 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.0 KB, free: 1056.2 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527317800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.3 KB, free: 1054.5 MB)
17/07/31 18:55:18 INFO scheduler.JobScheduler: Added jobs for time 1501527318000 ms
17/07/31 18:55:18 INFO scheduler.JobScheduler: Starting job streaming job 1501527318000 ms.0 from job set of time 1501527318000 ms
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527317800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.3 KB, free: 1054.5 MB)
17/07/31 18:55:18 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Got job 2 (runJob at PythonRDD.scala:441) with 1 output partitions
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Final stage: ResultStage 3 (runJob at PythonRDD.scala:441)
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Submitting ResultStage 3 (PythonRDD[88] at RDD at PythonRDD.scala:48), which has no missing parents
17/07/31 18:55:18 INFO memory.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/07/31 18:55:18 INFO memory.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.0.0.199:59442 (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:18 INFO spark.SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:996
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (PythonRDD[88] at RDD at PythonRDD.scala:48)
17/07/31 18:55:18 INFO cluster.YarnScheduler: Adding task set 3.0 with 1 tasks
17/07/31 18:55:18 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 3.0 (TID 71, ip-10-0-0-151.ec2.internal, executor 2, partition 0, NODE_LOCAL, 5800 bytes)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 5.3 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.3 KB, free: 1052.7 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.3 KB, free: 1052.8 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.1 KB, free: 1050.2 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.1 KB, free: 1050.2 MB)
17/07/31 18:55:18 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 3.0 (TID 71) in 893 ms on ip-10-0-0-151.ec2.internal (executor 2) (1/1)
17/07/31 18:55:18 INFO cluster.YarnScheduler: Removed TaskSet 3.0, whose tasks have all completed, from pool
17/07/31 18:55:18 INFO scheduler.DAGScheduler: ResultStage 3 (runJob at PythonRDD.scala:441) finished in 0.896 s
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Job 2 finished: runJob at PythonRDD.scala:441, took 0.913501 s
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527318800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.5 KB, free: 1049.3 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527318800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.5 KB, free: 1049.3 MB)
17/07/31 18:55:19 INFO scheduler.JobScheduler: Added jobs for time 1501527319000 ms
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 887.9 KB, free: 1048.4 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 887.9 KB, free: 1048.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1047.6 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1047.6 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1046.7 MB)
17/07/31 18:55:19 INFO spark.ContextCleaner: Cleaned shuffle 0
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 10.0.0.199:59442 in memory (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1988.0 B, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on 10.0.0.199:59442 in memory (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1956.0 B, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on 10.0.0.199:59442 in memory (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 5.3 KB, free: 1046.8 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.2 KB, free: 1045.0 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.2 KB, free: 1045.0 MB)
17/07/31 18:55:19 INFO execution.SparkSqlParser: Parsing command: spotdb.proxy
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527319800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527319800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
17/07/31 18:55:20 INFO metastore.ObjectStore: ObjectStore, initialize called
17/07/31 18:55:20 INFO scheduler.JobScheduler: Added jobs for time 1501527320000 ms
17/07/31 18:55:20 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing
17/07/31 18:55:20 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
17/07/31 18:55:20 INFO metastore.ObjectStore: Initialized ObjectStore
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_table start=1501527320011 end=1501527320059 duration=48 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_table start=1501527320177 end=1501527320179 duration=2 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_database: spotdb
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: spotdb
17/07/31 18:55:20 WARN metastore.ObjectStore: Failed to get database spotdb, returning NoSuchObjectException
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527320239 end=1501527320242 duration=3 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527318000 ms.0 from job set of time 1501527318000 ms
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 2.266 s for time 1501527318000 ms (execution: 2.249 s)
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 85 from persistence list
17/07/31 18:55:20 INFO scheduler.JobScheduler: Starting job streaming job 1501527319000 ms.0 from job set of time 1501527319000 ms
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 84 from persistence list
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527318000 ms.0
org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 71, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u"Database 'spotdb' not found;"
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[84] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527318000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 85
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527316000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527316000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 84
17/07/31 18:55:20 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Got job 3 (runJob at PythonRDD.scala:441) with 1 output partitions
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Final stage: ResultStage 4 (runJob at PythonRDD.scala:441)
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Submitting ResultStage 4 (PythonRDD[97] at RDD at PythonRDD.scala:48), which has no missing parents
Traceback (most recent call last):
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 148, in <module>
main()
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 77, in main
bluecoat_parse(args.zk,args.topic,args.db,args.db_table,args.num_of_workers,args.batch_size)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 144, in bluecoat_parse
ssc.awaitTermination()
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/context.py", line 206, in awaitTermination
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o48.awaitTermination.
: org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 71, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u"Database 'spotdb' not found;"
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/07/31 18:55:20 INFO memory.MemoryStore: Block broadcast_4 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/07/31 18:55:20 INFO memory.MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on 10.0.0.199:59442 (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:20 INFO spark.SparkContext: Created broadcast 4 from broadcast at DAGScheduler.scala:996
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 4 (PythonRDD[97] at RDD at PythonRDD.scala:48)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Adding task set 4.0 with 1 tasks
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 4.0 (TID 72, ip-10-0-0-151.ec2.internal, executor 2, partition 0, NODE_LOCAL, 5800 bytes)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 5.3 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527319000 ms.0 from job set of time 1501527319000 ms
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 1.329 s for time 1501527319000 ms (execution: 0.051 s)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Starting job streaming job 1501527320000 ms.0 from job set of time 1501527320000 ms
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527319000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy36.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 87 from persistence list
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 86 from persistence list
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 87
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[86] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527319000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 86
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527317000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527317000 ms
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317000 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.2 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317400 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 880.3 KB, free: 1045.0 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317200 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.4 KB, free: 1046.7 MB)
17/07/31 18:55:20 INFO streaming.StreamingContext: Invoking stop(stopGracefully=false) from shutdown hook
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317600 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.0 KB, free: 1047.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317000 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.2 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317200 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.4 KB, free: 1045.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317400 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 880.3 KB, free: 1046.7 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317600 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.0 KB, free: 1047.6 MB)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527320000 ms.0 from job set of time 1501527320000 ms
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 90 from persistence list
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 0.333 s for time 1501527320000 ms (execution: 0.003 s)
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527320000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy36.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 89 from persistence list
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 90
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 89
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[89] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527320000 ms
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: Sent stop signal to all 1 receivers
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317800 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.3 KB, free: 1049.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318000 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.1 KB, free: 1050.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318200 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.3 KB, free: 1051.0 MB)
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527318000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527318000 ms
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318400 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318600 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.1 KB, free: 1053.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317800 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.3 KB, free: 1049.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318000 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.1 KB, free: 1050.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318200 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.3 KB, free: 1051.0 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318400 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318600 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.1 KB, free: 1053.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 880.6 KB, free: 1052.7 MB)
17/07/31 18:55:20 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 880.6 KB, free: 1052.8 MB)
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 4.0 (TID 72) in 289 ms on ip-10-0-0-151.ec2.internal (executor 2) (1/1)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Removed TaskSet 4.0, whose tasks have all completed, from pool
17/07/31 18:55:20 ERROR scheduler.DAGScheduler: Failed to update accumulators for task 0
org.apache.spark.SparkException: EOF reached before Python server acknowledged
at org.apache.spark.api.python.PythonAccumulatorV2.merge(PythonRDD.scala:919)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$updateAccumulators$1.apply(DAGScheduler.scala:1088)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$updateAccumulators$1.apply(DAGScheduler.scala:1080)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.DAGScheduler.updateAccumulators(DAGScheduler.scala:1080)
at org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:1156)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1647)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/07/31 18:55:20 INFO scheduler.DAGScheduler: ResultStage 4 (runJob at PythonRDD.scala:441) finished in 0.292 s
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Job 3 finished: runJob at PythonRDD.scala:441, took 0.309062 s
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 2.0 (TID 70) in 43848 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/1)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool
17/07/31 18:55:20 INFO scheduler.DAGScheduler: ResultStage 2 (start at NativeMethodAccessorImpl.java:0) finished in 43.849 s
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: All of the receivers have deregistered successfully
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
17/07/31 18:55:20 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
17/07/31 18:55:20 INFO util.RecurringTimer: Stopped timer for JobGenerator after time 1501527320000
17/07/31 18:55:20 INFO scheduler.JobGenerator: Stopped JobGenerator
17/07/31 18:55:20 INFO scheduler.JobScheduler: Stopped JobScheduler
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19fd4723{/streaming,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@22116fc3{/streaming/batch,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@70d301d6{/static/streaming,null,UNAVAILABLE}
17/07/31 18:55:20 INFO streaming.StreamingContext: StreamingContext stopped successfully
17/07/31 18:55:20 INFO spark.SparkContext: Invoking stop() from shutdown hook
17/07/31 18:55:20 INFO server.ServerConnector: Stopped ServerConnector@188e6c7f{HTTP/1.1}{0.0.0.0:4040}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4f734944{/stages/stage/kill,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@b7d4da0{/jobs/job/kill,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ab71bb9{/api,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5d37ce06{/,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@27f55b38{/static,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1159f15e{/executors/threadDump/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ff970c2{/executors/threadDump,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@615b86f5{/executors/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19bb7285{/executors,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3b9882ec{/environment/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7b3691b6{/environment,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@423a3429{/storage/rdd/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@13016b86{/storage/rdd,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ea732f0{/storage/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2d4aa15a{/storage,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5cbdbf0f{/stages/pool/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14c62558{/stages/pool,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5c61bd1a{/stages/stage/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4eb72ecd{/stages/stage,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1bf090df{/stages/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14b58fc0{/stages,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@30e71b5d{/jobs/job/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@232864a3{/jobs/job,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1f4da763{/jobs/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1a3b1e79{/jobs,null,UNAVAILABLE}
17/07/31 18:55:20 INFO ui.SparkUI: Stopped Spark web UI at http://10.0.0.199:4040
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors
17/07/31 18:55:20 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
17/07/31 18:55:20 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
services=List(),
started=false)
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Stopped
17/07/31 18:55:20 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/07/31 18:55:20 INFO memory.MemoryStore: MemoryStore cleared
17/07/31 18:55:20 INFO storage.BlockManager: BlockManager stopped
17/07/31 18:55:20 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
17/07/31 18:55:20 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/07/31 18:55:20 INFO spark.SparkContext: Successfully stopped SparkContext
17/07/31 18:55:20 INFO util.ShutdownHookManager: Shutdown hook called
17/07/31 18:55:20 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40
17/07/31 18:55:20 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40/pyspark-506939c9-a852-404b-adea-6ad40ad0d73f
Closing worker...
Thanks
Deon Griessel
RE: Spot Ingest Proxy Error
Posted by Deon Griessel <dg...@searchtechnologies.com>.
I replaced the following line in bluecoat.py:
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
with
df.write.format("parquet").mode("append").insertInto(hive_table)
That seemed to do the trick, it then bombed on bad input data in the sample data provided in: Bluecoat_ProxySG_Sample.log
Seems like one of the records has a duration/time-taken which is larger than int and thus caused the python script to die.
Anyhow for now I removed the offending line and was able to get it to run through.
-Deon
From: Barona, Ricardo [mailto:ricardo.barona@intel.com]
Sent: Tuesday, August 1, 2017 11:14 AM
To: user@spot.incubator.apache.org; user@spot.apache.org
Subject: Re: Spot Ingest Proxy Error
Hi Dean,
Let me ask around.
From: Deon Griessel <dg...@searchtechnologies.com>>
Reply-To: "user@spot.incubator.apache.org<ma...@spot.incubator.apache.org>" <us...@spot.incubator.apache.org>>
Date: Tuesday, August 1, 2017 at 10:04 AM
To: "user@spot.incubator.apache.org<ma...@spot.incubator.apache.org>" <us...@spot.incubator.apache.org>>, "user@spot.apache.org<ma...@spot.apache.org>" <us...@spot.apache.org>>
Subject: RE: Spot Ingest Proxy Error
Ok, it seems like I got around the original error by enabling the hive service in Spark2 configuration in Cloudera Manager and restarting all services.
Now getting the following error:
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
Anybody seen this before? Full log below.
Thanks
---------------------------
2017-08-01 14:49:13,694 - SPOT.INGEST.WORKER - INFO - Initializing kafka instance
2017-08-01 14:49:13,694 - SPOT.INGEST.WRK.PROXY - INFO - Creating Spark Job for topic: SPOT-INGEST-proxy-14_49_11
2017-08-01 14:49:13,694 - SPOT.INGEST.WRK.PROXY - INFO - SPOT.Utils: Executing: spark2-submit --master yarn --driver-memory 2325m --num-executors 1 --conf spark.executor.memory=2325m --conf spark.executor.cores=1 --jars /home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar /home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py -zk ip-10-0-0-199.ec2.internal:2181 -t SPOT-INGEST-proxy-14_49_11 -db spotdb -dt proxy -w 1 -bs 1
17/08/01 14:49:14 INFO spark.SparkContext: Running Spark version 2.1.0.cloudera1
17/08/01 14:49:14 WARN spark.SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0
17/08/01 14:49:15 INFO spark.SecurityManager: Changing view acls to: spot-user
17/08/01 14:49:15 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/08/01 14:49:15 INFO spark.SecurityManager: Changing view acls groups to:
17/08/01 14:49:15 INFO spark.SecurityManager: Changing modify acls groups to:
17/08/01 14:49:15 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/08/01 14:49:15 INFO util.Utils: Successfully started service 'sparkDriver' on port 56925.
17/08/01 14:49:15 INFO spark.SparkEnv: Registering MapOutputTracker
17/08/01 14:49:15 INFO spark.SparkEnv: Registering BlockManagerMaster
17/08/01 14:49:15 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
17/08/01 14:49:15 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
17/08/01 14:49:15 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-23e4ad79-bd38-4e37-b688-a75f9565bda4
17/08/01 14:49:16 INFO memory.MemoryStore: MemoryStore started with capacity 1060.5 MB
17/08/01 14:49:16 INFO spark.SparkEnv: Registering OutputCommitCoordinator
17/08/01 14:49:16 INFO util.log: Logging initialized @2378ms
17/08/01 14:49:16 INFO server.Server: jetty-9.2.z-SNAPSHOT
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3b9882ec{/jobs,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3b9882ec%7b/jobs,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19bb7285{/jobs/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19bb7285%7b/jobs/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@615b86f5{/jobs/job,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@615b86f5%7b/jobs/job,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ff970c2{/jobs/job/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3ff970c2%7b/jobs/job/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1159f15e{/stages,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1159f15e%7b/stages,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27f55b38{/stages/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27f55b38%7b/stages/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5d37ce06{/stages/stage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5d37ce06%7b/stages/stage,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ab71bb9{/stages/stage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ab71bb9%7b/stages/stage/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b7d4da0{/stages/pool,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@b7d4da0%7b/stages/pool,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f734944{/stages/pool/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4f734944%7b/stages/pool/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19cf381c{/storage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19cf381c%7b/storage,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@736cc380{/storage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@736cc380%7b/storage/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@699743b3{/storage/rdd,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@699743b3%7b/storage/rdd,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@49ddd2ed{/storage/rdd/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@49ddd2ed%7b/storage/rdd/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@242eb7e{/environment,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@242eb7e%7b/environment,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@77729e5a{/environment/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@77729e5a%7b/environment/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5fa63fa{/executors,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5fa63fa%7b/executors,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@79b9b526{/executors/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@79b9b526%7b/executors/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@f241039{/executors/threadDump,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@f241039%7b/executors/threadDump,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@8a515f3{/executors/threadDump/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@8a515f3%7b/executors/threadDump/json,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@e4408f4{/static,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@e4408f4%7b/static,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14a7d27d{/,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14a7d27d%7b/,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@15ec1723{/api,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@15ec1723%7b/api,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ee30613{/jobs/job/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3ee30613%7b/jobs/job/kill,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2aa3e9a6{/stages/stage/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@2aa3e9a6%7b/stages/stage/kill,null,AVAILABLE%7d>
17/08/01 14:49:16 INFO server.ServerConnector: Started ServerConnector@37681deb{HTTP/1.1}{0.0.0.0:4040}<mailto:ServerConnector@37681deb%7bHTTP/1.1%7d%7b0.0.0.0:4040%7d>
17/08/01 14:49:16 INFO server.Server: Started @2532ms
17/08/01 14:49:16 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
17/08/01 14:49:16 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.0.0.199:4040
17/08/01 14:49:16 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/08/01 14:49:17 INFO client.RMProxy: Connecting to ResourceManager at ip-10-0-0-199.ec2.internal/10.0.0.199:8032
17/08/01 14:49:17 INFO yarn.Client: Requesting a new application from cluster with 3 NodeManagers
17/08/01 14:49:17 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (2725 MB per container)
17/08/01 14:49:17 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/08/01 14:49:17 INFO yarn.Client: Setting up container launch context for our AM
17/08/01 14:49:17 INFO yarn.Client: Setting up the launch environment for our AM container
17/08/01 14:49:17 INFO yarn.Client: Preparing resources for our AM container
17/08/01 14:49:19 INFO yarn.Client: Uploading resource file:/home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501598591600_0003/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar
17/08/01 14:49:19 INFO yarn.Client: Uploading resource file:/tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25/__spark_conf__4374658135010889941.zip -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501598591600_0003/__spark_conf__.zip
17/08/01 14:49:19 INFO spark.SecurityManager: Changing view acls to: spot-user
17/08/01 14:49:19 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/08/01 14:49:19 INFO spark.SecurityManager: Changing view acls groups to:
17/08/01 14:49:19 INFO spark.SecurityManager: Changing modify acls groups to:
17/08/01 14:49:19 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/08/01 14:49:19 INFO yarn.Client: Submitting application application_1501598591600_0003 to ResourceManager
17/08/01 14:49:19 INFO impl.YarnClientImpl: Submitted application application_1501598591600_0003
17/08/01 14:49:19 INFO cluster.SchedulerExtensionServices: Starting Yarn extension services with app application_1501598591600_0003 and attemptId None
17/08/01 14:49:20 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:20 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.users.spot-user
start time: 1501598959525
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003/
user: spot-user
17/08/01 14:49:21 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:22 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:23 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/08/01 14:49:23 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> ip-10-0-0-199.ec2.internal, PROXY_URI_BASES -> http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003), /proxy/application_1501598591600_0003
17/08/01 14:49:23 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/08/01 14:49:23 INFO yarn.Client: Application report for application_1501598591600_0003 (state: RUNNING)
17/08/01 14:49:23 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 10.0.0.151
ApplicationMaster RPC port: 0
queue: root.users.spot-user
start time: 1501598959525
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003/
user: spot-user
17/08/01 14:49:23 INFO cluster.YarnClientSchedulerBackend: Application application_1501598591600_0003 has started running.
17/08/01 14:49:23 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 53355.
17/08/01 14:49:23 INFO netty.NettyBlockTransferService: Server created on 10.0.0.199:53355
17/08/01 14:49:23 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/08/01 14:49:23 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManagerMasterEndpoint: Registering block manager 10.0.0.199:53355 with 1060.5 MB RAM, BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManager: external shuffle service port = 7337
17/08/01 14:49:23 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@52c2dab5{/metrics/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@52c2dab5%7b/metrics/json,null,AVAILABLE%7d>
17/08/01 14:49:23 INFO scheduler.EventLoggingListener: Logging events to hdfs://ip-10-0-0-199.ec2.internal:8020/user/spark/spark2ApplicationHistory/application_1501598591600_0003
17/08/01 14:49:23 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/08/01 14:49:27 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.229:37096) with ID 1
17/08/01 14:49:27 INFO spark.ExecutorAllocationManager: New executor 1 has registered (new total is 1)
17/08/01 14:49:27 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-229.ec2.internal:51770 with 1060.5 MB RAM, BlockManagerId(1, ip-10-0-0-229.ec2.internal, 51770, None)
17/08/01 14:49:27 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/08/01 14:49:27 INFO internal.SharedState: spark.sql.warehouse.dir is not set, but hive.metastore.warehouse.dir is set. Setting spark.sql.warehouse.dir to the value of hive.metastore.warehouse.dir ('/user/hive/warehouse').
17/08/01 14:49:27 INFO internal.SharedState: Warehouse path is '/user/hive/warehouse'.
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5554f0a0{/SQL,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5554f0a0%7b/SQL,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@44743601{/SQL/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@44743601%7b/SQL/json,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@144776d3{/SQL/execution,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@144776d3%7b/SQL/execution,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f3f8a4e{/SQL/execution/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4f3f8a4e%7b/SQL/execution/json,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27946137{/static/sql,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27946137%7b/static/sql,null,AVAILABLE%7d>
17/08/01 14:49:27 INFO hive.HiveUtils: Initializing HiveMetastoreConnection version 1.1.0 using file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-logging-1.1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-exec-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-exec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-1.1.0-cdh5.12.0-standalone.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-standalone.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-metastore-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-metastore.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-serde-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-serde.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-service-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-service.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/libfb303-0.9.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/libthrift-0.9.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/log4j-1.2.16.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-hadoop-compat.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-hadoop2-compat.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-protocol.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-server.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/htrace-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ST4-4.0.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-core-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-fate-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-start-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-trace-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/activation-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ant-1.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ant-launcher-1.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/antlr-2.7.7.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/antlr-runtime-3.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/apache-log4j-extras-1.2.17.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-3.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-commons-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-tree-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/avro.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/bonecp-0.8.0.RELEASE.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-avatica-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-core-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-linq4j-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-beanutils-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-beanutils-core-1.8.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-cli-1.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-codec-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-collections-3.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-compiler-2.7.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-compress-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-configuration-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-dbcp-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-digester-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-el-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-httpclient-3.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-io-2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-lang-2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-lang3-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-math-2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-pool-1.5.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-vfs2-2.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-client-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-framework-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-recipes-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-api-jdo-3.2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-core-3.2.10.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-rdbms-3.2.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/derby-10.11.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/eigenbase-properties-1.1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/findbugs-annotations-1.3.9-1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-annotation_1.0_spec-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-jaspic_1.0_spec-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-jta_1.1_spec-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/groovy-all-2.4.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/gson-2.2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/guava-14.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hamcrest-core-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/high-scale-lib-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-accumulo-handler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-accumulo-handler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-ant-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-ant.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-beeline-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-beeline.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-cli-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-cli.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-common-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-contrib-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-contrib.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hbase-handler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hbase-handler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hwi-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hwi.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-0.23-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-0.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-common-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-scheduler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-scheduler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-testutils-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-testutils.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/httpclient-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/httpcore-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-core-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ivy-2.0.0-rc2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/parquet-hadoop-bundle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/stringtemplate-3.2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/regexp-1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/xz-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/tempus-fugit-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/super-csv-2.2.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/stax-api-1.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/servlet-api-2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/opencsv-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-jvm-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-json-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-core-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-provider-svnexe-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-provider-svn-commons-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-api-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/mail-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/logredactor-1.0.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/junit-4.11.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jta-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jsr305-3.0.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jsp-api-2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jpam-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/joda-time-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jline-2.12.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jetty-all-server-7.6.0.v20120127.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jetty-all-7.6.0.v20120127.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jersey-servlet-1.14.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jersey-server-1.14.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jdo-api-3.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jcommander-1.32.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jasper-runtime-5.5.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jasper-compiler-5.5.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/janino-2.7.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jamon-runtime-2.3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-xc-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-jaxrs-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-databind-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-annotations-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/zookeeper.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/velocity-1.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/snappy-java-1.0.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/plexus-utils-1.5.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/paranamer-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/oro-2.0.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/activation-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/activation.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-i18n-2.0.0-M15.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-i18n.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-kerberos-codec-2.0.0-M15.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-kerberos-codec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-asn1-api-1.0.0-M20.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-asn1-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-util-1.0.0-M20.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-util.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/avro.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/aws-java-sdk-bundle-1.11.134.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/aws-java-sdk-bundle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/azure-data-lake-store-sdk-2.1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/azure-data-lake-store-sdk.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-core-1.8.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-cli-1.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-cli.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-codec-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-codec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-collections-3.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-collections.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-compress-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-compress.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-configuration-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-configuration.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-digester-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-digester.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-httpclient-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-httpclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-io-2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-io.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-lang-2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-lang.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-logging-1.1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-logging.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-math3-3.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-math3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-net-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-net.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-client-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-framework-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-framework.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-recipes-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-recipes.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/gson-2.2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/gson.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/guava-11.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/guava.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-annotations-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-auth-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-auth.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-aws-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-aws.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-azure-datalake-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-azure-datalake.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-hdfs-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-hdfs.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-app-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-app.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-core-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-jobclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-shuffle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-api-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-client-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-server-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-server-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/htrace-core4-4.0.1-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/htrace-core4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpclient-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpcore-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpcore.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-annotations-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-core-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-databind-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-databind.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-jaxrs-1.8.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-jaxrs.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-xc-1.8.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-xc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jaxb-api-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jaxb-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-client-1.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-core-1.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/zookeeper.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xz.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xz-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xmlenc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xmlenc-0.52.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xml-apis.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xml-apis-1.3.04.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xercesImpl.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xercesImpl-2.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/stax-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/stax-api-1.0-2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/snappy-java.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/snappy-java-1.0.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-log4j12.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-api-1.7.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/servlet-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/servlet-api-2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/protobuf-java.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/protobuf-java-2.5.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/paranamer.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/paranamer-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/netty.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/netty-3.10.5.Final.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/log4j.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/log4j-1.2.17.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/leveldbjni-all.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/leveldbjni-all-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jsr305.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jsr305-3.0.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jetty-util.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jetty-util-6.1.26.cloudera.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-core.jar
17/08/01 14:49:28 INFO session.SessionState: Created local directory: /tmp/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2_resources
17/08/01 14:49:28 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2
17/08/01 14:49:28 INFO session.SessionState: Created local directory: /tmp/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2
17/08/01 14:49:28 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2/_tmp_space.db
17/08/01 14:49:28 INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
17/08/01 14:49:28 INFO client.HiveClientImpl: Warehouse location for Hive client (version 1.1.0) is /user/hive/warehouse
17/08/01 14:49:29 INFO hive.metastore: Trying to connect to metastore with URI thrift://ip-10-0-0-8.ec2.internal:9083
17/08/01 14:49:29 INFO hive.metastore: Opened a connection to metastore, current connections: 1
17/08/01 14:49:29 INFO hive.metastore: Connected to metastore.
17/08/01 14:49:29 WARN streaming.StreamingContext: Dynamic Allocation is enabled for this application. Enabling Dynamic allocation for Spark Streaming applications can cause data loss if Write Ahead Log is not enabled for non-replayable sources like Flume. See the programming guide for details on how to enable the Write Ahead Log.
17/08/01 14:49:29 INFO spark.SparkContext: Starting job: start at NativeMethodAccessorImpl.java:0
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Registering RDD 1 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Got job 0 (start at NativeMethodAccessorImpl.java:0) with 20 output partitions
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/08/01 14:49:29 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 3.1 KB, free 1060.5 MB)
17/08/01 14:49:29 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 1988.0 B, free 1060.5 MB)
17/08/01 14:49:29 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.0.0.199:53355 (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:49:29 INFO spark.SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Submitting 50 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO cluster.YarnScheduler: Adding task set 0.0 with 50 tasks
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, ip-10-0-0-229.ec2.internal, executor 1, partition 1, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 664 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, ip-10-0-0-229.ec2.internal, executor 1, partition 2, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 78 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3, ip-10-0-0-229.ec2.internal, executor 1, partition 3, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 68 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 0.0 (TID 4, ip-10-0-0-229.ec2.internal, executor 1, partition 4, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 65 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 0.0 (TID 5, ip-10-0-0-229.ec2.internal, executor 1, partition 5, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 59 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/50)
17/08/01 14:49:30 INFO spark.ExecutorAllocationManager: Requesting 1 new executor because tasks are backlogged (new desired total will be 2)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 0.0 (TID 6, ip-10-0-0-229.ec2.internal, executor 1, partition 6, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 0.0 (TID 7, ip-10-0-0-229.ec2.internal, executor 1, partition 7, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 60 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 0.0 (TID 8, ip-10-0-0-229.ec2.internal, executor 1, partition 8, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 0.0 (TID 7) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 0.0 (TID 9, ip-10-0-0-229.ec2.internal, executor 1, partition 9, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 0.0 (TID 8) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 0.0 (TID 10, ip-10-0-0-229.ec2.internal, executor 1, partition 10, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 0.0 (TID 9) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 0.0 (TID 11, ip-10-0-0-229.ec2.internal, executor 1, partition 11, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 0.0 (TID 10) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 0.0 (TID 12, ip-10-0-0-229.ec2.internal, executor 1, partition 12, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 0.0 (TID 11) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 0.0 (TID 13, ip-10-0-0-229.ec2.internal, executor 1, partition 13, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 0.0 (TID 12) in 60 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 0.0 (TID 14, ip-10-0-0-229.ec2.internal, executor 1, partition 14, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 0.0 (TID 13) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 0.0 (TID 15, ip-10-0-0-229.ec2.internal, executor 1, partition 15, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 0.0 (TID 14) in 51 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 0.0 (TID 16, ip-10-0-0-229.ec2.internal, executor 1, partition 16, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 0.0 (TID 15) in 63 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 0.0 (TID 17, ip-10-0-0-229.ec2.internal, executor 1, partition 17, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 0.0 (TID 16) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 0.0 (TID 18, ip-10-0-0-229.ec2.internal, executor 1, partition 18, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 0.0 (TID 17) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 0.0 (TID 19, ip-10-0-0-229.ec2.internal, executor 1, partition 19, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 0.0 (TID 18) in 56 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 0.0 (TID 20, ip-10-0-0-229.ec2.internal, executor 1, partition 20, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 0.0 (TID 19) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 0.0 (TID 21, ip-10-0-0-229.ec2.internal, executor 1, partition 21, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 0.0 (TID 20) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (21/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 0.0 (TID 22, ip-10-0-0-229.ec2.internal, executor 1, partition 22, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 0.0 (TID 21) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (22/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 0.0 (TID 23, ip-10-0-0-229.ec2.internal, executor 1, partition 23, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 0.0 (TID 22) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (23/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 0.0 (TID 24, ip-10-0-0-229.ec2.internal, executor 1, partition 24, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 0.0 (TID 23) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (24/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 0.0 (TID 25, ip-10-0-0-229.ec2.internal, executor 1, partition 25, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 0.0 (TID 24) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (25/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 0.0 (TID 26, ip-10-0-0-229.ec2.internal, executor 1, partition 26, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 0.0 (TID 25) in 46 ms on ip-10-0-0-229.ec2.internal (executor 1) (26/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 0.0 (TID 27, ip-10-0-0-229.ec2.internal, executor 1, partition 27, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 0.0 (TID 26) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (27/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 0.0 (TID 28, ip-10-0-0-229.ec2.internal, executor 1, partition 28, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 0.0 (TID 27) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (28/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 0.0 (TID 29, ip-10-0-0-229.ec2.internal, executor 1, partition 29, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 0.0 (TID 28) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (29/50)
17/08/01 14:49:31 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 4)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 0.0 (TID 30, ip-10-0-0-229.ec2.internal, executor 1, partition 30, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 0.0 (TID 29) in 42 ms on ip-10-0-0-229.ec2.internal (executor 1) (30/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 0.0 (TID 31, ip-10-0-0-229.ec2.internal, executor 1, partition 31, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 0.0 (TID 30) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (31/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 0.0 (TID 32, ip-10-0-0-229.ec2.internal, executor 1, partition 32, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 0.0 (TID 31) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (32/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 0.0 (TID 33, ip-10-0-0-229.ec2.internal, executor 1, partition 33, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 0.0 (TID 32) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (33/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 0.0 (TID 34, ip-10-0-0-229.ec2.internal, executor 1, partition 34, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 0.0 (TID 33) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (34/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 0.0 (TID 35, ip-10-0-0-229.ec2.internal, executor 1, partition 35, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 0.0 (TID 34) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (35/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 0.0 (TID 36, ip-10-0-0-229.ec2.internal, executor 1, partition 36, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 0.0 (TID 35) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (36/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 0.0 (TID 37, ip-10-0-0-229.ec2.internal, executor 1, partition 37, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 0.0 (TID 36) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (37/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 0.0 (TID 38, ip-10-0-0-229.ec2.internal, executor 1, partition 38, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 0.0 (TID 37) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (38/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 0.0 (TID 39, ip-10-0-0-229.ec2.internal, executor 1, partition 39, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 0.0 (TID 38) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (39/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 0.0 (TID 40, ip-10-0-0-229.ec2.internal, executor 1, partition 40, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 0.0 (TID 39) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (40/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 0.0 (TID 41, ip-10-0-0-229.ec2.internal, executor 1, partition 41, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 0.0 (TID 40) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (41/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 0.0 (TID 42, ip-10-0-0-229.ec2.internal, executor 1, partition 42, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 0.0 (TID 41) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (42/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 0.0 (TID 43, ip-10-0-0-229.ec2.internal, executor 1, partition 43, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 0.0 (TID 42) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (43/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 0.0 (TID 44, ip-10-0-0-229.ec2.internal, executor 1, partition 44, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 0.0 (TID 43) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (44/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 0.0 (TID 45, ip-10-0-0-229.ec2.internal, executor 1, partition 45, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 0.0 (TID 44) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (45/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 0.0 (TID 46, ip-10-0-0-229.ec2.internal, executor 1, partition 46, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 0.0 (TID 45) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (46/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 0.0 (TID 47, ip-10-0-0-229.ec2.internal, executor 1, partition 47, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 0.0 (TID 46) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (47/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 0.0 (TID 48, ip-10-0-0-229.ec2.internal, executor 1, partition 48, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 0.0 (TID 47) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (48/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 0.0 (TID 49, ip-10-0-0-229.ec2.internal, executor 1, partition 49, PROCESS_LOCAL, 6676 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 0.0 (TID 48) in 28 ms on ip-10-0-0-229.ec2.internal (executor 1) (49/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 0.0 (TID 49) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (50/50)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/08/01 14:49:32 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (start at NativeMethodAccessorImpl.java:0) finished in 2.517 s
17/08/01 14:49:32 INFO scheduler.DAGScheduler: looking for newly runnable stages
17/08/01 14:49:32 INFO scheduler.DAGScheduler: running: Set()
17/08/01 14:49:32 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 1)
17/08/01 14:49:32 INFO scheduler.DAGScheduler: failed: Set()
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/08/01 14:49:32 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.2 KB, free 1060.5 MB)
17/08/01 14:49:32 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 1956.0 B, free 1060.5 MB)
17/08/01 14:49:32 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.0.0.199:53355 (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:49:32 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Submitting 20 missing tasks from ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Adding task set 1.0 with 20 tasks
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 50, ip-10-0-0-229.ec2.internal, executor 1, partition 0, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:49:32 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.0.0.229:37096
17/08/01 14:49:32 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 283 bytes
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.0 (TID 51, ip-10-0-0-229.ec2.internal, executor 1, partition 1, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 50) in 89 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 1.0 (TID 52, ip-10-0-0-229.ec2.internal, executor 1, partition 2, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 1.0 (TID 51) in 23 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 1.0 (TID 53, ip-10-0-0-229.ec2.internal, executor 1, partition 3, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 1.0 (TID 52) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 1.0 (TID 54, ip-10-0-0-229.ec2.internal, executor 1, partition 4, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 1.0 (TID 53) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 1.0 (TID 55, ip-10-0-0-229.ec2.internal, executor 1, partition 5, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 1.0 (TID 54) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 1.0 (TID 56, ip-10-0-0-229.ec2.internal, executor 1, partition 6, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 1.0 (TID 55) in 25 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 1.0 (TID 57, ip-10-0-0-229.ec2.internal, executor 1, partition 7, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 1.0 (TID 56) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 1.0 (TID 58, ip-10-0-0-229.ec2.internal, executor 1, partition 8, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 1.0 (TID 57) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 1.0 (TID 59, ip-10-0-0-229.ec2.internal, executor 1, partition 9, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 1.0 (TID 58) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 1.0 (TID 60, ip-10-0-0-229.ec2.internal, executor 1, partition 10, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 1.0 (TID 59) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 1.0 (TID 61, ip-10-0-0-229.ec2.internal, executor 1, partition 11, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 1.0 (TID 60) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 1.0 (TID 62, ip-10-0-0-229.ec2.internal, executor 1, partition 12, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 1.0 (TID 61) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 1.0 (TID 63, ip-10-0-0-229.ec2.internal, executor 1, partition 13, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 1.0 (TID 62) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 1.0 (TID 64, ip-10-0-0-229.ec2.internal, executor 1, partition 14, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 1.0 (TID 63) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 1.0 (TID 65, ip-10-0-0-229.ec2.internal, executor 1, partition 15, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 1.0 (TID 64) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 1.0 (TID 66, ip-10-0-0-229.ec2.internal, executor 1, partition 16, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 1.0 (TID 65) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 1.0 (TID 67, ip-10-0-0-229.ec2.internal, executor 1, partition 17, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 1.0 (TID 66) in 22 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 1.0 (TID 68, ip-10-0-0-229.ec2.internal, executor 1, partition 18, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 1.0 (TID 67) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 1.0 (TID 69, ip-10-0-0-229.ec2.internal, executor 1, partition 19, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 1.0 (TID 68) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 1.0 (TID 69) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/20)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool
17/08/01 14:49:32 INFO scheduler.DAGScheduler: ResultStage 1 (start at NativeMethodAccessorImpl.java:0) finished in 0.420 s
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Job 0 finished: start at NativeMethodAccessorImpl.java:0, took 3.286639 s
17/08/01 14:49:32 INFO scheduler.ReceiverTracker: Starting 1 receivers
17/08/01 14:49:32 INFO scheduler.ReceiverTracker: ReceiverTracker started
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Initialized and validated org.apache.spark.streaming.kafka.KafkaInputDStream@71871773
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Initialized and validated org.apache.spark.streaming.api.python.PythonTransformedDStream@6593d619
17/08/01 14:49:32 INFO dstream.ForEachDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO dstream.ForEachDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO dstream.ForEachDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO dstream.ForEachDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO dstream.ForEachDStream: Initialized and validated org.apache.spark.streaming.dstream.ForEachDStream@26c8ea4b
17/08/01 14:49:33 INFO util.RecurringTimer: Started timer for JobGenerator at time 1501598973000
17/08/01 14:49:33 INFO scheduler.JobGenerator: Started JobGenerator at 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Started JobScheduler
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@625d3c2d{/streaming,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@48a256f9{/streaming/json,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@39edaf54{/streaming/batch,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@648a544d{/streaming/batch/json,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@c773c0d{/static/streaming,null,AVAILABLE}
17/08/01 14:49:33 INFO streaming.StreamingContext: StreamingContext started
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Got job 1 (start at NativeMethodAccessorImpl.java:0) with 1 output partitions
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Final stage: ResultStage 2 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Missing parents: List()
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Submitting ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620), which has no missing parents
17/08/01 14:49:33 INFO scheduler.ReceiverTracker: Receiver 0 started
17/08/01 14:49:33 INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 80.8 KB, free 1060.4 MB)
17/08/01 14:49:33 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 29.1 KB, free 1060.4 MB)
17/08/01 14:49:33 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.0.0.199:53355 (size: 29.1 KB, free: 1060.5 MB)
17/08/01 14:49:33 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620)
17/08/01 14:49:33 INFO cluster.YarnScheduler: Adding task set 2.0 with 1 tasks
17/08/01 14:49:33 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 2.0 (TID 70, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 7067 bytes)
17/08/01 14:49:33 INFO scheduler.JobScheduler: Added jobs for time 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Starting job streaming job 1501598973000 ms.0 from job set of time 1501598973000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:33 INFO scheduler.JobScheduler: Finished job streaming job 1501598973000 ms.0 from job set of time 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Total delay: 0.102 s for time 1501598973000 ms (execution: 0.026 s)
17/08/01 14:49:33 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 29.1 KB, free: 1060.5 MB)
17/08/01 14:49:33 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/08/01 14:49:33 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/08/01 14:49:33 INFO scheduler.ReceiverTracker: Registered receiver for stream 0 from 10.0.0.229:37096
17/08/01 14:49:34 INFO scheduler.JobScheduler: Added jobs for time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.JobScheduler: Starting job streaming job 1501598974000 ms.0 from job set of time 1501598974000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:34 INFO scheduler.JobScheduler: Finished job streaming job 1501598974000 ms.0 from job set of time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598974000 ms (execution: 0.009 s)
17/08/01 14:49:34 INFO python.PythonRDD: Removing RDD 5 from persistence list
17/08/01 14:49:34 INFO storage.BlockManager: Removing RDD 5
17/08/01 14:49:34 INFO rdd.BlockRDD: Removing RDD 4 from persistence list
17/08/01 14:49:34 INFO storage.BlockManager: Removing RDD 4
17/08/01 14:49:34 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[4] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/08/01 14:49:34 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/08/01 14:49:34 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.127:44695) with ID 2
17/08/01 14:49:34 INFO spark.ExecutorAllocationManager: New executor 2 has registered (new total is 2)
17/08/01 14:49:34 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-127.ec2.internal:59267 with 1060.5 MB RAM, BlockManagerId(2, ip-10-0-0-127.ec2.internal, 59267, None)
17/08/01 14:49:35 INFO scheduler.JobScheduler: Added jobs for time 1501598975000 ms
17/08/01 14:49:35 INFO scheduler.JobScheduler: Starting job streaming job 1501598975000 ms.0 from job set of time 1501598975000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:35 INFO scheduler.JobScheduler: Finished job streaming job 1501598975000 ms.0 from job set of time 1501598975000 ms
17/08/01 14:49:35 INFO python.PythonRDD: Removing RDD 7 from persistence list
17/08/01 14:49:35 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501598975000 ms (execution: 0.009 s)
17/08/01 14:49:35 INFO rdd.BlockRDD: Removing RDD 6 from persistence list
17/08/01 14:49:35 INFO storage.BlockManager: Removing RDD 6
17/08/01 14:49:35 INFO storage.BlockManager: Removing RDD 7
17/08/01 14:49:35 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[6] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598975000 ms
17/08/01 14:49:35 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598973000 ms
17/08/01 14:49:35 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598973000 ms
17/08/01 14:49:36 INFO scheduler.JobScheduler: Added jobs for time 1501598976000 ms
17/08/01 14:49:36 INFO scheduler.JobScheduler: Starting job streaming job 1501598976000 ms.0 from job set of time 1501598976000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:36 INFO scheduler.JobScheduler: Finished job streaming job 1501598976000 ms.0 from job set of time 1501598976000 ms
17/08/01 14:49:36 INFO python.PythonRDD: Removing RDD 9 from persistence list
17/08/01 14:49:36 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501598976000 ms (execution: 0.008 s)
17/08/01 14:49:36 INFO storage.BlockManager: Removing RDD 9
17/08/01 14:49:36 INFO rdd.BlockRDD: Removing RDD 8 from persistence list
17/08/01 14:49:36 INFO storage.BlockManager: Removing RDD 8
17/08/01 14:49:36 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[8] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598976000 ms
17/08/01 14:49:36 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598974000 ms
17/08/01 14:49:36 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598974000 ms
17/08/01 14:49:37 INFO scheduler.JobScheduler: Added jobs for time 1501598977000 ms
17/08/01 14:49:37 INFO scheduler.JobScheduler: Starting job streaming job 1501598977000 ms.0 from job set of time 1501598977000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:37 INFO scheduler.JobScheduler: Finished job streaming job 1501598977000 ms.0 from job set of time 1501598977000 ms
17/08/01 14:49:37 INFO python.PythonRDD: Removing RDD 11 from persistence list
17/08/01 14:49:37 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598977000 ms (execution: 0.008 s)
17/08/01 14:49:37 INFO storage.BlockManager: Removing RDD 11
17/08/01 14:49:37 INFO rdd.BlockRDD: Removing RDD 10 from persistence list
17/08/01 14:49:37 INFO storage.BlockManager: Removing RDD 10
17/08/01 14:49:37 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[10] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598977000 ms
17/08/01 14:49:37 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598975000 ms
17/08/01 14:49:37 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598975000 ms
17/08/01 14:49:38 INFO scheduler.JobScheduler: Added jobs for time 1501598978000 ms
17/08/01 14:49:38 INFO scheduler.JobScheduler: Starting job streaming job 1501598978000 ms.0 from job set of time 1501598978000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:38 INFO scheduler.JobScheduler: Finished job streaming job 1501598978000 ms.0 from job set of time 1501598978000 ms
17/08/01 14:49:38 INFO python.PythonRDD: Removing RDD 13 from persistence list
17/08/01 14:49:38 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501598978000 ms (execution: 0.007 s)
17/08/01 14:49:38 INFO storage.BlockManager: Removing RDD 13
17/08/01 14:49:38 INFO rdd.BlockRDD: Removing RDD 12 from persistence list
17/08/01 14:49:38 INFO storage.BlockManager: Removing RDD 12
17/08/01 14:49:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[12] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598978000 ms
17/08/01 14:49:38 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598976000 ms
17/08/01 14:49:38 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598976000 ms
17/08/01 14:49:39 INFO scheduler.JobScheduler: Added jobs for time 1501598979000 ms
17/08/01 14:49:39 INFO scheduler.JobScheduler: Starting job streaming job 1501598979000 ms.0 from job set of time 1501598979000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:39 INFO scheduler.JobScheduler: Finished job streaming job 1501598979000 ms.0 from job set of time 1501598979000 ms
17/08/01 14:49:39 INFO python.PythonRDD: Removing RDD 15 from persistence list
17/08/01 14:49:39 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598979000 ms (execution: 0.007 s)
17/08/01 14:49:39 INFO storage.BlockManager: Removing RDD 15
17/08/01 14:49:39 INFO rdd.BlockRDD: Removing RDD 14 from persistence list
17/08/01 14:49:39 INFO storage.BlockManager: Removing RDD 14
17/08/01 14:49:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[14] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598979000 ms
17/08/01 14:49:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598977000 ms
17/08/01 14:49:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598977000 ms
17/08/01 14:49:40 INFO scheduler.JobScheduler: Added jobs for time 1501598980000 ms
17/08/01 14:49:40 INFO scheduler.JobScheduler: Starting job streaming job 1501598980000 ms.0 from job set of time 1501598980000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:40 INFO scheduler.JobScheduler: Finished job streaming job 1501598980000 ms.0 from job set of time 1501598980000 ms
17/08/01 14:49:40 INFO python.PythonRDD: Removing RDD 17 from persistence list
17/08/01 14:49:40 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598980000 ms (execution: 0.015 s)
17/08/01 14:49:40 INFO storage.BlockManager: Removing RDD 17
17/08/01 14:49:40 INFO rdd.BlockRDD: Removing RDD 16 from persistence list
17/08/01 14:49:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[16] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598980000 ms
17/08/01 14:49:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598978000 ms
17/08/01 14:49:40 INFO storage.BlockManager: Removing RDD 16
17/08/01 14:49:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598978000 ms
17/08/01 14:49:41 INFO scheduler.JobScheduler: Added jobs for time 1501598981000 ms
17/08/01 14:49:41 INFO scheduler.JobScheduler: Starting job streaming job 1501598981000 ms.0 from job set of time 1501598981000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:41 INFO scheduler.JobScheduler: Finished job streaming job 1501598981000 ms.0 from job set of time 1501598981000 ms
17/08/01 14:49:41 INFO python.PythonRDD: Removing RDD 19 from persistence list
17/08/01 14:49:41 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501598981000 ms (execution: 0.006 s)
17/08/01 14:49:41 INFO storage.BlockManager: Removing RDD 19
17/08/01 14:49:41 INFO rdd.BlockRDD: Removing RDD 18 from persistence list
17/08/01 14:49:41 INFO storage.BlockManager: Removing RDD 18
17/08/01 14:49:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[18] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598981000 ms
17/08/01 14:49:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598979000 ms
17/08/01 14:49:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598979000 ms
17/08/01 14:49:42 INFO scheduler.JobScheduler: Added jobs for time 1501598982000 ms
17/08/01 14:49:42 INFO scheduler.JobScheduler: Starting job streaming job 1501598982000 ms.0 from job set of time 1501598982000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:42 INFO scheduler.JobScheduler: Finished job streaming job 1501598982000 ms.0 from job set of time 1501598982000 ms
17/08/01 14:49:42 INFO python.PythonRDD: Removing RDD 21 from persistence list
17/08/01 14:49:42 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598982000 ms (execution: 0.008 s)
17/08/01 14:49:42 INFO storage.BlockManager: Removing RDD 21
17/08/01 14:49:42 INFO rdd.BlockRDD: Removing RDD 20 from persistence list
17/08/01 14:49:42 INFO storage.BlockManager: Removing RDD 20
17/08/01 14:49:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[20] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598982000 ms
17/08/01 14:49:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598980000 ms
17/08/01 14:49:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598980000 ms
17/08/01 14:49:43 INFO scheduler.JobScheduler: Added jobs for time 1501598983000 ms
17/08/01 14:49:43 INFO scheduler.JobScheduler: Starting job streaming job 1501598983000 ms.0 from job set of time 1501598983000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:43 INFO scheduler.JobScheduler: Finished job streaming job 1501598983000 ms.0 from job set of time 1501598983000 ms
17/08/01 14:49:43 INFO python.PythonRDD: Removing RDD 23 from persistence list
17/08/01 14:49:43 INFO scheduler.JobScheduler: Total delay: 0.036 s for time 1501598983000 ms (execution: 0.009 s)
17/08/01 14:49:43 INFO storage.BlockManager: Removing RDD 23
17/08/01 14:49:43 INFO rdd.BlockRDD: Removing RDD 22 from persistence list
17/08/01 14:49:43 INFO storage.BlockManager: Removing RDD 22
17/08/01 14:49:43 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[22] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598983000 ms
17/08/01 14:49:43 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598981000 ms
17/08/01 14:49:43 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598981000 ms
17/08/01 14:49:44 INFO scheduler.JobScheduler: Added jobs for time 1501598984000 ms
17/08/01 14:49:44 INFO scheduler.JobScheduler: Starting job streaming job 1501598984000 ms.0 from job set of time 1501598984000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:44 INFO scheduler.JobScheduler: Finished job streaming job 1501598984000 ms.0 from job set of time 1501598984000 ms
17/08/01 14:49:44 INFO python.PythonRDD: Removing RDD 25 from persistence list
17/08/01 14:49:44 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598984000 ms (execution: 0.007 s)
17/08/01 14:49:44 INFO storage.BlockManager: Removing RDD 25
17/08/01 14:49:44 INFO rdd.BlockRDD: Removing RDD 24 from persistence list
17/08/01 14:49:44 INFO storage.BlockManager: Removing RDD 24
17/08/01 14:49:44 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[24] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598984000 ms
17/08/01 14:49:44 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598982000 ms
17/08/01 14:49:44 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598982000 ms
17/08/01 14:49:45 INFO scheduler.JobScheduler: Added jobs for time 1501598985000 ms
17/08/01 14:49:45 INFO scheduler.JobScheduler: Starting job streaming job 1501598985000 ms.0 from job set of time 1501598985000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:45 INFO scheduler.JobScheduler: Finished job streaming job 1501598985000 ms.0 from job set of time 1501598985000 ms
17/08/01 14:49:45 INFO python.PythonRDD: Removing RDD 27 from persistence list
17/08/01 14:49:45 INFO scheduler.JobScheduler: Total delay: 0.033 s for time 1501598985000 ms (execution: 0.007 s)
17/08/01 14:49:45 INFO storage.BlockManager: Removing RDD 27
17/08/01 14:49:45 INFO rdd.BlockRDD: Removing RDD 26 from persistence list
17/08/01 14:49:45 INFO storage.BlockManager: Removing RDD 26
17/08/01 14:49:45 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[26] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598985000 ms
17/08/01 14:49:45 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598983000 ms
17/08/01 14:49:45 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598983000 ms
17/08/01 14:49:46 INFO scheduler.JobScheduler: Added jobs for time 1501598986000 ms
17/08/01 14:49:46 INFO scheduler.JobScheduler: Starting job streaming job 1501598986000 ms.0 from job set of time 1501598986000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:46 INFO scheduler.JobScheduler: Finished job streaming job 1501598986000 ms.0 from job set of time 1501598986000 ms
17/08/01 14:49:46 INFO python.PythonRDD: Removing RDD 29 from persistence list
17/08/01 14:49:46 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598986000 ms (execution: 0.008 s)
17/08/01 14:49:46 INFO storage.BlockManager: Removing RDD 29
17/08/01 14:49:46 INFO rdd.BlockRDD: Removing RDD 28 from persistence list
17/08/01 14:49:46 INFO storage.BlockManager: Removing RDD 28
17/08/01 14:49:46 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[28] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598986000 ms
17/08/01 14:49:46 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598984000 ms
17/08/01 14:49:46 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598984000 ms
17/08/01 14:49:47 INFO scheduler.JobScheduler: Added jobs for time 1501598987000 ms
17/08/01 14:49:47 INFO scheduler.JobScheduler: Starting job streaming job 1501598987000 ms.0 from job set of time 1501598987000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:47 INFO scheduler.JobScheduler: Finished job streaming job 1501598987000 ms.0 from job set of time 1501598987000 ms
17/08/01 14:49:47 INFO python.PythonRDD: Removing RDD 31 from persistence list
17/08/01 14:49:47 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598987000 ms (execution: 0.014 s)
17/08/01 14:49:47 INFO storage.BlockManager: Removing RDD 31
17/08/01 14:49:47 INFO rdd.BlockRDD: Removing RDD 30 from persistence list
17/08/01 14:49:47 INFO storage.BlockManager: Removing RDD 30
17/08/01 14:49:47 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[30] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598987000 ms
17/08/01 14:49:47 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598985000 ms
17/08/01 14:49:47 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598985000 ms
17/08/01 14:49:48 INFO scheduler.JobScheduler: Added jobs for time 1501598988000 ms
17/08/01 14:49:48 INFO scheduler.JobScheduler: Starting job streaming job 1501598988000 ms.0 from job set of time 1501598988000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:48 INFO scheduler.JobScheduler: Finished job streaming job 1501598988000 ms.0 from job set of time 1501598988000 ms
17/08/01 14:49:48 INFO python.PythonRDD: Removing RDD 33 from persistence list
17/08/01 14:49:48 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501598988000 ms (execution: 0.008 s)
17/08/01 14:49:48 INFO storage.BlockManager: Removing RDD 33
17/08/01 14:49:48 INFO rdd.BlockRDD: Removing RDD 32 from persistence list
17/08/01 14:49:48 INFO storage.BlockManager: Removing RDD 32
17/08/01 14:49:48 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[32] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598988000 ms
17/08/01 14:49:48 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598986000 ms
17/08/01 14:49:48 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598986000 ms
17/08/01 14:49:49 INFO scheduler.JobScheduler: Added jobs for time 1501598989000 ms
17/08/01 14:49:49 INFO scheduler.JobScheduler: Starting job streaming job 1501598989000 ms.0 from job set of time 1501598989000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:49 INFO scheduler.JobScheduler: Finished job streaming job 1501598989000 ms.0 from job set of time 1501598989000 ms
17/08/01 14:49:49 INFO python.PythonRDD: Removing RDD 35 from persistence list
17/08/01 14:49:49 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501598989000 ms (execution: 0.006 s)
17/08/01 14:49:49 INFO storage.BlockManager: Removing RDD 35
17/08/01 14:49:49 INFO rdd.BlockRDD: Removing RDD 34 from persistence list
17/08/01 14:49:49 INFO storage.BlockManager: Removing RDD 34
17/08/01 14:49:49 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[34] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598989000 ms
17/08/01 14:49:49 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598987000 ms
17/08/01 14:49:49 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598987000 ms
17/08/01 14:49:50 INFO scheduler.JobScheduler: Added jobs for time 1501598990000 ms
17/08/01 14:49:50 INFO scheduler.JobScheduler: Starting job streaming job 1501598990000 ms.0 from job set of time 1501598990000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:50 INFO scheduler.JobScheduler: Finished job streaming job 1501598990000 ms.0 from job set of time 1501598990000 ms
17/08/01 14:49:50 INFO python.PythonRDD: Removing RDD 37 from persistence list
17/08/01 14:49:50 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501598990000 ms (execution: 0.008 s)
17/08/01 14:49:50 INFO storage.BlockManager: Removing RDD 37
17/08/01 14:49:50 INFO rdd.BlockRDD: Removing RDD 36 from persistence list
17/08/01 14:49:50 INFO storage.BlockManager: Removing RDD 36
17/08/01 14:49:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[36] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598990000 ms
17/08/01 14:49:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598988000 ms
17/08/01 14:49:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598988000 ms
17/08/01 14:49:51 INFO scheduler.JobScheduler: Added jobs for time 1501598991000 ms
17/08/01 14:49:51 INFO scheduler.JobScheduler: Starting job streaming job 1501598991000 ms.0 from job set of time 1501598991000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:51 INFO scheduler.JobScheduler: Finished job streaming job 1501598991000 ms.0 from job set of time 1501598991000 ms
17/08/01 14:49:51 INFO python.PythonRDD: Removing RDD 39 from persistence list
17/08/01 14:49:51 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501598991000 ms (execution: 0.006 s)
17/08/01 14:49:51 INFO storage.BlockManager: Removing RDD 39
17/08/01 14:49:51 INFO rdd.BlockRDD: Removing RDD 38 from persistence list
17/08/01 14:49:51 INFO storage.BlockManager: Removing RDD 38
17/08/01 14:49:51 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[38] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598991000 ms
17/08/01 14:49:51 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598989000 ms
17/08/01 14:49:51 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598989000 ms
17/08/01 14:49:52 INFO scheduler.JobScheduler: Added jobs for time 1501598992000 ms
17/08/01 14:49:52 INFO scheduler.JobScheduler: Starting job streaming job 1501598992000 ms.0 from job set of time 1501598992000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:52 INFO scheduler.JobScheduler: Finished job streaming job 1501598992000 ms.0 from job set of time 1501598992000 ms
17/08/01 14:49:52 INFO python.PythonRDD: Removing RDD 41 from persistence list
17/08/01 14:49:52 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501598992000 ms (execution: 0.007 s)
17/08/01 14:49:52 INFO storage.BlockManager: Removing RDD 41
17/08/01 14:49:52 INFO rdd.BlockRDD: Removing RDD 40 from persistence list
17/08/01 14:49:52 INFO storage.BlockManager: Removing RDD 40
17/08/01 14:49:52 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[40] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598992000 ms
17/08/01 14:49:52 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598990000 ms
17/08/01 14:49:52 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598990000 ms
17/08/01 14:49:53 INFO scheduler.JobScheduler: Added jobs for time 1501598993000 ms
17/08/01 14:49:53 INFO scheduler.JobScheduler: Starting job streaming job 1501598993000 ms.0 from job set of time 1501598993000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:53 INFO scheduler.JobScheduler: Finished job streaming job 1501598993000 ms.0 from job set of time 1501598993000 ms
17/08/01 14:49:53 INFO python.PythonRDD: Removing RDD 43 from persistence list
17/08/01 14:49:53 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598993000 ms (execution: 0.008 s)
17/08/01 14:49:53 INFO storage.BlockManager: Removing RDD 43
17/08/01 14:49:53 INFO rdd.BlockRDD: Removing RDD 42 from persistence list
17/08/01 14:49:53 INFO storage.BlockManager: Removing RDD 42
17/08/01 14:49:53 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[42] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598993000 ms
17/08/01 14:49:53 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598991000 ms
17/08/01 14:49:53 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598991000 ms
17/08/01 14:49:54 INFO scheduler.JobScheduler: Added jobs for time 1501598994000 ms
17/08/01 14:49:54 INFO scheduler.JobScheduler: Starting job streaming job 1501598994000 ms.0 from job set of time 1501598994000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:54 INFO scheduler.JobScheduler: Finished job streaming job 1501598994000 ms.0 from job set of time 1501598994000 ms
17/08/01 14:49:54 INFO python.PythonRDD: Removing RDD 45 from persistence list
17/08/01 14:49:54 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501598994000 ms (execution: 0.006 s)
17/08/01 14:49:54 INFO storage.BlockManager: Removing RDD 45
17/08/01 14:49:54 INFO rdd.BlockRDD: Removing RDD 44 from persistence list
17/08/01 14:49:54 INFO storage.BlockManager: Removing RDD 44
17/08/01 14:49:54 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[44] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598994000 ms
17/08/01 14:49:54 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598992000 ms
17/08/01 14:49:54 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598992000 ms
17/08/01 14:49:55 INFO scheduler.JobScheduler: Added jobs for time 1501598995000 ms
17/08/01 14:49:55 INFO scheduler.JobScheduler: Starting job streaming job 1501598995000 ms.0 from job set of time 1501598995000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:55 INFO scheduler.JobScheduler: Finished job streaming job 1501598995000 ms.0 from job set of time 1501598995000 ms
17/08/01 14:49:55 INFO python.PythonRDD: Removing RDD 47 from persistence list
17/08/01 14:49:55 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598995000 ms (execution: 0.008 s)
17/08/01 14:49:55 INFO storage.BlockManager: Removing RDD 47
17/08/01 14:49:55 INFO rdd.BlockRDD: Removing RDD 46 from persistence list
17/08/01 14:49:55 INFO storage.BlockManager: Removing RDD 46
17/08/01 14:49:55 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[46] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598995000 ms
17/08/01 14:49:55 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598993000 ms
17/08/01 14:49:55 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598993000 ms
17/08/01 14:49:56 INFO scheduler.JobScheduler: Added jobs for time 1501598996000 ms
17/08/01 14:49:56 INFO scheduler.JobScheduler: Starting job streaming job 1501598996000 ms.0 from job set of time 1501598996000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:56 INFO scheduler.JobScheduler: Finished job streaming job 1501598996000 ms.0 from job set of time 1501598996000 ms
17/08/01 14:49:56 INFO python.PythonRDD: Removing RDD 49 from persistence list
17/08/01 14:49:56 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501598996000 ms (execution: 0.020 s)
17/08/01 14:49:56 INFO storage.BlockManager: Removing RDD 49
17/08/01 14:49:56 INFO rdd.BlockRDD: Removing RDD 48 from persistence list
17/08/01 14:49:56 INFO storage.BlockManager: Removing RDD 48
17/08/01 14:49:56 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[48] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598996000 ms
17/08/01 14:49:56 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598994000 ms
17/08/01 14:49:56 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598994000 ms
17/08/01 14:49:57 INFO scheduler.JobScheduler: Added jobs for time 1501598997000 ms
17/08/01 14:49:57 INFO scheduler.JobScheduler: Starting job streaming job 1501598997000 ms.0 from job set of time 1501598997000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:57 INFO scheduler.JobScheduler: Finished job streaming job 1501598997000 ms.0 from job set of time 1501598997000 ms
17/08/01 14:49:57 INFO python.PythonRDD: Removing RDD 51 from persistence list
17/08/01 14:49:57 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598997000 ms (execution: 0.007 s)
17/08/01 14:49:57 INFO storage.BlockManager: Removing RDD 51
17/08/01 14:49:57 INFO rdd.BlockRDD: Removing RDD 50 from persistence list
17/08/01 14:49:57 INFO storage.BlockManager: Removing RDD 50
17/08/01 14:49:57 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[50] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598997000 ms
17/08/01 14:49:57 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598995000 ms
17/08/01 14:49:57 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598995000 ms
17/08/01 14:49:58 INFO scheduler.JobScheduler: Added jobs for time 1501598998000 ms
17/08/01 14:49:58 INFO scheduler.JobScheduler: Starting job streaming job 1501598998000 ms.0 from job set of time 1501598998000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:58 INFO scheduler.JobScheduler: Finished job streaming job 1501598998000 ms.0 from job set of time 1501598998000 ms
17/08/01 14:49:58 INFO python.PythonRDD: Removing RDD 53 from persistence list
17/08/01 14:49:58 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501598998000 ms (execution: 0.008 s)
17/08/01 14:49:58 INFO storage.BlockManager: Removing RDD 53
17/08/01 14:49:58 INFO rdd.BlockRDD: Removing RDD 52 from persistence list
17/08/01 14:49:58 INFO storage.BlockManager: Removing RDD 52
17/08/01 14:49:58 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[52] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598998000 ms
17/08/01 14:49:58 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598996000 ms
17/08/01 14:49:58 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598996000 ms
17/08/01 14:49:59 INFO scheduler.JobScheduler: Added jobs for time 1501598999000 ms
17/08/01 14:49:59 INFO scheduler.JobScheduler: Starting job streaming job 1501598999000 ms.0 from job set of time 1501598999000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:59 INFO scheduler.JobScheduler: Finished job streaming job 1501598999000 ms.0 from job set of time 1501598999000 ms
17/08/01 14:49:59 INFO python.PythonRDD: Removing RDD 55 from persistence list
17/08/01 14:49:59 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598999000 ms (execution: 0.007 s)
17/08/01 14:49:59 INFO storage.BlockManager: Removing RDD 55
17/08/01 14:49:59 INFO rdd.BlockRDD: Removing RDD 54 from persistence list
17/08/01 14:49:59 INFO storage.BlockManager: Removing RDD 54
17/08/01 14:49:59 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[54] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598999000 ms
17/08/01 14:49:59 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598997000 ms
17/08/01 14:49:59 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598997000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Added jobs for time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Starting job streaming job 1501599000000 ms.0 from job set of time 1501599000000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:00 INFO scheduler.JobScheduler: Finished job streaming job 1501599000000 ms.0 from job set of time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599000000 ms (execution: 0.006 s)
17/08/01 14:50:00 INFO python.PythonRDD: Removing RDD 57 from persistence list
17/08/01 14:50:00 INFO storage.BlockManager: Removing RDD 57
17/08/01 14:50:00 INFO rdd.BlockRDD: Removing RDD 56 from persistence list
17/08/01 14:50:00 INFO storage.BlockManager: Removing RDD 56
17/08/01 14:50:00 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[56] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598998000 ms
17/08/01 14:50:00 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598998000 ms
17/08/01 14:50:01 INFO scheduler.JobScheduler: Added jobs for time 1501599001000 ms
17/08/01 14:50:01 INFO scheduler.JobScheduler: Starting job streaming job 1501599001000 ms.0 from job set of time 1501599001000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:01 INFO scheduler.JobScheduler: Finished job streaming job 1501599001000 ms.0 from job set of time 1501599001000 ms
17/08/01 14:50:01 INFO python.PythonRDD: Removing RDD 59 from persistence list
17/08/01 14:50:01 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599001000 ms (execution: 0.007 s)
17/08/01 14:50:01 INFO storage.BlockManager: Removing RDD 59
17/08/01 14:50:01 INFO rdd.BlockRDD: Removing RDD 58 from persistence list
17/08/01 14:50:01 INFO storage.BlockManager: Removing RDD 58
17/08/01 14:50:01 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[58] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599001000 ms
17/08/01 14:50:01 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598999000 ms
17/08/01 14:50:01 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598999000 ms
17/08/01 14:50:02 INFO scheduler.JobScheduler: Added jobs for time 1501599002000 ms
17/08/01 14:50:02 INFO scheduler.JobScheduler: Starting job streaming job 1501599002000 ms.0 from job set of time 1501599002000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:02 INFO scheduler.JobScheduler: Finished job streaming job 1501599002000 ms.0 from job set of time 1501599002000 ms
17/08/01 14:50:02 INFO python.PythonRDD: Removing RDD 61 from persistence list
17/08/01 14:50:02 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599002000 ms (execution: 0.006 s)
17/08/01 14:50:02 INFO storage.BlockManager: Removing RDD 61
17/08/01 14:50:02 INFO rdd.BlockRDD: Removing RDD 60 from persistence list
17/08/01 14:50:02 INFO storage.BlockManager: Removing RDD 60
17/08/01 14:50:02 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[60] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599002000 ms
17/08/01 14:50:02 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599000000 ms
17/08/01 14:50:02 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599000000 ms
17/08/01 14:50:03 INFO scheduler.JobScheduler: Added jobs for time 1501599003000 ms
17/08/01 14:50:03 INFO scheduler.JobScheduler: Starting job streaming job 1501599003000 ms.0 from job set of time 1501599003000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:03 INFO scheduler.JobScheduler: Finished job streaming job 1501599003000 ms.0 from job set of time 1501599003000 ms
17/08/01 14:50:03 INFO python.PythonRDD: Removing RDD 63 from persistence list
17/08/01 14:50:03 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599003000 ms (execution: 0.007 s)
17/08/01 14:50:03 INFO storage.BlockManager: Removing RDD 63
17/08/01 14:50:03 INFO rdd.BlockRDD: Removing RDD 62 from persistence list
17/08/01 14:50:03 INFO storage.BlockManager: Removing RDD 62
17/08/01 14:50:03 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[62] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599003000 ms
17/08/01 14:50:03 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599001000 ms
17/08/01 14:50:03 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599001000 ms
17/08/01 14:50:04 INFO scheduler.JobScheduler: Added jobs for time 1501599004000 ms
17/08/01 14:50:04 INFO scheduler.JobScheduler: Starting job streaming job 1501599004000 ms.0 from job set of time 1501599004000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:04 INFO scheduler.JobScheduler: Finished job streaming job 1501599004000 ms.0 from job set of time 1501599004000 ms
17/08/01 14:50:04 INFO python.PythonRDD: Removing RDD 65 from persistence list
17/08/01 14:50:04 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599004000 ms (execution: 0.006 s)
17/08/01 14:50:04 INFO storage.BlockManager: Removing RDD 65
17/08/01 14:50:04 INFO rdd.BlockRDD: Removing RDD 64 from persistence list
17/08/01 14:50:04 INFO storage.BlockManager: Removing RDD 64
17/08/01 14:50:04 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[64] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599004000 ms
17/08/01 14:50:04 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599002000 ms
17/08/01 14:50:04 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599002000 ms
17/08/01 14:50:05 INFO scheduler.JobScheduler: Added jobs for time 1501599005000 ms
17/08/01 14:50:05 INFO scheduler.JobScheduler: Starting job streaming job 1501599005000 ms.0 from job set of time 1501599005000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:05 INFO scheduler.JobScheduler: Finished job streaming job 1501599005000 ms.0 from job set of time 1501599005000 ms
17/08/01 14:50:05 INFO python.PythonRDD: Removing RDD 67 from persistence list
17/08/01 14:50:05 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501599005000 ms (execution: 0.013 s)
17/08/01 14:50:05 INFO storage.BlockManager: Removing RDD 67
17/08/01 14:50:05 INFO rdd.BlockRDD: Removing RDD 66 from persistence list
17/08/01 14:50:05 INFO storage.BlockManager: Removing RDD 66
17/08/01 14:50:05 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[66] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599005000 ms
17/08/01 14:50:05 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599003000 ms
17/08/01 14:50:05 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599003000 ms
17/08/01 14:50:06 INFO scheduler.JobScheduler: Added jobs for time 1501599006000 ms
17/08/01 14:50:06 INFO scheduler.JobScheduler: Starting job streaming job 1501599006000 ms.0 from job set of time 1501599006000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:06 INFO scheduler.JobScheduler: Finished job streaming job 1501599006000 ms.0 from job set of time 1501599006000 ms
17/08/01 14:50:06 INFO python.PythonRDD: Removing RDD 69 from persistence list
17/08/01 14:50:06 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599006000 ms (execution: 0.008 s)
17/08/01 14:50:06 INFO storage.BlockManager: Removing RDD 69
17/08/01 14:50:06 INFO rdd.BlockRDD: Removing RDD 68 from persistence list
17/08/01 14:50:06 INFO storage.BlockManager: Removing RDD 68
17/08/01 14:50:06 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[68] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599006000 ms
17/08/01 14:50:06 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599004000 ms
17/08/01 14:50:06 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599004000 ms
17/08/01 14:50:07 INFO scheduler.JobScheduler: Added jobs for time 1501599007000 ms
17/08/01 14:50:07 INFO scheduler.JobScheduler: Starting job streaming job 1501599007000 ms.0 from job set of time 1501599007000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:07 INFO scheduler.JobScheduler: Finished job streaming job 1501599007000 ms.0 from job set of time 1501599007000 ms
17/08/01 14:50:07 INFO python.PythonRDD: Removing RDD 71 from persistence list
17/08/01 14:50:07 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599007000 ms (execution: 0.009 s)
17/08/01 14:50:07 INFO rdd.BlockRDD: Removing RDD 70 from persistence list
17/08/01 14:50:07 INFO storage.BlockManager: Removing RDD 71
17/08/01 14:50:07 INFO storage.BlockManager: Removing RDD 70
17/08/01 14:50:07 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[70] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599007000 ms
17/08/01 14:50:07 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599005000 ms
17/08/01 14:50:07 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599005000 ms
17/08/01 14:50:08 INFO scheduler.JobScheduler: Added jobs for time 1501599008000 ms
17/08/01 14:50:08 INFO scheduler.JobScheduler: Starting job streaming job 1501599008000 ms.0 from job set of time 1501599008000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:08 INFO scheduler.JobScheduler: Finished job streaming job 1501599008000 ms.0 from job set of time 1501599008000 ms
17/08/01 14:50:08 INFO python.PythonRDD: Removing RDD 73 from persistence list
17/08/01 14:50:08 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599008000 ms (execution: 0.006 s)
17/08/01 14:50:08 INFO storage.BlockManager: Removing RDD 73
17/08/01 14:50:08 INFO rdd.BlockRDD: Removing RDD 72 from persistence list
17/08/01 14:50:08 INFO storage.BlockManager: Removing RDD 72
17/08/01 14:50:08 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[72] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599008000 ms
17/08/01 14:50:08 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599006000 ms
17/08/01 14:50:08 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599006000 ms
17/08/01 14:50:09 INFO scheduler.JobScheduler: Added jobs for time 1501599009000 ms
17/08/01 14:50:09 INFO scheduler.JobScheduler: Starting job streaming job 1501599009000 ms.0 from job set of time 1501599009000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:09 INFO scheduler.JobScheduler: Finished job streaming job 1501599009000 ms.0 from job set of time 1501599009000 ms
17/08/01 14:50:09 INFO python.PythonRDD: Removing RDD 75 from persistence list
17/08/01 14:50:09 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599009000 ms (execution: 0.005 s)
17/08/01 14:50:09 INFO storage.BlockManager: Removing RDD 75
17/08/01 14:50:09 INFO rdd.BlockRDD: Removing RDD 74 from persistence list
17/08/01 14:50:09 INFO storage.BlockManager: Removing RDD 74
17/08/01 14:50:09 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[74] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599009000 ms
17/08/01 14:50:09 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599007000 ms
17/08/01 14:50:09 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599007000 ms
17/08/01 14:50:10 INFO scheduler.JobScheduler: Added jobs for time 1501599010000 ms
17/08/01 14:50:10 INFO scheduler.JobScheduler: Starting job streaming job 1501599010000 ms.0 from job set of time 1501599010000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:10 INFO scheduler.JobScheduler: Finished job streaming job 1501599010000 ms.0 from job set of time 1501599010000 ms
17/08/01 14:50:10 INFO python.PythonRDD: Removing RDD 77 from persistence list
17/08/01 14:50:10 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599010000 ms (execution: 0.005 s)
17/08/01 14:50:10 INFO rdd.BlockRDD: Removing RDD 76 from persistence list
17/08/01 14:50:10 INFO storage.BlockManager: Removing RDD 76
17/08/01 14:50:10 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[76] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599010000 ms
17/08/01 14:50:10 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599008000 ms
17/08/01 14:50:10 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599008000 ms
17/08/01 14:50:10 INFO storage.BlockManager: Removing RDD 77
17/08/01 14:50:11 INFO scheduler.JobScheduler: Added jobs for time 1501599011000 ms
17/08/01 14:50:11 INFO scheduler.JobScheduler: Starting job streaming job 1501599011000 ms.0 from job set of time 1501599011000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:11 INFO scheduler.JobScheduler: Finished job streaming job 1501599011000 ms.0 from job set of time 1501599011000 ms
17/08/01 14:50:11 INFO python.PythonRDD: Removing RDD 79 from persistence list
17/08/01 14:50:11 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599011000 ms (execution: 0.007 s)
17/08/01 14:50:11 INFO storage.BlockManager: Removing RDD 79
17/08/01 14:50:11 INFO rdd.BlockRDD: Removing RDD 78 from persistence list
17/08/01 14:50:11 INFO storage.BlockManager: Removing RDD 78
17/08/01 14:50:11 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[78] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599011000 ms
17/08/01 14:50:11 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599009000 ms
17/08/01 14:50:11 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599009000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Added jobs for time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Starting job streaming job 1501599012000 ms.0 from job set of time 1501599012000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:12 INFO scheduler.JobScheduler: Finished job streaming job 1501599012000 ms.0 from job set of time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599012000 ms (execution: 0.015 s)
17/08/01 14:50:12 INFO python.PythonRDD: Removing RDD 81 from persistence list
17/08/01 14:50:12 INFO storage.BlockManager: Removing RDD 81
17/08/01 14:50:12 INFO rdd.BlockRDD: Removing RDD 80 from persistence list
17/08/01 14:50:12 INFO storage.BlockManager: Removing RDD 80
17/08/01 14:50:12 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[80] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599010000 ms
17/08/01 14:50:12 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599010000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Added jobs for time 1501599013000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Starting job streaming job 1501599013000 ms.0 from job set of time 1501599013000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:13 INFO scheduler.JobScheduler: Finished job streaming job 1501599013000 ms.0 from job set of time 1501599013000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501599013000 ms (execution: 0.008 s)
17/08/01 14:50:13 INFO python.PythonRDD: Removing RDD 83 from persistence list
17/08/01 14:50:13 INFO storage.BlockManager: Removing RDD 83
17/08/01 14:50:13 INFO rdd.BlockRDD: Removing RDD 82 from persistence list
17/08/01 14:50:13 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[82] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599013000 ms
17/08/01 14:50:13 INFO storage.BlockManager: Removing RDD 82
17/08/01 14:50:13 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599011000 ms
17/08/01 14:50:13 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599011000 ms
17/08/01 14:50:14 INFO scheduler.JobScheduler: Added jobs for time 1501599014000 ms
17/08/01 14:50:14 INFO scheduler.JobScheduler: Starting job streaming job 1501599014000 ms.0 from job set of time 1501599014000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:14 INFO scheduler.JobScheduler: Finished job streaming job 1501599014000 ms.0 from job set of time 1501599014000 ms
17/08/01 14:50:14 INFO python.PythonRDD: Removing RDD 85 from persistence list
17/08/01 14:50:14 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599014000 ms (execution: 0.007 s)
17/08/01 14:50:14 INFO rdd.BlockRDD: Removing RDD 84 from persistence list
17/08/01 14:50:14 INFO storage.BlockManager: Removing RDD 85
17/08/01 14:50:14 INFO storage.BlockManager: Removing RDD 84
17/08/01 14:50:14 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[84] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599014000 ms
17/08/01 14:50:14 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599012000 ms
17/08/01 14:50:14 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599012000 ms
17/08/01 14:50:15 INFO scheduler.JobScheduler: Added jobs for time 1501599015000 ms
17/08/01 14:50:15 INFO scheduler.JobScheduler: Starting job streaming job 1501599015000 ms.0 from job set of time 1501599015000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:15 INFO scheduler.JobScheduler: Finished job streaming job 1501599015000 ms.0 from job set of time 1501599015000 ms
17/08/01 14:50:15 INFO python.PythonRDD: Removing RDD 87 from persistence list
17/08/01 14:50:15 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599015000 ms (execution: 0.007 s)
17/08/01 14:50:15 INFO storage.BlockManager: Removing RDD 87
17/08/01 14:50:15 INFO rdd.BlockRDD: Removing RDD 86 from persistence list
17/08/01 14:50:15 INFO storage.BlockManager: Removing RDD 86
17/08/01 14:50:15 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[86] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599015000 ms
17/08/01 14:50:15 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599013000 ms
17/08/01 14:50:15 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599013000 ms
17/08/01 14:50:16 INFO scheduler.JobScheduler: Added jobs for time 1501599016000 ms
17/08/01 14:50:16 INFO scheduler.JobScheduler: Starting job streaming job 1501599016000 ms.0 from job set of time 1501599016000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:16 INFO scheduler.JobScheduler: Finished job streaming job 1501599016000 ms.0 from job set of time 1501599016000 ms
17/08/01 14:50:16 INFO python.PythonRDD: Removing RDD 89 from persistence list
17/08/01 14:50:16 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501599016000 ms (execution: 0.008 s)
17/08/01 14:50:16 INFO storage.BlockManager: Removing RDD 89
17/08/01 14:50:16 INFO rdd.BlockRDD: Removing RDD 88 from persistence list
17/08/01 14:50:16 INFO storage.BlockManager: Removing RDD 88
17/08/01 14:50:16 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[88] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599016000 ms
17/08/01 14:50:16 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599014000 ms
17/08/01 14:50:16 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599014000 ms
17/08/01 14:50:17 INFO scheduler.JobScheduler: Added jobs for time 1501599017000 ms
17/08/01 14:50:17 INFO scheduler.JobScheduler: Starting job streaming job 1501599017000 ms.0 from job set of time 1501599017000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:17 INFO scheduler.JobScheduler: Finished job streaming job 1501599017000 ms.0 from job set of time 1501599017000 ms
17/08/01 14:50:17 INFO python.PythonRDD: Removing RDD 91 from persistence list
17/08/01 14:50:17 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501599017000 ms (execution: 0.008 s)
17/08/01 14:50:17 INFO storage.BlockManager: Removing RDD 91
17/08/01 14:50:17 INFO rdd.BlockRDD: Removing RDD 90 from persistence list
17/08/01 14:50:17 INFO storage.BlockManager: Removing RDD 90
17/08/01 14:50:17 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[90] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599017000 ms
17/08/01 14:50:17 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599015000 ms
17/08/01 14:50:17 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599015000 ms
17/08/01 14:50:18 INFO scheduler.JobScheduler: Added jobs for time 1501599018000 ms
17/08/01 14:50:18 INFO scheduler.JobScheduler: Starting job streaming job 1501599018000 ms.0 from job set of time 1501599018000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:18 INFO scheduler.JobScheduler: Finished job streaming job 1501599018000 ms.0 from job set of time 1501599018000 ms
17/08/01 14:50:18 INFO python.PythonRDD: Removing RDD 93 from persistence list
17/08/01 14:50:18 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599018000 ms (execution: 0.006 s)
17/08/01 14:50:18 INFO storage.BlockManager: Removing RDD 93
17/08/01 14:50:18 INFO rdd.BlockRDD: Removing RDD 92 from persistence list
17/08/01 14:50:18 INFO storage.BlockManager: Removing RDD 92
17/08/01 14:50:18 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[92] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599018000 ms
17/08/01 14:50:18 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599016000 ms
17/08/01 14:50:18 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599016000 ms
17/08/01 14:50:19 INFO scheduler.JobScheduler: Added jobs for time 1501599019000 ms
17/08/01 14:50:19 INFO scheduler.JobScheduler: Starting job streaming job 1501599019000 ms.0 from job set of time 1501599019000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:19 INFO scheduler.JobScheduler: Finished job streaming job 1501599019000 ms.0 from job set of time 1501599019000 ms
17/08/01 14:50:19 INFO python.PythonRDD: Removing RDD 95 from persistence list
17/08/01 14:50:19 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599019000 ms (execution: 0.015 s)
17/08/01 14:50:19 INFO storage.BlockManager: Removing RDD 95
17/08/01 14:50:19 INFO rdd.BlockRDD: Removing RDD 94 from persistence list
17/08/01 14:50:19 INFO storage.BlockManager: Removing RDD 94
17/08/01 14:50:19 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[94] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599019000 ms
17/08/01 14:50:19 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599017000 ms
17/08/01 14:50:19 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599017000 ms
17/08/01 14:50:20 INFO scheduler.JobScheduler: Added jobs for time 1501599020000 ms
17/08/01 14:50:20 INFO scheduler.JobScheduler: Starting job streaming job 1501599020000 ms.0 from job set of time 1501599020000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:20 INFO scheduler.JobScheduler: Finished job streaming job 1501599020000 ms.0 from job set of time 1501599020000 ms
17/08/01 14:50:20 INFO python.PythonRDD: Removing RDD 97 from persistence list
17/08/01 14:50:20 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599020000 ms (execution: 0.006 s)
17/08/01 14:50:20 INFO storage.BlockManager: Removing RDD 97
17/08/01 14:50:20 INFO rdd.BlockRDD: Removing RDD 96 from persistence list
17/08/01 14:50:20 INFO storage.BlockManager: Removing RDD 96
17/08/01 14:50:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[96] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599020000 ms
17/08/01 14:50:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599018000 ms
17/08/01 14:50:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599018000 ms
17/08/01 14:50:21 INFO scheduler.JobScheduler: Added jobs for time 1501599021000 ms
17/08/01 14:50:21 INFO scheduler.JobScheduler: Starting job streaming job 1501599021000 ms.0 from job set of time 1501599021000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:21 INFO scheduler.JobScheduler: Finished job streaming job 1501599021000 ms.0 from job set of time 1501599021000 ms
17/08/01 14:50:21 INFO python.PythonRDD: Removing RDD 99 from persistence list
17/08/01 14:50:21 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599021000 ms (execution: 0.007 s)
17/08/01 14:50:21 INFO storage.BlockManager: Removing RDD 99
17/08/01 14:50:21 INFO rdd.BlockRDD: Removing RDD 98 from persistence list
17/08/01 14:50:21 INFO storage.BlockManager: Removing RDD 98
17/08/01 14:50:21 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[98] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599021000 ms
17/08/01 14:50:21 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599019000 ms
17/08/01 14:50:21 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599019000 ms
17/08/01 14:50:22 INFO scheduler.JobScheduler: Added jobs for time 1501599022000 ms
17/08/01 14:50:22 INFO scheduler.JobScheduler: Starting job streaming job 1501599022000 ms.0 from job set of time 1501599022000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:22 INFO scheduler.JobScheduler: Finished job streaming job 1501599022000 ms.0 from job set of time 1501599022000 ms
17/08/01 14:50:22 INFO python.PythonRDD: Removing RDD 101 from persistence list
17/08/01 14:50:22 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501599022000 ms (execution: 0.007 s)
17/08/01 14:50:22 INFO storage.BlockManager: Removing RDD 101
17/08/01 14:50:22 INFO rdd.BlockRDD: Removing RDD 100 from persistence list
17/08/01 14:50:22 INFO storage.BlockManager: Removing RDD 100
17/08/01 14:50:22 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[100] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599022000 ms
17/08/01 14:50:22 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599020000 ms
17/08/01 14:50:22 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599020000 ms
17/08/01 14:50:23 INFO scheduler.JobScheduler: Added jobs for time 1501599023000 ms
17/08/01 14:50:23 INFO scheduler.JobScheduler: Starting job streaming job 1501599023000 ms.0 from job set of time 1501599023000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:23 INFO scheduler.JobScheduler: Finished job streaming job 1501599023000 ms.0 from job set of time 1501599023000 ms
17/08/01 14:50:23 INFO python.PythonRDD: Removing RDD 103 from persistence list
17/08/01 14:50:23 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599023000 ms (execution: 0.007 s)
17/08/01 14:50:23 INFO storage.BlockManager: Removing RDD 103
17/08/01 14:50:23 INFO rdd.BlockRDD: Removing RDD 102 from persistence list
17/08/01 14:50:23 INFO storage.BlockManager: Removing RDD 102
17/08/01 14:50:23 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[102] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599023000 ms
17/08/01 14:50:23 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599021000 ms
17/08/01 14:50:23 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599021000 ms
17/08/01 14:50:24 INFO scheduler.JobScheduler: Added jobs for time 1501599024000 ms
17/08/01 14:50:24 INFO scheduler.JobScheduler: Starting job streaming job 1501599024000 ms.0 from job set of time 1501599024000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:24 INFO scheduler.JobScheduler: Finished job streaming job 1501599024000 ms.0 from job set of time 1501599024000 ms
17/08/01 14:50:24 INFO python.PythonRDD: Removing RDD 105 from persistence list
17/08/01 14:50:24 INFO scheduler.JobScheduler: Total delay: 0.035 s for time 1501599024000 ms (execution: 0.007 s)
17/08/01 14:50:24 INFO storage.BlockManager: Removing RDD 105
17/08/01 14:50:24 INFO rdd.BlockRDD: Removing RDD 104 from persistence list
17/08/01 14:50:24 INFO storage.BlockManager: Removing RDD 104
17/08/01 14:50:24 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[104] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599024000 ms
17/08/01 14:50:24 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599022000 ms
17/08/01 14:50:24 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599022000 ms
17/08/01 14:50:25 INFO scheduler.JobScheduler: Added jobs for time 1501599025000 ms
17/08/01 14:50:25 INFO scheduler.JobScheduler: Starting job streaming job 1501599025000 ms.0 from job set of time 1501599025000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:25 INFO scheduler.JobScheduler: Finished job streaming job 1501599025000 ms.0 from job set of time 1501599025000 ms
17/08/01 14:50:25 INFO python.PythonRDD: Removing RDD 107 from persistence list
17/08/01 14:50:25 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599025000 ms (execution: 0.007 s)
17/08/01 14:50:25 INFO storage.BlockManager: Removing RDD 107
17/08/01 14:50:25 INFO rdd.BlockRDD: Removing RDD 106 from persistence list
17/08/01 14:50:25 INFO storage.BlockManager: Removing RDD 106
17/08/01 14:50:25 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[106] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599025000 ms
17/08/01 14:50:25 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599023000 ms
17/08/01 14:50:25 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599023000 ms
17/08/01 14:50:26 INFO scheduler.JobScheduler: Added jobs for time 1501599026000 ms
17/08/01 14:50:26 INFO scheduler.JobScheduler: Starting job streaming job 1501599026000 ms.0 from job set of time 1501599026000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:26 INFO scheduler.JobScheduler: Finished job streaming job 1501599026000 ms.0 from job set of time 1501599026000 ms
17/08/01 14:50:26 INFO python.PythonRDD: Removing RDD 109 from persistence list
17/08/01 14:50:26 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599026000 ms (execution: 0.007 s)
17/08/01 14:50:26 INFO storage.BlockManager: Removing RDD 109
17/08/01 14:50:26 INFO rdd.BlockRDD: Removing RDD 108 from persistence list
17/08/01 14:50:26 INFO storage.BlockManager: Removing RDD 108
17/08/01 14:50:26 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[108] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599026000 ms
17/08/01 14:50:26 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599024000 ms
17/08/01 14:50:26 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599024000 ms
17/08/01 14:50:27 INFO scheduler.JobScheduler: Added jobs for time 1501599027000 ms
17/08/01 14:50:27 INFO scheduler.JobScheduler: Starting job streaming job 1501599027000 ms.0 from job set of time 1501599027000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:27 INFO scheduler.JobScheduler: Finished job streaming job 1501599027000 ms.0 from job set of time 1501599027000 ms
17/08/01 14:50:27 INFO python.PythonRDD: Removing RDD 111 from persistence list
17/08/01 14:50:27 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599027000 ms (execution: 0.007 s)
17/08/01 14:50:27 INFO storage.BlockManager: Removing RDD 111
17/08/01 14:50:27 INFO rdd.BlockRDD: Removing RDD 110 from persistence list
17/08/01 14:50:27 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[110] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599027000 ms
17/08/01 14:50:27 INFO storage.BlockManager: Removing RDD 110
17/08/01 14:50:27 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599025000 ms
17/08/01 14:50:27 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599025000 ms
17/08/01 14:50:28 INFO scheduler.JobScheduler: Added jobs for time 1501599028000 ms
17/08/01 14:50:28 INFO scheduler.JobScheduler: Starting job streaming job 1501599028000 ms.0 from job set of time 1501599028000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:28 INFO scheduler.JobScheduler: Finished job streaming job 1501599028000 ms.0 from job set of time 1501599028000 ms
17/08/01 14:50:28 INFO python.PythonRDD: Removing RDD 113 from persistence list
17/08/01 14:50:28 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599028000 ms (execution: 0.017 s)
17/08/01 14:50:28 INFO storage.BlockManager: Removing RDD 113
17/08/01 14:50:28 INFO rdd.BlockRDD: Removing RDD 112 from persistence list
17/08/01 14:50:28 INFO storage.BlockManager: Removing RDD 112
17/08/01 14:50:28 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[112] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599028000 ms
17/08/01 14:50:28 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599026000 ms
17/08/01 14:50:28 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599026000 ms
17/08/01 14:50:29 INFO scheduler.JobScheduler: Added jobs for time 1501599029000 ms
17/08/01 14:50:29 INFO scheduler.JobScheduler: Starting job streaming job 1501599029000 ms.0 from job set of time 1501599029000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:29 INFO scheduler.JobScheduler: Finished job streaming job 1501599029000 ms.0 from job set of time 1501599029000 ms
17/08/01 14:50:29 INFO python.PythonRDD: Removing RDD 115 from persistence list
17/08/01 14:50:29 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599029000 ms (execution: 0.007 s)
17/08/01 14:50:29 INFO storage.BlockManager: Removing RDD 115
17/08/01 14:50:29 INFO rdd.BlockRDD: Removing RDD 114 from persistence list
17/08/01 14:50:29 INFO storage.BlockManager: Removing RDD 114
17/08/01 14:50:29 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[114] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599029000 ms
17/08/01 14:50:29 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599027000 ms
17/08/01 14:50:29 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599027000 ms
17/08/01 14:50:30 INFO scheduler.JobScheduler: Added jobs for time 1501599030000 ms
17/08/01 14:50:30 INFO scheduler.JobScheduler: Starting job streaming job 1501599030000 ms.0 from job set of time 1501599030000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:30 INFO scheduler.JobScheduler: Finished job streaming job 1501599030000 ms.0 from job set of time 1501599030000 ms
17/08/01 14:50:30 INFO python.PythonRDD: Removing RDD 117 from persistence list
17/08/01 14:50:30 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599030000 ms (execution: 0.007 s)
17/08/01 14:50:30 INFO storage.BlockManager: Removing RDD 117
17/08/01 14:50:30 INFO rdd.BlockRDD: Removing RDD 116 from persistence list
17/08/01 14:50:30 INFO storage.BlockManager: Removing RDD 116
17/08/01 14:50:30 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[116] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599030000 ms
17/08/01 14:50:30 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599028000 ms
17/08/01 14:50:30 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599028000 ms
17/08/01 14:50:31 INFO scheduler.JobScheduler: Added jobs for time 1501599031000 ms
17/08/01 14:50:31 INFO scheduler.JobScheduler: Starting job streaming job 1501599031000 ms.0 from job set of time 1501599031000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:31 INFO scheduler.JobScheduler: Finished job streaming job 1501599031000 ms.0 from job set of time 1501599031000 ms
17/08/01 14:50:31 INFO python.PythonRDD: Removing RDD 119 from persistence list
17/08/01 14:50:31 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599031000 ms (execution: 0.006 s)
17/08/01 14:50:31 INFO rdd.BlockRDD: Removing RDD 118 from persistence list
17/08/01 14:50:31 INFO storage.BlockManager: Removing RDD 119
17/08/01 14:50:31 INFO storage.BlockManager: Removing RDD 118
17/08/01 14:50:31 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[118] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599031000 ms
17/08/01 14:50:31 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599029000 ms
17/08/01 14:50:31 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599029000 ms
17/08/01 14:50:32 INFO scheduler.JobScheduler: Added jobs for time 1501599032000 ms
17/08/01 14:50:32 INFO scheduler.JobScheduler: Starting job streaming job 1501599032000 ms.0 from job set of time 1501599032000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:32 INFO scheduler.JobScheduler: Finished job streaming job 1501599032000 ms.0 from job set of time 1501599032000 ms
17/08/01 14:50:32 INFO python.PythonRDD: Removing RDD 121 from persistence list
17/08/01 14:50:32 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599032000 ms (execution: 0.007 s)
17/08/01 14:50:32 INFO storage.BlockManager: Removing RDD 121
17/08/01 14:50:32 INFO rdd.BlockRDD: Removing RDD 120 from persistence list
17/08/01 14:50:32 INFO storage.BlockManager: Removing RDD 120
17/08/01 14:50:32 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[120] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599032000 ms
17/08/01 14:50:32 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599030000 ms
17/08/01 14:50:32 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599030000 ms
17/08/01 14:50:33 INFO scheduler.JobScheduler: Added jobs for time 1501599033000 ms
17/08/01 14:50:33 INFO scheduler.JobScheduler: Starting job streaming job 1501599033000 ms.0 from job set of time 1501599033000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:33 INFO scheduler.JobScheduler: Finished job streaming job 1501599033000 ms.0 from job set of time 1501599033000 ms
17/08/01 14:50:33 INFO python.PythonRDD: Removing RDD 123 from persistence list
17/08/01 14:50:33 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599033000 ms (execution: 0.005 s)
17/08/01 14:50:33 INFO storage.BlockManager: Removing RDD 123
17/08/01 14:50:33 INFO rdd.BlockRDD: Removing RDD 122 from persistence list
17/08/01 14:50:33 INFO storage.BlockManager: Removing RDD 122
17/08/01 14:50:33 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[122] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599033000 ms
17/08/01 14:50:33 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599031000 ms
17/08/01 14:50:33 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599031000 ms
17/08/01 14:50:34 INFO scheduler.JobScheduler: Added jobs for time 1501599034000 ms
17/08/01 14:50:34 INFO scheduler.JobScheduler: Starting job streaming job 1501599034000 ms.0 from job set of time 1501599034000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:34 INFO scheduler.JobScheduler: Finished job streaming job 1501599034000 ms.0 from job set of time 1501599034000 ms
17/08/01 14:50:34 INFO python.PythonRDD: Removing RDD 125 from persistence list
17/08/01 14:50:34 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599034000 ms (execution: 0.006 s)
17/08/01 14:50:34 INFO storage.BlockManager: Removing RDD 125
17/08/01 14:50:34 INFO rdd.BlockRDD: Removing RDD 124 from persistence list
17/08/01 14:50:34 INFO storage.BlockManager: Removing RDD 124
17/08/01 14:50:34 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[124] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599034000 ms
17/08/01 14:50:34 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599032000 ms
17/08/01 14:50:34 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599032000 ms
17/08/01 14:50:34 INFO spark.ExecutorAllocationManager: Request to remove executorIds: 2
17/08/01 14:50:34 INFO cluster.YarnClientSchedulerBackend: Requesting to kill executor(s) 2
17/08/01 14:50:34 INFO cluster.YarnClientSchedulerBackend: Actual list of executor(s) to be killed is 2
17/08/01 14:50:34 INFO spark.ExecutorAllocationManager: Removing executor 2 because it has been idle for 60 seconds (new desired total will be 1)
17/08/01 14:50:35 INFO scheduler.JobScheduler: Added jobs for time 1501599035000 ms
17/08/01 14:50:35 INFO scheduler.JobScheduler: Starting job streaming job 1501599035000 ms.0 from job set of time 1501599035000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:35 INFO scheduler.JobScheduler: Finished job streaming job 1501599035000 ms.0 from job set of time 1501599035000 ms
17/08/01 14:50:35 INFO python.PythonRDD: Removing RDD 127 from persistence list
17/08/01 14:50:35 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599035000 ms (execution: 0.013 s)
17/08/01 14:50:35 INFO storage.BlockManager: Removing RDD 127
17/08/01 14:50:35 INFO rdd.BlockRDD: Removing RDD 126 from persistence list
17/08/01 14:50:35 INFO storage.BlockManager: Removing RDD 126
17/08/01 14:50:35 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[126] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599035000 ms
17/08/01 14:50:35 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599033000 ms
17/08/01 14:50:35 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599033000 ms
17/08/01 14:50:36 INFO scheduler.JobScheduler: Added jobs for time 1501599036000 ms
17/08/01 14:50:36 INFO scheduler.JobScheduler: Starting job streaming job 1501599036000 ms.0 from job set of time 1501599036000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:36 INFO scheduler.JobScheduler: Finished job streaming job 1501599036000 ms.0 from job set of time 1501599036000 ms
17/08/01 14:50:36 INFO python.PythonRDD: Removing RDD 129 from persistence list
17/08/01 14:50:36 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599036000 ms (execution: 0.007 s)
17/08/01 14:50:36 INFO storage.BlockManager: Removing RDD 129
17/08/01 14:50:36 INFO rdd.BlockRDD: Removing RDD 128 from persistence list
17/08/01 14:50:36 INFO storage.BlockManager: Removing RDD 128
17/08/01 14:50:36 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[128] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599036000 ms
17/08/01 14:50:36 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599034000 ms
17/08/01 14:50:36 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599034000 ms
17/08/01 14:50:37 INFO scheduler.JobScheduler: Added jobs for time 1501599037000 ms
17/08/01 14:50:37 INFO scheduler.JobScheduler: Starting job streaming job 1501599037000 ms.0 from job set of time 1501599037000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:37 INFO scheduler.JobScheduler: Finished job streaming job 1501599037000 ms.0 from job set of time 1501599037000 ms
17/08/01 14:50:37 INFO python.PythonRDD: Removing RDD 131 from persistence list
17/08/01 14:50:37 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599037000 ms (execution: 0.006 s)
17/08/01 14:50:37 INFO storage.BlockManager: Removing RDD 131
17/08/01 14:50:37 INFO rdd.BlockRDD: Removing RDD 130 from persistence list
17/08/01 14:50:37 INFO storage.BlockManager: Removing RDD 130
17/08/01 14:50:37 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[130] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599037000 ms
17/08/01 14:50:37 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599035000 ms
17/08/01 14:50:37 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599035000 ms
17/08/01 14:50:38 INFO scheduler.JobScheduler: Added jobs for time 1501599038000 ms
17/08/01 14:50:38 INFO scheduler.JobScheduler: Starting job streaming job 1501599038000 ms.0 from job set of time 1501599038000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:38 INFO scheduler.JobScheduler: Finished job streaming job 1501599038000 ms.0 from job set of time 1501599038000 ms
17/08/01 14:50:38 INFO python.PythonRDD: Removing RDD 133 from persistence list
17/08/01 14:50:38 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599038000 ms (execution: 0.007 s)
17/08/01 14:50:38 INFO storage.BlockManager: Removing RDD 133
17/08/01 14:50:38 INFO rdd.BlockRDD: Removing RDD 132 from persistence list
17/08/01 14:50:38 INFO storage.BlockManager: Removing RDD 132
17/08/01 14:50:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[132] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599038000 ms
17/08/01 14:50:38 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599036000 ms
17/08/01 14:50:38 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599036000 ms
17/08/01 14:50:38 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Disabling executor 2.
17/08/01 14:50:38 INFO scheduler.DAGScheduler: Executor lost: 2 (epoch 1)
17/08/01 14:50:38 INFO storage.BlockManagerMasterEndpoint: Trying to remove executor 2 from BlockManagerMaster.
17/08/01 14:50:38 INFO storage.BlockManagerMasterEndpoint: Removing block manager BlockManagerId(2, ip-10-0-0-127.ec2.internal, 59267, None)
17/08/01 14:50:38 INFO storage.BlockManagerMaster: Removed 2 successfully in removeExecutor
17/08/01 14:50:38 INFO cluster.YarnScheduler: Executor 2 on ip-10-0-0-127.ec2.internal killed by driver.
17/08/01 14:50:38 INFO spark.ExecutorAllocationManager: Existing executor 2 has been removed (new total is 1)
17/08/01 14:50:39 INFO scheduler.JobScheduler: Added jobs for time 1501599039000 ms
17/08/01 14:50:39 INFO scheduler.JobScheduler: Starting job streaming job 1501599039000 ms.0 from job set of time 1501599039000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:39 INFO scheduler.JobScheduler: Finished job streaming job 1501599039000 ms.0 from job set of time 1501599039000 ms
17/08/01 14:50:39 INFO python.PythonRDD: Removing RDD 135 from persistence list
17/08/01 14:50:39 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599039000 ms (execution: 0.006 s)
17/08/01 14:50:39 INFO storage.BlockManager: Removing RDD 135
17/08/01 14:50:39 INFO rdd.BlockRDD: Removing RDD 134 from persistence list
17/08/01 14:50:39 INFO storage.BlockManager: Removing RDD 134
17/08/01 14:50:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[134] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599039000 ms
17/08/01 14:50:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599037000 ms
17/08/01 14:50:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599037000 ms
17/08/01 14:50:40 INFO scheduler.JobScheduler: Added jobs for time 1501599040000 ms
17/08/01 14:50:40 INFO scheduler.JobScheduler: Starting job streaming job 1501599040000 ms.0 from job set of time 1501599040000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:40 INFO scheduler.JobScheduler: Finished job streaming job 1501599040000 ms.0 from job set of time 1501599040000 ms
17/08/01 14:50:40 INFO python.PythonRDD: Removing RDD 137 from persistence list
17/08/01 14:50:40 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599040000 ms (execution: 0.006 s)
17/08/01 14:50:40 INFO storage.BlockManager: Removing RDD 137
17/08/01 14:50:40 INFO rdd.BlockRDD: Removing RDD 136 from persistence list
17/08/01 14:50:40 INFO storage.BlockManager: Removing RDD 136
17/08/01 14:50:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[136] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599040000 ms
17/08/01 14:50:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599038000 ms
17/08/01 14:50:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599038000 ms
17/08/01 14:50:41 INFO scheduler.JobScheduler: Added jobs for time 1501599041000 ms
17/08/01 14:50:41 INFO scheduler.JobScheduler: Starting job streaming job 1501599041000 ms.0 from job set of time 1501599041000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:41 INFO scheduler.JobScheduler: Finished job streaming job 1501599041000 ms.0 from job set of time 1501599041000 ms
17/08/01 14:50:41 INFO python.PythonRDD: Removing RDD 139 from persistence list
17/08/01 14:50:41 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599041000 ms (execution: 0.006 s)
17/08/01 14:50:41 INFO storage.BlockManager: Removing RDD 139
17/08/01 14:50:41 INFO rdd.BlockRDD: Removing RDD 138 from persistence list
17/08/01 14:50:41 INFO storage.BlockManager: Removing RDD 138
17/08/01 14:50:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[138] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599041000 ms
17/08/01 14:50:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599039000 ms
17/08/01 14:50:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599039000 ms
17/08/01 14:50:42 INFO scheduler.JobScheduler: Added jobs for time 1501599042000 ms
17/08/01 14:50:42 INFO scheduler.JobScheduler: Starting job streaming job 1501599042000 ms.0 from job set of time 1501599042000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:42 INFO scheduler.JobScheduler: Finished job streaming job 1501599042000 ms.0 from job set of time 1501599042000 ms
17/08/01 14:50:42 INFO python.PythonRDD: Removing RDD 141 from persistence list
17/08/01 14:50:42 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599042000 ms (execution: 0.014 s)
17/08/01 14:50:42 INFO storage.BlockManager: Removing RDD 141
17/08/01 14:50:42 INFO rdd.BlockRDD: Removing RDD 140 from persistence list
17/08/01 14:50:42 INFO storage.BlockManager: Removing RDD 140
17/08/01 14:50:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[140] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599042000 ms
17/08/01 14:50:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599040000 ms
17/08/01 14:50:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599040000 ms
17/08/01 14:50:42 INFO storage.BlockManagerInfo: Added input-0-1501599042400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1059.6 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599042800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1057.9 MB)
17/08/01 14:50:43 INFO scheduler.JobScheduler: Added jobs for time 1501599043000 ms
17/08/01 14:50:43 INFO scheduler.JobScheduler: Starting job streaming job 1501599043000 ms.0 from job set of time 1501599043000 ms
17/08/01 14:50:43 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Got job 2 (runJob at PythonRDD.scala:441) with 1 output partitions
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Final stage: ResultStage 3 (runJob at PythonRDD.scala:441)
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Missing parents: List()
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Submitting ResultStage 3 (PythonRDD[146] at RDD at PythonRDD.scala:48), which has no missing parents
17/08/01 14:50:43 INFO memory.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/08/01 14:50:43 INFO memory.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.0.0.199:53355 (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:43 INFO spark.SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:996
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (PythonRDD[146] at RDD at PythonRDD.scala:48)
17/08/01 14:50:43 INFO cluster.YarnScheduler: Adding task set 3.0 with 1 tasks
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.3 KB, free: 1057.0 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1056.2 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.4 KB, free: 1055.3 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 878.9 KB, free: 1054.5 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599043800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1052.7 MB)
17/08/01 14:50:44 INFO scheduler.JobScheduler: Added jobs for time 1501599044000 ms
17/08/01 14:50:44 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 2)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1051.9 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1051.0 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1050.2 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.5 KB, free: 1049.3 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599044800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 887.9 KB, free: 1048.4 MB)
17/08/01 14:50:45 INFO scheduler.JobScheduler: Added jobs for time 1501599045000 ms
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1047.6 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1046.7 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1045.9 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.0 KB, free: 1044.1 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599045800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1043.3 MB)
17/08/01 14:50:46 INFO scheduler.JobScheduler: Added jobs for time 1501599046000 ms
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.6 KB, free: 1042.4 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.0 KB, free: 1041.6 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1040.7 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1039.9 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599046800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1039.0 MB)
17/08/01 14:50:47 INFO scheduler.JobScheduler: Added jobs for time 1501599047000 ms
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1038.1 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 878.9 KB, free: 1037.3 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1035.6 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1034.7 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599047800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1033.8 MB)
17/08/01 14:50:48 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.127:44785) with ID 3
17/08/01 14:50:48 INFO spark.ExecutorAllocationManager: New executor 3 has registered (new total is 2)
17/08/01 14:50:48 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 3.0 (TID 71, ip-10-0-0-127.ec2.internal, executor 3, partition 0, RACK_LOCAL, 5800 bytes)
17/08/01 14:50:48 INFO scheduler.JobScheduler: Added jobs for time 1501599048000 ms
17/08/01 14:50:48 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-127.ec2.internal:51258 with 1060.5 MB RAM, BlockManagerId(3, ip-10-0-0-127.ec2.internal, 51258, None)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1033.0 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on ip-10-0-0-127.ec2.internal:51258 (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1032.1 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.5 KB, free: 1031.3 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1030.4 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599048800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1029.5 MB)
17/08/01 14:50:49 INFO scheduler.JobScheduler: Added jobs for time 1501599049000 ms
17/08/01 14:50:49 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 3.0 (TID 71) in 1023 ms on ip-10-0-0-127.ec2.internal (executor 3) (1/1)
17/08/01 14:50:49 INFO cluster.YarnScheduler: Removed TaskSet 3.0, whose tasks have all completed, from pool
17/08/01 14:50:49 INFO scheduler.DAGScheduler: ResultStage 3 (runJob at PythonRDD.scala:441) finished in 5.985 s
17/08/01 14:50:49 INFO scheduler.DAGScheduler: Job 2 finished: runJob at PythonRDD.scala:441, took 6.003911 s
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.0 KB, free: 1027.8 MB)
17/08/01 14:50:49 INFO spark.ContextCleaner: Cleaned shuffle 0
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 10.0.0.199:53355 in memory (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1988.0 B, free: 1027.8 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on 10.0.0.199:53355 in memory (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1956.0 B, free: 1027.8 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on 10.0.0.199:53355 in memory (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on ip-10-0-0-127.ec2.internal:51258 in memory (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1027.0 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1026.1 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1025.3 MB)
17/08/01 14:50:49 INFO execution.SparkSqlParser: Parsing command: spotdb.proxy
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599049800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1024.4 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Added jobs for time 1501599050000 ms
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1023.5 MB)
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1022.7 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599043000 ms.0 from job set of time 1501599043000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 143 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 7.504 s for time 1501599043000 ms (execution: 7.487 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599044000 ms.0 from job set of time 1501599044000 ms
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 142 from persistence list
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599043000 ms.0
org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 69, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[142] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599043000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599041000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599041000 ms
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 142
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 143
Traceback (most recent call last):
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 148, in <module>
main()
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 77, in main
bluecoat_parse(args.zk,args.topic,args.db,args.db_table,args.num_of_workers,args.batch_size)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 144, in bluecoat_parse
ssc.awaitTermination()
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/context.py", line 206, in awaitTermination
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o50.awaitTermination.
: org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 69, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599044000 ms.0 from job set of time 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 6.542 s for time 1501599044000 ms (execution: 0.037 s)
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 145 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599045000 ms.0 from job set of time 1501599045000 ms
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599044000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 145
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 144 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 144
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[144] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599042000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599042000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599042400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1023.5 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599045000 ms.0 from job set of time 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 5.547 s for time 1501599045000 ms (execution: 0.003 s)
17/08/01 14:50:50 INFO streaming.StreamingContext: Invoking stop(stopGracefully=false) from shutdown hook
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599046000 ms.0 from job set of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599046000 ms.0 from job set of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 4.559 s for time 1501599046000 ms (execution: 0.004 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599047000 ms.0 from job set of time 1501599047000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 148 from persistence list
17/08/01 14:50:50 INFO scheduler.ReceiverTracker: Sent stop signal to all 1 receivers
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 148
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599047000 ms.0 from job set of time 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 3.562 s for time 1501599047000 ms (execution: 0.002 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599048000 ms.0 from job set of time 1501599048000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599048000 ms.0 from job set of time 1501599048000 ms
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 147 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 2.571 s for time 1501599048000 ms (execution: 0.001 s)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 147
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[147] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599049000 ms.0 from job set of time 1501599049000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599049000 ms.0 from job set of time 1501599049000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 1.572 s for time 1501599049000 ms (execution: 0.001 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599050000 ms.0 from job set of time 1501599050000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599050000 ms.0 from job set of time 1501599050000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 0.574 s for time 1501599050000 ms (execution: 0.001 s)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599045000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599046000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599047000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599048000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599049000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599042800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1025.3 MB)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599050000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.3 KB, free: 1026.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1027.0 MB)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599043000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599043000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 150 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 878.9 KB, free: 1027.8 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.4 KB, free: 1028.7 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 150
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 149 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 149
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[149] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599044000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 152 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 152
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1030.4 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1031.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1032.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1033.0 MB)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 151 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 151
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[151] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599045000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 154 from persistence list
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 153 from persistence list
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[153] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599048000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.5 KB, free: 1033.8 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 153
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 154
17/08/01 14:50:50 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1034.7 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599046000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 156 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 887.9 KB, free: 1035.6 MB)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 155 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.4 KB, free: 1034.7 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 155
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 156
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[155] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599049000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1035.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1036.4 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599047000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 158 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 158
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 157 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 157
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[157] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599050000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.6 KB, free: 1037.3 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599048000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599048000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1038.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.0 KB, free: 1039.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.0 KB, free: 1040.7 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1041.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1042.4 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1043.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 878.9 KB, free: 1044.2 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1045.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1046.7 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1047.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1048.5 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.5 KB, free: 1049.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1050.2 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1051.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1051.9 MB)
17/08/01 14:50:51 ERROR scheduler.JobScheduler: Error generating jobs for time 1501599051000 ms
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 2.0 (TID 70) in 77942 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/1)
17/08/01 14:50:51 INFO cluster.YarnScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool
17/08/01 14:50:51 INFO scheduler.DAGScheduler: ResultStage 2 (start at NativeMethodAccessorImpl.java:0) finished in 77.943 s
17/08/01 14:50:51 INFO scheduler.ReceiverTracker: All of the receivers have deregistered successfully
17/08/01 14:50:51 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
17/08/01 14:50:51 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
17/08/01 14:50:51 INFO util.RecurringTimer: Stopped timer for JobGenerator after time 1501599051000
17/08/01 14:50:51 INFO scheduler.JobGenerator: Stopped JobGenerator
17/08/01 14:50:51 INFO scheduler.JobScheduler: Stopped JobScheduler
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@625d3c2d{/streaming,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@39edaf54{/streaming/batch,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@c773c0d{/static/streaming,null,UNAVAILABLE}
17/08/01 14:50:51 INFO streaming.StreamingContext: StreamingContext stopped successfully
17/08/01 14:50:51 INFO spark.SparkContext: Invoking stop() from shutdown hook
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 INFO server.ServerConnector: Stopped ServerConnector@37681deb{HTTP/1.1}{0.0.0.0:4040}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2aa3e9a6{/stages/stage/kill,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ee30613{/jobs/job/kill,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@15ec1723{/api,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14a7d27d{/,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@e4408f4{/static,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@8a515f3{/executors/threadDump/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@f241039{/executors/threadDump,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@79b9b526{/executors/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5fa63fa{/executors,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@77729e5a{/environment/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@242eb7e{/environment,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@49ddd2ed{/storage/rdd/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@699743b3{/storage/rdd,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@736cc380{/storage/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19cf381c{/storage,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4f734944{/stages/pool/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@b7d4da0{/stages/pool,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ab71bb9{/stages/stage/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5d37ce06{/stages/stage,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@27f55b38{/stages/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1159f15e{/stages,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ff970c2{/jobs/job/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@615b86f5{/jobs/job,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19bb7285{/jobs/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3b9882ec{/jobs,null,UNAVAILABLE}
17/08/01 14:50:51 INFO ui.SparkUI: Stopped Spark web UI at http://10.0.0.199:4040
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors
17/08/01 14:50:51 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
17/08/01 14:50:51 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
services=List(),
started=false)
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Stopped
17/08/01 14:50:51 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/08/01 14:50:51 INFO memory.MemoryStore: MemoryStore cleared
17/08/01 14:50:51 INFO storage.BlockManager: BlockManager stopped
17/08/01 14:50:51 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
17/08/01 14:50:51 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/08/01 14:50:51 INFO spark.SparkContext: Successfully stopped SparkContext
17/08/01 14:50:51 INFO util.ShutdownHookManager: Shutdown hook called
17/08/01 14:50:51 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25
17/08/01 14:50:51 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25/pyspark-c2380580-feb5-47ef-bc69-7b2c9ba38770
Closing worker...
From: Deon Griessel [mailto:dgriessel@searchtechnologies.com]
Sent: Monday, July 31, 2017 4:53 PM
To: user@spot.apache.org
Subject: Spot Ingest Proxy Error
I am trying to run the spark-ingest for the sample proxy log: Bluecoat_ProxySG_Sample.log
I run bash ./start_ingest_standalone.sh proxy 1
Then add the demo log to my collector_path. The worker dies, seems that it cannot find the hive metastore, probably a spark/hive configuration issue. Also, notice that the ip=unknown-ip-addr.
Would appreciate any help on this.
This is what I’m seeing in the worker screen output:
2017-07-31 18:54:15,855 - SPOT.INGEST.WORKER - INFO - Initializing kafka instance
2017-07-31 18:54:15,855 - SPOT.INGEST.WRK.PROXY - INFO - Creating Spark Job for topic: SPOT-INGEST-proxy-18_54_13
2017-07-31 18:54:15,856 - SPOT.INGEST.WRK.PROXY - INFO - SPOT.Utils: Executing: spark2-submit --master yarn --driver-memory 2325m --num-executors 1 --conf spark.executor.memory=2325m --conf spark.executor.cores=1 --jars /home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar /home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py -zk ip-10-0-0-199.ec2.internal:2181 -t SPOT-INGEST-proxy-18_54_13 -db spotdb -dt proxy -w 1 -bs 1
17/07/31 18:54:17 INFO spark.SparkContext: Running Spark version 2.1.0.cloudera1
17/07/31 18:54:17 WARN spark.SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0
17/07/31 18:54:17 INFO spark.SecurityManager: Changing view acls to: spot-user
17/07/31 18:54:17 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/07/31 18:54:17 INFO spark.SecurityManager: Changing view acls groups to:
17/07/31 18:54:17 INFO spark.SecurityManager: Changing modify acls groups to:
17/07/31 18:54:17 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/07/31 18:54:18 INFO util.Utils: Successfully started service 'sparkDriver' on port 54401.
17/07/31 18:54:18 INFO spark.SparkEnv: Registering MapOutputTracker
17/07/31 18:54:18 INFO spark.SparkEnv: Registering BlockManagerMaster
17/07/31 18:54:18 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
17/07/31 18:54:18 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
17/07/31 18:54:18 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-1db8ad37-9fcc-4799-b9e0-a206b6041d04
17/07/31 18:54:18 INFO memory.MemoryStore: MemoryStore started with capacity 1060.5 MB
17/07/31 18:54:18 INFO spark.SparkEnv: Registering OutputCommitCoordinator
17/07/31 18:54:18 INFO util.log: Logging initialized @2379ms
17/07/31 18:54:18 INFO server.Server: jetty-9.2.z-SNAPSHOT
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1a3b1e79{/jobs,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1a3b1e79%7b/jobs,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1f4da763{/jobs/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1f4da763%7b/jobs/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@232864a3{/jobs/job,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@232864a3%7b/jobs/job,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@30e71b5d{/jobs/job/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@30e71b5d%7b/jobs/job/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14b58fc0{/stages,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14b58fc0%7b/stages,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1bf090df{/stages/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1bf090df%7b/stages/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4eb72ecd{/stages/stage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4eb72ecd%7b/stages/stage,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5c61bd1a{/stages/stage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5c61bd1a%7b/stages/stage/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14c62558{/stages/pool,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14c62558%7b/stages/pool,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5cbdbf0f{/stages/pool/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5cbdbf0f%7b/stages/pool/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2d4aa15a{/storage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@2d4aa15a%7b/storage,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ea732f0{/storage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ea732f0%7b/storage/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@13016b86{/storage/rdd,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@13016b86%7b/storage/rdd,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@423a3429{/storage/rdd/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@423a3429%7b/storage/rdd/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7b3691b6{/environment,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7b3691b6%7b/environment,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3b9882ec{/environment/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3b9882ec%7b/environment/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19bb7285{/executors,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19bb7285%7b/executors,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@615b86f5{/executors/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@615b86f5%7b/executors/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ff970c2{/executors/threadDump,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3ff970c2%7b/executors/threadDump,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1159f15e{/executors/threadDump/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1159f15e%7b/executors/threadDump/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27f55b38{/static,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27f55b38%7b/static,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5d37ce06{/,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5d37ce06%7b/,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ab71bb9{/api,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ab71bb9%7b/api,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b7d4da0{/jobs/job/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@b7d4da0%7b/jobs/job/kill,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f734944{/stages/stage/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4f734944%7b/stages/stage/kill,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO server.ServerConnector: Started ServerConnector@188e6c7f{HTTP/1.1}{0.0.0.0:4040}<mailto:ServerConnector@188e6c7f%7bHTTP/1.1%7d%7b0.0.0.0:4040%7d>
17/07/31 18:54:18 INFO server.Server: Started @2532ms
17/07/31 18:54:18 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
17/07/31 18:54:18 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.0.0.199:4040
17/07/31 18:54:18 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/07/31 18:54:19 INFO client.RMProxy: Connecting to ResourceManager at ip-10-0-0-199.ec2.internal/10.0.0.199:8032
17/07/31 18:54:19 INFO yarn.Client: Requesting a new application from cluster with 3 NodeManagers
17/07/31 18:54:19 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (2725 MB per container)
17/07/31 18:54:19 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/07/31 18:54:19 INFO yarn.Client: Setting up container launch context for our AM
17/07/31 18:54:19 INFO yarn.Client: Setting up the launch environment for our AM container
17/07/31 18:54:19 INFO yarn.Client: Preparing resources for our AM container
17/07/31 18:54:21 INFO yarn.Client: Uploading resource file:/home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501523941584_0005/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar
17/07/31 18:54:21 INFO yarn.Client: Uploading resource file:/tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40/__spark_conf__5838957838871043110.zip -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501523941584_0005/__spark_conf__.zip
17/07/31 18:54:21 INFO spark.SecurityManager: Changing view acls to: spot-user
17/07/31 18:54:21 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/07/31 18:54:21 INFO spark.SecurityManager: Changing view acls groups to:
17/07/31 18:54:21 INFO spark.SecurityManager: Changing modify acls groups to:
17/07/31 18:54:21 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/07/31 18:54:21 INFO yarn.Client: Submitting application application_1501523941584_0005 to ResourceManager
17/07/31 18:54:21 INFO impl.YarnClientImpl: Submitted application application_1501523941584_0005
17/07/31 18:54:21 INFO cluster.SchedulerExtensionServices: Starting Yarn extension services with app application_1501523941584_0005 and attemptId None
17/07/31 18:54:22 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:22 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.users.spot-user
start time: 1501527261635
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005/
user: spot-user
17/07/31 18:54:23 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:24 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:25 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/07/31 18:54:25 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> ip-10-0-0-199.ec2.internal, PROXY_URI_BASES -> http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005), /proxy/application_1501523941584_0005
17/07/31 18:54:25 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/07/31 18:54:25 INFO yarn.Client: Application report for application_1501523941584_0005 (state: RUNNING)
17/07/31 18:54:25 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 10.0.0.127
ApplicationMaster RPC port: 0
queue: root.users.spot-user
start time: 1501527261635
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005/
user: spot-user
17/07/31 18:54:25 INFO cluster.YarnClientSchedulerBackend: Application application_1501523941584_0005 has started running.
17/07/31 18:54:25 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 59442.
17/07/31 18:54:25 INFO netty.NettyBlockTransferService: Server created on 10.0.0.199:59442
17/07/31 18:54:25 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/07/31 18:54:25 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManagerMasterEndpoint: Registering block manager 10.0.0.199:59442 with 1060.5 MB RAM, BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManager: external shuffle service port = 7337
17/07/31 18:54:25 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3e93262b{/metrics/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3e93262b%7b/metrics/json,null,AVAILABLE%7d>
17/07/31 18:54:25 INFO scheduler.EventLoggingListener: Logging events to hdfs://ip-10-0-0-199.ec2.internal:8020/user/spark/spark2ApplicationHistory/application_1501523941584_0005
17/07/31 18:54:25 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/07/31 18:54:28 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.229:51685) with ID 1
17/07/31 18:54:28 INFO spark.ExecutorAllocationManager: New executor 1 has registered (new total is 1)
17/07/31 18:54:28 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-229.ec2.internal:35659 with 1060.5 MB RAM, BlockManagerId(1, ip-10-0-0-229.ec2.internal, 35659, None)
17/07/31 18:54:28 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/07/31 18:54:29 INFO internal.SharedState: Warehouse path is 'file:/home/spot-user/spot-ingest/spark-warehouse'.
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7a9c6710{/SQL,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7a9c6710%7b/SQL,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@141768e4{/SQL/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@141768e4%7b/SQL/json,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1e367d45{/SQL/execution,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1e367d45%7b/SQL/execution,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5554f0a0{/SQL/execution/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5554f0a0%7b/SQL/execution/json,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@144776d3{/static/sql,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@144776d3%7b/static/sql,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO hive.HiveUtils: Initializing HiveMetastoreConnection version 1.1.0 using Spark classes.
17/07/31 18:54:30 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
17/07/31 18:54:30 INFO metastore.ObjectStore: ObjectStore, initialize called
17/07/31 18:54:30 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
17/07/31 18:54:30 INFO DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
17/07/31 18:54:30 WARN bonecp.BoneCPConfig: Max Connections < 1. Setting to 20
17/07/31 18:54:31 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
17/07/31 18:54:32 WARN bonecp.BoneCPConfig: Max Connections < 1. Setting to 20
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0<ma...@0>" since the connection used is closing
17/07/31 18:54:32 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
17/07/31 18:54:32 INFO metastore.ObjectStore: Initialized ObjectStore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: Added admin role in metastore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: Added public role in metastore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_all_functions from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_all_functions
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_all_functions
17/07/31 18:54:33 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_all_functions start=1501527273291 end=1501527273356 duration=65 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=0 error=false>
17/07/31 18:54:33 INFO session.SessionState: Created local directory: /tmp/aaecce13-256b-4b91-b661-9046c9c418c6_resources
17/07/31 18:54:33 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6
17/07/31 18:54:33 INFO session.SessionState: Created local directory: /tmp/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6
17/07/31 18:54:33 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6/_tmp_space.db
17/07/31 18:54:33 INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
17/07/31 18:54:33 INFO client.HiveClientImpl: Warehouse location for Hive client (version 1.1.0) is file:/home/spot-user/spot-ingest/spark-warehouse
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_database: default
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: default
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527273406 end=1501527273408 duration=2 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=0 error=false>
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_database: global_temp
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: global_temp
17/07/31 18:54:33 WARN metastore.ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527273409 end=1501527273412 duration=3 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=-1 error=true>
17/07/31 18:54:33 WARN streaming.StreamingContext: Dynamic Allocation is enabled for this application. Enabling Dynamic allocation for Spark Streaming applications can cause data loss if Write Ahead Log is not enabled for non-replayable sources like Flume. See the programming guide for details on how to enable the Write Ahead Log.
17/07/31 18:54:33 INFO spark.SparkContext: Starting job: start at NativeMethodAccessorImpl.java:0
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Registering RDD 1 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Got job 0 (start at NativeMethodAccessorImpl.java:0) with 20 output partitions
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/07/31 18:54:33 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 3.1 KB, free 1060.5 MB)
17/07/31 18:54:33 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 1988.0 B, free 1060.5 MB)
17/07/31 18:54:33 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.0.0.199:59442 (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:54:33 INFO spark.SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Submitting 50 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO cluster.YarnScheduler: Adding task set 0.0 with 50 tasks
17/07/31 18:54:33 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, ip-10-0-0-229.ec2.internal, executor 1, partition 1, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 669 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, ip-10-0-0-229.ec2.internal, executor 1, partition 2, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 77 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3, ip-10-0-0-229.ec2.internal, executor 1, partition 3, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 72 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 0.0 (TID 4, ip-10-0-0-229.ec2.internal, executor 1, partition 4, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 69 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 0.0 (TID 5, ip-10-0-0-229.ec2.internal, executor 1, partition 5, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 59 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 0.0 (TID 6, ip-10-0-0-229.ec2.internal, executor 1, partition 6, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 0.0 (TID 7, ip-10-0-0-229.ec2.internal, executor 1, partition 7, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 0.0 (TID 8, ip-10-0-0-229.ec2.internal, executor 1, partition 8, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 0.0 (TID 7) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/50)
17/07/31 18:54:34 INFO spark.ExecutorAllocationManager: Requesting 1 new executor because tasks are backlogged (new desired total will be 2)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 0.0 (TID 9, ip-10-0-0-229.ec2.internal, executor 1, partition 9, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 0.0 (TID 8) in 58 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 0.0 (TID 10, ip-10-0-0-229.ec2.internal, executor 1, partition 10, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 0.0 (TID 9) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 0.0 (TID 11, ip-10-0-0-229.ec2.internal, executor 1, partition 11, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 0.0 (TID 10) in 52 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 0.0 (TID 12, ip-10-0-0-229.ec2.internal, executor 1, partition 12, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 0.0 (TID 11) in 58 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 0.0 (TID 13, ip-10-0-0-229.ec2.internal, executor 1, partition 13, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 0.0 (TID 12) in 51 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 0.0 (TID 14, ip-10-0-0-229.ec2.internal, executor 1, partition 14, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 0.0 (TID 13) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 0.0 (TID 15, ip-10-0-0-229.ec2.internal, executor 1, partition 15, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 0.0 (TID 14) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 0.0 (TID 16, ip-10-0-0-229.ec2.internal, executor 1, partition 16, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 0.0 (TID 15) in 63 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 0.0 (TID 17, ip-10-0-0-229.ec2.internal, executor 1, partition 17, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 0.0 (TID 16) in 50 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 0.0 (TID 18, ip-10-0-0-229.ec2.internal, executor 1, partition 18, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 0.0 (TID 17) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 0.0 (TID 19, ip-10-0-0-229.ec2.internal, executor 1, partition 19, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 0.0 (TID 18) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 0.0 (TID 20, ip-10-0-0-229.ec2.internal, executor 1, partition 20, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 0.0 (TID 19) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 0.0 (TID 21, ip-10-0-0-229.ec2.internal, executor 1, partition 21, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 0.0 (TID 20) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (21/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 0.0 (TID 22, ip-10-0-0-229.ec2.internal, executor 1, partition 22, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 0.0 (TID 21) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (22/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 0.0 (TID 23, ip-10-0-0-229.ec2.internal, executor 1, partition 23, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 0.0 (TID 22) in 47 ms on ip-10-0-0-229.ec2.internal (executor 1) (23/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 0.0 (TID 24, ip-10-0-0-229.ec2.internal, executor 1, partition 24, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 0.0 (TID 23) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (24/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 0.0 (TID 25, ip-10-0-0-229.ec2.internal, executor 1, partition 25, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 0.0 (TID 24) in 46 ms on ip-10-0-0-229.ec2.internal (executor 1) (25/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 0.0 (TID 26, ip-10-0-0-229.ec2.internal, executor 1, partition 26, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 0.0 (TID 25) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (26/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 0.0 (TID 27, ip-10-0-0-229.ec2.internal, executor 1, partition 27, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 0.0 (TID 26) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (27/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 0.0 (TID 28, ip-10-0-0-229.ec2.internal, executor 1, partition 28, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 0.0 (TID 27) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (28/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 0.0 (TID 29, ip-10-0-0-229.ec2.internal, executor 1, partition 29, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 0.0 (TID 28) in 39 ms on ip-10-0-0-229.ec2.internal (executor 1) (29/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 0.0 (TID 30, ip-10-0-0-229.ec2.internal, executor 1, partition 30, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 0.0 (TID 29) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (30/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 0.0 (TID 31, ip-10-0-0-229.ec2.internal, executor 1, partition 31, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 0.0 (TID 30) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (31/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 0.0 (TID 32, ip-10-0-0-229.ec2.internal, executor 1, partition 32, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 0.0 (TID 31) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (32/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 0.0 (TID 33, ip-10-0-0-229.ec2.internal, executor 1, partition 33, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 0.0 (TID 32) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (33/50)
17/07/31 18:54:35 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 4)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 0.0 (TID 34, ip-10-0-0-229.ec2.internal, executor 1, partition 34, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 0.0 (TID 33) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (34/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 0.0 (TID 35, ip-10-0-0-229.ec2.internal, executor 1, partition 35, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 0.0 (TID 34) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (35/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 0.0 (TID 36, ip-10-0-0-229.ec2.internal, executor 1, partition 36, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 0.0 (TID 35) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (36/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 0.0 (TID 37, ip-10-0-0-229.ec2.internal, executor 1, partition 37, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 0.0 (TID 36) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (37/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 0.0 (TID 38, ip-10-0-0-229.ec2.internal, executor 1, partition 38, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 0.0 (TID 37) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (38/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 0.0 (TID 39, ip-10-0-0-229.ec2.internal, executor 1, partition 39, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 0.0 (TID 38) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (39/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 0.0 (TID 40, ip-10-0-0-229.ec2.internal, executor 1, partition 40, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 0.0 (TID 39) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (40/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 0.0 (TID 41, ip-10-0-0-229.ec2.internal, executor 1, partition 41, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 0.0 (TID 40) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (41/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 0.0 (TID 42, ip-10-0-0-229.ec2.internal, executor 1, partition 42, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 0.0 (TID 41) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (42/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 0.0 (TID 43, ip-10-0-0-229.ec2.internal, executor 1, partition 43, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 0.0 (TID 42) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (43/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 0.0 (TID 44, ip-10-0-0-229.ec2.internal, executor 1, partition 44, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 0.0 (TID 43) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (44/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 0.0 (TID 45, ip-10-0-0-229.ec2.internal, executor 1, partition 45, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 0.0 (TID 44) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (45/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 0.0 (TID 46, ip-10-0-0-229.ec2.internal, executor 1, partition 46, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 0.0 (TID 45) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (46/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 0.0 (TID 47, ip-10-0-0-229.ec2.internal, executor 1, partition 47, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 0.0 (TID 46) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (47/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 0.0 (TID 48, ip-10-0-0-229.ec2.internal, executor 1, partition 48, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 0.0 (TID 47) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (48/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 0.0 (TID 49, ip-10-0-0-229.ec2.internal, executor 1, partition 49, PROCESS_LOCAL, 6676 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 0.0 (TID 48) in 29 ms on ip-10-0-0-229.ec2.internal (executor 1) (49/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 0.0 (TID 49) in 27 ms on ip-10-0-0-229.ec2.internal (executor 1) (50/50)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (start at NativeMethodAccessorImpl.java:0) finished in 2.499 s
17/07/31 18:54:36 INFO cluster.YarnScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/07/31 18:54:36 INFO scheduler.DAGScheduler: looking for newly runnable stages
17/07/31 18:54:36 INFO scheduler.DAGScheduler: running: Set()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 1)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: failed: Set()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.2 KB, free 1060.5 MB)
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 1956.0 B, free 1060.5 MB)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.0.0.199:59442 (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting 20 missing tasks from ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Adding task set 1.0 with 20 tasks
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 50, ip-10-0-0-229.ec2.internal, executor 1, partition 0, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.0.0.229:51685
17/07/31 18:54:36 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 283 bytes
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.0 (TID 51, ip-10-0-0-229.ec2.internal, executor 1, partition 1, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 50) in 91 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 1.0 (TID 52, ip-10-0-0-229.ec2.internal, executor 1, partition 2, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 1.0 (TID 51) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 1.0 (TID 53, ip-10-0-0-229.ec2.internal, executor 1, partition 3, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 1.0 (TID 52) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 1.0 (TID 54, ip-10-0-0-229.ec2.internal, executor 1, partition 4, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 1.0 (TID 53) in 24 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 1.0 (TID 55, ip-10-0-0-229.ec2.internal, executor 1, partition 5, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 1.0 (TID 54) in 24 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 1.0 (TID 56, ip-10-0-0-229.ec2.internal, executor 1, partition 6, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 1.0 (TID 55) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 1.0 (TID 57, ip-10-0-0-229.ec2.internal, executor 1, partition 7, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 1.0 (TID 56) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 1.0 (TID 58, ip-10-0-0-229.ec2.internal, executor 1, partition 8, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 1.0 (TID 57) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 1.0 (TID 59, ip-10-0-0-229.ec2.internal, executor 1, partition 9, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 1.0 (TID 58) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 1.0 (TID 60, ip-10-0-0-229.ec2.internal, executor 1, partition 10, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 1.0 (TID 59) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 1.0 (TID 61, ip-10-0-0-229.ec2.internal, executor 1, partition 11, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 1.0 (TID 60) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 1.0 (TID 62, ip-10-0-0-229.ec2.internal, executor 1, partition 12, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 1.0 (TID 61) in 16 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 1.0 (TID 63, ip-10-0-0-229.ec2.internal, executor 1, partition 13, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 1.0 (TID 62) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 1.0 (TID 64, ip-10-0-0-229.ec2.internal, executor 1, partition 14, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 1.0 (TID 63) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 1.0 (TID 65, ip-10-0-0-229.ec2.internal, executor 1, partition 15, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 1.0 (TID 64) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 1.0 (TID 66, ip-10-0-0-229.ec2.internal, executor 1, partition 16, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 1.0 (TID 65) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 1.0 (TID 67, ip-10-0-0-229.ec2.internal, executor 1, partition 17, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 1.0 (TID 66) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 1.0 (TID 68, ip-10-0-0-229.ec2.internal, executor 1, partition 18, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 1.0 (TID 67) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 1.0 (TID 69, ip-10-0-0-229.ec2.internal, executor 1, partition 19, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 1.0 (TID 68) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 1.0 (TID 69) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/20)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool
17/07/31 18:54:36 INFO scheduler.DAGScheduler: ResultStage 1 (start at NativeMethodAccessorImpl.java:0) finished in 0.416 s
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Job 0 finished: start at NativeMethodAccessorImpl.java:0, took 3.184418 s
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: Starting 1 receivers
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: ReceiverTracker started
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Initialized and validated org.apache.spark.streaming.kafka.KafkaInputDStream@7b67308f<ma...@7b67308f>
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Initialized and validated org.apache.spark.streaming.api.python.PythonTransformedDStream@33a53b94<ma...@33a53b94>
17/07/31 18:54:36 INFO dstream.ForEachDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO dstream.ForEachDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO dstream.ForEachDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO dstream.ForEachDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO dstream.ForEachDStream: Initialized and validated org.apache.spark.streaming.dstream.ForEachDStream@24ea8c09<ma...@24ea8c09>
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Got job 1 (start at NativeMethodAccessorImpl.java:0) with 1 output partitions
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Final stage: ResultStage 2 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620), which has no missing parents
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: Receiver 0 started
17/07/31 18:54:36 INFO util.RecurringTimer: Started timer for JobGenerator at time 1501527277000
17/07/31 18:54:36 INFO scheduler.JobGenerator: Started JobGenerator at 1501527277000 ms
17/07/31 18:54:36 INFO scheduler.JobScheduler: Started JobScheduler
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19fd4723{/streaming,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19fd4723%7b/streaming,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@43b0e42{/streaming/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@43b0e42%7b/streaming/json,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@22116fc3{/streaming/batch,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@22116fc3%7b/streaming/batch,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27399762{/streaming/batch/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27399762%7b/streaming/batch/json,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@70d301d6{/static/streaming,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@70d301d6%7b/static/streaming,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO streaming.StreamingContext: StreamingContext started
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 74.4 KB, free 1060.4 MB)
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 27.3 KB, free 1060.4 MB)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.0.0.199:59442 (size: 27.3 KB, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Adding task set 2.0 with 1 tasks
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 2.0 (TID 70, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 7067 bytes)
17/07/31 18:54:37 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 27.3 KB, free: 1060.5 MB)
17/07/31 18:54:37 INFO scheduler.JobScheduler: Added jobs for time 1501527277000 ms
17/07/31 18:54:37 INFO scheduler.JobScheduler: Starting job streaming job 1501527277000 ms.0 from job set of time 1501527277000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:37 INFO scheduler.JobScheduler: Finished job streaming job 1501527277000 ms.0 from job set of time 1501527277000 ms
17/07/31 18:54:37 INFO scheduler.JobScheduler: Total delay: 0.071 s for time 1501527277000 ms (execution: 0.014 s)
17/07/31 18:54:37 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/07/31 18:54:37 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/07/31 18:54:37 INFO scheduler.ReceiverTracker: Registered receiver for stream 0 from 10.0.0.229:51685
17/07/31 18:54:38 INFO scheduler.JobScheduler: Added jobs for time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.JobScheduler: Starting job streaming job 1501527278000 ms.0 from job set of time 1501527278000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:38 INFO scheduler.JobScheduler: Finished job streaming job 1501527278000 ms.0 from job set of time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.JobScheduler: Total delay: 0.045 s for time 1501527278000 ms (execution: 0.010 s)
17/07/31 18:54:38 INFO python.PythonRDD: Removing RDD 5 from persistence list
17/07/31 18:54:38 INFO rdd.BlockRDD: Removing RDD 4 from persistence list
17/07/31 18:54:38 INFO storage.BlockManager: Removing RDD 5
17/07/31 18:54:38 INFO storage.BlockManager: Removing RDD 4
17/07/31 18:54:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[4] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/07/31 18:54:38 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/07/31 18:54:38 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.151:57862) with ID 2
17/07/31 18:54:38 INFO spark.ExecutorAllocationManager: New executor 2 has registered (new total is 2)
17/07/31 18:54:38 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-151.ec2.internal:54365 with 1060.5 MB RAM, BlockManagerId(2, ip-10-0-0-151.ec2.internal, 54365, None)
17/07/31 18:54:39 INFO scheduler.JobScheduler: Added jobs for time 1501527279000 ms
17/07/31 18:54:39 INFO scheduler.JobScheduler: Starting job streaming job 1501527279000 ms.0 from job set of time 1501527279000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:39 INFO scheduler.JobScheduler: Finished job streaming job 1501527279000 ms.0 from job set of time 1501527279000 ms
17/07/31 18:54:39 INFO python.PythonRDD: Removing RDD 7 from persistence list
17/07/31 18:54:39 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527279000 ms (execution: 0.006 s)
17/07/31 18:54:39 INFO storage.BlockManager: Removing RDD 7
17/07/31 18:54:39 INFO rdd.BlockRDD: Removing RDD 6 from persistence list
17/07/31 18:54:39 INFO storage.BlockManager: Removing RDD 6
17/07/31 18:54:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[6] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527279000 ms
17/07/31 18:54:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527277000 ms
17/07/31 18:54:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527277000 ms
17/07/31 18:54:40 INFO scheduler.JobScheduler: Added jobs for time 1501527280000 ms
17/07/31 18:54:40 INFO scheduler.JobScheduler: Starting job streaming job 1501527280000 ms.0 from job set of time 1501527280000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:40 INFO scheduler.JobScheduler: Finished job streaming job 1501527280000 ms.0 from job set of time 1501527280000 ms
17/07/31 18:54:40 INFO python.PythonRDD: Removing RDD 9 from persistence list
17/07/31 18:54:40 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501527280000 ms (execution: 0.009 s)
17/07/31 18:54:40 INFO storage.BlockManager: Removing RDD 9
17/07/31 18:54:40 INFO rdd.BlockRDD: Removing RDD 8 from persistence list
17/07/31 18:54:40 INFO storage.BlockManager: Removing RDD 8
17/07/31 18:54:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[8] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527280000 ms
17/07/31 18:54:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527278000 ms
17/07/31 18:54:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527278000 ms
17/07/31 18:54:41 INFO scheduler.JobScheduler: Added jobs for time 1501527281000 ms
17/07/31 18:54:41 INFO scheduler.JobScheduler: Starting job streaming job 1501527281000 ms.0 from job set of time 1501527281000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:41 INFO scheduler.JobScheduler: Finished job streaming job 1501527281000 ms.0 from job set of time 1501527281000 ms
17/07/31 18:54:41 INFO python.PythonRDD: Removing RDD 11 from persistence list
17/07/31 18:54:41 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527281000 ms (execution: 0.006 s)
17/07/31 18:54:41 INFO storage.BlockManager: Removing RDD 11
17/07/31 18:54:41 INFO rdd.BlockRDD: Removing RDD 10 from persistence list
17/07/31 18:54:41 INFO storage.BlockManager: Removing RDD 10
17/07/31 18:54:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[10] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527281000 ms
17/07/31 18:54:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527279000 ms
17/07/31 18:54:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527279000 ms
17/07/31 18:54:42 INFO scheduler.JobScheduler: Added jobs for time 1501527282000 ms
17/07/31 18:54:42 INFO scheduler.JobScheduler: Starting job streaming job 1501527282000 ms.0 from job set of time 1501527282000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:42 INFO scheduler.JobScheduler: Finished job streaming job 1501527282000 ms.0 from job set of time 1501527282000 ms
17/07/31 18:54:42 INFO python.PythonRDD: Removing RDD 13 from persistence list
17/07/31 18:54:42 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527282000 ms (execution: 0.006 s)
17/07/31 18:54:42 INFO storage.BlockManager: Removing RDD 13
17/07/31 18:54:42 INFO rdd.BlockRDD: Removing RDD 12 from persistence list
17/07/31 18:54:42 INFO storage.BlockManager: Removing RDD 12
17/07/31 18:54:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[12] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527282000 ms
17/07/31 18:54:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527280000 ms
17/07/31 18:54:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527280000 ms
17/07/31 18:54:43 INFO scheduler.JobScheduler: Added jobs for time 1501527283000 ms
17/07/31 18:54:43 INFO scheduler.JobScheduler: Starting job streaming job 1501527283000 ms.0 from job set of time 1501527283000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:43 INFO scheduler.JobScheduler: Finished job streaming job 1501527283000 ms.0 from job set of time 1501527283000 ms
17/07/31 18:54:43 INFO python.PythonRDD: Removing RDD 15 from persistence list
17/07/31 18:54:43 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527283000 ms (execution: 0.008 s)
17/07/31 18:54:43 INFO storage.BlockManager: Removing RDD 15
17/07/31 18:54:43 INFO rdd.BlockRDD: Removing RDD 14 from persistence list
17/07/31 18:54:43 INFO storage.BlockManager: Removing RDD 14
17/07/31 18:54:43 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[14] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527283000 ms
17/07/31 18:54:43 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527281000 ms
17/07/31 18:54:43 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527281000 ms
17/07/31 18:54:44 INFO scheduler.JobScheduler: Added jobs for time 1501527284000 ms
17/07/31 18:54:44 INFO scheduler.JobScheduler: Starting job streaming job 1501527284000 ms.0 from job set of time 1501527284000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:44 INFO scheduler.JobScheduler: Finished job streaming job 1501527284000 ms.0 from job set of time 1501527284000 ms
17/07/31 18:54:44 INFO python.PythonRDD: Removing RDD 17 from persistence list
17/07/31 18:54:44 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527284000 ms (execution: 0.016 s)
17/07/31 18:54:44 INFO storage.BlockManager: Removing RDD 17
17/07/31 18:54:44 INFO rdd.BlockRDD: Removing RDD 16 from persistence list
17/07/31 18:54:44 INFO storage.BlockManager: Removing RDD 16
17/07/31 18:54:44 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[16] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527284000 ms
17/07/31 18:54:44 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527282000 ms
17/07/31 18:54:44 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527282000 ms
17/07/31 18:54:45 INFO scheduler.JobScheduler: Added jobs for time 1501527285000 ms
17/07/31 18:54:45 INFO scheduler.JobScheduler: Starting job streaming job 1501527285000 ms.0 from job set of time 1501527285000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:45 INFO scheduler.JobScheduler: Finished job streaming job 1501527285000 ms.0 from job set of time 1501527285000 ms
17/07/31 18:54:45 INFO python.PythonRDD: Removing RDD 19 from persistence list
17/07/31 18:54:45 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527285000 ms (execution: 0.007 s)
17/07/31 18:54:45 INFO rdd.BlockRDD: Removing RDD 18 from persistence list
17/07/31 18:54:45 INFO storage.BlockManager: Removing RDD 19
17/07/31 18:54:45 INFO storage.BlockManager: Removing RDD 18
17/07/31 18:54:45 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[18] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527285000 ms
17/07/31 18:54:45 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527283000 ms
17/07/31 18:54:45 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527283000 ms
17/07/31 18:54:46 INFO scheduler.JobScheduler: Added jobs for time 1501527286000 ms
17/07/31 18:54:46 INFO scheduler.JobScheduler: Starting job streaming job 1501527286000 ms.0 from job set of time 1501527286000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:46 INFO scheduler.JobScheduler: Finished job streaming job 1501527286000 ms.0 from job set of time 1501527286000 ms
17/07/31 18:54:46 INFO python.PythonRDD: Removing RDD 21 from persistence list
17/07/31 18:54:46 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527286000 ms (execution: 0.008 s)
17/07/31 18:54:46 INFO storage.BlockManager: Removing RDD 21
17/07/31 18:54:46 INFO rdd.BlockRDD: Removing RDD 20 from persistence list
17/07/31 18:54:46 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[20] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527286000 ms
17/07/31 18:54:46 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527284000 ms
17/07/31 18:54:46 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527284000 ms
17/07/31 18:54:46 INFO storage.BlockManager: Removing RDD 20
17/07/31 18:54:47 INFO scheduler.JobScheduler: Added jobs for time 1501527287000 ms
17/07/31 18:54:47 INFO scheduler.JobScheduler: Starting job streaming job 1501527287000 ms.0 from job set of time 1501527287000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:47 INFO scheduler.JobScheduler: Finished job streaming job 1501527287000 ms.0 from job set of time 1501527287000 ms
17/07/31 18:54:47 INFO python.PythonRDD: Removing RDD 23 from persistence list
17/07/31 18:54:47 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527287000 ms (execution: 0.006 s)
17/07/31 18:54:47 INFO storage.BlockManager: Removing RDD 23
17/07/31 18:54:47 INFO rdd.BlockRDD: Removing RDD 22 from persistence list
17/07/31 18:54:47 INFO storage.BlockManager: Removing RDD 22
17/07/31 18:54:47 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[22] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527287000 ms
17/07/31 18:54:47 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527285000 ms
17/07/31 18:54:47 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527285000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Added jobs for time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Starting job streaming job 1501527288000 ms.0 from job set of time 1501527288000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:48 INFO scheduler.JobScheduler: Finished job streaming job 1501527288000 ms.0 from job set of time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527288000 ms (execution: 0.008 s)
17/07/31 18:54:48 INFO python.PythonRDD: Removing RDD 25 from persistence list
17/07/31 18:54:48 INFO storage.BlockManager: Removing RDD 25
17/07/31 18:54:48 INFO rdd.BlockRDD: Removing RDD 24 from persistence list
17/07/31 18:54:48 INFO storage.BlockManager: Removing RDD 24
17/07/31 18:54:48 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[24] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527286000 ms
17/07/31 18:54:48 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527286000 ms
17/07/31 18:54:49 INFO scheduler.JobScheduler: Added jobs for time 1501527289000 ms
17/07/31 18:54:49 INFO scheduler.JobScheduler: Starting job streaming job 1501527289000 ms.0 from job set of time 1501527289000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:49 INFO scheduler.JobScheduler: Finished job streaming job 1501527289000 ms.0 from job set of time 1501527289000 ms
17/07/31 18:54:49 INFO python.PythonRDD: Removing RDD 27 from persistence list
17/07/31 18:54:49 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501527289000 ms (execution: 0.008 s)
17/07/31 18:54:49 INFO storage.BlockManager: Removing RDD 27
17/07/31 18:54:49 INFO rdd.BlockRDD: Removing RDD 26 from persistence list
17/07/31 18:54:49 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[26] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527289000 ms
17/07/31 18:54:49 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527287000 ms
17/07/31 18:54:49 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527287000 ms
17/07/31 18:54:49 INFO storage.BlockManager: Removing RDD 26
17/07/31 18:54:50 INFO scheduler.JobScheduler: Added jobs for time 1501527290000 ms
17/07/31 18:54:50 INFO scheduler.JobScheduler: Starting job streaming job 1501527290000 ms.0 from job set of time 1501527290000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:50 INFO scheduler.JobScheduler: Finished job streaming job 1501527290000 ms.0 from job set of time 1501527290000 ms
17/07/31 18:54:50 INFO python.PythonRDD: Removing RDD 29 from persistence list
17/07/31 18:54:50 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527290000 ms (execution: 0.008 s)
17/07/31 18:54:50 INFO storage.BlockManager: Removing RDD 29
17/07/31 18:54:50 INFO rdd.BlockRDD: Removing RDD 28 from persistence list
17/07/31 18:54:50 INFO storage.BlockManager: Removing RDD 28
17/07/31 18:54:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[28] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527290000 ms
17/07/31 18:54:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527288000 ms
17/07/31 18:54:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527288000 ms
17/07/31 18:54:51 INFO scheduler.JobScheduler: Added jobs for time 1501527291000 ms
17/07/31 18:54:51 INFO scheduler.JobScheduler: Starting job streaming job 1501527291000 ms.0 from job set of time 1501527291000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:51 INFO scheduler.JobScheduler: Finished job streaming job 1501527291000 ms.0 from job set of time 1501527291000 ms
17/07/31 18:54:51 INFO python.PythonRDD: Removing RDD 31 from persistence list
17/07/31 18:54:51 INFO scheduler.JobScheduler: Total delay: 0.033 s for time 1501527291000 ms (execution: 0.018 s)
17/07/31 18:54:51 INFO storage.BlockManager: Removing RDD 31
17/07/31 18:54:51 INFO rdd.BlockRDD: Removing RDD 30 from persistence list
17/07/31 18:54:51 INFO storage.BlockManager: Removing RDD 30
17/07/31 18:54:51 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[30] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527291000 ms
17/07/31 18:54:51 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527289000 ms
17/07/31 18:54:51 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527289000 ms
17/07/31 18:54:52 INFO scheduler.JobScheduler: Added jobs for time 1501527292000 ms
17/07/31 18:54:52 INFO scheduler.JobScheduler: Starting job streaming job 1501527292000 ms.0 from job set of time 1501527292000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:52 INFO scheduler.JobScheduler: Finished job streaming job 1501527292000 ms.0 from job set of time 1501527292000 ms
17/07/31 18:54:52 INFO python.PythonRDD: Removing RDD 33 from persistence list
17/07/31 18:54:52 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501527292000 ms (execution: 0.008 s)
17/07/31 18:54:52 INFO storage.BlockManager: Removing RDD 33
17/07/31 18:54:52 INFO rdd.BlockRDD: Removing RDD 32 from persistence list
17/07/31 18:54:52 INFO storage.BlockManager: Removing RDD 32
17/07/31 18:54:52 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[32] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527292000 ms
17/07/31 18:54:52 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527290000 ms
17/07/31 18:54:52 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527290000 ms
17/07/31 18:54:53 INFO scheduler.JobScheduler: Added jobs for time 1501527293000 ms
17/07/31 18:54:53 INFO scheduler.JobScheduler: Starting job streaming job 1501527293000 ms.0 from job set of time 1501527293000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:53 INFO scheduler.JobScheduler: Finished job streaming job 1501527293000 ms.0 from job set of time 1501527293000 ms
17/07/31 18:54:53 INFO python.PythonRDD: Removing RDD 35 from persistence list
17/07/31 18:54:53 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501527293000 ms (execution: 0.008 s)
17/07/31 18:54:53 INFO storage.BlockManager: Removing RDD 35
17/07/31 18:54:53 INFO rdd.BlockRDD: Removing RDD 34 from persistence list
17/07/31 18:54:53 INFO storage.BlockManager: Removing RDD 34
17/07/31 18:54:53 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[34] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527293000 ms
17/07/31 18:54:53 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527291000 ms
17/07/31 18:54:53 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527291000 ms
17/07/31 18:54:54 INFO scheduler.JobScheduler: Added jobs for time 1501527294000 ms
17/07/31 18:54:54 INFO scheduler.JobScheduler: Starting job streaming job 1501527294000 ms.0 from job set of time 1501527294000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:54 INFO scheduler.JobScheduler: Finished job streaming job 1501527294000 ms.0 from job set of time 1501527294000 ms
17/07/31 18:54:54 INFO python.PythonRDD: Removing RDD 37 from persistence list
17/07/31 18:54:54 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527294000 ms (execution: 0.006 s)
17/07/31 18:54:54 INFO storage.BlockManager: Removing RDD 37
17/07/31 18:54:54 INFO rdd.BlockRDD: Removing RDD 36 from persistence list
17/07/31 18:54:54 INFO storage.BlockManager: Removing RDD 36
17/07/31 18:54:54 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[36] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527294000 ms
17/07/31 18:54:54 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527292000 ms
17/07/31 18:54:54 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527292000 ms
17/07/31 18:54:55 INFO scheduler.JobScheduler: Added jobs for time 1501527295000 ms
17/07/31 18:54:55 INFO scheduler.JobScheduler: Starting job streaming job 1501527295000 ms.0 from job set of time 1501527295000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:55 INFO scheduler.JobScheduler: Finished job streaming job 1501527295000 ms.0 from job set of time 1501527295000 ms
17/07/31 18:54:55 INFO python.PythonRDD: Removing RDD 39 from persistence list
17/07/31 18:54:55 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527295000 ms (execution: 0.007 s)
17/07/31 18:54:55 INFO storage.BlockManager: Removing RDD 39
17/07/31 18:54:55 INFO rdd.BlockRDD: Removing RDD 38 from persistence list
17/07/31 18:54:55 INFO storage.BlockManager: Removing RDD 38
17/07/31 18:54:55 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[38] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527295000 ms
17/07/31 18:54:55 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527293000 ms
17/07/31 18:54:55 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527293000 ms
17/07/31 18:54:56 INFO scheduler.JobScheduler: Added jobs for time 1501527296000 ms
17/07/31 18:54:56 INFO scheduler.JobScheduler: Starting job streaming job 1501527296000 ms.0 from job set of time 1501527296000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:56 INFO scheduler.JobScheduler: Finished job streaming job 1501527296000 ms.0 from job set of time 1501527296000 ms
17/07/31 18:54:56 INFO python.PythonRDD: Removing RDD 41 from persistence list
17/07/31 18:54:56 INFO scheduler.JobScheduler: Total delay: 0.031 s for time 1501527296000 ms (execution: 0.007 s)
17/07/31 18:54:56 INFO storage.BlockManager: Removing RDD 41
17/07/31 18:54:56 INFO rdd.BlockRDD: Removing RDD 40 from persistence list
17/07/31 18:54:56 INFO storage.BlockManager: Removing RDD 40
17/07/31 18:54:56 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[40] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527296000 ms
17/07/31 18:54:56 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527294000 ms
17/07/31 18:54:56 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527294000 ms
17/07/31 18:54:57 INFO scheduler.JobScheduler: Added jobs for time 1501527297000 ms
17/07/31 18:54:57 INFO scheduler.JobScheduler: Starting job streaming job 1501527297000 ms.0 from job set of time 1501527297000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:57 INFO scheduler.JobScheduler: Finished job streaming job 1501527297000 ms.0 from job set of time 1501527297000 ms
17/07/31 18:54:57 INFO python.PythonRDD: Removing RDD 43 from persistence list
17/07/31 18:54:57 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527297000 ms (execution: 0.008 s)
17/07/31 18:54:57 INFO storage.BlockManager: Removing RDD 43
17/07/31 18:54:57 INFO rdd.BlockRDD: Removing RDD 42 from persistence list
17/07/31 18:54:57 INFO storage.BlockManager: Removing RDD 42
17/07/31 18:54:57 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[42] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527297000 ms
17/07/31 18:54:57 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527295000 ms
17/07/31 18:54:57 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527295000 ms
17/07/31 18:54:58 INFO scheduler.JobScheduler: Added jobs for time 1501527298000 ms
17/07/31 18:54:58 INFO scheduler.JobScheduler: Starting job streaming job 1501527298000 ms.0 from job set of time 1501527298000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:58 INFO scheduler.JobScheduler: Finished job streaming job 1501527298000 ms.0 from job set of time 1501527298000 ms
17/07/31 18:54:58 INFO python.PythonRDD: Removing RDD 45 from persistence list
17/07/31 18:54:58 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527298000 ms (execution: 0.008 s)
17/07/31 18:54:58 INFO storage.BlockManager: Removing RDD 45
17/07/31 18:54:58 INFO rdd.BlockRDD: Removing RDD 44 from persistence list
17/07/31 18:54:58 INFO storage.BlockManager: Removing RDD 44
17/07/31 18:54:58 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[44] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527298000 ms
17/07/31 18:54:58 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527296000 ms
17/07/31 18:54:58 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527296000 ms
17/07/31 18:54:59 INFO scheduler.JobScheduler: Added jobs for time 1501527299000 ms
17/07/31 18:54:59 INFO scheduler.JobScheduler: Starting job streaming job 1501527299000 ms.0 from job set of time 1501527299000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:59 INFO scheduler.JobScheduler: Finished job streaming job 1501527299000 ms.0 from job set of time 1501527299000 ms
17/07/31 18:54:59 INFO python.PythonRDD: Removing RDD 47 from persistence list
17/07/31 18:54:59 INFO scheduler.JobScheduler: Total delay: 0.031 s for time 1501527299000 ms (execution: 0.009 s)
17/07/31 18:54:59 INFO storage.BlockManager: Removing RDD 47
17/07/31 18:54:59 INFO rdd.BlockRDD: Removing RDD 46 from persistence list
17/07/31 18:54:59 INFO storage.BlockManager: Removing RDD 46
17/07/31 18:54:59 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[46] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527299000 ms
17/07/31 18:54:59 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527297000 ms
17/07/31 18:54:59 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527297000 ms
17/07/31 18:55:00 INFO scheduler.JobScheduler: Added jobs for time 1501527300000 ms
17/07/31 18:55:00 INFO scheduler.JobScheduler: Starting job streaming job 1501527300000 ms.0 from job set of time 1501527300000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:00 INFO scheduler.JobScheduler: Finished job streaming job 1501527300000 ms.0 from job set of time 1501527300000 ms
17/07/31 18:55:00 INFO python.PythonRDD: Removing RDD 49 from persistence list
17/07/31 18:55:00 INFO scheduler.JobScheduler: Total delay: 0.035 s for time 1501527300000 ms (execution: 0.021 s)
17/07/31 18:55:00 INFO storage.BlockManager: Removing RDD 49
17/07/31 18:55:00 INFO rdd.BlockRDD: Removing RDD 48 from persistence list
17/07/31 18:55:00 INFO storage.BlockManager: Removing RDD 48
17/07/31 18:55:00 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[48] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527300000 ms
17/07/31 18:55:00 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527298000 ms
17/07/31 18:55:00 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527298000 ms
17/07/31 18:55:01 INFO scheduler.JobScheduler: Added jobs for time 1501527301000 ms
17/07/31 18:55:01 INFO scheduler.JobScheduler: Starting job streaming job 1501527301000 ms.0 from job set of time 1501527301000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:01 INFO scheduler.JobScheduler: Finished job streaming job 1501527301000 ms.0 from job set of time 1501527301000 ms
17/07/31 18:55:01 INFO python.PythonRDD: Removing RDD 51 from persistence list
17/07/31 18:55:01 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527301000 ms (execution: 0.008 s)
17/07/31 18:55:01 INFO storage.BlockManager: Removing RDD 51
17/07/31 18:55:01 INFO rdd.BlockRDD: Removing RDD 50 from persistence list
17/07/31 18:55:01 INFO storage.BlockManager: Removing RDD 50
17/07/31 18:55:01 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[50] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527301000 ms
17/07/31 18:55:01 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527299000 ms
17/07/31 18:55:01 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527299000 ms
17/07/31 18:55:02 INFO scheduler.JobScheduler: Added jobs for time 1501527302000 ms
17/07/31 18:55:02 INFO scheduler.JobScheduler: Starting job streaming job 1501527302000 ms.0 from job set of time 1501527302000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:02 INFO scheduler.JobScheduler: Finished job streaming job 1501527302000 ms.0 from job set of time 1501527302000 ms
17/07/31 18:55:02 INFO python.PythonRDD: Removing RDD 53 from persistence list
17/07/31 18:55:02 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501527302000 ms (execution: 0.009 s)
17/07/31 18:55:02 INFO storage.BlockManager: Removing RDD 53
17/07/31 18:55:02 INFO rdd.BlockRDD: Removing RDD 52 from persistence list
17/07/31 18:55:02 INFO storage.BlockManager: Removing RDD 52
17/07/31 18:55:02 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[52] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527302000 ms
17/07/31 18:55:02 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527300000 ms
17/07/31 18:55:02 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527300000 ms
17/07/31 18:55:03 INFO scheduler.JobScheduler: Added jobs for time 1501527303000 ms
17/07/31 18:55:03 INFO scheduler.JobScheduler: Starting job streaming job 1501527303000 ms.0 from job set of time 1501527303000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:03 INFO scheduler.JobScheduler: Finished job streaming job 1501527303000 ms.0 from job set of time 1501527303000 ms
17/07/31 18:55:03 INFO python.PythonRDD: Removing RDD 55 from persistence list
17/07/31 18:55:03 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527303000 ms (execution: 0.008 s)
17/07/31 18:55:03 INFO storage.BlockManager: Removing RDD 55
17/07/31 18:55:03 INFO rdd.BlockRDD: Removing RDD 54 from persistence list
17/07/31 18:55:03 INFO storage.BlockManager: Removing RDD 54
17/07/31 18:55:03 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[54] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527303000 ms
17/07/31 18:55:03 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527301000 ms
17/07/31 18:55:03 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527301000 ms
17/07/31 18:55:04 INFO scheduler.JobScheduler: Added jobs for time 1501527304000 ms
17/07/31 18:55:04 INFO scheduler.JobScheduler: Starting job streaming job 1501527304000 ms.0 from job set of time 1501527304000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:04 INFO scheduler.JobScheduler: Finished job streaming job 1501527304000 ms.0 from job set of time 1501527304000 ms
17/07/31 18:55:04 INFO python.PythonRDD: Removing RDD 57 from persistence list
17/07/31 18:55:04 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501527304000 ms (execution: 0.008 s)
17/07/31 18:55:04 INFO storage.BlockManager: Removing RDD 57
17/07/31 18:55:04 INFO rdd.BlockRDD: Removing RDD 56 from persistence list
17/07/31 18:55:04 INFO storage.BlockManager: Removing RDD 56
17/07/31 18:55:04 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[56] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527304000 ms
17/07/31 18:55:04 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527302000 ms
17/07/31 18:55:04 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527302000 ms
17/07/31 18:55:05 INFO scheduler.JobScheduler: Added jobs for time 1501527305000 ms
17/07/31 18:55:05 INFO scheduler.JobScheduler: Starting job streaming job 1501527305000 ms.0 from job set of time 1501527305000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:05 INFO scheduler.JobScheduler: Finished job streaming job 1501527305000 ms.0 from job set of time 1501527305000 ms
17/07/31 18:55:05 INFO python.PythonRDD: Removing RDD 59 from persistence list
17/07/31 18:55:05 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527305000 ms (execution: 0.008 s)
17/07/31 18:55:05 INFO storage.BlockManager: Removing RDD 59
17/07/31 18:55:05 INFO rdd.BlockRDD: Removing RDD 58 from persistence list
17/07/31 18:55:05 INFO storage.BlockManager: Removing RDD 58
17/07/31 18:55:05 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[58] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527305000 ms
17/07/31 18:55:05 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527303000 ms
17/07/31 18:55:05 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527303000 ms
17/07/31 18:55:06 INFO scheduler.JobScheduler: Added jobs for time 1501527306000 ms
17/07/31 18:55:06 INFO scheduler.JobScheduler: Starting job streaming job 1501527306000 ms.0 from job set of time 1501527306000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:06 INFO scheduler.JobScheduler: Finished job streaming job 1501527306000 ms.0 from job set of time 1501527306000 ms
17/07/31 18:55:06 INFO python.PythonRDD: Removing RDD 61 from persistence list
17/07/31 18:55:06 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527306000 ms (execution: 0.007 s)
17/07/31 18:55:06 INFO storage.BlockManager: Removing RDD 61
17/07/31 18:55:06 INFO rdd.BlockRDD: Removing RDD 60 from persistence list
17/07/31 18:55:06 INFO storage.BlockManager: Removing RDD 60
17/07/31 18:55:06 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[60] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527306000 ms
17/07/31 18:55:06 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527304000 ms
17/07/31 18:55:06 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527304000 ms
17/07/31 18:55:07 INFO scheduler.JobScheduler: Added jobs for time 1501527307000 ms
17/07/31 18:55:07 INFO scheduler.JobScheduler: Starting job streaming job 1501527307000 ms.0 from job set of time 1501527307000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:07 INFO scheduler.JobScheduler: Finished job streaming job 1501527307000 ms.0 from job set of time 1501527307000 ms
17/07/31 18:55:07 INFO python.PythonRDD: Removing RDD 63 from persistence list
17/07/31 18:55:07 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501527307000 ms (execution: 0.007 s)
17/07/31 18:55:07 INFO storage.BlockManager: Removing RDD 63
17/07/31 18:55:07 INFO rdd.BlockRDD: Removing RDD 62 from persistence list
17/07/31 18:55:07 INFO storage.BlockManager: Removing RDD 62
17/07/31 18:55:07 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[62] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527307000 ms
17/07/31 18:55:07 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527305000 ms
17/07/31 18:55:07 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527305000 ms
17/07/31 18:55:08 INFO scheduler.JobScheduler: Added jobs for time 1501527308000 ms
17/07/31 18:55:08 INFO scheduler.JobScheduler: Starting job streaming job 1501527308000 ms.0 from job set of time 1501527308000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:08 INFO scheduler.JobScheduler: Finished job streaming job 1501527308000 ms.0 from job set of time 1501527308000 ms
17/07/31 18:55:08 INFO python.PythonRDD: Removing RDD 65 from persistence list
17/07/31 18:55:08 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527308000 ms (execution: 0.008 s)
17/07/31 18:55:08 INFO storage.BlockManager: Removing RDD 65
17/07/31 18:55:08 INFO rdd.BlockRDD: Removing RDD 64 from persistence list
17/07/31 18:55:08 INFO storage.BlockManager: Removing RDD 64
17/07/31 18:55:08 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[64] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527308000 ms
17/07/31 18:55:08 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527306000 ms
17/07/31 18:55:08 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527306000 ms
17/07/31 18:55:09 INFO scheduler.JobScheduler: Added jobs for time 1501527309000 ms
17/07/31 18:55:09 INFO scheduler.JobScheduler: Starting job streaming job 1501527309000 ms.0 from job set of time 1501527309000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:09 INFO scheduler.JobScheduler: Finished job streaming job 1501527309000 ms.0 from job set of time 1501527309000 ms
17/07/31 18:55:09 INFO python.PythonRDD: Removing RDD 67 from persistence list
17/07/31 18:55:09 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527309000 ms (execution: 0.015 s)
17/07/31 18:55:09 INFO storage.BlockManager: Removing RDD 67
17/07/31 18:55:09 INFO rdd.BlockRDD: Removing RDD 66 from persistence list
17/07/31 18:55:09 INFO storage.BlockManager: Removing RDD 66
17/07/31 18:55:09 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[66] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527309000 ms
17/07/31 18:55:09 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527307000 ms
17/07/31 18:55:09 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527307000 ms
17/07/31 18:55:10 INFO scheduler.JobScheduler: Added jobs for time 1501527310000 ms
17/07/31 18:55:10 INFO scheduler.JobScheduler: Starting job streaming job 1501527310000 ms.0 from job set of time 1501527310000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:10 INFO scheduler.JobScheduler: Finished job streaming job 1501527310000 ms.0 from job set of time 1501527310000 ms
17/07/31 18:55:10 INFO python.PythonRDD: Removing RDD 69 from persistence list
17/07/31 18:55:10 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527310000 ms (execution: 0.007 s)
17/07/31 18:55:10 INFO storage.BlockManager: Removing RDD 69
17/07/31 18:55:10 INFO rdd.BlockRDD: Removing RDD 68 from persistence list
17/07/31 18:55:10 INFO storage.BlockManager: Removing RDD 68
17/07/31 18:55:10 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[68] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527310000 ms
17/07/31 18:55:10 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527308000 ms
17/07/31 18:55:10 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527308000 ms
17/07/31 18:55:11 INFO scheduler.JobScheduler: Added jobs for time 1501527311000 ms
17/07/31 18:55:11 INFO scheduler.JobScheduler: Starting job streaming job 1501527311000 ms.0 from job set of time 1501527311000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:11 INFO scheduler.JobScheduler: Finished job streaming job 1501527311000 ms.0 from job set of time 1501527311000 ms
17/07/31 18:55:11 INFO python.PythonRDD: Removing RDD 71 from persistence list
17/07/31 18:55:11 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527311000 ms (execution: 0.007 s)
17/07/31 18:55:11 INFO storage.BlockManager: Removing RDD 71
17/07/31 18:55:11 INFO rdd.BlockRDD: Removing RDD 70 from persistence list
17/07/31 18:55:11 INFO storage.BlockManager: Removing RDD 70
17/07/31 18:55:11 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[70] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527311000 ms
17/07/31 18:55:11 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527309000 ms
17/07/31 18:55:11 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527309000 ms
17/07/31 18:55:12 INFO scheduler.JobScheduler: Added jobs for time 1501527312000 ms
17/07/31 18:55:12 INFO scheduler.JobScheduler: Starting job streaming job 1501527312000 ms.0 from job set of time 1501527312000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:12 INFO scheduler.JobScheduler: Finished job streaming job 1501527312000 ms.0 from job set of time 1501527312000 ms
17/07/31 18:55:12 INFO python.PythonRDD: Removing RDD 73 from persistence list
17/07/31 18:55:12 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501527312000 ms (execution: 0.008 s)
17/07/31 18:55:12 INFO storage.BlockManager: Removing RDD 73
17/07/31 18:55:12 INFO rdd.BlockRDD: Removing RDD 72 from persistence list
17/07/31 18:55:12 INFO storage.BlockManager: Removing RDD 72
17/07/31 18:55:12 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[72] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527312000 ms
17/07/31 18:55:12 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527310000 ms
17/07/31 18:55:12 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527310000 ms
17/07/31 18:55:13 INFO scheduler.JobScheduler: Added jobs for time 1501527313000 ms
17/07/31 18:55:13 INFO scheduler.JobScheduler: Starting job streaming job 1501527313000 ms.0 from job set of time 1501527313000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:13 INFO scheduler.JobScheduler: Finished job streaming job 1501527313000 ms.0 from job set of time 1501527313000 ms
17/07/31 18:55:13 INFO python.PythonRDD: Removing RDD 75 from persistence list
17/07/31 18:55:13 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527313000 ms (execution: 0.007 s)
17/07/31 18:55:13 INFO storage.BlockManager: Removing RDD 75
17/07/31 18:55:13 INFO rdd.BlockRDD: Removing RDD 74 from persistence list
17/07/31 18:55:13 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[74] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527313000 ms
17/07/31 18:55:13 INFO storage.BlockManager: Removing RDD 74
17/07/31 18:55:13 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527311000 ms
17/07/31 18:55:13 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527311000 ms
17/07/31 18:55:14 INFO scheduler.JobScheduler: Added jobs for time 1501527314000 ms
17/07/31 18:55:14 INFO scheduler.JobScheduler: Starting job streaming job 1501527314000 ms.0 from job set of time 1501527314000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:14 INFO scheduler.JobScheduler: Finished job streaming job 1501527314000 ms.0 from job set of time 1501527314000 ms
17/07/31 18:55:14 INFO python.PythonRDD: Removing RDD 77 from persistence list
17/07/31 18:55:14 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527314000 ms (execution: 0.007 s)
17/07/31 18:55:14 INFO storage.BlockManager: Removing RDD 77
17/07/31 18:55:14 INFO rdd.BlockRDD: Removing RDD 76 from persistence list
17/07/31 18:55:14 INFO storage.BlockManager: Removing RDD 76
17/07/31 18:55:14 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[76] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527314000 ms
17/07/31 18:55:14 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527312000 ms
17/07/31 18:55:14 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527312000 ms
17/07/31 18:55:15 INFO scheduler.JobScheduler: Added jobs for time 1501527315000 ms
17/07/31 18:55:15 INFO scheduler.JobScheduler: Starting job streaming job 1501527315000 ms.0 from job set of time 1501527315000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:15 INFO scheduler.JobScheduler: Finished job streaming job 1501527315000 ms.0 from job set of time 1501527315000 ms
17/07/31 18:55:15 INFO python.PythonRDD: Removing RDD 79 from persistence list
17/07/31 18:55:15 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527315000 ms (execution: 0.007 s)
17/07/31 18:55:15 INFO storage.BlockManager: Removing RDD 79
17/07/31 18:55:15 INFO rdd.BlockRDD: Removing RDD 78 from persistence list
17/07/31 18:55:15 INFO storage.BlockManager: Removing RDD 78
17/07/31 18:55:15 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[78] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527315000 ms
17/07/31 18:55:15 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527313000 ms
17/07/31 18:55:15 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527313000 ms
17/07/31 18:55:16 INFO scheduler.JobScheduler: Added jobs for time 1501527316000 ms
17/07/31 18:55:16 INFO scheduler.JobScheduler: Starting job streaming job 1501527316000 ms.0 from job set of time 1501527316000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:16 INFO scheduler.JobScheduler: Finished job streaming job 1501527316000 ms.0 from job set of time 1501527316000 ms
17/07/31 18:55:16 INFO python.PythonRDD: Removing RDD 81 from persistence list
17/07/31 18:55:16 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501527316000 ms (execution: 0.014 s)
17/07/31 18:55:16 INFO storage.BlockManager: Removing RDD 81
17/07/31 18:55:16 INFO rdd.BlockRDD: Removing RDD 80 from persistence list
17/07/31 18:55:16 INFO storage.BlockManager: Removing RDD 80
17/07/31 18:55:16 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[80] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527316000 ms
17/07/31 18:55:16 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527314000 ms
17/07/31 18:55:16 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527314000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Added jobs for time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Starting job streaming job 1501527317000 ms.0 from job set of time 1501527317000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:17 INFO scheduler.JobScheduler: Finished job streaming job 1501527317000 ms.0 from job set of time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527317000 ms (execution: 0.011 s)
17/07/31 18:55:17 INFO python.PythonRDD: Removing RDD 83 from persistence list
17/07/31 18:55:17 INFO storage.BlockManager: Removing RDD 83
17/07/31 18:55:17 INFO rdd.BlockRDD: Removing RDD 82 from persistence list
17/07/31 18:55:17 INFO storage.BlockManager: Removing RDD 82
17/07/31 18:55:17 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[82] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527315000 ms
17/07/31 18:55:17 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527315000 ms
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1059.6 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1059.6 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.4 KB, free: 1057.9 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.4 KB, free: 1057.9 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 880.3 KB, free: 1057.0 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 880.3 KB, free: 1057.1 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.0 KB, free: 1056.2 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.0 KB, free: 1056.2 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527317800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.3 KB, free: 1054.5 MB)
17/07/31 18:55:18 INFO scheduler.JobScheduler: Added jobs for time 1501527318000 ms
17/07/31 18:55:18 INFO scheduler.JobScheduler: Starting job streaming job 1501527318000 ms.0 from job set of time 1501527318000 ms
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527317800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.3 KB, free: 1054.5 MB)
17/07/31 18:55:18 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Got job 2 (runJob at PythonRDD.scala:441) with 1 output partitions
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Final stage: ResultStage 3 (runJob at PythonRDD.scala:441)
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Submitting ResultStage 3 (PythonRDD[88] at RDD at PythonRDD.scala:48), which has no missing parents
17/07/31 18:55:18 INFO memory.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/07/31 18:55:18 INFO memory.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.0.0.199:59442 (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:18 INFO spark.SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:996
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (PythonRDD[88] at RDD at PythonRDD.scala:48)
17/07/31 18:55:18 INFO cluster.YarnScheduler: Adding task set 3.0 with 1 tasks
17/07/31 18:55:18 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 3.0 (TID 71, ip-10-0-0-151.ec2.internal, executor 2, partition 0, NODE_LOCAL, 5800 bytes)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 5.3 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.3 KB, free: 1052.7 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.3 KB, free: 1052.8 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.1 KB, free: 1050.2 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.1 KB, free: 1050.2 MB)
17/07/31 18:55:18 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 3.0 (TID 71) in 893 ms on ip-10-0-0-151.ec2.internal (executor 2) (1/1)
17/07/31 18:55:18 INFO cluster.YarnScheduler: Removed TaskSet 3.0, whose tasks have all completed, from pool
17/07/31 18:55:18 INFO scheduler.DAGScheduler: ResultStage 3 (runJob at PythonRDD.scala:441) finished in 0.896 s
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Job 2 finished: runJob at PythonRDD.scala:441, took 0.913501 s
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527318800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.5 KB, free: 1049.3 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527318800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.5 KB, free: 1049.3 MB)
17/07/31 18:55:19 INFO scheduler.JobScheduler: Added jobs for time 1501527319000 ms
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 887.9 KB, free: 1048.4 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 887.9 KB, free: 1048.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1047.6 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1047.6 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1046.7 MB)
17/07/31 18:55:19 INFO spark.ContextCleaner: Cleaned shuffle 0
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 10.0.0.199:59442 in memory (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1988.0 B, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on 10.0.0.199:59442 in memory (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1956.0 B, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on 10.0.0.199:59442 in memory (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 5.3 KB, free: 1046.8 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.2 KB, free: 1045.0 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.2 KB, free: 1045.0 MB)
17/07/31 18:55:19 INFO execution.SparkSqlParser: Parsing command: spotdb.proxy
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527319800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527319800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
17/07/31 18:55:20 INFO metastore.ObjectStore: ObjectStore, initialize called
17/07/31 18:55:20 INFO scheduler.JobScheduler: Added jobs for time 1501527320000 ms
17/07/31 18:55:20 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing
17/07/31 18:55:20 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
17/07/31 18:55:20 INFO metastore.ObjectStore: Initialized ObjectStore
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_table start=1501527320011 end=1501527320059 duration=48 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_table start=1501527320177 end=1501527320179 duration=2 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_database: spotdb
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: spotdb
17/07/31 18:55:20 WARN metastore.ObjectStore: Failed to get database spotdb, returning NoSuchObjectException
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527320239 end=1501527320242 duration=3 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527318000 ms.0 from job set of time 1501527318000 ms
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 2.266 s for time 1501527318000 ms (execution: 2.249 s)
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 85 from persistence list
17/07/31 18:55:20 INFO scheduler.JobScheduler: Starting job streaming job 1501527319000 ms.0 from job set of time 1501527319000 ms
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 84 from persistence list
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527318000 ms.0
org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 71, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u"Database 'spotdb' not found;"
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[84] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527318000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 85
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527316000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527316000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 84
17/07/31 18:55:20 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Got job 3 (runJob at PythonRDD.scala:441) with 1 output partitions
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Final stage: ResultStage 4 (runJob at PythonRDD.scala:441)
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Submitting ResultStage 4 (PythonRDD[97] at RDD at PythonRDD.scala:48), which has no missing parents
Traceback (most recent call last):
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 148, in <module>
main()
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 77, in main
bluecoat_parse(args.zk,args.topic,args.db,args.db_table,args.num_of_workers,args.batch_size)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 144, in bluecoat_parse
ssc.awaitTermination()
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/context.py", line 206, in awaitTermination
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o48.awaitTermination.
: org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 71, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u"Database 'spotdb' not found;"
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/07/31 18:55:20 INFO memory.MemoryStore: Block broadcast_4 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/07/31 18:55:20 INFO memory.MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on 10.0.0.199:59442 (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:20 INFO spark.SparkContext: Created broadcast 4 from broadcast at DAGScheduler.scala:996
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 4 (PythonRDD[97] at RDD at PythonRDD.scala:48)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Adding task set 4.0 with 1 tasks
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 4.0 (TID 72, ip-10-0-0-151.ec2.internal, executor 2, partition 0, NODE_LOCAL, 5800 bytes)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 5.3 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527319000 ms.0 from job set of time 1501527319000 ms
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 1.329 s for time 1501527319000 ms (execution: 0.051 s)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Starting job streaming job 1501527320000 ms.0 from job set of time 1501527320000 ms
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527319000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy36.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 87 from persistence list
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 86 from persistence list
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 87
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[86] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527319000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 86
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527317000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527317000 ms
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317000 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.2 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317400 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 880.3 KB, free: 1045.0 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317200 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.4 KB, free: 1046.7 MB)
17/07/31 18:55:20 INFO streaming.StreamingContext: Invoking stop(stopGracefully=false) from shutdown hook
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317600 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.0 KB, free: 1047.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317000 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.2 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317200 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.4 KB, free: 1045.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317400 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 880.3 KB, free: 1046.7 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317600 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.0 KB, free: 1047.6 MB)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527320000 ms.0 from job set of time 1501527320000 ms
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 90 from persistence list
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 0.333 s for time 1501527320000 ms (execution: 0.003 s)
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527320000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy36.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 89 from persistence list
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 90
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 89
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[89] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527320000 ms
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: Sent stop signal to all 1 receivers
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317800 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.3 KB, free: 1049.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318000 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.1 KB, free: 1050.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318200 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.3 KB, free: 1051.0 MB)
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527318000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527318000 ms
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318400 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318600 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.1 KB, free: 1053.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317800 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.3 KB, free: 1049.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318000 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.1 KB, free: 1050.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318200 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.3 KB, free: 1051.0 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318400 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318600 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.1 KB, free: 1053.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 880.6 KB, free: 1052.7 MB)
17/07/31 18:55:20 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 880.6 KB, free: 1052.8 MB)
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 4.0 (TID 72) in 289 ms on ip-10-0-0-151.ec2.internal (executor 2) (1/1)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Removed TaskSet 4.0, whose tasks have all completed, from pool
17/07/31 18:55:20 ERROR scheduler.DAGScheduler: Failed to update accumulators for task 0
org.apache.spark.SparkException: EOF reached before Python server acknowledged
at org.apache.spark.api.python.PythonAccumulatorV2.merge(PythonRDD.scala:919)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$updateAccumulators$1.apply(DAGScheduler.scala:1088)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$updateAccumulators$1.apply(DAGScheduler.scala:1080)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.DAGScheduler.updateAccumulators(DAGScheduler.scala:1080)
at org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:1156)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1647)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/07/31 18:55:20 INFO scheduler.DAGScheduler: ResultStage 4 (runJob at PythonRDD.scala:441) finished in 0.292 s
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Job 3 finished: runJob at PythonRDD.scala:441, took 0.309062 s
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 2.0 (TID 70) in 43848 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/1)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool
17/07/31 18:55:20 INFO scheduler.DAGScheduler: ResultStage 2 (start at NativeMethodAccessorImpl.java:0) finished in 43.849 s
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: All of the receivers have deregistered successfully
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
17/07/31 18:55:20 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
17/07/31 18:55:20 INFO util.RecurringTimer: Stopped timer for JobGenerator after time 1501527320000
17/07/31 18:55:20 INFO scheduler.JobGenerator: Stopped JobGenerator
17/07/31 18:55:20 INFO scheduler.JobScheduler: Stopped JobScheduler
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19fd4723{/streaming,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@22116fc3{/streaming/batch,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@70d301d6{/static/streaming,null,UNAVAILABLE}
17/07/31 18:55:20 INFO streaming.StreamingContext: StreamingContext stopped successfully
17/07/31 18:55:20 INFO spark.SparkContext: Invoking stop() from shutdown hook
17/07/31 18:55:20 INFO server.ServerConnector: Stopped ServerConnector@188e6c7f{HTTP/1.1}{0.0.0.0:4040}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4f734944{/stages/stage/kill,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@b7d4da0{/jobs/job/kill,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ab71bb9{/api,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5d37ce06{/,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@27f55b38{/static,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1159f15e{/executors/threadDump/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ff970c2{/executors/threadDump,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@615b86f5{/executors/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19bb7285{/executors,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3b9882ec{/environment/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7b3691b6{/environment,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@423a3429{/storage/rdd/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@13016b86{/storage/rdd,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ea732f0{/storage/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2d4aa15a{/storage,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5cbdbf0f{/stages/pool/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14c62558{/stages/pool,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5c61bd1a{/stages/stage/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4eb72ecd{/stages/stage,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1bf090df{/stages/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14b58fc0{/stages,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@30e71b5d{/jobs/job/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@232864a3{/jobs/job,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1f4da763{/jobs/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1a3b1e79{/jobs,null,UNAVAILABLE}
17/07/31 18:55:20 INFO ui.SparkUI: Stopped Spark web UI at http://10.0.0.199:4040
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors
17/07/31 18:55:20 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
17/07/31 18:55:20 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
services=List(),
started=false)
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Stopped
17/07/31 18:55:20 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/07/31 18:55:20 INFO memory.MemoryStore: MemoryStore cleared
17/07/31 18:55:20 INFO storage.BlockManager: BlockManager stopped
17/07/31 18:55:20 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
17/07/31 18:55:20 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/07/31 18:55:20 INFO spark.SparkContext: Successfully stopped SparkContext
17/07/31 18:55:20 INFO util.ShutdownHookManager: Shutdown hook called
17/07/31 18:55:20 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40
17/07/31 18:55:20 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40/pyspark-506939c9-a852-404b-adea-6ad40ad0d73f
Closing worker...
Thanks
Deon Griessel
Re: Spot Ingest Proxy Error
Posted by "Barona, Ricardo" <ri...@intel.com>.
Hi Dean,
Let me ask around.
From: Deon Griessel <dg...@searchtechnologies.com>
Reply-To: "user@spot.incubator.apache.org" <us...@spot.incubator.apache.org>
Date: Tuesday, August 1, 2017 at 10:04 AM
To: "user@spot.incubator.apache.org" <us...@spot.incubator.apache.org>, "user@spot.apache.org" <us...@spot.apache.org>
Subject: RE: Spot Ingest Proxy Error
Ok, it seems like I got around the original error by enabling the hive service in Spark2 configuration in Cloudera Manager and restarting all services.
Now getting the following error:
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
Anybody seen this before? Full log below.
Thanks
---------------------------
2017-08-01 14:49:13,694 - SPOT.INGEST.WORKER - INFO - Initializing kafka instance
2017-08-01 14:49:13,694 - SPOT.INGEST.WRK.PROXY - INFO - Creating Spark Job for topic: SPOT-INGEST-proxy-14_49_11
2017-08-01 14:49:13,694 - SPOT.INGEST.WRK.PROXY - INFO - SPOT.Utils: Executing: spark2-submit --master yarn --driver-memory 2325m --num-executors 1 --conf spark.executor.memory=2325m --conf spark.executor.cores=1 --jars /home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar /home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py -zk ip-10-0-0-199.ec2.internal:2181 -t SPOT-INGEST-proxy-14_49_11 -db spotdb -dt proxy -w 1 -bs 1
17/08/01 14:49:14 INFO spark.SparkContext: Running Spark version 2.1.0.cloudera1
17/08/01 14:49:14 WARN spark.SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0
17/08/01 14:49:15 INFO spark.SecurityManager: Changing view acls to: spot-user
17/08/01 14:49:15 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/08/01 14:49:15 INFO spark.SecurityManager: Changing view acls groups to:
17/08/01 14:49:15 INFO spark.SecurityManager: Changing modify acls groups to:
17/08/01 14:49:15 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/08/01 14:49:15 INFO util.Utils: Successfully started service 'sparkDriver' on port 56925.
17/08/01 14:49:15 INFO spark.SparkEnv: Registering MapOutputTracker
17/08/01 14:49:15 INFO spark.SparkEnv: Registering BlockManagerMaster
17/08/01 14:49:15 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
17/08/01 14:49:15 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
17/08/01 14:49:15 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-23e4ad79-bd38-4e37-b688-a75f9565bda4
17/08/01 14:49:16 INFO memory.MemoryStore: MemoryStore started with capacity 1060.5 MB
17/08/01 14:49:16 INFO spark.SparkEnv: Registering OutputCommitCoordinator
17/08/01 14:49:16 INFO util.log: Logging initialized @2378ms
17/08/01 14:49:16 INFO server.Server: jetty-9.2.z-SNAPSHOT
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3b9882ec{/jobs,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19bb7285{/jobs/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@615b86f5{/jobs/job,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ff970c2{/jobs/job/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1159f15e{/stages,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27f55b38{/stages/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5d37ce06{/stages/stage,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ab71bb9{/stages/stage/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b7d4da0{/stages/pool,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f734944{/stages/pool/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19cf381c{/storage,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@736cc380{/storage/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@699743b3{/storage/rdd,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@49ddd2ed{/storage/rdd/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@242eb7e{/environment,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@77729e5a{/environment/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5fa63fa{/executors,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@79b9b526{/executors/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@f241039{/executors/threadDump,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@8a515f3{/executors/threadDump/json,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@e4408f4{/static,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14a7d27d{/,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@15ec1723{/api,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ee30613{/jobs/job/kill,null,AVAILABLE}
17/08/01 14:49:16 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2aa3e9a6{/stages/stage/kill,null,AVAILABLE}
17/08/01 14:49:16 INFO server.ServerConnector: Started ServerConnector@37681deb{HTTP/1.1}{0.0.0.0:4040}
17/08/01 14:49:16 INFO server.Server: Started @2532ms
17/08/01 14:49:16 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
17/08/01 14:49:16 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.0.0.199:4040
17/08/01 14:49:16 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/08/01 14:49:17 INFO client.RMProxy: Connecting to ResourceManager at ip-10-0-0-199.ec2.internal/10.0.0.199:8032
17/08/01 14:49:17 INFO yarn.Client: Requesting a new application from cluster with 3 NodeManagers
17/08/01 14:49:17 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (2725 MB per container)
17/08/01 14:49:17 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/08/01 14:49:17 INFO yarn.Client: Setting up container launch context for our AM
17/08/01 14:49:17 INFO yarn.Client: Setting up the launch environment for our AM container
17/08/01 14:49:17 INFO yarn.Client: Preparing resources for our AM container
17/08/01 14:49:19 INFO yarn.Client: Uploading resource file:/home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501598591600_0003/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar
17/08/01 14:49:19 INFO yarn.Client: Uploading resource file:/tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25/__spark_conf__4374658135010889941.zip -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501598591600_0003/__spark_conf__.zip
17/08/01 14:49:19 INFO spark.SecurityManager: Changing view acls to: spot-user
17/08/01 14:49:19 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/08/01 14:49:19 INFO spark.SecurityManager: Changing view acls groups to:
17/08/01 14:49:19 INFO spark.SecurityManager: Changing modify acls groups to:
17/08/01 14:49:19 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/08/01 14:49:19 INFO yarn.Client: Submitting application application_1501598591600_0003 to ResourceManager
17/08/01 14:49:19 INFO impl.YarnClientImpl: Submitted application application_1501598591600_0003
17/08/01 14:49:19 INFO cluster.SchedulerExtensionServices: Starting Yarn extension services with app application_1501598591600_0003 and attemptId None
17/08/01 14:49:20 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:20 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.users.spot-user
start time: 1501598959525
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003/
user: spot-user
17/08/01 14:49:21 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:22 INFO yarn.Client: Application report for application_1501598591600_0003 (state: ACCEPTED)
17/08/01 14:49:23 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/08/01 14:49:23 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> ip-10-0-0-199.ec2.internal, PROXY_URI_BASES -> http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003), /proxy/application_1501598591600_0003
17/08/01 14:49:23 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/08/01 14:49:23 INFO yarn.Client: Application report for application_1501598591600_0003 (state: RUNNING)
17/08/01 14:49:23 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 10.0.0.151
ApplicationMaster RPC port: 0
queue: root.users.spot-user
start time: 1501598959525
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501598591600_0003/
user: spot-user
17/08/01 14:49:23 INFO cluster.YarnClientSchedulerBackend: Application application_1501598591600_0003 has started running.
17/08/01 14:49:23 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 53355.
17/08/01 14:49:23 INFO netty.NettyBlockTransferService: Server created on 10.0.0.199:53355
17/08/01 14:49:23 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/08/01 14:49:23 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManagerMasterEndpoint: Registering block manager 10.0.0.199:53355 with 1060.5 MB RAM, BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO storage.BlockManager: external shuffle service port = 7337
17/08/01 14:49:23 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 10.0.0.199, 53355, None)
17/08/01 14:49:23 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@52c2dab5{/metrics/json,null,AVAILABLE}
17/08/01 14:49:23 INFO scheduler.EventLoggingListener: Logging events to hdfs://ip-10-0-0-199.ec2.internal:8020/user/spark/spark2ApplicationHistory/application_1501598591600_0003
17/08/01 14:49:23 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/08/01 14:49:27 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.229:37096) with ID 1
17/08/01 14:49:27 INFO spark.ExecutorAllocationManager: New executor 1 has registered (new total is 1)
17/08/01 14:49:27 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-229.ec2.internal:51770 with 1060.5 MB RAM, BlockManagerId(1, ip-10-0-0-229.ec2.internal, 51770, None)
17/08/01 14:49:27 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/08/01 14:49:27 INFO internal.SharedState: spark.sql.warehouse.dir is not set, but hive.metastore.warehouse.dir is set. Setting spark.sql.warehouse.dir to the value of hive.metastore.warehouse.dir ('/user/hive/warehouse').
17/08/01 14:49:27 INFO internal.SharedState: Warehouse path is '/user/hive/warehouse'.
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5554f0a0{/SQL,null,AVAILABLE}
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@44743601{/SQL/json,null,AVAILABLE}
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@144776d3{/SQL/execution,null,AVAILABLE}
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f3f8a4e{/SQL/execution/json,null,AVAILABLE}
17/08/01 14:49:27 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27946137{/static/sql,null,AVAILABLE}
17/08/01 14:49:27 INFO hive.HiveUtils: Initializing HiveMetastoreConnection version 1.1.0 using file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-logging-1.1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-exec-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-exec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-1.1.0-cdh5.12.0-standalone.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc-standalone.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-jdbc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-metastore-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-metastore.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-serde-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-serde.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-service-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-service.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/libfb303-0.9.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/libthrift-0.9.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/log4j-1.2.16.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-hadoop-compat.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-hadoop2-compat.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-protocol.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-server.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/htrace-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ST4-4.0.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-core-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-fate-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-start-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/accumulo-trace-1.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/activation-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ant-1.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ant-launcher-1.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/antlr-2.7.7.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/antlr-runtime-3.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/apache-log4j-extras-1.2.17.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-3.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-commons-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/asm-tree-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/avro.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/bonecp-0.8.0.RELEASE.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-avatica-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-core-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/calcite-linq4j-1.0.0-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-beanutils-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-beanutils-core-1.8.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-cli-1.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-codec-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-collections-3.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-compiler-2.7.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-compress-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-configuration-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-dbcp-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-digester-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-el-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-httpclient-3.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-io-2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-lang-2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-lang3-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-math-2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-pool-1.5.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/commons-vfs2-2.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-client-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-framework-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/curator-recipes-2.6.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-api-jdo-3.2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-core-3.2.10.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/datanucleus-rdbms-3.2.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/derby-10.11.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/eigenbase-properties-1.1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/findbugs-annotations-1.3.9-1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-annotation_1.0_spec-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-jaspic_1.0_spec-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/geronimo-jta_1.1_spec-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/groovy-all-2.4.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/gson-2.2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/guava-14.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hamcrest-core-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hbase-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/high-scale-lib-1.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-accumulo-handler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-accumulo-handler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-ant-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-ant.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-beeline-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-beeline.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-cli-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-cli.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-common-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-contrib-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-contrib.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hbase-handler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hbase-handler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hwi-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-hwi.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-0.23-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-0.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-common-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-scheduler-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims-scheduler.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-shims.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-testutils-1.1.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/hive-testutils.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/httpclient-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/httpcore-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-core-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/ivy-2.0.0-rc2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/parquet-hadoop-bundle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/stringtemplate-3.2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/regexp-1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/xz-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/tempus-fugit-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/super-csv-2.2.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/stax-api-1.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/servlet-api-2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/opencsv-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-jvm-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-json-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/metrics-core-3.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-provider-svnexe-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-provider-svn-commons-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/maven-scm-api-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/mail-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/logredactor-1.0.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/junit-4.11.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jta-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jsr305-3.0.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jsp-api-2.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jpam-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/joda-time-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jline-2.12.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jetty-all-server-7.6.0.v20120127.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jetty-all-7.6.0.v20120127.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jersey-servlet-1.14.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jersey-server-1.14.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jdo-api-3.0.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jcommander-1.32.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jasper-runtime-5.5.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jasper-compiler-5.5.23.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/janino-2.7.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jamon-runtime-2.3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-xc-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-jaxrs-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-databind-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/jackson-annotations-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/zookeeper.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/velocity-1.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/snappy-java-1.0.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/plexus-utils-1.5.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/paranamer-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/../hive/lib/oro-2.0.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/activation-1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/activation.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-i18n-2.0.0-M15.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-i18n.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-kerberos-codec-2.0.0-M15.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/apacheds-kerberos-codec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-asn1-api-1.0.0-M20.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-asn1-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-util-1.0.0-M20.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/api-util.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/avro.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/aws-java-sdk-bundle-1.11.134.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/aws-java-sdk-bundle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/azure-data-lake-store-sdk-2.1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/azure-data-lake-store-sdk.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-1.9.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-core-1.8.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-beanutils.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-cli-1.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-cli.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-codec-1.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-codec.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-collections-3.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-collections.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-compress-1.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-compress.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-configuration-1.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-configuration.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-digester-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-digester.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-httpclient-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-httpclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-io-2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-io.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-lang-2.6.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-lang.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-logging-1.1.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-logging.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-math3-3.1.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-math3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-net-3.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/commons-net.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-client-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-framework-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-framework.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-recipes-2.7.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/curator-recipes.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/gson-2.2.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/gson.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/guava-11.0.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/guava.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-annotations-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-auth-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-auth.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-aws-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-aws.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-azure-datalake-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-azure-datalake.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-hdfs-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-hdfs.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-app-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-app.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-core-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-jobclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-mapreduce-client-shuffle.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-api-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-client-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-server-common-2.6.0-cdh5.12.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/hadoop-yarn-server-common.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/htrace-core4-4.0.1-incubating.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/htrace-core4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpclient-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpclient.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpcore-4.2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/httpcore.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-annotations-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-annotations.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-core-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-core.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-databind-2.2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-databind.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-jaxrs-1.8.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-jaxrs.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-xc-1.8.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jackson-xc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jaxb-api-2.2.2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jaxb-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-client-1.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-client.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-core-1.9.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/zookeeper.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xz.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xz-1.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xmlenc.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xmlenc-0.52.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xml-apis.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xml-apis-1.3.04.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xercesImpl.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/xercesImpl-2.9.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/stax-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/stax-api-1.0-2.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/snappy-java.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/snappy-java-1.0.4.1.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-log4j12.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/slf4j-api-1.7.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/servlet-api.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/servlet-api-2.5.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/protobuf-java.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/protobuf-java-2.5.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/paranamer.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/paranamer-2.3.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/netty.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/netty-3.10.5.Final.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/log4j.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/log4j-1.2.17.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/leveldbjni-all.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/leveldbjni-all-1.8.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jsr305.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jsr305-3.0.0.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jetty-util.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jetty-util-6.1.26.cloudera.4.jar:file:/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/client/jersey-core.jar
17/08/01 14:49:28 INFO session.SessionState: Created local directory: /tmp/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2_resources
17/08/01 14:49:28 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2
17/08/01 14:49:28 INFO session.SessionState: Created local directory: /tmp/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2
17/08/01 14:49:28 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/222b8db6-d3ed-4f13-a1de-8ac0b1cfcbd2/_tmp_space.db
17/08/01 14:49:28 INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
17/08/01 14:49:28 INFO client.HiveClientImpl: Warehouse location for Hive client (version 1.1.0) is /user/hive/warehouse
17/08/01 14:49:29 INFO hive.metastore: Trying to connect to metastore with URI thrift://ip-10-0-0-8.ec2.internal:9083
17/08/01 14:49:29 INFO hive.metastore: Opened a connection to metastore, current connections: 1
17/08/01 14:49:29 INFO hive.metastore: Connected to metastore.
17/08/01 14:49:29 WARN streaming.StreamingContext: Dynamic Allocation is enabled for this application. Enabling Dynamic allocation for Spark Streaming applications can cause data loss if Write Ahead Log is not enabled for non-replayable sources like Flume. See the programming guide for details on how to enable the Write Ahead Log.
17/08/01 14:49:29 INFO spark.SparkContext: Starting job: start at NativeMethodAccessorImpl.java:0
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Registering RDD 1 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Got job 0 (start at NativeMethodAccessorImpl.java:0) with 20 output partitions
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/08/01 14:49:29 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 3.1 KB, free 1060.5 MB)
17/08/01 14:49:29 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 1988.0 B, free 1060.5 MB)
17/08/01 14:49:29 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.0.0.199:53355 (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:49:29 INFO spark.SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:29 INFO scheduler.DAGScheduler: Submitting 50 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:29 INFO cluster.YarnScheduler: Adding task set 0.0 with 50 tasks
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, ip-10-0-0-229.ec2.internal, executor 1, partition 1, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 664 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, ip-10-0-0-229.ec2.internal, executor 1, partition 2, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 78 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3, ip-10-0-0-229.ec2.internal, executor 1, partition 3, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 68 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 0.0 (TID 4, ip-10-0-0-229.ec2.internal, executor 1, partition 4, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 65 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 0.0 (TID 5, ip-10-0-0-229.ec2.internal, executor 1, partition 5, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 59 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/50)
17/08/01 14:49:30 INFO spark.ExecutorAllocationManager: Requesting 1 new executor because tasks are backlogged (new desired total will be 2)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 0.0 (TID 6, ip-10-0-0-229.ec2.internal, executor 1, partition 6, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 0.0 (TID 7, ip-10-0-0-229.ec2.internal, executor 1, partition 7, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 60 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/50)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 0.0 (TID 8, ip-10-0-0-229.ec2.internal, executor 1, partition 8, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:30 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 0.0 (TID 7) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 0.0 (TID 9, ip-10-0-0-229.ec2.internal, executor 1, partition 9, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 0.0 (TID 8) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 0.0 (TID 10, ip-10-0-0-229.ec2.internal, executor 1, partition 10, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 0.0 (TID 9) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 0.0 (TID 11, ip-10-0-0-229.ec2.internal, executor 1, partition 11, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 0.0 (TID 10) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 0.0 (TID 12, ip-10-0-0-229.ec2.internal, executor 1, partition 12, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 0.0 (TID 11) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 0.0 (TID 13, ip-10-0-0-229.ec2.internal, executor 1, partition 13, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 0.0 (TID 12) in 60 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 0.0 (TID 14, ip-10-0-0-229.ec2.internal, executor 1, partition 14, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 0.0 (TID 13) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 0.0 (TID 15, ip-10-0-0-229.ec2.internal, executor 1, partition 15, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 0.0 (TID 14) in 51 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 0.0 (TID 16, ip-10-0-0-229.ec2.internal, executor 1, partition 16, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 0.0 (TID 15) in 63 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 0.0 (TID 17, ip-10-0-0-229.ec2.internal, executor 1, partition 17, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 0.0 (TID 16) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 0.0 (TID 18, ip-10-0-0-229.ec2.internal, executor 1, partition 18, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 0.0 (TID 17) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 0.0 (TID 19, ip-10-0-0-229.ec2.internal, executor 1, partition 19, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 0.0 (TID 18) in 56 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 0.0 (TID 20, ip-10-0-0-229.ec2.internal, executor 1, partition 20, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 0.0 (TID 19) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 0.0 (TID 21, ip-10-0-0-229.ec2.internal, executor 1, partition 21, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 0.0 (TID 20) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (21/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 0.0 (TID 22, ip-10-0-0-229.ec2.internal, executor 1, partition 22, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 0.0 (TID 21) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (22/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 0.0 (TID 23, ip-10-0-0-229.ec2.internal, executor 1, partition 23, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 0.0 (TID 22) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (23/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 0.0 (TID 24, ip-10-0-0-229.ec2.internal, executor 1, partition 24, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 0.0 (TID 23) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (24/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 0.0 (TID 25, ip-10-0-0-229.ec2.internal, executor 1, partition 25, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 0.0 (TID 24) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (25/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 0.0 (TID 26, ip-10-0-0-229.ec2.internal, executor 1, partition 26, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 0.0 (TID 25) in 46 ms on ip-10-0-0-229.ec2.internal (executor 1) (26/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 0.0 (TID 27, ip-10-0-0-229.ec2.internal, executor 1, partition 27, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 0.0 (TID 26) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (27/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 0.0 (TID 28, ip-10-0-0-229.ec2.internal, executor 1, partition 28, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 0.0 (TID 27) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (28/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 0.0 (TID 29, ip-10-0-0-229.ec2.internal, executor 1, partition 29, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 0.0 (TID 28) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (29/50)
17/08/01 14:49:31 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 4)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 0.0 (TID 30, ip-10-0-0-229.ec2.internal, executor 1, partition 30, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 0.0 (TID 29) in 42 ms on ip-10-0-0-229.ec2.internal (executor 1) (30/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 0.0 (TID 31, ip-10-0-0-229.ec2.internal, executor 1, partition 31, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 0.0 (TID 30) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (31/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 0.0 (TID 32, ip-10-0-0-229.ec2.internal, executor 1, partition 32, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 0.0 (TID 31) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (32/50)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 0.0 (TID 33, ip-10-0-0-229.ec2.internal, executor 1, partition 33, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:31 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 0.0 (TID 32) in 48 ms on ip-10-0-0-229.ec2.internal (executor 1) (33/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 0.0 (TID 34, ip-10-0-0-229.ec2.internal, executor 1, partition 34, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 0.0 (TID 33) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (34/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 0.0 (TID 35, ip-10-0-0-229.ec2.internal, executor 1, partition 35, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 0.0 (TID 34) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (35/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 0.0 (TID 36, ip-10-0-0-229.ec2.internal, executor 1, partition 36, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 0.0 (TID 35) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (36/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 0.0 (TID 37, ip-10-0-0-229.ec2.internal, executor 1, partition 37, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 0.0 (TID 36) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (37/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 0.0 (TID 38, ip-10-0-0-229.ec2.internal, executor 1, partition 38, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 0.0 (TID 37) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (38/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 0.0 (TID 39, ip-10-0-0-229.ec2.internal, executor 1, partition 39, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 0.0 (TID 38) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (39/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 0.0 (TID 40, ip-10-0-0-229.ec2.internal, executor 1, partition 40, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 0.0 (TID 39) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (40/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 0.0 (TID 41, ip-10-0-0-229.ec2.internal, executor 1, partition 41, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 0.0 (TID 40) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (41/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 0.0 (TID 42, ip-10-0-0-229.ec2.internal, executor 1, partition 42, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 0.0 (TID 41) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (42/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 0.0 (TID 43, ip-10-0-0-229.ec2.internal, executor 1, partition 43, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 0.0 (TID 42) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (43/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 0.0 (TID 44, ip-10-0-0-229.ec2.internal, executor 1, partition 44, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 0.0 (TID 43) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (44/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 0.0 (TID 45, ip-10-0-0-229.ec2.internal, executor 1, partition 45, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 0.0 (TID 44) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (45/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 0.0 (TID 46, ip-10-0-0-229.ec2.internal, executor 1, partition 46, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 0.0 (TID 45) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (46/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 0.0 (TID 47, ip-10-0-0-229.ec2.internal, executor 1, partition 47, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 0.0 (TID 46) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (47/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 0.0 (TID 48, ip-10-0-0-229.ec2.internal, executor 1, partition 48, PROCESS_LOCAL, 6709 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 0.0 (TID 47) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (48/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 0.0 (TID 49, ip-10-0-0-229.ec2.internal, executor 1, partition 49, PROCESS_LOCAL, 6676 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 0.0 (TID 48) in 28 ms on ip-10-0-0-229.ec2.internal (executor 1) (49/50)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 0.0 (TID 49) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (50/50)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/08/01 14:49:32 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (start at NativeMethodAccessorImpl.java:0) finished in 2.517 s
17/08/01 14:49:32 INFO scheduler.DAGScheduler: looking for newly runnable stages
17/08/01 14:49:32 INFO scheduler.DAGScheduler: running: Set()
17/08/01 14:49:32 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 1)
17/08/01 14:49:32 INFO scheduler.DAGScheduler: failed: Set()
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/08/01 14:49:32 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.2 KB, free 1060.5 MB)
17/08/01 14:49:32 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 1956.0 B, free 1060.5 MB)
17/08/01 14:49:32 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.0.0.199:53355 (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:49:32 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Submitting 20 missing tasks from ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Adding task set 1.0 with 20 tasks
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 50, ip-10-0-0-229.ec2.internal, executor 1, partition 0, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:49:32 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.0.0.229:37096
17/08/01 14:49:32 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 283 bytes
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.0 (TID 51, ip-10-0-0-229.ec2.internal, executor 1, partition 1, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 50) in 89 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 1.0 (TID 52, ip-10-0-0-229.ec2.internal, executor 1, partition 2, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 1.0 (TID 51) in 23 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 1.0 (TID 53, ip-10-0-0-229.ec2.internal, executor 1, partition 3, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 1.0 (TID 52) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 1.0 (TID 54, ip-10-0-0-229.ec2.internal, executor 1, partition 4, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 1.0 (TID 53) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 1.0 (TID 55, ip-10-0-0-229.ec2.internal, executor 1, partition 5, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 1.0 (TID 54) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 1.0 (TID 56, ip-10-0-0-229.ec2.internal, executor 1, partition 6, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 1.0 (TID 55) in 25 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 1.0 (TID 57, ip-10-0-0-229.ec2.internal, executor 1, partition 7, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 1.0 (TID 56) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 1.0 (TID 58, ip-10-0-0-229.ec2.internal, executor 1, partition 8, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 1.0 (TID 57) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 1.0 (TID 59, ip-10-0-0-229.ec2.internal, executor 1, partition 9, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 1.0 (TID 58) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 1.0 (TID 60, ip-10-0-0-229.ec2.internal, executor 1, partition 10, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 1.0 (TID 59) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 1.0 (TID 61, ip-10-0-0-229.ec2.internal, executor 1, partition 11, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 1.0 (TID 60) in 21 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 1.0 (TID 62, ip-10-0-0-229.ec2.internal, executor 1, partition 12, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 1.0 (TID 61) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 1.0 (TID 63, ip-10-0-0-229.ec2.internal, executor 1, partition 13, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 1.0 (TID 62) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 1.0 (TID 64, ip-10-0-0-229.ec2.internal, executor 1, partition 14, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 1.0 (TID 63) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 1.0 (TID 65, ip-10-0-0-229.ec2.internal, executor 1, partition 15, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 1.0 (TID 64) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 1.0 (TID 66, ip-10-0-0-229.ec2.internal, executor 1, partition 16, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 1.0 (TID 65) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 1.0 (TID 67, ip-10-0-0-229.ec2.internal, executor 1, partition 17, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 1.0 (TID 66) in 22 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 1.0 (TID 68, ip-10-0-0-229.ec2.internal, executor 1, partition 18, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 1.0 (TID 67) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 1.0 (TID 69, ip-10-0-0-229.ec2.internal, executor 1, partition 19, NODE_LOCAL, 6628 bytes)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 1.0 (TID 68) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/20)
17/08/01 14:49:32 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 1.0 (TID 69) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/20)
17/08/01 14:49:32 INFO cluster.YarnScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool
17/08/01 14:49:32 INFO scheduler.DAGScheduler: ResultStage 1 (start at NativeMethodAccessorImpl.java:0) finished in 0.420 s
17/08/01 14:49:32 INFO scheduler.DAGScheduler: Job 0 finished: start at NativeMethodAccessorImpl.java:0, took 3.286639 s
17/08/01 14:49:32 INFO scheduler.ReceiverTracker: Starting 1 receivers
17/08/01 14:49:32 INFO scheduler.ReceiverTracker: ReceiverTracker started
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO kafka.KafkaInputDStream: Initialized and validated org.apache.spark.streaming.kafka.KafkaInputDStream@71871773
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO python.PythonTransformedDStream: Initialized and validated org.apache.spark.streaming.api.python.PythonTransformedDStream@6593d619
17/08/01 14:49:32 INFO dstream.ForEachDStream: Slide time = 1000 ms
17/08/01 14:49:32 INFO dstream.ForEachDStream: Storage level = Serialized 1x Replicated
17/08/01 14:49:32 INFO dstream.ForEachDStream: Checkpoint interval = null
17/08/01 14:49:32 INFO dstream.ForEachDStream: Remember interval = 1000 ms
17/08/01 14:49:32 INFO dstream.ForEachDStream: Initialized and validated org.apache.spark.streaming.dstream.ForEachDStream@26c8ea4b
17/08/01 14:49:33 INFO util.RecurringTimer: Started timer for JobGenerator at time 1501598973000
17/08/01 14:49:33 INFO scheduler.JobGenerator: Started JobGenerator at 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Started JobScheduler
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@625d3c2d{/streaming,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@48a256f9{/streaming/json,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@39edaf54{/streaming/batch,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@648a544d{/streaming/batch/json,null,AVAILABLE}
17/08/01 14:49:33 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@c773c0d{/static/streaming,null,AVAILABLE}
17/08/01 14:49:33 INFO streaming.StreamingContext: StreamingContext started
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Got job 1 (start at NativeMethodAccessorImpl.java:0) with 1 output partitions
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Final stage: ResultStage 2 (start at NativeMethodAccessorImpl.java:0)
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Missing parents: List()
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Submitting ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620), which has no missing parents
17/08/01 14:49:33 INFO scheduler.ReceiverTracker: Receiver 0 started
17/08/01 14:49:33 INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 80.8 KB, free 1060.4 MB)
17/08/01 14:49:33 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 29.1 KB, free 1060.4 MB)
17/08/01 14:49:33 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.0.0.199:53355 (size: 29.1 KB, free: 1060.5 MB)
17/08/01 14:49:33 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:996
17/08/01 14:49:33 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620)
17/08/01 14:49:33 INFO cluster.YarnScheduler: Adding task set 2.0 with 1 tasks
17/08/01 14:49:33 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 2.0 (TID 70, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 7067 bytes)
17/08/01 14:49:33 INFO scheduler.JobScheduler: Added jobs for time 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Starting job streaming job 1501598973000 ms.0 from job set of time 1501598973000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:33 INFO scheduler.JobScheduler: Finished job streaming job 1501598973000 ms.0 from job set of time 1501598973000 ms
17/08/01 14:49:33 INFO scheduler.JobScheduler: Total delay: 0.102 s for time 1501598973000 ms (execution: 0.026 s)
17/08/01 14:49:33 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 29.1 KB, free: 1060.5 MB)
17/08/01 14:49:33 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/08/01 14:49:33 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/08/01 14:49:33 INFO scheduler.ReceiverTracker: Registered receiver for stream 0 from 10.0.0.229:37096
17/08/01 14:49:34 INFO scheduler.JobScheduler: Added jobs for time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.JobScheduler: Starting job streaming job 1501598974000 ms.0 from job set of time 1501598974000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:34 INFO scheduler.JobScheduler: Finished job streaming job 1501598974000 ms.0 from job set of time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598974000 ms (execution: 0.009 s)
17/08/01 14:49:34 INFO python.PythonRDD: Removing RDD 5 from persistence list
17/08/01 14:49:34 INFO storage.BlockManager: Removing RDD 5
17/08/01 14:49:34 INFO rdd.BlockRDD: Removing RDD 4 from persistence list
17/08/01 14:49:34 INFO storage.BlockManager: Removing RDD 4
17/08/01 14:49:34 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[4] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598974000 ms
17/08/01 14:49:34 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/08/01 14:49:34 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/08/01 14:49:34 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.127:44695) with ID 2
17/08/01 14:49:34 INFO spark.ExecutorAllocationManager: New executor 2 has registered (new total is 2)
17/08/01 14:49:34 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-127.ec2.internal:59267 with 1060.5 MB RAM, BlockManagerId(2, ip-10-0-0-127.ec2.internal, 59267, None)
17/08/01 14:49:35 INFO scheduler.JobScheduler: Added jobs for time 1501598975000 ms
17/08/01 14:49:35 INFO scheduler.JobScheduler: Starting job streaming job 1501598975000 ms.0 from job set of time 1501598975000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:35 INFO scheduler.JobScheduler: Finished job streaming job 1501598975000 ms.0 from job set of time 1501598975000 ms
17/08/01 14:49:35 INFO python.PythonRDD: Removing RDD 7 from persistence list
17/08/01 14:49:35 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501598975000 ms (execution: 0.009 s)
17/08/01 14:49:35 INFO rdd.BlockRDD: Removing RDD 6 from persistence list
17/08/01 14:49:35 INFO storage.BlockManager: Removing RDD 6
17/08/01 14:49:35 INFO storage.BlockManager: Removing RDD 7
17/08/01 14:49:35 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[6] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598975000 ms
17/08/01 14:49:35 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598973000 ms
17/08/01 14:49:35 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598973000 ms
17/08/01 14:49:36 INFO scheduler.JobScheduler: Added jobs for time 1501598976000 ms
17/08/01 14:49:36 INFO scheduler.JobScheduler: Starting job streaming job 1501598976000 ms.0 from job set of time 1501598976000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:36 INFO scheduler.JobScheduler: Finished job streaming job 1501598976000 ms.0 from job set of time 1501598976000 ms
17/08/01 14:49:36 INFO python.PythonRDD: Removing RDD 9 from persistence list
17/08/01 14:49:36 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501598976000 ms (execution: 0.008 s)
17/08/01 14:49:36 INFO storage.BlockManager: Removing RDD 9
17/08/01 14:49:36 INFO rdd.BlockRDD: Removing RDD 8 from persistence list
17/08/01 14:49:36 INFO storage.BlockManager: Removing RDD 8
17/08/01 14:49:36 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[8] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598976000 ms
17/08/01 14:49:36 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598974000 ms
17/08/01 14:49:36 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598974000 ms
17/08/01 14:49:37 INFO scheduler.JobScheduler: Added jobs for time 1501598977000 ms
17/08/01 14:49:37 INFO scheduler.JobScheduler: Starting job streaming job 1501598977000 ms.0 from job set of time 1501598977000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:37 INFO scheduler.JobScheduler: Finished job streaming job 1501598977000 ms.0 from job set of time 1501598977000 ms
17/08/01 14:49:37 INFO python.PythonRDD: Removing RDD 11 from persistence list
17/08/01 14:49:37 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598977000 ms (execution: 0.008 s)
17/08/01 14:49:37 INFO storage.BlockManager: Removing RDD 11
17/08/01 14:49:37 INFO rdd.BlockRDD: Removing RDD 10 from persistence list
17/08/01 14:49:37 INFO storage.BlockManager: Removing RDD 10
17/08/01 14:49:37 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[10] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598977000 ms
17/08/01 14:49:37 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598975000 ms
17/08/01 14:49:37 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598975000 ms
17/08/01 14:49:38 INFO scheduler.JobScheduler: Added jobs for time 1501598978000 ms
17/08/01 14:49:38 INFO scheduler.JobScheduler: Starting job streaming job 1501598978000 ms.0 from job set of time 1501598978000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:38 INFO scheduler.JobScheduler: Finished job streaming job 1501598978000 ms.0 from job set of time 1501598978000 ms
17/08/01 14:49:38 INFO python.PythonRDD: Removing RDD 13 from persistence list
17/08/01 14:49:38 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501598978000 ms (execution: 0.007 s)
17/08/01 14:49:38 INFO storage.BlockManager: Removing RDD 13
17/08/01 14:49:38 INFO rdd.BlockRDD: Removing RDD 12 from persistence list
17/08/01 14:49:38 INFO storage.BlockManager: Removing RDD 12
17/08/01 14:49:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[12] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598978000 ms
17/08/01 14:49:38 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598976000 ms
17/08/01 14:49:38 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598976000 ms
17/08/01 14:49:39 INFO scheduler.JobScheduler: Added jobs for time 1501598979000 ms
17/08/01 14:49:39 INFO scheduler.JobScheduler: Starting job streaming job 1501598979000 ms.0 from job set of time 1501598979000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:39 INFO scheduler.JobScheduler: Finished job streaming job 1501598979000 ms.0 from job set of time 1501598979000 ms
17/08/01 14:49:39 INFO python.PythonRDD: Removing RDD 15 from persistence list
17/08/01 14:49:39 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598979000 ms (execution: 0.007 s)
17/08/01 14:49:39 INFO storage.BlockManager: Removing RDD 15
17/08/01 14:49:39 INFO rdd.BlockRDD: Removing RDD 14 from persistence list
17/08/01 14:49:39 INFO storage.BlockManager: Removing RDD 14
17/08/01 14:49:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[14] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598979000 ms
17/08/01 14:49:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598977000 ms
17/08/01 14:49:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598977000 ms
17/08/01 14:49:40 INFO scheduler.JobScheduler: Added jobs for time 1501598980000 ms
17/08/01 14:49:40 INFO scheduler.JobScheduler: Starting job streaming job 1501598980000 ms.0 from job set of time 1501598980000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:40 INFO scheduler.JobScheduler: Finished job streaming job 1501598980000 ms.0 from job set of time 1501598980000 ms
17/08/01 14:49:40 INFO python.PythonRDD: Removing RDD 17 from persistence list
17/08/01 14:49:40 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598980000 ms (execution: 0.015 s)
17/08/01 14:49:40 INFO storage.BlockManager: Removing RDD 17
17/08/01 14:49:40 INFO rdd.BlockRDD: Removing RDD 16 from persistence list
17/08/01 14:49:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[16] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598980000 ms
17/08/01 14:49:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598978000 ms
17/08/01 14:49:40 INFO storage.BlockManager: Removing RDD 16
17/08/01 14:49:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598978000 ms
17/08/01 14:49:41 INFO scheduler.JobScheduler: Added jobs for time 1501598981000 ms
17/08/01 14:49:41 INFO scheduler.JobScheduler: Starting job streaming job 1501598981000 ms.0 from job set of time 1501598981000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:41 INFO scheduler.JobScheduler: Finished job streaming job 1501598981000 ms.0 from job set of time 1501598981000 ms
17/08/01 14:49:41 INFO python.PythonRDD: Removing RDD 19 from persistence list
17/08/01 14:49:41 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501598981000 ms (execution: 0.006 s)
17/08/01 14:49:41 INFO storage.BlockManager: Removing RDD 19
17/08/01 14:49:41 INFO rdd.BlockRDD: Removing RDD 18 from persistence list
17/08/01 14:49:41 INFO storage.BlockManager: Removing RDD 18
17/08/01 14:49:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[18] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598981000 ms
17/08/01 14:49:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598979000 ms
17/08/01 14:49:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598979000 ms
17/08/01 14:49:42 INFO scheduler.JobScheduler: Added jobs for time 1501598982000 ms
17/08/01 14:49:42 INFO scheduler.JobScheduler: Starting job streaming job 1501598982000 ms.0 from job set of time 1501598982000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:42 INFO scheduler.JobScheduler: Finished job streaming job 1501598982000 ms.0 from job set of time 1501598982000 ms
17/08/01 14:49:42 INFO python.PythonRDD: Removing RDD 21 from persistence list
17/08/01 14:49:42 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598982000 ms (execution: 0.008 s)
17/08/01 14:49:42 INFO storage.BlockManager: Removing RDD 21
17/08/01 14:49:42 INFO rdd.BlockRDD: Removing RDD 20 from persistence list
17/08/01 14:49:42 INFO storage.BlockManager: Removing RDD 20
17/08/01 14:49:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[20] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598982000 ms
17/08/01 14:49:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598980000 ms
17/08/01 14:49:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598980000 ms
17/08/01 14:49:43 INFO scheduler.JobScheduler: Added jobs for time 1501598983000 ms
17/08/01 14:49:43 INFO scheduler.JobScheduler: Starting job streaming job 1501598983000 ms.0 from job set of time 1501598983000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:43 INFO scheduler.JobScheduler: Finished job streaming job 1501598983000 ms.0 from job set of time 1501598983000 ms
17/08/01 14:49:43 INFO python.PythonRDD: Removing RDD 23 from persistence list
17/08/01 14:49:43 INFO scheduler.JobScheduler: Total delay: 0.036 s for time 1501598983000 ms (execution: 0.009 s)
17/08/01 14:49:43 INFO storage.BlockManager: Removing RDD 23
17/08/01 14:49:43 INFO rdd.BlockRDD: Removing RDD 22 from persistence list
17/08/01 14:49:43 INFO storage.BlockManager: Removing RDD 22
17/08/01 14:49:43 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[22] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598983000 ms
17/08/01 14:49:43 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598981000 ms
17/08/01 14:49:43 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598981000 ms
17/08/01 14:49:44 INFO scheduler.JobScheduler: Added jobs for time 1501598984000 ms
17/08/01 14:49:44 INFO scheduler.JobScheduler: Starting job streaming job 1501598984000 ms.0 from job set of time 1501598984000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:44 INFO scheduler.JobScheduler: Finished job streaming job 1501598984000 ms.0 from job set of time 1501598984000 ms
17/08/01 14:49:44 INFO python.PythonRDD: Removing RDD 25 from persistence list
17/08/01 14:49:44 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598984000 ms (execution: 0.007 s)
17/08/01 14:49:44 INFO storage.BlockManager: Removing RDD 25
17/08/01 14:49:44 INFO rdd.BlockRDD: Removing RDD 24 from persistence list
17/08/01 14:49:44 INFO storage.BlockManager: Removing RDD 24
17/08/01 14:49:44 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[24] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598984000 ms
17/08/01 14:49:44 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598982000 ms
17/08/01 14:49:44 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598982000 ms
17/08/01 14:49:45 INFO scheduler.JobScheduler: Added jobs for time 1501598985000 ms
17/08/01 14:49:45 INFO scheduler.JobScheduler: Starting job streaming job 1501598985000 ms.0 from job set of time 1501598985000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:45 INFO scheduler.JobScheduler: Finished job streaming job 1501598985000 ms.0 from job set of time 1501598985000 ms
17/08/01 14:49:45 INFO python.PythonRDD: Removing RDD 27 from persistence list
17/08/01 14:49:45 INFO scheduler.JobScheduler: Total delay: 0.033 s for time 1501598985000 ms (execution: 0.007 s)
17/08/01 14:49:45 INFO storage.BlockManager: Removing RDD 27
17/08/01 14:49:45 INFO rdd.BlockRDD: Removing RDD 26 from persistence list
17/08/01 14:49:45 INFO storage.BlockManager: Removing RDD 26
17/08/01 14:49:45 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[26] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598985000 ms
17/08/01 14:49:45 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598983000 ms
17/08/01 14:49:45 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598983000 ms
17/08/01 14:49:46 INFO scheduler.JobScheduler: Added jobs for time 1501598986000 ms
17/08/01 14:49:46 INFO scheduler.JobScheduler: Starting job streaming job 1501598986000 ms.0 from job set of time 1501598986000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:46 INFO scheduler.JobScheduler: Finished job streaming job 1501598986000 ms.0 from job set of time 1501598986000 ms
17/08/01 14:49:46 INFO python.PythonRDD: Removing RDD 29 from persistence list
17/08/01 14:49:46 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598986000 ms (execution: 0.008 s)
17/08/01 14:49:46 INFO storage.BlockManager: Removing RDD 29
17/08/01 14:49:46 INFO rdd.BlockRDD: Removing RDD 28 from persistence list
17/08/01 14:49:46 INFO storage.BlockManager: Removing RDD 28
17/08/01 14:49:46 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[28] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598986000 ms
17/08/01 14:49:46 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598984000 ms
17/08/01 14:49:46 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598984000 ms
17/08/01 14:49:47 INFO scheduler.JobScheduler: Added jobs for time 1501598987000 ms
17/08/01 14:49:47 INFO scheduler.JobScheduler: Starting job streaming job 1501598987000 ms.0 from job set of time 1501598987000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:47 INFO scheduler.JobScheduler: Finished job streaming job 1501598987000 ms.0 from job set of time 1501598987000 ms
17/08/01 14:49:47 INFO python.PythonRDD: Removing RDD 31 from persistence list
17/08/01 14:49:47 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598987000 ms (execution: 0.014 s)
17/08/01 14:49:47 INFO storage.BlockManager: Removing RDD 31
17/08/01 14:49:47 INFO rdd.BlockRDD: Removing RDD 30 from persistence list
17/08/01 14:49:47 INFO storage.BlockManager: Removing RDD 30
17/08/01 14:49:47 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[30] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598987000 ms
17/08/01 14:49:47 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598985000 ms
17/08/01 14:49:47 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598985000 ms
17/08/01 14:49:48 INFO scheduler.JobScheduler: Added jobs for time 1501598988000 ms
17/08/01 14:49:48 INFO scheduler.JobScheduler: Starting job streaming job 1501598988000 ms.0 from job set of time 1501598988000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:48 INFO scheduler.JobScheduler: Finished job streaming job 1501598988000 ms.0 from job set of time 1501598988000 ms
17/08/01 14:49:48 INFO python.PythonRDD: Removing RDD 33 from persistence list
17/08/01 14:49:48 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501598988000 ms (execution: 0.008 s)
17/08/01 14:49:48 INFO storage.BlockManager: Removing RDD 33
17/08/01 14:49:48 INFO rdd.BlockRDD: Removing RDD 32 from persistence list
17/08/01 14:49:48 INFO storage.BlockManager: Removing RDD 32
17/08/01 14:49:48 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[32] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598988000 ms
17/08/01 14:49:48 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598986000 ms
17/08/01 14:49:48 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598986000 ms
17/08/01 14:49:49 INFO scheduler.JobScheduler: Added jobs for time 1501598989000 ms
17/08/01 14:49:49 INFO scheduler.JobScheduler: Starting job streaming job 1501598989000 ms.0 from job set of time 1501598989000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:49 INFO scheduler.JobScheduler: Finished job streaming job 1501598989000 ms.0 from job set of time 1501598989000 ms
17/08/01 14:49:49 INFO python.PythonRDD: Removing RDD 35 from persistence list
17/08/01 14:49:49 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501598989000 ms (execution: 0.006 s)
17/08/01 14:49:49 INFO storage.BlockManager: Removing RDD 35
17/08/01 14:49:49 INFO rdd.BlockRDD: Removing RDD 34 from persistence list
17/08/01 14:49:49 INFO storage.BlockManager: Removing RDD 34
17/08/01 14:49:49 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[34] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598989000 ms
17/08/01 14:49:49 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598987000 ms
17/08/01 14:49:49 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598987000 ms
17/08/01 14:49:50 INFO scheduler.JobScheduler: Added jobs for time 1501598990000 ms
17/08/01 14:49:50 INFO scheduler.JobScheduler: Starting job streaming job 1501598990000 ms.0 from job set of time 1501598990000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:50 INFO scheduler.JobScheduler: Finished job streaming job 1501598990000 ms.0 from job set of time 1501598990000 ms
17/08/01 14:49:50 INFO python.PythonRDD: Removing RDD 37 from persistence list
17/08/01 14:49:50 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501598990000 ms (execution: 0.008 s)
17/08/01 14:49:50 INFO storage.BlockManager: Removing RDD 37
17/08/01 14:49:50 INFO rdd.BlockRDD: Removing RDD 36 from persistence list
17/08/01 14:49:50 INFO storage.BlockManager: Removing RDD 36
17/08/01 14:49:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[36] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598990000 ms
17/08/01 14:49:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598988000 ms
17/08/01 14:49:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598988000 ms
17/08/01 14:49:51 INFO scheduler.JobScheduler: Added jobs for time 1501598991000 ms
17/08/01 14:49:51 INFO scheduler.JobScheduler: Starting job streaming job 1501598991000 ms.0 from job set of time 1501598991000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:51 INFO scheduler.JobScheduler: Finished job streaming job 1501598991000 ms.0 from job set of time 1501598991000 ms
17/08/01 14:49:51 INFO python.PythonRDD: Removing RDD 39 from persistence list
17/08/01 14:49:51 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501598991000 ms (execution: 0.006 s)
17/08/01 14:49:51 INFO storage.BlockManager: Removing RDD 39
17/08/01 14:49:51 INFO rdd.BlockRDD: Removing RDD 38 from persistence list
17/08/01 14:49:51 INFO storage.BlockManager: Removing RDD 38
17/08/01 14:49:51 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[38] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598991000 ms
17/08/01 14:49:51 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598989000 ms
17/08/01 14:49:51 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598989000 ms
17/08/01 14:49:52 INFO scheduler.JobScheduler: Added jobs for time 1501598992000 ms
17/08/01 14:49:52 INFO scheduler.JobScheduler: Starting job streaming job 1501598992000 ms.0 from job set of time 1501598992000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:52 INFO scheduler.JobScheduler: Finished job streaming job 1501598992000 ms.0 from job set of time 1501598992000 ms
17/08/01 14:49:52 INFO python.PythonRDD: Removing RDD 41 from persistence list
17/08/01 14:49:52 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501598992000 ms (execution: 0.007 s)
17/08/01 14:49:52 INFO storage.BlockManager: Removing RDD 41
17/08/01 14:49:52 INFO rdd.BlockRDD: Removing RDD 40 from persistence list
17/08/01 14:49:52 INFO storage.BlockManager: Removing RDD 40
17/08/01 14:49:52 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[40] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598992000 ms
17/08/01 14:49:52 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598990000 ms
17/08/01 14:49:52 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598990000 ms
17/08/01 14:49:53 INFO scheduler.JobScheduler: Added jobs for time 1501598993000 ms
17/08/01 14:49:53 INFO scheduler.JobScheduler: Starting job streaming job 1501598993000 ms.0 from job set of time 1501598993000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:53 INFO scheduler.JobScheduler: Finished job streaming job 1501598993000 ms.0 from job set of time 1501598993000 ms
17/08/01 14:49:53 INFO python.PythonRDD: Removing RDD 43 from persistence list
17/08/01 14:49:53 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598993000 ms (execution: 0.008 s)
17/08/01 14:49:53 INFO storage.BlockManager: Removing RDD 43
17/08/01 14:49:53 INFO rdd.BlockRDD: Removing RDD 42 from persistence list
17/08/01 14:49:53 INFO storage.BlockManager: Removing RDD 42
17/08/01 14:49:53 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[42] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598993000 ms
17/08/01 14:49:53 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598991000 ms
17/08/01 14:49:53 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598991000 ms
17/08/01 14:49:54 INFO scheduler.JobScheduler: Added jobs for time 1501598994000 ms
17/08/01 14:49:54 INFO scheduler.JobScheduler: Starting job streaming job 1501598994000 ms.0 from job set of time 1501598994000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:54 INFO scheduler.JobScheduler: Finished job streaming job 1501598994000 ms.0 from job set of time 1501598994000 ms
17/08/01 14:49:54 INFO python.PythonRDD: Removing RDD 45 from persistence list
17/08/01 14:49:54 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501598994000 ms (execution: 0.006 s)
17/08/01 14:49:54 INFO storage.BlockManager: Removing RDD 45
17/08/01 14:49:54 INFO rdd.BlockRDD: Removing RDD 44 from persistence list
17/08/01 14:49:54 INFO storage.BlockManager: Removing RDD 44
17/08/01 14:49:54 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[44] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598994000 ms
17/08/01 14:49:54 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598992000 ms
17/08/01 14:49:54 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598992000 ms
17/08/01 14:49:55 INFO scheduler.JobScheduler: Added jobs for time 1501598995000 ms
17/08/01 14:49:55 INFO scheduler.JobScheduler: Starting job streaming job 1501598995000 ms.0 from job set of time 1501598995000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:55 INFO scheduler.JobScheduler: Finished job streaming job 1501598995000 ms.0 from job set of time 1501598995000 ms
17/08/01 14:49:55 INFO python.PythonRDD: Removing RDD 47 from persistence list
17/08/01 14:49:55 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598995000 ms (execution: 0.008 s)
17/08/01 14:49:55 INFO storage.BlockManager: Removing RDD 47
17/08/01 14:49:55 INFO rdd.BlockRDD: Removing RDD 46 from persistence list
17/08/01 14:49:55 INFO storage.BlockManager: Removing RDD 46
17/08/01 14:49:55 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[46] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598995000 ms
17/08/01 14:49:55 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598993000 ms
17/08/01 14:49:55 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598993000 ms
17/08/01 14:49:56 INFO scheduler.JobScheduler: Added jobs for time 1501598996000 ms
17/08/01 14:49:56 INFO scheduler.JobScheduler: Starting job streaming job 1501598996000 ms.0 from job set of time 1501598996000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:56 INFO scheduler.JobScheduler: Finished job streaming job 1501598996000 ms.0 from job set of time 1501598996000 ms
17/08/01 14:49:56 INFO python.PythonRDD: Removing RDD 49 from persistence list
17/08/01 14:49:56 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501598996000 ms (execution: 0.020 s)
17/08/01 14:49:56 INFO storage.BlockManager: Removing RDD 49
17/08/01 14:49:56 INFO rdd.BlockRDD: Removing RDD 48 from persistence list
17/08/01 14:49:56 INFO storage.BlockManager: Removing RDD 48
17/08/01 14:49:56 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[48] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598996000 ms
17/08/01 14:49:56 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598994000 ms
17/08/01 14:49:56 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598994000 ms
17/08/01 14:49:57 INFO scheduler.JobScheduler: Added jobs for time 1501598997000 ms
17/08/01 14:49:57 INFO scheduler.JobScheduler: Starting job streaming job 1501598997000 ms.0 from job set of time 1501598997000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:57 INFO scheduler.JobScheduler: Finished job streaming job 1501598997000 ms.0 from job set of time 1501598997000 ms
17/08/01 14:49:57 INFO python.PythonRDD: Removing RDD 51 from persistence list
17/08/01 14:49:57 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501598997000 ms (execution: 0.007 s)
17/08/01 14:49:57 INFO storage.BlockManager: Removing RDD 51
17/08/01 14:49:57 INFO rdd.BlockRDD: Removing RDD 50 from persistence list
17/08/01 14:49:57 INFO storage.BlockManager: Removing RDD 50
17/08/01 14:49:57 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[50] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598997000 ms
17/08/01 14:49:57 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598995000 ms
17/08/01 14:49:57 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598995000 ms
17/08/01 14:49:58 INFO scheduler.JobScheduler: Added jobs for time 1501598998000 ms
17/08/01 14:49:58 INFO scheduler.JobScheduler: Starting job streaming job 1501598998000 ms.0 from job set of time 1501598998000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:58 INFO scheduler.JobScheduler: Finished job streaming job 1501598998000 ms.0 from job set of time 1501598998000 ms
17/08/01 14:49:58 INFO python.PythonRDD: Removing RDD 53 from persistence list
17/08/01 14:49:58 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501598998000 ms (execution: 0.008 s)
17/08/01 14:49:58 INFO storage.BlockManager: Removing RDD 53
17/08/01 14:49:58 INFO rdd.BlockRDD: Removing RDD 52 from persistence list
17/08/01 14:49:58 INFO storage.BlockManager: Removing RDD 52
17/08/01 14:49:58 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[52] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598998000 ms
17/08/01 14:49:58 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598996000 ms
17/08/01 14:49:58 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598996000 ms
17/08/01 14:49:59 INFO scheduler.JobScheduler: Added jobs for time 1501598999000 ms
17/08/01 14:49:59 INFO scheduler.JobScheduler: Starting job streaming job 1501598999000 ms.0 from job set of time 1501598999000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:49:59 INFO scheduler.JobScheduler: Finished job streaming job 1501598999000 ms.0 from job set of time 1501598999000 ms
17/08/01 14:49:59 INFO python.PythonRDD: Removing RDD 55 from persistence list
17/08/01 14:49:59 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501598999000 ms (execution: 0.007 s)
17/08/01 14:49:59 INFO storage.BlockManager: Removing RDD 55
17/08/01 14:49:59 INFO rdd.BlockRDD: Removing RDD 54 from persistence list
17/08/01 14:49:59 INFO storage.BlockManager: Removing RDD 54
17/08/01 14:49:59 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[54] at createStream at NativeMethodAccessorImpl.java:0 of time 1501598999000 ms
17/08/01 14:49:59 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598997000 ms
17/08/01 14:49:59 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598997000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Added jobs for time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Starting job streaming job 1501599000000 ms.0 from job set of time 1501599000000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:00 INFO scheduler.JobScheduler: Finished job streaming job 1501599000000 ms.0 from job set of time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599000000 ms (execution: 0.006 s)
17/08/01 14:50:00 INFO python.PythonRDD: Removing RDD 57 from persistence list
17/08/01 14:50:00 INFO storage.BlockManager: Removing RDD 57
17/08/01 14:50:00 INFO rdd.BlockRDD: Removing RDD 56 from persistence list
17/08/01 14:50:00 INFO storage.BlockManager: Removing RDD 56
17/08/01 14:50:00 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[56] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599000000 ms
17/08/01 14:50:00 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598998000 ms
17/08/01 14:50:00 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598998000 ms
17/08/01 14:50:01 INFO scheduler.JobScheduler: Added jobs for time 1501599001000 ms
17/08/01 14:50:01 INFO scheduler.JobScheduler: Starting job streaming job 1501599001000 ms.0 from job set of time 1501599001000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:01 INFO scheduler.JobScheduler: Finished job streaming job 1501599001000 ms.0 from job set of time 1501599001000 ms
17/08/01 14:50:01 INFO python.PythonRDD: Removing RDD 59 from persistence list
17/08/01 14:50:01 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599001000 ms (execution: 0.007 s)
17/08/01 14:50:01 INFO storage.BlockManager: Removing RDD 59
17/08/01 14:50:01 INFO rdd.BlockRDD: Removing RDD 58 from persistence list
17/08/01 14:50:01 INFO storage.BlockManager: Removing RDD 58
17/08/01 14:50:01 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[58] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599001000 ms
17/08/01 14:50:01 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501598999000 ms
17/08/01 14:50:01 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501598999000 ms
17/08/01 14:50:02 INFO scheduler.JobScheduler: Added jobs for time 1501599002000 ms
17/08/01 14:50:02 INFO scheduler.JobScheduler: Starting job streaming job 1501599002000 ms.0 from job set of time 1501599002000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:02 INFO scheduler.JobScheduler: Finished job streaming job 1501599002000 ms.0 from job set of time 1501599002000 ms
17/08/01 14:50:02 INFO python.PythonRDD: Removing RDD 61 from persistence list
17/08/01 14:50:02 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599002000 ms (execution: 0.006 s)
17/08/01 14:50:02 INFO storage.BlockManager: Removing RDD 61
17/08/01 14:50:02 INFO rdd.BlockRDD: Removing RDD 60 from persistence list
17/08/01 14:50:02 INFO storage.BlockManager: Removing RDD 60
17/08/01 14:50:02 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[60] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599002000 ms
17/08/01 14:50:02 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599000000 ms
17/08/01 14:50:02 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599000000 ms
17/08/01 14:50:03 INFO scheduler.JobScheduler: Added jobs for time 1501599003000 ms
17/08/01 14:50:03 INFO scheduler.JobScheduler: Starting job streaming job 1501599003000 ms.0 from job set of time 1501599003000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:03 INFO scheduler.JobScheduler: Finished job streaming job 1501599003000 ms.0 from job set of time 1501599003000 ms
17/08/01 14:50:03 INFO python.PythonRDD: Removing RDD 63 from persistence list
17/08/01 14:50:03 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599003000 ms (execution: 0.007 s)
17/08/01 14:50:03 INFO storage.BlockManager: Removing RDD 63
17/08/01 14:50:03 INFO rdd.BlockRDD: Removing RDD 62 from persistence list
17/08/01 14:50:03 INFO storage.BlockManager: Removing RDD 62
17/08/01 14:50:03 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[62] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599003000 ms
17/08/01 14:50:03 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599001000 ms
17/08/01 14:50:03 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599001000 ms
17/08/01 14:50:04 INFO scheduler.JobScheduler: Added jobs for time 1501599004000 ms
17/08/01 14:50:04 INFO scheduler.JobScheduler: Starting job streaming job 1501599004000 ms.0 from job set of time 1501599004000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:04 INFO scheduler.JobScheduler: Finished job streaming job 1501599004000 ms.0 from job set of time 1501599004000 ms
17/08/01 14:50:04 INFO python.PythonRDD: Removing RDD 65 from persistence list
17/08/01 14:50:04 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599004000 ms (execution: 0.006 s)
17/08/01 14:50:04 INFO storage.BlockManager: Removing RDD 65
17/08/01 14:50:04 INFO rdd.BlockRDD: Removing RDD 64 from persistence list
17/08/01 14:50:04 INFO storage.BlockManager: Removing RDD 64
17/08/01 14:50:04 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[64] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599004000 ms
17/08/01 14:50:04 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599002000 ms
17/08/01 14:50:04 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599002000 ms
17/08/01 14:50:05 INFO scheduler.JobScheduler: Added jobs for time 1501599005000 ms
17/08/01 14:50:05 INFO scheduler.JobScheduler: Starting job streaming job 1501599005000 ms.0 from job set of time 1501599005000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:05 INFO scheduler.JobScheduler: Finished job streaming job 1501599005000 ms.0 from job set of time 1501599005000 ms
17/08/01 14:50:05 INFO python.PythonRDD: Removing RDD 67 from persistence list
17/08/01 14:50:05 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501599005000 ms (execution: 0.013 s)
17/08/01 14:50:05 INFO storage.BlockManager: Removing RDD 67
17/08/01 14:50:05 INFO rdd.BlockRDD: Removing RDD 66 from persistence list
17/08/01 14:50:05 INFO storage.BlockManager: Removing RDD 66
17/08/01 14:50:05 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[66] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599005000 ms
17/08/01 14:50:05 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599003000 ms
17/08/01 14:50:05 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599003000 ms
17/08/01 14:50:06 INFO scheduler.JobScheduler: Added jobs for time 1501599006000 ms
17/08/01 14:50:06 INFO scheduler.JobScheduler: Starting job streaming job 1501599006000 ms.0 from job set of time 1501599006000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:06 INFO scheduler.JobScheduler: Finished job streaming job 1501599006000 ms.0 from job set of time 1501599006000 ms
17/08/01 14:50:06 INFO python.PythonRDD: Removing RDD 69 from persistence list
17/08/01 14:50:06 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599006000 ms (execution: 0.008 s)
17/08/01 14:50:06 INFO storage.BlockManager: Removing RDD 69
17/08/01 14:50:06 INFO rdd.BlockRDD: Removing RDD 68 from persistence list
17/08/01 14:50:06 INFO storage.BlockManager: Removing RDD 68
17/08/01 14:50:06 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[68] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599006000 ms
17/08/01 14:50:06 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599004000 ms
17/08/01 14:50:06 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599004000 ms
17/08/01 14:50:07 INFO scheduler.JobScheduler: Added jobs for time 1501599007000 ms
17/08/01 14:50:07 INFO scheduler.JobScheduler: Starting job streaming job 1501599007000 ms.0 from job set of time 1501599007000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:07 INFO scheduler.JobScheduler: Finished job streaming job 1501599007000 ms.0 from job set of time 1501599007000 ms
17/08/01 14:50:07 INFO python.PythonRDD: Removing RDD 71 from persistence list
17/08/01 14:50:07 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599007000 ms (execution: 0.009 s)
17/08/01 14:50:07 INFO rdd.BlockRDD: Removing RDD 70 from persistence list
17/08/01 14:50:07 INFO storage.BlockManager: Removing RDD 71
17/08/01 14:50:07 INFO storage.BlockManager: Removing RDD 70
17/08/01 14:50:07 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[70] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599007000 ms
17/08/01 14:50:07 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599005000 ms
17/08/01 14:50:07 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599005000 ms
17/08/01 14:50:08 INFO scheduler.JobScheduler: Added jobs for time 1501599008000 ms
17/08/01 14:50:08 INFO scheduler.JobScheduler: Starting job streaming job 1501599008000 ms.0 from job set of time 1501599008000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:08 INFO scheduler.JobScheduler: Finished job streaming job 1501599008000 ms.0 from job set of time 1501599008000 ms
17/08/01 14:50:08 INFO python.PythonRDD: Removing RDD 73 from persistence list
17/08/01 14:50:08 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599008000 ms (execution: 0.006 s)
17/08/01 14:50:08 INFO storage.BlockManager: Removing RDD 73
17/08/01 14:50:08 INFO rdd.BlockRDD: Removing RDD 72 from persistence list
17/08/01 14:50:08 INFO storage.BlockManager: Removing RDD 72
17/08/01 14:50:08 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[72] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599008000 ms
17/08/01 14:50:08 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599006000 ms
17/08/01 14:50:08 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599006000 ms
17/08/01 14:50:09 INFO scheduler.JobScheduler: Added jobs for time 1501599009000 ms
17/08/01 14:50:09 INFO scheduler.JobScheduler: Starting job streaming job 1501599009000 ms.0 from job set of time 1501599009000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:09 INFO scheduler.JobScheduler: Finished job streaming job 1501599009000 ms.0 from job set of time 1501599009000 ms
17/08/01 14:50:09 INFO python.PythonRDD: Removing RDD 75 from persistence list
17/08/01 14:50:09 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599009000 ms (execution: 0.005 s)
17/08/01 14:50:09 INFO storage.BlockManager: Removing RDD 75
17/08/01 14:50:09 INFO rdd.BlockRDD: Removing RDD 74 from persistence list
17/08/01 14:50:09 INFO storage.BlockManager: Removing RDD 74
17/08/01 14:50:09 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[74] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599009000 ms
17/08/01 14:50:09 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599007000 ms
17/08/01 14:50:09 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599007000 ms
17/08/01 14:50:10 INFO scheduler.JobScheduler: Added jobs for time 1501599010000 ms
17/08/01 14:50:10 INFO scheduler.JobScheduler: Starting job streaming job 1501599010000 ms.0 from job set of time 1501599010000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:10 INFO scheduler.JobScheduler: Finished job streaming job 1501599010000 ms.0 from job set of time 1501599010000 ms
17/08/01 14:50:10 INFO python.PythonRDD: Removing RDD 77 from persistence list
17/08/01 14:50:10 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599010000 ms (execution: 0.005 s)
17/08/01 14:50:10 INFO rdd.BlockRDD: Removing RDD 76 from persistence list
17/08/01 14:50:10 INFO storage.BlockManager: Removing RDD 76
17/08/01 14:50:10 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[76] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599010000 ms
17/08/01 14:50:10 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599008000 ms
17/08/01 14:50:10 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599008000 ms
17/08/01 14:50:10 INFO storage.BlockManager: Removing RDD 77
17/08/01 14:50:11 INFO scheduler.JobScheduler: Added jobs for time 1501599011000 ms
17/08/01 14:50:11 INFO scheduler.JobScheduler: Starting job streaming job 1501599011000 ms.0 from job set of time 1501599011000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:11 INFO scheduler.JobScheduler: Finished job streaming job 1501599011000 ms.0 from job set of time 1501599011000 ms
17/08/01 14:50:11 INFO python.PythonRDD: Removing RDD 79 from persistence list
17/08/01 14:50:11 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599011000 ms (execution: 0.007 s)
17/08/01 14:50:11 INFO storage.BlockManager: Removing RDD 79
17/08/01 14:50:11 INFO rdd.BlockRDD: Removing RDD 78 from persistence list
17/08/01 14:50:11 INFO storage.BlockManager: Removing RDD 78
17/08/01 14:50:11 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[78] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599011000 ms
17/08/01 14:50:11 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599009000 ms
17/08/01 14:50:11 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599009000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Added jobs for time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Starting job streaming job 1501599012000 ms.0 from job set of time 1501599012000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:12 INFO scheduler.JobScheduler: Finished job streaming job 1501599012000 ms.0 from job set of time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599012000 ms (execution: 0.015 s)
17/08/01 14:50:12 INFO python.PythonRDD: Removing RDD 81 from persistence list
17/08/01 14:50:12 INFO storage.BlockManager: Removing RDD 81
17/08/01 14:50:12 INFO rdd.BlockRDD: Removing RDD 80 from persistence list
17/08/01 14:50:12 INFO storage.BlockManager: Removing RDD 80
17/08/01 14:50:12 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[80] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599012000 ms
17/08/01 14:50:12 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599010000 ms
17/08/01 14:50:12 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599010000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Added jobs for time 1501599013000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Starting job streaming job 1501599013000 ms.0 from job set of time 1501599013000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:13 INFO scheduler.JobScheduler: Finished job streaming job 1501599013000 ms.0 from job set of time 1501599013000 ms
17/08/01 14:50:13 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501599013000 ms (execution: 0.008 s)
17/08/01 14:50:13 INFO python.PythonRDD: Removing RDD 83 from persistence list
17/08/01 14:50:13 INFO storage.BlockManager: Removing RDD 83
17/08/01 14:50:13 INFO rdd.BlockRDD: Removing RDD 82 from persistence list
17/08/01 14:50:13 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[82] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599013000 ms
17/08/01 14:50:13 INFO storage.BlockManager: Removing RDD 82
17/08/01 14:50:13 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599011000 ms
17/08/01 14:50:13 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599011000 ms
17/08/01 14:50:14 INFO scheduler.JobScheduler: Added jobs for time 1501599014000 ms
17/08/01 14:50:14 INFO scheduler.JobScheduler: Starting job streaming job 1501599014000 ms.0 from job set of time 1501599014000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:14 INFO scheduler.JobScheduler: Finished job streaming job 1501599014000 ms.0 from job set of time 1501599014000 ms
17/08/01 14:50:14 INFO python.PythonRDD: Removing RDD 85 from persistence list
17/08/01 14:50:14 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599014000 ms (execution: 0.007 s)
17/08/01 14:50:14 INFO rdd.BlockRDD: Removing RDD 84 from persistence list
17/08/01 14:50:14 INFO storage.BlockManager: Removing RDD 85
17/08/01 14:50:14 INFO storage.BlockManager: Removing RDD 84
17/08/01 14:50:14 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[84] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599014000 ms
17/08/01 14:50:14 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599012000 ms
17/08/01 14:50:14 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599012000 ms
17/08/01 14:50:15 INFO scheduler.JobScheduler: Added jobs for time 1501599015000 ms
17/08/01 14:50:15 INFO scheduler.JobScheduler: Starting job streaming job 1501599015000 ms.0 from job set of time 1501599015000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:15 INFO scheduler.JobScheduler: Finished job streaming job 1501599015000 ms.0 from job set of time 1501599015000 ms
17/08/01 14:50:15 INFO python.PythonRDD: Removing RDD 87 from persistence list
17/08/01 14:50:15 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599015000 ms (execution: 0.007 s)
17/08/01 14:50:15 INFO storage.BlockManager: Removing RDD 87
17/08/01 14:50:15 INFO rdd.BlockRDD: Removing RDD 86 from persistence list
17/08/01 14:50:15 INFO storage.BlockManager: Removing RDD 86
17/08/01 14:50:15 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[86] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599015000 ms
17/08/01 14:50:15 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599013000 ms
17/08/01 14:50:15 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599013000 ms
17/08/01 14:50:16 INFO scheduler.JobScheduler: Added jobs for time 1501599016000 ms
17/08/01 14:50:16 INFO scheduler.JobScheduler: Starting job streaming job 1501599016000 ms.0 from job set of time 1501599016000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:16 INFO scheduler.JobScheduler: Finished job streaming job 1501599016000 ms.0 from job set of time 1501599016000 ms
17/08/01 14:50:16 INFO python.PythonRDD: Removing RDD 89 from persistence list
17/08/01 14:50:16 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501599016000 ms (execution: 0.008 s)
17/08/01 14:50:16 INFO storage.BlockManager: Removing RDD 89
17/08/01 14:50:16 INFO rdd.BlockRDD: Removing RDD 88 from persistence list
17/08/01 14:50:16 INFO storage.BlockManager: Removing RDD 88
17/08/01 14:50:16 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[88] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599016000 ms
17/08/01 14:50:16 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599014000 ms
17/08/01 14:50:16 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599014000 ms
17/08/01 14:50:17 INFO scheduler.JobScheduler: Added jobs for time 1501599017000 ms
17/08/01 14:50:17 INFO scheduler.JobScheduler: Starting job streaming job 1501599017000 ms.0 from job set of time 1501599017000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:17 INFO scheduler.JobScheduler: Finished job streaming job 1501599017000 ms.0 from job set of time 1501599017000 ms
17/08/01 14:50:17 INFO python.PythonRDD: Removing RDD 91 from persistence list
17/08/01 14:50:17 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501599017000 ms (execution: 0.008 s)
17/08/01 14:50:17 INFO storage.BlockManager: Removing RDD 91
17/08/01 14:50:17 INFO rdd.BlockRDD: Removing RDD 90 from persistence list
17/08/01 14:50:17 INFO storage.BlockManager: Removing RDD 90
17/08/01 14:50:17 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[90] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599017000 ms
17/08/01 14:50:17 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599015000 ms
17/08/01 14:50:17 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599015000 ms
17/08/01 14:50:18 INFO scheduler.JobScheduler: Added jobs for time 1501599018000 ms
17/08/01 14:50:18 INFO scheduler.JobScheduler: Starting job streaming job 1501599018000 ms.0 from job set of time 1501599018000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:18 INFO scheduler.JobScheduler: Finished job streaming job 1501599018000 ms.0 from job set of time 1501599018000 ms
17/08/01 14:50:18 INFO python.PythonRDD: Removing RDD 93 from persistence list
17/08/01 14:50:18 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599018000 ms (execution: 0.006 s)
17/08/01 14:50:18 INFO storage.BlockManager: Removing RDD 93
17/08/01 14:50:18 INFO rdd.BlockRDD: Removing RDD 92 from persistence list
17/08/01 14:50:18 INFO storage.BlockManager: Removing RDD 92
17/08/01 14:50:18 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[92] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599018000 ms
17/08/01 14:50:18 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599016000 ms
17/08/01 14:50:18 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599016000 ms
17/08/01 14:50:19 INFO scheduler.JobScheduler: Added jobs for time 1501599019000 ms
17/08/01 14:50:19 INFO scheduler.JobScheduler: Starting job streaming job 1501599019000 ms.0 from job set of time 1501599019000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:19 INFO scheduler.JobScheduler: Finished job streaming job 1501599019000 ms.0 from job set of time 1501599019000 ms
17/08/01 14:50:19 INFO python.PythonRDD: Removing RDD 95 from persistence list
17/08/01 14:50:19 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599019000 ms (execution: 0.015 s)
17/08/01 14:50:19 INFO storage.BlockManager: Removing RDD 95
17/08/01 14:50:19 INFO rdd.BlockRDD: Removing RDD 94 from persistence list
17/08/01 14:50:19 INFO storage.BlockManager: Removing RDD 94
17/08/01 14:50:19 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[94] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599019000 ms
17/08/01 14:50:19 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599017000 ms
17/08/01 14:50:19 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599017000 ms
17/08/01 14:50:20 INFO scheduler.JobScheduler: Added jobs for time 1501599020000 ms
17/08/01 14:50:20 INFO scheduler.JobScheduler: Starting job streaming job 1501599020000 ms.0 from job set of time 1501599020000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:20 INFO scheduler.JobScheduler: Finished job streaming job 1501599020000 ms.0 from job set of time 1501599020000 ms
17/08/01 14:50:20 INFO python.PythonRDD: Removing RDD 97 from persistence list
17/08/01 14:50:20 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599020000 ms (execution: 0.006 s)
17/08/01 14:50:20 INFO storage.BlockManager: Removing RDD 97
17/08/01 14:50:20 INFO rdd.BlockRDD: Removing RDD 96 from persistence list
17/08/01 14:50:20 INFO storage.BlockManager: Removing RDD 96
17/08/01 14:50:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[96] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599020000 ms
17/08/01 14:50:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599018000 ms
17/08/01 14:50:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599018000 ms
17/08/01 14:50:21 INFO scheduler.JobScheduler: Added jobs for time 1501599021000 ms
17/08/01 14:50:21 INFO scheduler.JobScheduler: Starting job streaming job 1501599021000 ms.0 from job set of time 1501599021000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:21 INFO scheduler.JobScheduler: Finished job streaming job 1501599021000 ms.0 from job set of time 1501599021000 ms
17/08/01 14:50:21 INFO python.PythonRDD: Removing RDD 99 from persistence list
17/08/01 14:50:21 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599021000 ms (execution: 0.007 s)
17/08/01 14:50:21 INFO storage.BlockManager: Removing RDD 99
17/08/01 14:50:21 INFO rdd.BlockRDD: Removing RDD 98 from persistence list
17/08/01 14:50:21 INFO storage.BlockManager: Removing RDD 98
17/08/01 14:50:21 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[98] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599021000 ms
17/08/01 14:50:21 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599019000 ms
17/08/01 14:50:21 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599019000 ms
17/08/01 14:50:22 INFO scheduler.JobScheduler: Added jobs for time 1501599022000 ms
17/08/01 14:50:22 INFO scheduler.JobScheduler: Starting job streaming job 1501599022000 ms.0 from job set of time 1501599022000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:22 INFO scheduler.JobScheduler: Finished job streaming job 1501599022000 ms.0 from job set of time 1501599022000 ms
17/08/01 14:50:22 INFO python.PythonRDD: Removing RDD 101 from persistence list
17/08/01 14:50:22 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501599022000 ms (execution: 0.007 s)
17/08/01 14:50:22 INFO storage.BlockManager: Removing RDD 101
17/08/01 14:50:22 INFO rdd.BlockRDD: Removing RDD 100 from persistence list
17/08/01 14:50:22 INFO storage.BlockManager: Removing RDD 100
17/08/01 14:50:22 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[100] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599022000 ms
17/08/01 14:50:22 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599020000 ms
17/08/01 14:50:22 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599020000 ms
17/08/01 14:50:23 INFO scheduler.JobScheduler: Added jobs for time 1501599023000 ms
17/08/01 14:50:23 INFO scheduler.JobScheduler: Starting job streaming job 1501599023000 ms.0 from job set of time 1501599023000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:23 INFO scheduler.JobScheduler: Finished job streaming job 1501599023000 ms.0 from job set of time 1501599023000 ms
17/08/01 14:50:23 INFO python.PythonRDD: Removing RDD 103 from persistence list
17/08/01 14:50:23 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599023000 ms (execution: 0.007 s)
17/08/01 14:50:23 INFO storage.BlockManager: Removing RDD 103
17/08/01 14:50:23 INFO rdd.BlockRDD: Removing RDD 102 from persistence list
17/08/01 14:50:23 INFO storage.BlockManager: Removing RDD 102
17/08/01 14:50:23 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[102] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599023000 ms
17/08/01 14:50:23 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599021000 ms
17/08/01 14:50:23 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599021000 ms
17/08/01 14:50:24 INFO scheduler.JobScheduler: Added jobs for time 1501599024000 ms
17/08/01 14:50:24 INFO scheduler.JobScheduler: Starting job streaming job 1501599024000 ms.0 from job set of time 1501599024000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:24 INFO scheduler.JobScheduler: Finished job streaming job 1501599024000 ms.0 from job set of time 1501599024000 ms
17/08/01 14:50:24 INFO python.PythonRDD: Removing RDD 105 from persistence list
17/08/01 14:50:24 INFO scheduler.JobScheduler: Total delay: 0.035 s for time 1501599024000 ms (execution: 0.007 s)
17/08/01 14:50:24 INFO storage.BlockManager: Removing RDD 105
17/08/01 14:50:24 INFO rdd.BlockRDD: Removing RDD 104 from persistence list
17/08/01 14:50:24 INFO storage.BlockManager: Removing RDD 104
17/08/01 14:50:24 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[104] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599024000 ms
17/08/01 14:50:24 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599022000 ms
17/08/01 14:50:24 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599022000 ms
17/08/01 14:50:25 INFO scheduler.JobScheduler: Added jobs for time 1501599025000 ms
17/08/01 14:50:25 INFO scheduler.JobScheduler: Starting job streaming job 1501599025000 ms.0 from job set of time 1501599025000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:25 INFO scheduler.JobScheduler: Finished job streaming job 1501599025000 ms.0 from job set of time 1501599025000 ms
17/08/01 14:50:25 INFO python.PythonRDD: Removing RDD 107 from persistence list
17/08/01 14:50:25 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599025000 ms (execution: 0.007 s)
17/08/01 14:50:25 INFO storage.BlockManager: Removing RDD 107
17/08/01 14:50:25 INFO rdd.BlockRDD: Removing RDD 106 from persistence list
17/08/01 14:50:25 INFO storage.BlockManager: Removing RDD 106
17/08/01 14:50:25 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[106] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599025000 ms
17/08/01 14:50:25 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599023000 ms
17/08/01 14:50:25 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599023000 ms
17/08/01 14:50:26 INFO scheduler.JobScheduler: Added jobs for time 1501599026000 ms
17/08/01 14:50:26 INFO scheduler.JobScheduler: Starting job streaming job 1501599026000 ms.0 from job set of time 1501599026000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:26 INFO scheduler.JobScheduler: Finished job streaming job 1501599026000 ms.0 from job set of time 1501599026000 ms
17/08/01 14:50:26 INFO python.PythonRDD: Removing RDD 109 from persistence list
17/08/01 14:50:26 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599026000 ms (execution: 0.007 s)
17/08/01 14:50:26 INFO storage.BlockManager: Removing RDD 109
17/08/01 14:50:26 INFO rdd.BlockRDD: Removing RDD 108 from persistence list
17/08/01 14:50:26 INFO storage.BlockManager: Removing RDD 108
17/08/01 14:50:26 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[108] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599026000 ms
17/08/01 14:50:26 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599024000 ms
17/08/01 14:50:26 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599024000 ms
17/08/01 14:50:27 INFO scheduler.JobScheduler: Added jobs for time 1501599027000 ms
17/08/01 14:50:27 INFO scheduler.JobScheduler: Starting job streaming job 1501599027000 ms.0 from job set of time 1501599027000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:27 INFO scheduler.JobScheduler: Finished job streaming job 1501599027000 ms.0 from job set of time 1501599027000 ms
17/08/01 14:50:27 INFO python.PythonRDD: Removing RDD 111 from persistence list
17/08/01 14:50:27 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599027000 ms (execution: 0.007 s)
17/08/01 14:50:27 INFO storage.BlockManager: Removing RDD 111
17/08/01 14:50:27 INFO rdd.BlockRDD: Removing RDD 110 from persistence list
17/08/01 14:50:27 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[110] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599027000 ms
17/08/01 14:50:27 INFO storage.BlockManager: Removing RDD 110
17/08/01 14:50:27 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599025000 ms
17/08/01 14:50:27 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599025000 ms
17/08/01 14:50:28 INFO scheduler.JobScheduler: Added jobs for time 1501599028000 ms
17/08/01 14:50:28 INFO scheduler.JobScheduler: Starting job streaming job 1501599028000 ms.0 from job set of time 1501599028000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:28 INFO scheduler.JobScheduler: Finished job streaming job 1501599028000 ms.0 from job set of time 1501599028000 ms
17/08/01 14:50:28 INFO python.PythonRDD: Removing RDD 113 from persistence list
17/08/01 14:50:28 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501599028000 ms (execution: 0.017 s)
17/08/01 14:50:28 INFO storage.BlockManager: Removing RDD 113
17/08/01 14:50:28 INFO rdd.BlockRDD: Removing RDD 112 from persistence list
17/08/01 14:50:28 INFO storage.BlockManager: Removing RDD 112
17/08/01 14:50:28 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[112] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599028000 ms
17/08/01 14:50:28 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599026000 ms
17/08/01 14:50:28 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599026000 ms
17/08/01 14:50:29 INFO scheduler.JobScheduler: Added jobs for time 1501599029000 ms
17/08/01 14:50:29 INFO scheduler.JobScheduler: Starting job streaming job 1501599029000 ms.0 from job set of time 1501599029000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:29 INFO scheduler.JobScheduler: Finished job streaming job 1501599029000 ms.0 from job set of time 1501599029000 ms
17/08/01 14:50:29 INFO python.PythonRDD: Removing RDD 115 from persistence list
17/08/01 14:50:29 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501599029000 ms (execution: 0.007 s)
17/08/01 14:50:29 INFO storage.BlockManager: Removing RDD 115
17/08/01 14:50:29 INFO rdd.BlockRDD: Removing RDD 114 from persistence list
17/08/01 14:50:29 INFO storage.BlockManager: Removing RDD 114
17/08/01 14:50:29 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[114] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599029000 ms
17/08/01 14:50:29 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599027000 ms
17/08/01 14:50:29 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599027000 ms
17/08/01 14:50:30 INFO scheduler.JobScheduler: Added jobs for time 1501599030000 ms
17/08/01 14:50:30 INFO scheduler.JobScheduler: Starting job streaming job 1501599030000 ms.0 from job set of time 1501599030000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:30 INFO scheduler.JobScheduler: Finished job streaming job 1501599030000 ms.0 from job set of time 1501599030000 ms
17/08/01 14:50:30 INFO python.PythonRDD: Removing RDD 117 from persistence list
17/08/01 14:50:30 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501599030000 ms (execution: 0.007 s)
17/08/01 14:50:30 INFO storage.BlockManager: Removing RDD 117
17/08/01 14:50:30 INFO rdd.BlockRDD: Removing RDD 116 from persistence list
17/08/01 14:50:30 INFO storage.BlockManager: Removing RDD 116
17/08/01 14:50:30 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[116] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599030000 ms
17/08/01 14:50:30 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599028000 ms
17/08/01 14:50:30 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599028000 ms
17/08/01 14:50:31 INFO scheduler.JobScheduler: Added jobs for time 1501599031000 ms
17/08/01 14:50:31 INFO scheduler.JobScheduler: Starting job streaming job 1501599031000 ms.0 from job set of time 1501599031000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:31 INFO scheduler.JobScheduler: Finished job streaming job 1501599031000 ms.0 from job set of time 1501599031000 ms
17/08/01 14:50:31 INFO python.PythonRDD: Removing RDD 119 from persistence list
17/08/01 14:50:31 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599031000 ms (execution: 0.006 s)
17/08/01 14:50:31 INFO rdd.BlockRDD: Removing RDD 118 from persistence list
17/08/01 14:50:31 INFO storage.BlockManager: Removing RDD 119
17/08/01 14:50:31 INFO storage.BlockManager: Removing RDD 118
17/08/01 14:50:31 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[118] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599031000 ms
17/08/01 14:50:31 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599029000 ms
17/08/01 14:50:31 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599029000 ms
17/08/01 14:50:32 INFO scheduler.JobScheduler: Added jobs for time 1501599032000 ms
17/08/01 14:50:32 INFO scheduler.JobScheduler: Starting job streaming job 1501599032000 ms.0 from job set of time 1501599032000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:32 INFO scheduler.JobScheduler: Finished job streaming job 1501599032000 ms.0 from job set of time 1501599032000 ms
17/08/01 14:50:32 INFO python.PythonRDD: Removing RDD 121 from persistence list
17/08/01 14:50:32 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599032000 ms (execution: 0.007 s)
17/08/01 14:50:32 INFO storage.BlockManager: Removing RDD 121
17/08/01 14:50:32 INFO rdd.BlockRDD: Removing RDD 120 from persistence list
17/08/01 14:50:32 INFO storage.BlockManager: Removing RDD 120
17/08/01 14:50:32 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[120] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599032000 ms
17/08/01 14:50:32 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599030000 ms
17/08/01 14:50:32 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599030000 ms
17/08/01 14:50:33 INFO scheduler.JobScheduler: Added jobs for time 1501599033000 ms
17/08/01 14:50:33 INFO scheduler.JobScheduler: Starting job streaming job 1501599033000 ms.0 from job set of time 1501599033000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:33 INFO scheduler.JobScheduler: Finished job streaming job 1501599033000 ms.0 from job set of time 1501599033000 ms
17/08/01 14:50:33 INFO python.PythonRDD: Removing RDD 123 from persistence list
17/08/01 14:50:33 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599033000 ms (execution: 0.005 s)
17/08/01 14:50:33 INFO storage.BlockManager: Removing RDD 123
17/08/01 14:50:33 INFO rdd.BlockRDD: Removing RDD 122 from persistence list
17/08/01 14:50:33 INFO storage.BlockManager: Removing RDD 122
17/08/01 14:50:33 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[122] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599033000 ms
17/08/01 14:50:33 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599031000 ms
17/08/01 14:50:33 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599031000 ms
17/08/01 14:50:34 INFO scheduler.JobScheduler: Added jobs for time 1501599034000 ms
17/08/01 14:50:34 INFO scheduler.JobScheduler: Starting job streaming job 1501599034000 ms.0 from job set of time 1501599034000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:34 INFO scheduler.JobScheduler: Finished job streaming job 1501599034000 ms.0 from job set of time 1501599034000 ms
17/08/01 14:50:34 INFO python.PythonRDD: Removing RDD 125 from persistence list
17/08/01 14:50:34 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599034000 ms (execution: 0.006 s)
17/08/01 14:50:34 INFO storage.BlockManager: Removing RDD 125
17/08/01 14:50:34 INFO rdd.BlockRDD: Removing RDD 124 from persistence list
17/08/01 14:50:34 INFO storage.BlockManager: Removing RDD 124
17/08/01 14:50:34 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[124] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599034000 ms
17/08/01 14:50:34 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599032000 ms
17/08/01 14:50:34 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599032000 ms
17/08/01 14:50:34 INFO spark.ExecutorAllocationManager: Request to remove executorIds: 2
17/08/01 14:50:34 INFO cluster.YarnClientSchedulerBackend: Requesting to kill executor(s) 2
17/08/01 14:50:34 INFO cluster.YarnClientSchedulerBackend: Actual list of executor(s) to be killed is 2
17/08/01 14:50:34 INFO spark.ExecutorAllocationManager: Removing executor 2 because it has been idle for 60 seconds (new desired total will be 1)
17/08/01 14:50:35 INFO scheduler.JobScheduler: Added jobs for time 1501599035000 ms
17/08/01 14:50:35 INFO scheduler.JobScheduler: Starting job streaming job 1501599035000 ms.0 from job set of time 1501599035000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:35 INFO scheduler.JobScheduler: Finished job streaming job 1501599035000 ms.0 from job set of time 1501599035000 ms
17/08/01 14:50:35 INFO python.PythonRDD: Removing RDD 127 from persistence list
17/08/01 14:50:35 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599035000 ms (execution: 0.013 s)
17/08/01 14:50:35 INFO storage.BlockManager: Removing RDD 127
17/08/01 14:50:35 INFO rdd.BlockRDD: Removing RDD 126 from persistence list
17/08/01 14:50:35 INFO storage.BlockManager: Removing RDD 126
17/08/01 14:50:35 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[126] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599035000 ms
17/08/01 14:50:35 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599033000 ms
17/08/01 14:50:35 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599033000 ms
17/08/01 14:50:36 INFO scheduler.JobScheduler: Added jobs for time 1501599036000 ms
17/08/01 14:50:36 INFO scheduler.JobScheduler: Starting job streaming job 1501599036000 ms.0 from job set of time 1501599036000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:36 INFO scheduler.JobScheduler: Finished job streaming job 1501599036000 ms.0 from job set of time 1501599036000 ms
17/08/01 14:50:36 INFO python.PythonRDD: Removing RDD 129 from persistence list
17/08/01 14:50:36 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501599036000 ms (execution: 0.007 s)
17/08/01 14:50:36 INFO storage.BlockManager: Removing RDD 129
17/08/01 14:50:36 INFO rdd.BlockRDD: Removing RDD 128 from persistence list
17/08/01 14:50:36 INFO storage.BlockManager: Removing RDD 128
17/08/01 14:50:36 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[128] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599036000 ms
17/08/01 14:50:36 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599034000 ms
17/08/01 14:50:36 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599034000 ms
17/08/01 14:50:37 INFO scheduler.JobScheduler: Added jobs for time 1501599037000 ms
17/08/01 14:50:37 INFO scheduler.JobScheduler: Starting job streaming job 1501599037000 ms.0 from job set of time 1501599037000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:37 INFO scheduler.JobScheduler: Finished job streaming job 1501599037000 ms.0 from job set of time 1501599037000 ms
17/08/01 14:50:37 INFO python.PythonRDD: Removing RDD 131 from persistence list
17/08/01 14:50:37 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599037000 ms (execution: 0.006 s)
17/08/01 14:50:37 INFO storage.BlockManager: Removing RDD 131
17/08/01 14:50:37 INFO rdd.BlockRDD: Removing RDD 130 from persistence list
17/08/01 14:50:37 INFO storage.BlockManager: Removing RDD 130
17/08/01 14:50:37 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[130] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599037000 ms
17/08/01 14:50:37 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599035000 ms
17/08/01 14:50:37 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599035000 ms
17/08/01 14:50:38 INFO scheduler.JobScheduler: Added jobs for time 1501599038000 ms
17/08/01 14:50:38 INFO scheduler.JobScheduler: Starting job streaming job 1501599038000 ms.0 from job set of time 1501599038000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:38 INFO scheduler.JobScheduler: Finished job streaming job 1501599038000 ms.0 from job set of time 1501599038000 ms
17/08/01 14:50:38 INFO python.PythonRDD: Removing RDD 133 from persistence list
17/08/01 14:50:38 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501599038000 ms (execution: 0.007 s)
17/08/01 14:50:38 INFO storage.BlockManager: Removing RDD 133
17/08/01 14:50:38 INFO rdd.BlockRDD: Removing RDD 132 from persistence list
17/08/01 14:50:38 INFO storage.BlockManager: Removing RDD 132
17/08/01 14:50:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[132] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599038000 ms
17/08/01 14:50:38 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599036000 ms
17/08/01 14:50:38 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599036000 ms
17/08/01 14:50:38 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Disabling executor 2.
17/08/01 14:50:38 INFO scheduler.DAGScheduler: Executor lost: 2 (epoch 1)
17/08/01 14:50:38 INFO storage.BlockManagerMasterEndpoint: Trying to remove executor 2 from BlockManagerMaster.
17/08/01 14:50:38 INFO storage.BlockManagerMasterEndpoint: Removing block manager BlockManagerId(2, ip-10-0-0-127.ec2.internal, 59267, None)
17/08/01 14:50:38 INFO storage.BlockManagerMaster: Removed 2 successfully in removeExecutor
17/08/01 14:50:38 INFO cluster.YarnScheduler: Executor 2 on ip-10-0-0-127.ec2.internal killed by driver.
17/08/01 14:50:38 INFO spark.ExecutorAllocationManager: Existing executor 2 has been removed (new total is 1)
17/08/01 14:50:39 INFO scheduler.JobScheduler: Added jobs for time 1501599039000 ms
17/08/01 14:50:39 INFO scheduler.JobScheduler: Starting job streaming job 1501599039000 ms.0 from job set of time 1501599039000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:39 INFO scheduler.JobScheduler: Finished job streaming job 1501599039000 ms.0 from job set of time 1501599039000 ms
17/08/01 14:50:39 INFO python.PythonRDD: Removing RDD 135 from persistence list
17/08/01 14:50:39 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599039000 ms (execution: 0.006 s)
17/08/01 14:50:39 INFO storage.BlockManager: Removing RDD 135
17/08/01 14:50:39 INFO rdd.BlockRDD: Removing RDD 134 from persistence list
17/08/01 14:50:39 INFO storage.BlockManager: Removing RDD 134
17/08/01 14:50:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[134] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599039000 ms
17/08/01 14:50:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599037000 ms
17/08/01 14:50:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599037000 ms
17/08/01 14:50:40 INFO scheduler.JobScheduler: Added jobs for time 1501599040000 ms
17/08/01 14:50:40 INFO scheduler.JobScheduler: Starting job streaming job 1501599040000 ms.0 from job set of time 1501599040000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:40 INFO scheduler.JobScheduler: Finished job streaming job 1501599040000 ms.0 from job set of time 1501599040000 ms
17/08/01 14:50:40 INFO python.PythonRDD: Removing RDD 137 from persistence list
17/08/01 14:50:40 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501599040000 ms (execution: 0.006 s)
17/08/01 14:50:40 INFO storage.BlockManager: Removing RDD 137
17/08/01 14:50:40 INFO rdd.BlockRDD: Removing RDD 136 from persistence list
17/08/01 14:50:40 INFO storage.BlockManager: Removing RDD 136
17/08/01 14:50:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[136] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599040000 ms
17/08/01 14:50:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599038000 ms
17/08/01 14:50:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599038000 ms
17/08/01 14:50:41 INFO scheduler.JobScheduler: Added jobs for time 1501599041000 ms
17/08/01 14:50:41 INFO scheduler.JobScheduler: Starting job streaming job 1501599041000 ms.0 from job set of time 1501599041000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:41 INFO scheduler.JobScheduler: Finished job streaming job 1501599041000 ms.0 from job set of time 1501599041000 ms
17/08/01 14:50:41 INFO python.PythonRDD: Removing RDD 139 from persistence list
17/08/01 14:50:41 INFO scheduler.JobScheduler: Total delay: 0.020 s for time 1501599041000 ms (execution: 0.006 s)
17/08/01 14:50:41 INFO storage.BlockManager: Removing RDD 139
17/08/01 14:50:41 INFO rdd.BlockRDD: Removing RDD 138 from persistence list
17/08/01 14:50:41 INFO storage.BlockManager: Removing RDD 138
17/08/01 14:50:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[138] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599041000 ms
17/08/01 14:50:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599039000 ms
17/08/01 14:50:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599039000 ms
17/08/01 14:50:42 INFO scheduler.JobScheduler: Added jobs for time 1501599042000 ms
17/08/01 14:50:42 INFO scheduler.JobScheduler: Starting job streaming job 1501599042000 ms.0 from job set of time 1501599042000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-14_49_11------------------------
17/08/01 14:50:42 INFO scheduler.JobScheduler: Finished job streaming job 1501599042000 ms.0 from job set of time 1501599042000 ms
17/08/01 14:50:42 INFO python.PythonRDD: Removing RDD 141 from persistence list
17/08/01 14:50:42 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501599042000 ms (execution: 0.014 s)
17/08/01 14:50:42 INFO storage.BlockManager: Removing RDD 141
17/08/01 14:50:42 INFO rdd.BlockRDD: Removing RDD 140 from persistence list
17/08/01 14:50:42 INFO storage.BlockManager: Removing RDD 140
17/08/01 14:50:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[140] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599042000 ms
17/08/01 14:50:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599040000 ms
17/08/01 14:50:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599040000 ms
17/08/01 14:50:42 INFO storage.BlockManagerInfo: Added input-0-1501599042400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1059.6 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599042800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1057.9 MB)
17/08/01 14:50:43 INFO scheduler.JobScheduler: Added jobs for time 1501599043000 ms
17/08/01 14:50:43 INFO scheduler.JobScheduler: Starting job streaming job 1501599043000 ms.0 from job set of time 1501599043000 ms
17/08/01 14:50:43 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Got job 2 (runJob at PythonRDD.scala:441) with 1 output partitions
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Final stage: ResultStage 3 (runJob at PythonRDD.scala:441)
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Missing parents: List()
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Submitting ResultStage 3 (PythonRDD[146] at RDD at PythonRDD.scala:48), which has no missing parents
17/08/01 14:50:43 INFO memory.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/08/01 14:50:43 INFO memory.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.0.0.199:53355 (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:43 INFO spark.SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:996
17/08/01 14:50:43 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (PythonRDD[146] at RDD at PythonRDD.scala:48)
17/08/01 14:50:43 INFO cluster.YarnScheduler: Adding task set 3.0 with 1 tasks
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.3 KB, free: 1057.0 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1056.2 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.4 KB, free: 1055.3 MB)
17/08/01 14:50:43 INFO storage.BlockManagerInfo: Added input-0-1501599043600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 878.9 KB, free: 1054.5 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599043800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1052.7 MB)
17/08/01 14:50:44 INFO scheduler.JobScheduler: Added jobs for time 1501599044000 ms
17/08/01 14:50:44 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 2)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1051.9 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1051.0 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1050.2 MB)
17/08/01 14:50:44 INFO storage.BlockManagerInfo: Added input-0-1501599044600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.5 KB, free: 1049.3 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599044800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 887.9 KB, free: 1048.4 MB)
17/08/01 14:50:45 INFO scheduler.JobScheduler: Added jobs for time 1501599045000 ms
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1047.6 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1046.7 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1045.9 MB)
17/08/01 14:50:45 INFO storage.BlockManagerInfo: Added input-0-1501599045600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.0 KB, free: 1044.1 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599045800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1043.3 MB)
17/08/01 14:50:46 INFO scheduler.JobScheduler: Added jobs for time 1501599046000 ms
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.6 KB, free: 1042.4 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 880.0 KB, free: 1041.6 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1040.7 MB)
17/08/01 14:50:46 INFO storage.BlockManagerInfo: Added input-0-1501599046600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1039.9 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599046800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1039.0 MB)
17/08/01 14:50:47 INFO scheduler.JobScheduler: Added jobs for time 1501599047000 ms
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1038.1 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 878.9 KB, free: 1037.3 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.4 KB, free: 1035.6 MB)
17/08/01 14:50:47 INFO storage.BlockManagerInfo: Added input-0-1501599047600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1034.7 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599047800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1033.8 MB)
17/08/01 14:50:48 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.127:44785) with ID 3
17/08/01 14:50:48 INFO spark.ExecutorAllocationManager: New executor 3 has registered (new total is 2)
17/08/01 14:50:48 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 3.0 (TID 71, ip-10-0-0-127.ec2.internal, executor 3, partition 0, RACK_LOCAL, 5800 bytes)
17/08/01 14:50:48 INFO scheduler.JobScheduler: Added jobs for time 1501599048000 ms
17/08/01 14:50:48 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-127.ec2.internal:51258 with 1060.5 MB RAM, BlockManagerId(3, ip-10-0-0-127.ec2.internal, 51258, None)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1033.0 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on ip-10-0-0-127.ec2.internal:51258 (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1032.1 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.5 KB, free: 1031.3 MB)
17/08/01 14:50:48 INFO storage.BlockManagerInfo: Added input-0-1501599048600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.1 KB, free: 1030.4 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599048800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1029.5 MB)
17/08/01 14:50:49 INFO scheduler.JobScheduler: Added jobs for time 1501599049000 ms
17/08/01 14:50:49 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 3.0 (TID 71) in 1023 ms on ip-10-0-0-127.ec2.internal (executor 3) (1/1)
17/08/01 14:50:49 INFO cluster.YarnScheduler: Removed TaskSet 3.0, whose tasks have all completed, from pool
17/08/01 14:50:49 INFO scheduler.DAGScheduler: ResultStage 3 (runJob at PythonRDD.scala:441) finished in 5.985 s
17/08/01 14:50:49 INFO scheduler.DAGScheduler: Job 2 finished: runJob at PythonRDD.scala:441, took 6.003911 s
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 1758.0 KB, free: 1027.8 MB)
17/08/01 14:50:49 INFO spark.ContextCleaner: Cleaned shuffle 0
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 10.0.0.199:53355 in memory (size: 1988.0 B, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1988.0 B, free: 1027.8 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on 10.0.0.199:53355 in memory (size: 1956.0 B, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1956.0 B, free: 1027.8 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on 10.0.0.199:53355 in memory (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on ip-10-0-0-127.ec2.internal:51258 in memory (size: 5.3 KB, free: 1060.5 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1027.0 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.3 KB, free: 1026.1 MB)
17/08/01 14:50:49 INFO storage.BlockManagerInfo: Added input-0-1501599049600 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1025.3 MB)
17/08/01 14:50:49 INFO execution.SparkSqlParser: Parsing command: spotdb.proxy
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599049800 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1024.4 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Added jobs for time 1501599050000 ms
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050000 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.2 KB, free: 1023.5 MB)
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: int
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO parser.CatalystSqlParser: Parsing command: string
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050200 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.0 KB, free: 1022.7 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599043000 ms.0 from job set of time 1501599043000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 143 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 7.504 s for time 1501599043000 ms (execution: 7.487 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599044000 ms.0 from job set of time 1501599044000 ms
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 142 from persistence list
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599043000 ms.0
org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 69, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[142] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599043000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599041000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599041000 ms
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 142
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 143
Traceback (most recent call last):
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 148, in <module>
main()
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 77, in main
bluecoat_parse(args.zk,args.topic,args.db,args.db_table,args.num_of_workers,args.batch_size)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 144, in bluecoat_parse
ssc.awaitTermination()
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/context.py", line 206, in awaitTermination
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o50.awaitTermination.
: org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 69, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u'Saving data in the Hive serde table `spotdb`.`proxy` is not supported yet. Please use the insertInto() API as an alternative..;'
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599044000 ms.0 from job set of time 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 6.542 s for time 1501599044000 ms (execution: 0.037 s)
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 145 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599045000 ms.0 from job set of time 1501599045000 ms
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599044000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 145
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 144 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 144
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[144] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599042000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599042000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599042400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1023.5 MB)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599045000 ms.0 from job set of time 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 5.547 s for time 1501599045000 ms (execution: 0.003 s)
17/08/01 14:50:50 INFO streaming.StreamingContext: Invoking stop(stopGracefully=false) from shutdown hook
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599046000 ms.0 from job set of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599046000 ms.0 from job set of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 4.559 s for time 1501599046000 ms (execution: 0.004 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599047000 ms.0 from job set of time 1501599047000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 148 from persistence list
17/08/01 14:50:50 INFO scheduler.ReceiverTracker: Sent stop signal to all 1 receivers
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 148
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599047000 ms.0 from job set of time 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 3.562 s for time 1501599047000 ms (execution: 0.002 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599048000 ms.0 from job set of time 1501599048000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599048000 ms.0 from job set of time 1501599048000 ms
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 147 from persistence list
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 2.571 s for time 1501599048000 ms (execution: 0.001 s)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 147
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[147] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599049000 ms.0 from job set of time 1501599049000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599049000 ms.0 from job set of time 1501599049000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 1.572 s for time 1501599049000 ms (execution: 0.001 s)
17/08/01 14:50:50 INFO scheduler.JobScheduler: Starting job streaming job 1501599050000 ms.0 from job set of time 1501599050000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Finished job streaming job 1501599050000 ms.0 from job set of time 1501599050000 ms
17/08/01 14:50:50 INFO scheduler.JobScheduler: Total delay: 0.574 s for time 1501599050000 ms (execution: 0.001 s)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599045000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599046000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599047000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599048000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599049000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599042800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1025.3 MB)
17/08/01 14:50:50 ERROR scheduler.JobScheduler: Error running job streaming job 1501599050000 ms.0
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.3 KB, free: 1026.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1027.0 MB)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599043000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599043000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 150 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 878.9 KB, free: 1027.8 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.4 KB, free: 1028.7 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 150
17/08/01 14:50:50 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 149 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 149
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[149] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599044000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599044000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 152 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 152
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599043800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1030.4 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1031.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1032.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1033.0 MB)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 151 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 151
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[151] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599045000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599045000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 154 from persistence list
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 153 from persistence list
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[153] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599048000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.5 KB, free: 1033.8 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 153
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 154
17/08/01 14:50:50 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1034.7 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599046000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599046000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 156 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599044800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 887.9 KB, free: 1035.6 MB)
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 155 from persistence list
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Added input-0-1501599050400 in memory on ip-10-0-0-229.ec2.internal:51770 (size: 879.4 KB, free: 1034.7 MB)
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 155
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 156
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[155] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599049000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1035.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1036.4 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599047000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599047000 ms
17/08/01 14:50:50 INFO python.PythonRDD: Removing RDD 158 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 158
17/08/01 14:50:50 INFO rdd.BlockRDD: Removing RDD 157 from persistence list
17/08/01 14:50:50 INFO storage.BlockManager: Removing RDD 157
17/08/01 14:50:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[157] at createStream at NativeMethodAccessorImpl.java:0 of time 1501599050000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.6 KB, free: 1037.3 MB)
17/08/01 14:50:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501599048000 ms
17/08/01 14:50:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501599048000 ms
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1038.1 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 880.0 KB, free: 1039.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599045600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.0 KB, free: 1040.7 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1041.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1042.4 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599046800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1043.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 878.9 KB, free: 1044.2 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.2 KB, free: 1045.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 1758.4 KB, free: 1046.7 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.3 KB, free: 1047.6 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599047800 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1048.5 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048400 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.5 KB, free: 1049.3 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048200 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1050.2 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048000 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.0 KB, free: 1051.0 MB)
17/08/01 14:50:50 INFO storage.BlockManagerInfo: Removed input-0-1501599048600 on ip-10-0-0-229.ec2.internal:51770 in memory (size: 879.1 KB, free: 1051.9 MB)
17/08/01 14:50:51 ERROR scheduler.JobScheduler: Error generating jobs for time 1501599051000 ms
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 ERROR python.PythonDStream$$anon$1: Cannot connect to Python process. It's probably dead. Stopping StreamingContext.
py4j.Py4JException: Cannot obtain a new communication channel
at py4j.CallbackClient.sendCommand(CallbackClient.java:340)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy38.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonTransformedDStream.compute(PythonDStream.scala:246)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/08/01 14:50:51 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 2.0 (TID 70) in 77942 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/1)
17/08/01 14:50:51 INFO cluster.YarnScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool
17/08/01 14:50:51 INFO scheduler.DAGScheduler: ResultStage 2 (start at NativeMethodAccessorImpl.java:0) finished in 77.943 s
17/08/01 14:50:51 INFO scheduler.ReceiverTracker: All of the receivers have deregistered successfully
17/08/01 14:50:51 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
17/08/01 14:50:51 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
17/08/01 14:50:51 INFO util.RecurringTimer: Stopped timer for JobGenerator after time 1501599051000
17/08/01 14:50:51 INFO scheduler.JobGenerator: Stopped JobGenerator
17/08/01 14:50:51 INFO scheduler.JobScheduler: Stopped JobScheduler
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@625d3c2d{/streaming,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@39edaf54{/streaming/batch,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@c773c0d{/static/streaming,null,UNAVAILABLE}
17/08/01 14:50:51 INFO streaming.StreamingContext: StreamingContext stopped successfully
17/08/01 14:50:51 INFO spark.SparkContext: Invoking stop() from shutdown hook
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 WARN streaming.StreamingContext: StreamingContext has already been stopped
17/08/01 14:50:51 INFO server.ServerConnector: Stopped ServerConnector@37681deb{HTTP/1.1}{0.0.0.0:4040}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2aa3e9a6{/stages/stage/kill,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ee30613{/jobs/job/kill,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@15ec1723{/api,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14a7d27d{/,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@e4408f4{/static,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@8a515f3{/executors/threadDump/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@f241039{/executors/threadDump,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@79b9b526{/executors/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5fa63fa{/executors,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@77729e5a{/environment/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@242eb7e{/environment,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@49ddd2ed{/storage/rdd/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@699743b3{/storage/rdd,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@736cc380{/storage/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19cf381c{/storage,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4f734944{/stages/pool/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@b7d4da0{/stages/pool,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ab71bb9{/stages/stage/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5d37ce06{/stages/stage,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@27f55b38{/stages/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1159f15e{/stages,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ff970c2{/jobs/job/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@615b86f5{/jobs/job,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19bb7285{/jobs/json,null,UNAVAILABLE}
17/08/01 14:50:51 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3b9882ec{/jobs,null,UNAVAILABLE}
17/08/01 14:50:51 INFO ui.SparkUI: Stopped Spark web UI at http://10.0.0.199:4040
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors
17/08/01 14:50:51 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
17/08/01 14:50:51 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
services=List(),
started=false)
17/08/01 14:50:51 INFO cluster.YarnClientSchedulerBackend: Stopped
17/08/01 14:50:51 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/08/01 14:50:51 INFO memory.MemoryStore: MemoryStore cleared
17/08/01 14:50:51 INFO storage.BlockManager: BlockManager stopped
17/08/01 14:50:51 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
17/08/01 14:50:51 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/08/01 14:50:51 INFO spark.SparkContext: Successfully stopped SparkContext
17/08/01 14:50:51 INFO util.ShutdownHookManager: Shutdown hook called
17/08/01 14:50:51 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25
17/08/01 14:50:51 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-ef37f80d-b239-4cac-b02d-8eebb4342e25/pyspark-c2380580-feb5-47ef-bc69-7b2c9ba38770
Closing worker...
From: Deon Griessel [mailto:dgriessel@searchtechnologies.com]
Sent: Monday, July 31, 2017 4:53 PM
To: user@spot.apache.org
Subject: Spot Ingest Proxy Error
I am trying to run the spark-ingest for the sample proxy log: Bluecoat_ProxySG_Sample.log
I run bash ./start_ingest_standalone.sh proxy 1
Then add the demo log to my collector_path. The worker dies, seems that it cannot find the hive metastore, probably a spark/hive configuration issue. Also, notice that the ip=unknown-ip-addr.
Would appreciate any help on this.
This is what I’m seeing in the worker screen output:
2017-07-31 18:54:15,855 - SPOT.INGEST.WORKER - INFO - Initializing kafka instance
2017-07-31 18:54:15,855 - SPOT.INGEST.WRK.PROXY - INFO - Creating Spark Job for topic: SPOT-INGEST-proxy-18_54_13
2017-07-31 18:54:15,856 - SPOT.INGEST.WRK.PROXY - INFO - SPOT.Utils: Executing: spark2-submit --master yarn --driver-memory 2325m --num-executors 1 --conf spark.executor.memory=2325m --conf spark.executor.cores=1 --jars /home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar /home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py -zk ip-10-0-0-199.ec2.internal:2181 -t SPOT-INGEST-proxy-18_54_13 -db spotdb -dt proxy -w 1 -bs 1
17/07/31 18:54:17 INFO spark.SparkContext: Running Spark version 2.1.0.cloudera1
17/07/31 18:54:17 WARN spark.SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0
17/07/31 18:54:17 INFO spark.SecurityManager: Changing view acls to: spot-user
17/07/31 18:54:17 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/07/31 18:54:17 INFO spark.SecurityManager: Changing view acls groups to:
17/07/31 18:54:17 INFO spark.SecurityManager: Changing modify acls groups to:
17/07/31 18:54:17 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/07/31 18:54:18 INFO util.Utils: Successfully started service 'sparkDriver' on port 54401.
17/07/31 18:54:18 INFO spark.SparkEnv: Registering MapOutputTracker
17/07/31 18:54:18 INFO spark.SparkEnv: Registering BlockManagerMaster
17/07/31 18:54:18 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
17/07/31 18:54:18 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
17/07/31 18:54:18 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-1db8ad37-9fcc-4799-b9e0-a206b6041d04
17/07/31 18:54:18 INFO memory.MemoryStore: MemoryStore started with capacity 1060.5 MB
17/07/31 18:54:18 INFO spark.SparkEnv: Registering OutputCommitCoordinator
17/07/31 18:54:18 INFO util.log: Logging initialized @2379ms
17/07/31 18:54:18 INFO server.Server: jetty-9.2.z-SNAPSHOT
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1a3b1e79{/jobs,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1a3b1e79%7b/jobs,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1f4da763{/jobs/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1f4da763%7b/jobs/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@232864a3{/jobs/job,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@232864a3%7b/jobs/job,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@30e71b5d{/jobs/job/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@30e71b5d%7b/jobs/job/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14b58fc0{/stages,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14b58fc0%7b/stages,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1bf090df{/stages/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1bf090df%7b/stages/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4eb72ecd{/stages/stage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4eb72ecd%7b/stages/stage,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5c61bd1a{/stages/stage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5c61bd1a%7b/stages/stage/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14c62558{/stages/pool,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@14c62558%7b/stages/pool,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5cbdbf0f{/stages/pool/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5cbdbf0f%7b/stages/pool/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2d4aa15a{/storage,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@2d4aa15a%7b/storage,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ea732f0{/storage/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ea732f0%7b/storage/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@13016b86{/storage/rdd,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@13016b86%7b/storage/rdd,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@423a3429{/storage/rdd/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@423a3429%7b/storage/rdd/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7b3691b6{/environment,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7b3691b6%7b/environment,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3b9882ec{/environment/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3b9882ec%7b/environment/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19bb7285{/executors,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19bb7285%7b/executors,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@615b86f5{/executors/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@615b86f5%7b/executors/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3ff970c2{/executors/threadDump,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3ff970c2%7b/executors/threadDump,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1159f15e{/executors/threadDump/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1159f15e%7b/executors/threadDump/json,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27f55b38{/static,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27f55b38%7b/static,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5d37ce06{/,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5d37ce06%7b/,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7ab71bb9{/api,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7ab71bb9%7b/api,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b7d4da0{/jobs/job/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@b7d4da0%7b/jobs/job/kill,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4f734944{/stages/stage/kill,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@4f734944%7b/stages/stage/kill,null,AVAILABLE%7d>
17/07/31 18:54:18 INFO server.ServerConnector: Started ServerConnector@188e6c7f{HTTP/1.1}{0.0.0.0:4040}<mailto:ServerConnector@188e6c7f%7bHTTP/1.1%7d%7b0.0.0.0:4040%7d>
17/07/31 18:54:18 INFO server.Server: Started @2532ms
17/07/31 18:54:18 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
17/07/31 18:54:18 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.0.0.199:4040
17/07/31 18:54:18 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/07/31 18:54:19 INFO client.RMProxy: Connecting to ResourceManager at ip-10-0-0-199.ec2.internal/10.0.0.199:8032
17/07/31 18:54:19 INFO yarn.Client: Requesting a new application from cluster with 3 NodeManagers
17/07/31 18:54:19 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (2725 MB per container)
17/07/31 18:54:19 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/07/31 18:54:19 INFO yarn.Client: Setting up container launch context for our AM
17/07/31 18:54:19 INFO yarn.Client: Setting up the launch environment for our AM container
17/07/31 18:54:19 INFO yarn.Client: Preparing resources for our AM container
17/07/31 18:54:21 INFO yarn.Client: Uploading resource file:/home/spot-user/spot-ingest/common/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501523941584_0005/spark-streaming-kafka-0-8-assembly_2.11-2.0.0.jar
17/07/31 18:54:21 INFO yarn.Client: Uploading resource file:/tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40/__spark_conf__5838957838871043110.zip -> hdfs://ip-10-0-0-199.ec2.internal:8020/user/spot-user/.sparkStaging/application_1501523941584_0005/__spark_conf__.zip
17/07/31 18:54:21 INFO spark.SecurityManager: Changing view acls to: spot-user
17/07/31 18:54:21 INFO spark.SecurityManager: Changing modify acls to: spot-user
17/07/31 18:54:21 INFO spark.SecurityManager: Changing view acls groups to:
17/07/31 18:54:21 INFO spark.SecurityManager: Changing modify acls groups to:
17/07/31 18:54:21 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spot-user); groups with view permissions: Set(); users with modify permissions: Set(spot-user); groups with modify permissions: Set()
17/07/31 18:54:21 INFO yarn.Client: Submitting application application_1501523941584_0005 to ResourceManager
17/07/31 18:54:21 INFO impl.YarnClientImpl: Submitted application application_1501523941584_0005
17/07/31 18:54:21 INFO cluster.SchedulerExtensionServices: Starting Yarn extension services with app application_1501523941584_0005 and attemptId None
17/07/31 18:54:22 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:22 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.users.spot-user
start time: 1501527261635
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005/
user: spot-user
17/07/31 18:54:23 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:24 INFO yarn.Client: Application report for application_1501523941584_0005 (state: ACCEPTED)
17/07/31 18:54:25 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/07/31 18:54:25 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> ip-10-0-0-199.ec2.internal, PROXY_URI_BASES -> http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005), /proxy/application_1501523941584_0005
17/07/31 18:54:25 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/07/31 18:54:25 INFO yarn.Client: Application report for application_1501523941584_0005 (state: RUNNING)
17/07/31 18:54:25 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 10.0.0.127
ApplicationMaster RPC port: 0
queue: root.users.spot-user
start time: 1501527261635
final status: UNDEFINED
tracking URL: http://ip-10-0-0-199.ec2.internal:8088/proxy/application_1501523941584_0005/
user: spot-user
17/07/31 18:54:25 INFO cluster.YarnClientSchedulerBackend: Application application_1501523941584_0005 has started running.
17/07/31 18:54:25 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 59442.
17/07/31 18:54:25 INFO netty.NettyBlockTransferService: Server created on 10.0.0.199:59442
17/07/31 18:54:25 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/07/31 18:54:25 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManagerMasterEndpoint: Registering block manager 10.0.0.199:59442 with 1060.5 MB RAM, BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO storage.BlockManager: external shuffle service port = 7337
17/07/31 18:54:25 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 10.0.0.199, 59442, None)
17/07/31 18:54:25 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3e93262b{/metrics/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@3e93262b%7b/metrics/json,null,AVAILABLE%7d>
17/07/31 18:54:25 INFO scheduler.EventLoggingListener: Logging events to hdfs://ip-10-0-0-199.ec2.internal:8020/user/spark/spark2ApplicationHistory/application_1501523941584_0005
17/07/31 18:54:25 INFO util.Utils: Using initial executors = 1, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
17/07/31 18:54:28 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.229:51685) with ID 1
17/07/31 18:54:28 INFO spark.ExecutorAllocationManager: New executor 1 has registered (new total is 1)
17/07/31 18:54:28 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-229.ec2.internal:35659 with 1060.5 MB RAM, BlockManagerId(1, ip-10-0-0-229.ec2.internal, 35659, None)
17/07/31 18:54:28 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/07/31 18:54:29 INFO internal.SharedState: Warehouse path is 'file:/home/spot-user/spot-ingest/spark-warehouse'.
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7a9c6710{/SQL,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@7a9c6710%7b/SQL,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@141768e4{/SQL/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@141768e4%7b/SQL/json,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1e367d45{/SQL/execution,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@1e367d45%7b/SQL/execution,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5554f0a0{/SQL/execution/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@5554f0a0%7b/SQL/execution/json,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@144776d3{/static/sql,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@144776d3%7b/static/sql,null,AVAILABLE%7d>
17/07/31 18:54:29 INFO hive.HiveUtils: Initializing HiveMetastoreConnection version 1.1.0 using Spark classes.
17/07/31 18:54:30 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
17/07/31 18:54:30 INFO metastore.ObjectStore: ObjectStore, initialize called
17/07/31 18:54:30 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
17/07/31 18:54:30 INFO DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
17/07/31 18:54:30 WARN bonecp.BoneCPConfig: Max Connections < 1. Setting to 20
17/07/31 18:54:31 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
17/07/31 18:54:32 WARN bonecp.BoneCPConfig: Max Connections < 1. Setting to 20
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:32 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0<ma...@0>" since the connection used is closing
17/07/31 18:54:32 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
17/07/31 18:54:32 INFO metastore.ObjectStore: Initialized ObjectStore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: Added admin role in metastore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: Added public role in metastore
17/07/31 18:54:33 INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_all_functions from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_all_functions
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_all_functions
17/07/31 18:54:33 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table.
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_all_functions start=1501527273291 end=1501527273356 duration=65 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=0 error=false>
17/07/31 18:54:33 INFO session.SessionState: Created local directory: /tmp/aaecce13-256b-4b91-b661-9046c9c418c6_resources
17/07/31 18:54:33 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6
17/07/31 18:54:33 INFO session.SessionState: Created local directory: /tmp/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6
17/07/31 18:54:33 INFO session.SessionState: Created HDFS directory: /tmp/hive/spot-user/aaecce13-256b-4b91-b661-9046c9c418c6/_tmp_space.db
17/07/31 18:54:33 INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
17/07/31 18:54:33 INFO client.HiveClientImpl: Warehouse location for Hive client (version 1.1.0) is file:/home/spot-user/spot-ingest/spark-warehouse
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_database: default
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: default
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527273406 end=1501527273408 duration=2 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=0 error=false>
17/07/31 18:54:33 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:54:33 INFO metastore.HiveMetaStore: 0: get_database: global_temp
17/07/31 18:54:33 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: global_temp
17/07/31 18:54:33 WARN metastore.ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
17/07/31 18:54:33 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527273409 end=1501527273412 duration=3 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0 retryCount=-1 error=true>
17/07/31 18:54:33 WARN streaming.StreamingContext: Dynamic Allocation is enabled for this application. Enabling Dynamic allocation for Spark Streaming applications can cause data loss if Write Ahead Log is not enabled for non-replayable sources like Flume. See the programming guide for details on how to enable the Write Ahead Log.
17/07/31 18:54:33 INFO spark.SparkContext: Starting job: start at NativeMethodAccessorImpl.java:0
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Registering RDD 1 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Got job 0 (start at NativeMethodAccessorImpl.java:0) with 20 output partitions
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/07/31 18:54:33 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 3.1 KB, free 1060.5 MB)
17/07/31 18:54:33 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 1988.0 B, free 1060.5 MB)
17/07/31 18:54:33 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.0.0.199:59442 (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:54:33 INFO spark.SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:33 INFO scheduler.DAGScheduler: Submitting 50 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[1] at start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:33 INFO cluster.YarnScheduler: Adding task set 0.0 with 50 tasks
17/07/31 18:54:33 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, ip-10-0-0-229.ec2.internal, executor 1, partition 1, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 669 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, ip-10-0-0-229.ec2.internal, executor 1, partition 2, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 77 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3, ip-10-0-0-229.ec2.internal, executor 1, partition 3, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 72 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 0.0 (TID 4, ip-10-0-0-229.ec2.internal, executor 1, partition 4, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 69 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 0.0 (TID 5, ip-10-0-0-229.ec2.internal, executor 1, partition 5, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 59 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 0.0 (TID 6, ip-10-0-0-229.ec2.internal, executor 1, partition 6, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 0.0 (TID 7, ip-10-0-0-229.ec2.internal, executor 1, partition 7, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 0.0 (TID 8, ip-10-0-0-229.ec2.internal, executor 1, partition 8, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 0.0 (TID 7) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/50)
17/07/31 18:54:34 INFO spark.ExecutorAllocationManager: Requesting 1 new executor because tasks are backlogged (new desired total will be 2)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 0.0 (TID 9, ip-10-0-0-229.ec2.internal, executor 1, partition 9, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 0.0 (TID 8) in 58 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/50)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 0.0 (TID 10, ip-10-0-0-229.ec2.internal, executor 1, partition 10, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:34 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 0.0 (TID 9) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 0.0 (TID 11, ip-10-0-0-229.ec2.internal, executor 1, partition 11, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 0.0 (TID 10) in 52 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 0.0 (TID 12, ip-10-0-0-229.ec2.internal, executor 1, partition 12, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 0.0 (TID 11) in 58 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 0.0 (TID 13, ip-10-0-0-229.ec2.internal, executor 1, partition 13, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 0.0 (TID 12) in 51 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 0.0 (TID 14, ip-10-0-0-229.ec2.internal, executor 1, partition 14, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 0.0 (TID 13) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 0.0 (TID 15, ip-10-0-0-229.ec2.internal, executor 1, partition 15, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 0.0 (TID 14) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 0.0 (TID 16, ip-10-0-0-229.ec2.internal, executor 1, partition 16, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 0.0 (TID 15) in 63 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 0.0 (TID 17, ip-10-0-0-229.ec2.internal, executor 1, partition 17, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 0.0 (TID 16) in 50 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 0.0 (TID 18, ip-10-0-0-229.ec2.internal, executor 1, partition 18, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 0.0 (TID 17) in 55 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 0.0 (TID 19, ip-10-0-0-229.ec2.internal, executor 1, partition 19, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 0.0 (TID 18) in 57 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 0.0 (TID 20, ip-10-0-0-229.ec2.internal, executor 1, partition 20, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 0.0 (TID 19) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 0.0 (TID 21, ip-10-0-0-229.ec2.internal, executor 1, partition 21, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 0.0 (TID 20) in 53 ms on ip-10-0-0-229.ec2.internal (executor 1) (21/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 0.0 (TID 22, ip-10-0-0-229.ec2.internal, executor 1, partition 22, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 0.0 (TID 21) in 54 ms on ip-10-0-0-229.ec2.internal (executor 1) (22/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 0.0 (TID 23, ip-10-0-0-229.ec2.internal, executor 1, partition 23, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 0.0 (TID 22) in 47 ms on ip-10-0-0-229.ec2.internal (executor 1) (23/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 0.0 (TID 24, ip-10-0-0-229.ec2.internal, executor 1, partition 24, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 0.0 (TID 23) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (24/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 0.0 (TID 25, ip-10-0-0-229.ec2.internal, executor 1, partition 25, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 0.0 (TID 24) in 46 ms on ip-10-0-0-229.ec2.internal (executor 1) (25/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 0.0 (TID 26, ip-10-0-0-229.ec2.internal, executor 1, partition 26, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 0.0 (TID 25) in 49 ms on ip-10-0-0-229.ec2.internal (executor 1) (26/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 0.0 (TID 27, ip-10-0-0-229.ec2.internal, executor 1, partition 27, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 0.0 (TID 26) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (27/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 0.0 (TID 28, ip-10-0-0-229.ec2.internal, executor 1, partition 28, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 0.0 (TID 27) in 40 ms on ip-10-0-0-229.ec2.internal (executor 1) (28/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 0.0 (TID 29, ip-10-0-0-229.ec2.internal, executor 1, partition 29, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 0.0 (TID 28) in 39 ms on ip-10-0-0-229.ec2.internal (executor 1) (29/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 0.0 (TID 30, ip-10-0-0-229.ec2.internal, executor 1, partition 30, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 0.0 (TID 29) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (30/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 0.0 (TID 31, ip-10-0-0-229.ec2.internal, executor 1, partition 31, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 0.0 (TID 30) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (31/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 0.0 (TID 32, ip-10-0-0-229.ec2.internal, executor 1, partition 32, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 0.0 (TID 31) in 38 ms on ip-10-0-0-229.ec2.internal (executor 1) (32/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 0.0 (TID 33, ip-10-0-0-229.ec2.internal, executor 1, partition 33, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 0.0 (TID 32) in 43 ms on ip-10-0-0-229.ec2.internal (executor 1) (33/50)
17/07/31 18:54:35 INFO spark.ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 4)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 0.0 (TID 34, ip-10-0-0-229.ec2.internal, executor 1, partition 34, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 0.0 (TID 33) in 41 ms on ip-10-0-0-229.ec2.internal (executor 1) (34/50)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 0.0 (TID 35, ip-10-0-0-229.ec2.internal, executor 1, partition 35, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:35 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 0.0 (TID 34) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (35/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 0.0 (TID 36, ip-10-0-0-229.ec2.internal, executor 1, partition 36, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 0.0 (TID 35) in 45 ms on ip-10-0-0-229.ec2.internal (executor 1) (36/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 0.0 (TID 37, ip-10-0-0-229.ec2.internal, executor 1, partition 37, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 0.0 (TID 36) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (37/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 0.0 (TID 38, ip-10-0-0-229.ec2.internal, executor 1, partition 38, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 0.0 (TID 37) in 37 ms on ip-10-0-0-229.ec2.internal (executor 1) (38/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 0.0 (TID 39, ip-10-0-0-229.ec2.internal, executor 1, partition 39, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 0.0 (TID 38) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (39/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 0.0 (TID 40, ip-10-0-0-229.ec2.internal, executor 1, partition 40, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 0.0 (TID 39) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (40/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 0.0 (TID 41, ip-10-0-0-229.ec2.internal, executor 1, partition 41, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 0.0 (TID 40) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (41/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 0.0 (TID 42, ip-10-0-0-229.ec2.internal, executor 1, partition 42, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 0.0 (TID 41) in 35 ms on ip-10-0-0-229.ec2.internal (executor 1) (42/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 0.0 (TID 43, ip-10-0-0-229.ec2.internal, executor 1, partition 43, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 0.0 (TID 42) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (43/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 0.0 (TID 44, ip-10-0-0-229.ec2.internal, executor 1, partition 44, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 0.0 (TID 43) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (44/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 0.0 (TID 45, ip-10-0-0-229.ec2.internal, executor 1, partition 45, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 0.0 (TID 44) in 31 ms on ip-10-0-0-229.ec2.internal (executor 1) (45/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 0.0 (TID 46, ip-10-0-0-229.ec2.internal, executor 1, partition 46, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 0.0 (TID 45) in 36 ms on ip-10-0-0-229.ec2.internal (executor 1) (46/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 0.0 (TID 47, ip-10-0-0-229.ec2.internal, executor 1, partition 47, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 0.0 (TID 46) in 34 ms on ip-10-0-0-229.ec2.internal (executor 1) (47/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 0.0 (TID 48, ip-10-0-0-229.ec2.internal, executor 1, partition 48, PROCESS_LOCAL, 6709 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 0.0 (TID 47) in 44 ms on ip-10-0-0-229.ec2.internal (executor 1) (48/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 0.0 (TID 49, ip-10-0-0-229.ec2.internal, executor 1, partition 49, PROCESS_LOCAL, 6676 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 0.0 (TID 48) in 29 ms on ip-10-0-0-229.ec2.internal (executor 1) (49/50)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 0.0 (TID 49) in 27 ms on ip-10-0-0-229.ec2.internal (executor 1) (50/50)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (start at NativeMethodAccessorImpl.java:0) finished in 2.499 s
17/07/31 18:54:36 INFO cluster.YarnScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/07/31 18:54:36 INFO scheduler.DAGScheduler: looking for newly runnable stages
17/07/31 18:54:36 INFO scheduler.DAGScheduler: running: Set()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 1)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: failed: Set()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0), which has no missing parents
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.2 KB, free 1060.5 MB)
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 1956.0 B, free 1060.5 MB)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.0.0.199:59442 (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting 20 missing tasks from ResultStage 1 (ShuffledRDD[2] at start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Adding task set 1.0 with 20 tasks
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 50, ip-10-0-0-229.ec2.internal, executor 1, partition 0, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.0.0.229:51685
17/07/31 18:54:36 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 283 bytes
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.0 (TID 51, ip-10-0-0-229.ec2.internal, executor 1, partition 1, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 50) in 91 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 1.0 (TID 52, ip-10-0-0-229.ec2.internal, executor 1, partition 2, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 1.0 (TID 51) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (2/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 1.0 (TID 53, ip-10-0-0-229.ec2.internal, executor 1, partition 3, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 1.0 (TID 52) in 20 ms on ip-10-0-0-229.ec2.internal (executor 1) (3/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 1.0 (TID 54, ip-10-0-0-229.ec2.internal, executor 1, partition 4, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 1.0 (TID 53) in 24 ms on ip-10-0-0-229.ec2.internal (executor 1) (4/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 1.0 (TID 55, ip-10-0-0-229.ec2.internal, executor 1, partition 5, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 1.0 (TID 54) in 24 ms on ip-10-0-0-229.ec2.internal (executor 1) (5/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 1.0 (TID 56, ip-10-0-0-229.ec2.internal, executor 1, partition 6, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 1.0 (TID 55) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (6/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 1.0 (TID 57, ip-10-0-0-229.ec2.internal, executor 1, partition 7, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 1.0 (TID 56) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (7/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 1.0 (TID 58, ip-10-0-0-229.ec2.internal, executor 1, partition 8, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 1.0 (TID 57) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (8/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 1.0 (TID 59, ip-10-0-0-229.ec2.internal, executor 1, partition 9, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 1.0 (TID 58) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (9/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 1.0 (TID 60, ip-10-0-0-229.ec2.internal, executor 1, partition 10, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 1.0 (TID 59) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (10/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 1.0 (TID 61, ip-10-0-0-229.ec2.internal, executor 1, partition 11, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 1.0 (TID 60) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (11/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 1.0 (TID 62, ip-10-0-0-229.ec2.internal, executor 1, partition 12, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 1.0 (TID 61) in 16 ms on ip-10-0-0-229.ec2.internal (executor 1) (12/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 1.0 (TID 63, ip-10-0-0-229.ec2.internal, executor 1, partition 13, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 1.0 (TID 62) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (13/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 1.0 (TID 64, ip-10-0-0-229.ec2.internal, executor 1, partition 14, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 1.0 (TID 63) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (14/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 1.0 (TID 65, ip-10-0-0-229.ec2.internal, executor 1, partition 15, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 1.0 (TID 64) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (15/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 1.0 (TID 66, ip-10-0-0-229.ec2.internal, executor 1, partition 16, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 1.0 (TID 65) in 18 ms on ip-10-0-0-229.ec2.internal (executor 1) (16/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 1.0 (TID 67, ip-10-0-0-229.ec2.internal, executor 1, partition 17, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 1.0 (TID 66) in 17 ms on ip-10-0-0-229.ec2.internal (executor 1) (17/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 1.0 (TID 68, ip-10-0-0-229.ec2.internal, executor 1, partition 18, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 1.0 (TID 67) in 19 ms on ip-10-0-0-229.ec2.internal (executor 1) (18/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 1.0 (TID 69, ip-10-0-0-229.ec2.internal, executor 1, partition 19, NODE_LOCAL, 6628 bytes)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 1.0 (TID 68) in 14 ms on ip-10-0-0-229.ec2.internal (executor 1) (19/20)
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 1.0 (TID 69) in 15 ms on ip-10-0-0-229.ec2.internal (executor 1) (20/20)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool
17/07/31 18:54:36 INFO scheduler.DAGScheduler: ResultStage 1 (start at NativeMethodAccessorImpl.java:0) finished in 0.416 s
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Job 0 finished: start at NativeMethodAccessorImpl.java:0, took 3.184418 s
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: Starting 1 receivers
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: ReceiverTracker started
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO kafka.KafkaInputDStream: Initialized and validated org.apache.spark.streaming.kafka.KafkaInputDStream@7b67308f<ma...@7b67308f>
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO python.PythonTransformedDStream: Initialized and validated org.apache.spark.streaming.api.python.PythonTransformedDStream@33a53b94<ma...@33a53b94>
17/07/31 18:54:36 INFO dstream.ForEachDStream: Slide time = 1000 ms
17/07/31 18:54:36 INFO dstream.ForEachDStream: Storage level = Serialized 1x Replicated
17/07/31 18:54:36 INFO dstream.ForEachDStream: Checkpoint interval = null
17/07/31 18:54:36 INFO dstream.ForEachDStream: Remember interval = 1000 ms
17/07/31 18:54:36 INFO dstream.ForEachDStream: Initialized and validated org.apache.spark.streaming.dstream.ForEachDStream@24ea8c09<ma...@24ea8c09>
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Got job 1 (start at NativeMethodAccessorImpl.java:0) with 1 output partitions
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Final stage: ResultStage 2 (start at NativeMethodAccessorImpl.java:0)
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620), which has no missing parents
17/07/31 18:54:36 INFO scheduler.ReceiverTracker: Receiver 0 started
17/07/31 18:54:36 INFO util.RecurringTimer: Started timer for JobGenerator at time 1501527277000
17/07/31 18:54:36 INFO scheduler.JobGenerator: Started JobGenerator at 1501527277000 ms
17/07/31 18:54:36 INFO scheduler.JobScheduler: Started JobScheduler
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@19fd4723{/streaming,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@19fd4723%7b/streaming,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@43b0e42{/streaming/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@43b0e42%7b/streaming/json,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@22116fc3{/streaming/batch,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@22116fc3%7b/streaming/batch,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27399762{/streaming/batch/json,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@27399762%7b/streaming/batch/json,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@70d301d6{/static/streaming,null,AVAILABLE}<mailto:o.s.j.s.ServletContextHandler@70d301d6%7b/static/streaming,null,AVAILABLE%7d>
17/07/31 18:54:36 INFO streaming.StreamingContext: StreamingContext started
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 74.4 KB, free 1060.4 MB)
17/07/31 18:54:36 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 27.3 KB, free 1060.4 MB)
17/07/31 18:54:36 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.0.0.199:59442 (size: 27.3 KB, free: 1060.5 MB)
17/07/31 18:54:36 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:996
17/07/31 18:54:36 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (Receiver 0 ParallelCollectionRDD[3] at makeRDD at ReceiverTracker.scala:620)
17/07/31 18:54:36 INFO cluster.YarnScheduler: Adding task set 2.0 with 1 tasks
17/07/31 18:54:36 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 2.0 (TID 70, ip-10-0-0-229.ec2.internal, executor 1, partition 0, PROCESS_LOCAL, 7067 bytes)
17/07/31 18:54:37 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 27.3 KB, free: 1060.5 MB)
17/07/31 18:54:37 INFO scheduler.JobScheduler: Added jobs for time 1501527277000 ms
17/07/31 18:54:37 INFO scheduler.JobScheduler: Starting job streaming job 1501527277000 ms.0 from job set of time 1501527277000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:37 INFO scheduler.JobScheduler: Finished job streaming job 1501527277000 ms.0 from job set of time 1501527277000 ms
17/07/31 18:54:37 INFO scheduler.JobScheduler: Total delay: 0.071 s for time 1501527277000 ms (execution: 0.014 s)
17/07/31 18:54:37 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/07/31 18:54:37 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/07/31 18:54:37 INFO scheduler.ReceiverTracker: Registered receiver for stream 0 from 10.0.0.229:51685
17/07/31 18:54:38 INFO scheduler.JobScheduler: Added jobs for time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.JobScheduler: Starting job streaming job 1501527278000 ms.0 from job set of time 1501527278000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:38 INFO scheduler.JobScheduler: Finished job streaming job 1501527278000 ms.0 from job set of time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.JobScheduler: Total delay: 0.045 s for time 1501527278000 ms (execution: 0.010 s)
17/07/31 18:54:38 INFO python.PythonRDD: Removing RDD 5 from persistence list
17/07/31 18:54:38 INFO rdd.BlockRDD: Removing RDD 4 from persistence list
17/07/31 18:54:38 INFO storage.BlockManager: Removing RDD 5
17/07/31 18:54:38 INFO storage.BlockManager: Removing RDD 4
17/07/31 18:54:38 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[4] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527278000 ms
17/07/31 18:54:38 INFO scheduler.ReceivedBlockTracker: Deleting batches:
17/07/31 18:54:38 INFO scheduler.InputInfoTracker: remove old batch metadata:
17/07/31 18:54:38 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (10.0.0.151:57862) with ID 2
17/07/31 18:54:38 INFO spark.ExecutorAllocationManager: New executor 2 has registered (new total is 2)
17/07/31 18:54:38 INFO storage.BlockManagerMasterEndpoint: Registering block manager ip-10-0-0-151.ec2.internal:54365 with 1060.5 MB RAM, BlockManagerId(2, ip-10-0-0-151.ec2.internal, 54365, None)
17/07/31 18:54:39 INFO scheduler.JobScheduler: Added jobs for time 1501527279000 ms
17/07/31 18:54:39 INFO scheduler.JobScheduler: Starting job streaming job 1501527279000 ms.0 from job set of time 1501527279000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:39 INFO scheduler.JobScheduler: Finished job streaming job 1501527279000 ms.0 from job set of time 1501527279000 ms
17/07/31 18:54:39 INFO python.PythonRDD: Removing RDD 7 from persistence list
17/07/31 18:54:39 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527279000 ms (execution: 0.006 s)
17/07/31 18:54:39 INFO storage.BlockManager: Removing RDD 7
17/07/31 18:54:39 INFO rdd.BlockRDD: Removing RDD 6 from persistence list
17/07/31 18:54:39 INFO storage.BlockManager: Removing RDD 6
17/07/31 18:54:39 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[6] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527279000 ms
17/07/31 18:54:39 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527277000 ms
17/07/31 18:54:39 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527277000 ms
17/07/31 18:54:40 INFO scheduler.JobScheduler: Added jobs for time 1501527280000 ms
17/07/31 18:54:40 INFO scheduler.JobScheduler: Starting job streaming job 1501527280000 ms.0 from job set of time 1501527280000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:40 INFO scheduler.JobScheduler: Finished job streaming job 1501527280000 ms.0 from job set of time 1501527280000 ms
17/07/31 18:54:40 INFO python.PythonRDD: Removing RDD 9 from persistence list
17/07/31 18:54:40 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501527280000 ms (execution: 0.009 s)
17/07/31 18:54:40 INFO storage.BlockManager: Removing RDD 9
17/07/31 18:54:40 INFO rdd.BlockRDD: Removing RDD 8 from persistence list
17/07/31 18:54:40 INFO storage.BlockManager: Removing RDD 8
17/07/31 18:54:40 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[8] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527280000 ms
17/07/31 18:54:40 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527278000 ms
17/07/31 18:54:40 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527278000 ms
17/07/31 18:54:41 INFO scheduler.JobScheduler: Added jobs for time 1501527281000 ms
17/07/31 18:54:41 INFO scheduler.JobScheduler: Starting job streaming job 1501527281000 ms.0 from job set of time 1501527281000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:41 INFO scheduler.JobScheduler: Finished job streaming job 1501527281000 ms.0 from job set of time 1501527281000 ms
17/07/31 18:54:41 INFO python.PythonRDD: Removing RDD 11 from persistence list
17/07/31 18:54:41 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527281000 ms (execution: 0.006 s)
17/07/31 18:54:41 INFO storage.BlockManager: Removing RDD 11
17/07/31 18:54:41 INFO rdd.BlockRDD: Removing RDD 10 from persistence list
17/07/31 18:54:41 INFO storage.BlockManager: Removing RDD 10
17/07/31 18:54:41 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[10] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527281000 ms
17/07/31 18:54:41 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527279000 ms
17/07/31 18:54:41 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527279000 ms
17/07/31 18:54:42 INFO scheduler.JobScheduler: Added jobs for time 1501527282000 ms
17/07/31 18:54:42 INFO scheduler.JobScheduler: Starting job streaming job 1501527282000 ms.0 from job set of time 1501527282000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:42 INFO scheduler.JobScheduler: Finished job streaming job 1501527282000 ms.0 from job set of time 1501527282000 ms
17/07/31 18:54:42 INFO python.PythonRDD: Removing RDD 13 from persistence list
17/07/31 18:54:42 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527282000 ms (execution: 0.006 s)
17/07/31 18:54:42 INFO storage.BlockManager: Removing RDD 13
17/07/31 18:54:42 INFO rdd.BlockRDD: Removing RDD 12 from persistence list
17/07/31 18:54:42 INFO storage.BlockManager: Removing RDD 12
17/07/31 18:54:42 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[12] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527282000 ms
17/07/31 18:54:42 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527280000 ms
17/07/31 18:54:42 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527280000 ms
17/07/31 18:54:43 INFO scheduler.JobScheduler: Added jobs for time 1501527283000 ms
17/07/31 18:54:43 INFO scheduler.JobScheduler: Starting job streaming job 1501527283000 ms.0 from job set of time 1501527283000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:43 INFO scheduler.JobScheduler: Finished job streaming job 1501527283000 ms.0 from job set of time 1501527283000 ms
17/07/31 18:54:43 INFO python.PythonRDD: Removing RDD 15 from persistence list
17/07/31 18:54:43 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527283000 ms (execution: 0.008 s)
17/07/31 18:54:43 INFO storage.BlockManager: Removing RDD 15
17/07/31 18:54:43 INFO rdd.BlockRDD: Removing RDD 14 from persistence list
17/07/31 18:54:43 INFO storage.BlockManager: Removing RDD 14
17/07/31 18:54:43 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[14] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527283000 ms
17/07/31 18:54:43 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527281000 ms
17/07/31 18:54:43 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527281000 ms
17/07/31 18:54:44 INFO scheduler.JobScheduler: Added jobs for time 1501527284000 ms
17/07/31 18:54:44 INFO scheduler.JobScheduler: Starting job streaming job 1501527284000 ms.0 from job set of time 1501527284000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:44 INFO scheduler.JobScheduler: Finished job streaming job 1501527284000 ms.0 from job set of time 1501527284000 ms
17/07/31 18:54:44 INFO python.PythonRDD: Removing RDD 17 from persistence list
17/07/31 18:54:44 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527284000 ms (execution: 0.016 s)
17/07/31 18:54:44 INFO storage.BlockManager: Removing RDD 17
17/07/31 18:54:44 INFO rdd.BlockRDD: Removing RDD 16 from persistence list
17/07/31 18:54:44 INFO storage.BlockManager: Removing RDD 16
17/07/31 18:54:44 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[16] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527284000 ms
17/07/31 18:54:44 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527282000 ms
17/07/31 18:54:44 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527282000 ms
17/07/31 18:54:45 INFO scheduler.JobScheduler: Added jobs for time 1501527285000 ms
17/07/31 18:54:45 INFO scheduler.JobScheduler: Starting job streaming job 1501527285000 ms.0 from job set of time 1501527285000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:45 INFO scheduler.JobScheduler: Finished job streaming job 1501527285000 ms.0 from job set of time 1501527285000 ms
17/07/31 18:54:45 INFO python.PythonRDD: Removing RDD 19 from persistence list
17/07/31 18:54:45 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527285000 ms (execution: 0.007 s)
17/07/31 18:54:45 INFO rdd.BlockRDD: Removing RDD 18 from persistence list
17/07/31 18:54:45 INFO storage.BlockManager: Removing RDD 19
17/07/31 18:54:45 INFO storage.BlockManager: Removing RDD 18
17/07/31 18:54:45 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[18] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527285000 ms
17/07/31 18:54:45 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527283000 ms
17/07/31 18:54:45 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527283000 ms
17/07/31 18:54:46 INFO scheduler.JobScheduler: Added jobs for time 1501527286000 ms
17/07/31 18:54:46 INFO scheduler.JobScheduler: Starting job streaming job 1501527286000 ms.0 from job set of time 1501527286000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:46 INFO scheduler.JobScheduler: Finished job streaming job 1501527286000 ms.0 from job set of time 1501527286000 ms
17/07/31 18:54:46 INFO python.PythonRDD: Removing RDD 21 from persistence list
17/07/31 18:54:46 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527286000 ms (execution: 0.008 s)
17/07/31 18:54:46 INFO storage.BlockManager: Removing RDD 21
17/07/31 18:54:46 INFO rdd.BlockRDD: Removing RDD 20 from persistence list
17/07/31 18:54:46 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[20] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527286000 ms
17/07/31 18:54:46 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527284000 ms
17/07/31 18:54:46 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527284000 ms
17/07/31 18:54:46 INFO storage.BlockManager: Removing RDD 20
17/07/31 18:54:47 INFO scheduler.JobScheduler: Added jobs for time 1501527287000 ms
17/07/31 18:54:47 INFO scheduler.JobScheduler: Starting job streaming job 1501527287000 ms.0 from job set of time 1501527287000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:47 INFO scheduler.JobScheduler: Finished job streaming job 1501527287000 ms.0 from job set of time 1501527287000 ms
17/07/31 18:54:47 INFO python.PythonRDD: Removing RDD 23 from persistence list
17/07/31 18:54:47 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527287000 ms (execution: 0.006 s)
17/07/31 18:54:47 INFO storage.BlockManager: Removing RDD 23
17/07/31 18:54:47 INFO rdd.BlockRDD: Removing RDD 22 from persistence list
17/07/31 18:54:47 INFO storage.BlockManager: Removing RDD 22
17/07/31 18:54:47 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[22] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527287000 ms
17/07/31 18:54:47 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527285000 ms
17/07/31 18:54:47 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527285000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Added jobs for time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Starting job streaming job 1501527288000 ms.0 from job set of time 1501527288000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:48 INFO scheduler.JobScheduler: Finished job streaming job 1501527288000 ms.0 from job set of time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527288000 ms (execution: 0.008 s)
17/07/31 18:54:48 INFO python.PythonRDD: Removing RDD 25 from persistence list
17/07/31 18:54:48 INFO storage.BlockManager: Removing RDD 25
17/07/31 18:54:48 INFO rdd.BlockRDD: Removing RDD 24 from persistence list
17/07/31 18:54:48 INFO storage.BlockManager: Removing RDD 24
17/07/31 18:54:48 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[24] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527288000 ms
17/07/31 18:54:48 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527286000 ms
17/07/31 18:54:48 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527286000 ms
17/07/31 18:54:49 INFO scheduler.JobScheduler: Added jobs for time 1501527289000 ms
17/07/31 18:54:49 INFO scheduler.JobScheduler: Starting job streaming job 1501527289000 ms.0 from job set of time 1501527289000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:49 INFO scheduler.JobScheduler: Finished job streaming job 1501527289000 ms.0 from job set of time 1501527289000 ms
17/07/31 18:54:49 INFO python.PythonRDD: Removing RDD 27 from persistence list
17/07/31 18:54:49 INFO scheduler.JobScheduler: Total delay: 0.034 s for time 1501527289000 ms (execution: 0.008 s)
17/07/31 18:54:49 INFO storage.BlockManager: Removing RDD 27
17/07/31 18:54:49 INFO rdd.BlockRDD: Removing RDD 26 from persistence list
17/07/31 18:54:49 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[26] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527289000 ms
17/07/31 18:54:49 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527287000 ms
17/07/31 18:54:49 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527287000 ms
17/07/31 18:54:49 INFO storage.BlockManager: Removing RDD 26
17/07/31 18:54:50 INFO scheduler.JobScheduler: Added jobs for time 1501527290000 ms
17/07/31 18:54:50 INFO scheduler.JobScheduler: Starting job streaming job 1501527290000 ms.0 from job set of time 1501527290000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:50 INFO scheduler.JobScheduler: Finished job streaming job 1501527290000 ms.0 from job set of time 1501527290000 ms
17/07/31 18:54:50 INFO python.PythonRDD: Removing RDD 29 from persistence list
17/07/31 18:54:50 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527290000 ms (execution: 0.008 s)
17/07/31 18:54:50 INFO storage.BlockManager: Removing RDD 29
17/07/31 18:54:50 INFO rdd.BlockRDD: Removing RDD 28 from persistence list
17/07/31 18:54:50 INFO storage.BlockManager: Removing RDD 28
17/07/31 18:54:50 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[28] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527290000 ms
17/07/31 18:54:50 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527288000 ms
17/07/31 18:54:50 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527288000 ms
17/07/31 18:54:51 INFO scheduler.JobScheduler: Added jobs for time 1501527291000 ms
17/07/31 18:54:51 INFO scheduler.JobScheduler: Starting job streaming job 1501527291000 ms.0 from job set of time 1501527291000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:51 INFO scheduler.JobScheduler: Finished job streaming job 1501527291000 ms.0 from job set of time 1501527291000 ms
17/07/31 18:54:51 INFO python.PythonRDD: Removing RDD 31 from persistence list
17/07/31 18:54:51 INFO scheduler.JobScheduler: Total delay: 0.033 s for time 1501527291000 ms (execution: 0.018 s)
17/07/31 18:54:51 INFO storage.BlockManager: Removing RDD 31
17/07/31 18:54:51 INFO rdd.BlockRDD: Removing RDD 30 from persistence list
17/07/31 18:54:51 INFO storage.BlockManager: Removing RDD 30
17/07/31 18:54:51 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[30] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527291000 ms
17/07/31 18:54:51 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527289000 ms
17/07/31 18:54:51 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527289000 ms
17/07/31 18:54:52 INFO scheduler.JobScheduler: Added jobs for time 1501527292000 ms
17/07/31 18:54:52 INFO scheduler.JobScheduler: Starting job streaming job 1501527292000 ms.0 from job set of time 1501527292000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:52 INFO scheduler.JobScheduler: Finished job streaming job 1501527292000 ms.0 from job set of time 1501527292000 ms
17/07/31 18:54:52 INFO python.PythonRDD: Removing RDD 33 from persistence list
17/07/31 18:54:52 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501527292000 ms (execution: 0.008 s)
17/07/31 18:54:52 INFO storage.BlockManager: Removing RDD 33
17/07/31 18:54:52 INFO rdd.BlockRDD: Removing RDD 32 from persistence list
17/07/31 18:54:52 INFO storage.BlockManager: Removing RDD 32
17/07/31 18:54:52 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[32] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527292000 ms
17/07/31 18:54:52 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527290000 ms
17/07/31 18:54:52 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527290000 ms
17/07/31 18:54:53 INFO scheduler.JobScheduler: Added jobs for time 1501527293000 ms
17/07/31 18:54:53 INFO scheduler.JobScheduler: Starting job streaming job 1501527293000 ms.0 from job set of time 1501527293000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:53 INFO scheduler.JobScheduler: Finished job streaming job 1501527293000 ms.0 from job set of time 1501527293000 ms
17/07/31 18:54:53 INFO python.PythonRDD: Removing RDD 35 from persistence list
17/07/31 18:54:53 INFO scheduler.JobScheduler: Total delay: 0.026 s for time 1501527293000 ms (execution: 0.008 s)
17/07/31 18:54:53 INFO storage.BlockManager: Removing RDD 35
17/07/31 18:54:53 INFO rdd.BlockRDD: Removing RDD 34 from persistence list
17/07/31 18:54:53 INFO storage.BlockManager: Removing RDD 34
17/07/31 18:54:53 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[34] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527293000 ms
17/07/31 18:54:53 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527291000 ms
17/07/31 18:54:53 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527291000 ms
17/07/31 18:54:54 INFO scheduler.JobScheduler: Added jobs for time 1501527294000 ms
17/07/31 18:54:54 INFO scheduler.JobScheduler: Starting job streaming job 1501527294000 ms.0 from job set of time 1501527294000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:54 INFO scheduler.JobScheduler: Finished job streaming job 1501527294000 ms.0 from job set of time 1501527294000 ms
17/07/31 18:54:54 INFO python.PythonRDD: Removing RDD 37 from persistence list
17/07/31 18:54:54 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527294000 ms (execution: 0.006 s)
17/07/31 18:54:54 INFO storage.BlockManager: Removing RDD 37
17/07/31 18:54:54 INFO rdd.BlockRDD: Removing RDD 36 from persistence list
17/07/31 18:54:54 INFO storage.BlockManager: Removing RDD 36
17/07/31 18:54:54 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[36] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527294000 ms
17/07/31 18:54:54 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527292000 ms
17/07/31 18:54:54 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527292000 ms
17/07/31 18:54:55 INFO scheduler.JobScheduler: Added jobs for time 1501527295000 ms
17/07/31 18:54:55 INFO scheduler.JobScheduler: Starting job streaming job 1501527295000 ms.0 from job set of time 1501527295000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:55 INFO scheduler.JobScheduler: Finished job streaming job 1501527295000 ms.0 from job set of time 1501527295000 ms
17/07/31 18:54:55 INFO python.PythonRDD: Removing RDD 39 from persistence list
17/07/31 18:54:55 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527295000 ms (execution: 0.007 s)
17/07/31 18:54:55 INFO storage.BlockManager: Removing RDD 39
17/07/31 18:54:55 INFO rdd.BlockRDD: Removing RDD 38 from persistence list
17/07/31 18:54:55 INFO storage.BlockManager: Removing RDD 38
17/07/31 18:54:55 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[38] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527295000 ms
17/07/31 18:54:55 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527293000 ms
17/07/31 18:54:55 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527293000 ms
17/07/31 18:54:56 INFO scheduler.JobScheduler: Added jobs for time 1501527296000 ms
17/07/31 18:54:56 INFO scheduler.JobScheduler: Starting job streaming job 1501527296000 ms.0 from job set of time 1501527296000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:56 INFO scheduler.JobScheduler: Finished job streaming job 1501527296000 ms.0 from job set of time 1501527296000 ms
17/07/31 18:54:56 INFO python.PythonRDD: Removing RDD 41 from persistence list
17/07/31 18:54:56 INFO scheduler.JobScheduler: Total delay: 0.031 s for time 1501527296000 ms (execution: 0.007 s)
17/07/31 18:54:56 INFO storage.BlockManager: Removing RDD 41
17/07/31 18:54:56 INFO rdd.BlockRDD: Removing RDD 40 from persistence list
17/07/31 18:54:56 INFO storage.BlockManager: Removing RDD 40
17/07/31 18:54:56 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[40] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527296000 ms
17/07/31 18:54:56 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527294000 ms
17/07/31 18:54:56 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527294000 ms
17/07/31 18:54:57 INFO scheduler.JobScheduler: Added jobs for time 1501527297000 ms
17/07/31 18:54:57 INFO scheduler.JobScheduler: Starting job streaming job 1501527297000 ms.0 from job set of time 1501527297000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:57 INFO scheduler.JobScheduler: Finished job streaming job 1501527297000 ms.0 from job set of time 1501527297000 ms
17/07/31 18:54:57 INFO python.PythonRDD: Removing RDD 43 from persistence list
17/07/31 18:54:57 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527297000 ms (execution: 0.008 s)
17/07/31 18:54:57 INFO storage.BlockManager: Removing RDD 43
17/07/31 18:54:57 INFO rdd.BlockRDD: Removing RDD 42 from persistence list
17/07/31 18:54:57 INFO storage.BlockManager: Removing RDD 42
17/07/31 18:54:57 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[42] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527297000 ms
17/07/31 18:54:57 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527295000 ms
17/07/31 18:54:57 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527295000 ms
17/07/31 18:54:58 INFO scheduler.JobScheduler: Added jobs for time 1501527298000 ms
17/07/31 18:54:58 INFO scheduler.JobScheduler: Starting job streaming job 1501527298000 ms.0 from job set of time 1501527298000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:58 INFO scheduler.JobScheduler: Finished job streaming job 1501527298000 ms.0 from job set of time 1501527298000 ms
17/07/31 18:54:58 INFO python.PythonRDD: Removing RDD 45 from persistence list
17/07/31 18:54:58 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527298000 ms (execution: 0.008 s)
17/07/31 18:54:58 INFO storage.BlockManager: Removing RDD 45
17/07/31 18:54:58 INFO rdd.BlockRDD: Removing RDD 44 from persistence list
17/07/31 18:54:58 INFO storage.BlockManager: Removing RDD 44
17/07/31 18:54:58 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[44] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527298000 ms
17/07/31 18:54:58 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527296000 ms
17/07/31 18:54:58 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527296000 ms
17/07/31 18:54:59 INFO scheduler.JobScheduler: Added jobs for time 1501527299000 ms
17/07/31 18:54:59 INFO scheduler.JobScheduler: Starting job streaming job 1501527299000 ms.0 from job set of time 1501527299000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:54:59 INFO scheduler.JobScheduler: Finished job streaming job 1501527299000 ms.0 from job set of time 1501527299000 ms
17/07/31 18:54:59 INFO python.PythonRDD: Removing RDD 47 from persistence list
17/07/31 18:54:59 INFO scheduler.JobScheduler: Total delay: 0.031 s for time 1501527299000 ms (execution: 0.009 s)
17/07/31 18:54:59 INFO storage.BlockManager: Removing RDD 47
17/07/31 18:54:59 INFO rdd.BlockRDD: Removing RDD 46 from persistence list
17/07/31 18:54:59 INFO storage.BlockManager: Removing RDD 46
17/07/31 18:54:59 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[46] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527299000 ms
17/07/31 18:54:59 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527297000 ms
17/07/31 18:54:59 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527297000 ms
17/07/31 18:55:00 INFO scheduler.JobScheduler: Added jobs for time 1501527300000 ms
17/07/31 18:55:00 INFO scheduler.JobScheduler: Starting job streaming job 1501527300000 ms.0 from job set of time 1501527300000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:00 INFO scheduler.JobScheduler: Finished job streaming job 1501527300000 ms.0 from job set of time 1501527300000 ms
17/07/31 18:55:00 INFO python.PythonRDD: Removing RDD 49 from persistence list
17/07/31 18:55:00 INFO scheduler.JobScheduler: Total delay: 0.035 s for time 1501527300000 ms (execution: 0.021 s)
17/07/31 18:55:00 INFO storage.BlockManager: Removing RDD 49
17/07/31 18:55:00 INFO rdd.BlockRDD: Removing RDD 48 from persistence list
17/07/31 18:55:00 INFO storage.BlockManager: Removing RDD 48
17/07/31 18:55:00 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[48] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527300000 ms
17/07/31 18:55:00 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527298000 ms
17/07/31 18:55:00 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527298000 ms
17/07/31 18:55:01 INFO scheduler.JobScheduler: Added jobs for time 1501527301000 ms
17/07/31 18:55:01 INFO scheduler.JobScheduler: Starting job streaming job 1501527301000 ms.0 from job set of time 1501527301000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:01 INFO scheduler.JobScheduler: Finished job streaming job 1501527301000 ms.0 from job set of time 1501527301000 ms
17/07/31 18:55:01 INFO python.PythonRDD: Removing RDD 51 from persistence list
17/07/31 18:55:01 INFO scheduler.JobScheduler: Total delay: 0.024 s for time 1501527301000 ms (execution: 0.008 s)
17/07/31 18:55:01 INFO storage.BlockManager: Removing RDD 51
17/07/31 18:55:01 INFO rdd.BlockRDD: Removing RDD 50 from persistence list
17/07/31 18:55:01 INFO storage.BlockManager: Removing RDD 50
17/07/31 18:55:01 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[50] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527301000 ms
17/07/31 18:55:01 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527299000 ms
17/07/31 18:55:01 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527299000 ms
17/07/31 18:55:02 INFO scheduler.JobScheduler: Added jobs for time 1501527302000 ms
17/07/31 18:55:02 INFO scheduler.JobScheduler: Starting job streaming job 1501527302000 ms.0 from job set of time 1501527302000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:02 INFO scheduler.JobScheduler: Finished job streaming job 1501527302000 ms.0 from job set of time 1501527302000 ms
17/07/31 18:55:02 INFO python.PythonRDD: Removing RDD 53 from persistence list
17/07/31 18:55:02 INFO scheduler.JobScheduler: Total delay: 0.025 s for time 1501527302000 ms (execution: 0.009 s)
17/07/31 18:55:02 INFO storage.BlockManager: Removing RDD 53
17/07/31 18:55:02 INFO rdd.BlockRDD: Removing RDD 52 from persistence list
17/07/31 18:55:02 INFO storage.BlockManager: Removing RDD 52
17/07/31 18:55:02 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[52] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527302000 ms
17/07/31 18:55:02 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527300000 ms
17/07/31 18:55:02 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527300000 ms
17/07/31 18:55:03 INFO scheduler.JobScheduler: Added jobs for time 1501527303000 ms
17/07/31 18:55:03 INFO scheduler.JobScheduler: Starting job streaming job 1501527303000 ms.0 from job set of time 1501527303000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:03 INFO scheduler.JobScheduler: Finished job streaming job 1501527303000 ms.0 from job set of time 1501527303000 ms
17/07/31 18:55:03 INFO python.PythonRDD: Removing RDD 55 from persistence list
17/07/31 18:55:03 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527303000 ms (execution: 0.008 s)
17/07/31 18:55:03 INFO storage.BlockManager: Removing RDD 55
17/07/31 18:55:03 INFO rdd.BlockRDD: Removing RDD 54 from persistence list
17/07/31 18:55:03 INFO storage.BlockManager: Removing RDD 54
17/07/31 18:55:03 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[54] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527303000 ms
17/07/31 18:55:03 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527301000 ms
17/07/31 18:55:03 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527301000 ms
17/07/31 18:55:04 INFO scheduler.JobScheduler: Added jobs for time 1501527304000 ms
17/07/31 18:55:04 INFO scheduler.JobScheduler: Starting job streaming job 1501527304000 ms.0 from job set of time 1501527304000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:04 INFO scheduler.JobScheduler: Finished job streaming job 1501527304000 ms.0 from job set of time 1501527304000 ms
17/07/31 18:55:04 INFO python.PythonRDD: Removing RDD 57 from persistence list
17/07/31 18:55:04 INFO scheduler.JobScheduler: Total delay: 0.027 s for time 1501527304000 ms (execution: 0.008 s)
17/07/31 18:55:04 INFO storage.BlockManager: Removing RDD 57
17/07/31 18:55:04 INFO rdd.BlockRDD: Removing RDD 56 from persistence list
17/07/31 18:55:04 INFO storage.BlockManager: Removing RDD 56
17/07/31 18:55:04 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[56] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527304000 ms
17/07/31 18:55:04 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527302000 ms
17/07/31 18:55:04 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527302000 ms
17/07/31 18:55:05 INFO scheduler.JobScheduler: Added jobs for time 1501527305000 ms
17/07/31 18:55:05 INFO scheduler.JobScheduler: Starting job streaming job 1501527305000 ms.0 from job set of time 1501527305000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:05 INFO scheduler.JobScheduler: Finished job streaming job 1501527305000 ms.0 from job set of time 1501527305000 ms
17/07/31 18:55:05 INFO python.PythonRDD: Removing RDD 59 from persistence list
17/07/31 18:55:05 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527305000 ms (execution: 0.008 s)
17/07/31 18:55:05 INFO storage.BlockManager: Removing RDD 59
17/07/31 18:55:05 INFO rdd.BlockRDD: Removing RDD 58 from persistence list
17/07/31 18:55:05 INFO storage.BlockManager: Removing RDD 58
17/07/31 18:55:05 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[58] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527305000 ms
17/07/31 18:55:05 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527303000 ms
17/07/31 18:55:05 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527303000 ms
17/07/31 18:55:06 INFO scheduler.JobScheduler: Added jobs for time 1501527306000 ms
17/07/31 18:55:06 INFO scheduler.JobScheduler: Starting job streaming job 1501527306000 ms.0 from job set of time 1501527306000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:06 INFO scheduler.JobScheduler: Finished job streaming job 1501527306000 ms.0 from job set of time 1501527306000 ms
17/07/31 18:55:06 INFO python.PythonRDD: Removing RDD 61 from persistence list
17/07/31 18:55:06 INFO scheduler.JobScheduler: Total delay: 0.022 s for time 1501527306000 ms (execution: 0.007 s)
17/07/31 18:55:06 INFO storage.BlockManager: Removing RDD 61
17/07/31 18:55:06 INFO rdd.BlockRDD: Removing RDD 60 from persistence list
17/07/31 18:55:06 INFO storage.BlockManager: Removing RDD 60
17/07/31 18:55:06 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[60] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527306000 ms
17/07/31 18:55:06 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527304000 ms
17/07/31 18:55:06 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527304000 ms
17/07/31 18:55:07 INFO scheduler.JobScheduler: Added jobs for time 1501527307000 ms
17/07/31 18:55:07 INFO scheduler.JobScheduler: Starting job streaming job 1501527307000 ms.0 from job set of time 1501527307000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:07 INFO scheduler.JobScheduler: Finished job streaming job 1501527307000 ms.0 from job set of time 1501527307000 ms
17/07/31 18:55:07 INFO python.PythonRDD: Removing RDD 63 from persistence list
17/07/31 18:55:07 INFO scheduler.JobScheduler: Total delay: 0.028 s for time 1501527307000 ms (execution: 0.007 s)
17/07/31 18:55:07 INFO storage.BlockManager: Removing RDD 63
17/07/31 18:55:07 INFO rdd.BlockRDD: Removing RDD 62 from persistence list
17/07/31 18:55:07 INFO storage.BlockManager: Removing RDD 62
17/07/31 18:55:07 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[62] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527307000 ms
17/07/31 18:55:07 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527305000 ms
17/07/31 18:55:07 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527305000 ms
17/07/31 18:55:08 INFO scheduler.JobScheduler: Added jobs for time 1501527308000 ms
17/07/31 18:55:08 INFO scheduler.JobScheduler: Starting job streaming job 1501527308000 ms.0 from job set of time 1501527308000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:08 INFO scheduler.JobScheduler: Finished job streaming job 1501527308000 ms.0 from job set of time 1501527308000 ms
17/07/31 18:55:08 INFO python.PythonRDD: Removing RDD 65 from persistence list
17/07/31 18:55:08 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527308000 ms (execution: 0.008 s)
17/07/31 18:55:08 INFO storage.BlockManager: Removing RDD 65
17/07/31 18:55:08 INFO rdd.BlockRDD: Removing RDD 64 from persistence list
17/07/31 18:55:08 INFO storage.BlockManager: Removing RDD 64
17/07/31 18:55:08 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[64] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527308000 ms
17/07/31 18:55:08 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527306000 ms
17/07/31 18:55:08 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527306000 ms
17/07/31 18:55:09 INFO scheduler.JobScheduler: Added jobs for time 1501527309000 ms
17/07/31 18:55:09 INFO scheduler.JobScheduler: Starting job streaming job 1501527309000 ms.0 from job set of time 1501527309000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:09 INFO scheduler.JobScheduler: Finished job streaming job 1501527309000 ms.0 from job set of time 1501527309000 ms
17/07/31 18:55:09 INFO python.PythonRDD: Removing RDD 67 from persistence list
17/07/31 18:55:09 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527309000 ms (execution: 0.015 s)
17/07/31 18:55:09 INFO storage.BlockManager: Removing RDD 67
17/07/31 18:55:09 INFO rdd.BlockRDD: Removing RDD 66 from persistence list
17/07/31 18:55:09 INFO storage.BlockManager: Removing RDD 66
17/07/31 18:55:09 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[66] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527309000 ms
17/07/31 18:55:09 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527307000 ms
17/07/31 18:55:09 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527307000 ms
17/07/31 18:55:10 INFO scheduler.JobScheduler: Added jobs for time 1501527310000 ms
17/07/31 18:55:10 INFO scheduler.JobScheduler: Starting job streaming job 1501527310000 ms.0 from job set of time 1501527310000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:10 INFO scheduler.JobScheduler: Finished job streaming job 1501527310000 ms.0 from job set of time 1501527310000 ms
17/07/31 18:55:10 INFO python.PythonRDD: Removing RDD 69 from persistence list
17/07/31 18:55:10 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527310000 ms (execution: 0.007 s)
17/07/31 18:55:10 INFO storage.BlockManager: Removing RDD 69
17/07/31 18:55:10 INFO rdd.BlockRDD: Removing RDD 68 from persistence list
17/07/31 18:55:10 INFO storage.BlockManager: Removing RDD 68
17/07/31 18:55:10 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[68] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527310000 ms
17/07/31 18:55:10 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527308000 ms
17/07/31 18:55:10 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527308000 ms
17/07/31 18:55:11 INFO scheduler.JobScheduler: Added jobs for time 1501527311000 ms
17/07/31 18:55:11 INFO scheduler.JobScheduler: Starting job streaming job 1501527311000 ms.0 from job set of time 1501527311000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:11 INFO scheduler.JobScheduler: Finished job streaming job 1501527311000 ms.0 from job set of time 1501527311000 ms
17/07/31 18:55:11 INFO python.PythonRDD: Removing RDD 71 from persistence list
17/07/31 18:55:11 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527311000 ms (execution: 0.007 s)
17/07/31 18:55:11 INFO storage.BlockManager: Removing RDD 71
17/07/31 18:55:11 INFO rdd.BlockRDD: Removing RDD 70 from persistence list
17/07/31 18:55:11 INFO storage.BlockManager: Removing RDD 70
17/07/31 18:55:11 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[70] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527311000 ms
17/07/31 18:55:11 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527309000 ms
17/07/31 18:55:11 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527309000 ms
17/07/31 18:55:12 INFO scheduler.JobScheduler: Added jobs for time 1501527312000 ms
17/07/31 18:55:12 INFO scheduler.JobScheduler: Starting job streaming job 1501527312000 ms.0 from job set of time 1501527312000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:12 INFO scheduler.JobScheduler: Finished job streaming job 1501527312000 ms.0 from job set of time 1501527312000 ms
17/07/31 18:55:12 INFO python.PythonRDD: Removing RDD 73 from persistence list
17/07/31 18:55:12 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501527312000 ms (execution: 0.008 s)
17/07/31 18:55:12 INFO storage.BlockManager: Removing RDD 73
17/07/31 18:55:12 INFO rdd.BlockRDD: Removing RDD 72 from persistence list
17/07/31 18:55:12 INFO storage.BlockManager: Removing RDD 72
17/07/31 18:55:12 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[72] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527312000 ms
17/07/31 18:55:12 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527310000 ms
17/07/31 18:55:12 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527310000 ms
17/07/31 18:55:13 INFO scheduler.JobScheduler: Added jobs for time 1501527313000 ms
17/07/31 18:55:13 INFO scheduler.JobScheduler: Starting job streaming job 1501527313000 ms.0 from job set of time 1501527313000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:13 INFO scheduler.JobScheduler: Finished job streaming job 1501527313000 ms.0 from job set of time 1501527313000 ms
17/07/31 18:55:13 INFO python.PythonRDD: Removing RDD 75 from persistence list
17/07/31 18:55:13 INFO scheduler.JobScheduler: Total delay: 0.023 s for time 1501527313000 ms (execution: 0.007 s)
17/07/31 18:55:13 INFO storage.BlockManager: Removing RDD 75
17/07/31 18:55:13 INFO rdd.BlockRDD: Removing RDD 74 from persistence list
17/07/31 18:55:13 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[74] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527313000 ms
17/07/31 18:55:13 INFO storage.BlockManager: Removing RDD 74
17/07/31 18:55:13 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527311000 ms
17/07/31 18:55:13 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527311000 ms
17/07/31 18:55:14 INFO scheduler.JobScheduler: Added jobs for time 1501527314000 ms
17/07/31 18:55:14 INFO scheduler.JobScheduler: Starting job streaming job 1501527314000 ms.0 from job set of time 1501527314000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:14 INFO scheduler.JobScheduler: Finished job streaming job 1501527314000 ms.0 from job set of time 1501527314000 ms
17/07/31 18:55:14 INFO python.PythonRDD: Removing RDD 77 from persistence list
17/07/31 18:55:14 INFO scheduler.JobScheduler: Total delay: 0.032 s for time 1501527314000 ms (execution: 0.007 s)
17/07/31 18:55:14 INFO storage.BlockManager: Removing RDD 77
17/07/31 18:55:14 INFO rdd.BlockRDD: Removing RDD 76 from persistence list
17/07/31 18:55:14 INFO storage.BlockManager: Removing RDD 76
17/07/31 18:55:14 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[76] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527314000 ms
17/07/31 18:55:14 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527312000 ms
17/07/31 18:55:14 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527312000 ms
17/07/31 18:55:15 INFO scheduler.JobScheduler: Added jobs for time 1501527315000 ms
17/07/31 18:55:15 INFO scheduler.JobScheduler: Starting job streaming job 1501527315000 ms.0 from job set of time 1501527315000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:15 INFO scheduler.JobScheduler: Finished job streaming job 1501527315000 ms.0 from job set of time 1501527315000 ms
17/07/31 18:55:15 INFO python.PythonRDD: Removing RDD 79 from persistence list
17/07/31 18:55:15 INFO scheduler.JobScheduler: Total delay: 0.021 s for time 1501527315000 ms (execution: 0.007 s)
17/07/31 18:55:15 INFO storage.BlockManager: Removing RDD 79
17/07/31 18:55:15 INFO rdd.BlockRDD: Removing RDD 78 from persistence list
17/07/31 18:55:15 INFO storage.BlockManager: Removing RDD 78
17/07/31 18:55:15 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[78] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527315000 ms
17/07/31 18:55:15 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527313000 ms
17/07/31 18:55:15 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527313000 ms
17/07/31 18:55:16 INFO scheduler.JobScheduler: Added jobs for time 1501527316000 ms
17/07/31 18:55:16 INFO scheduler.JobScheduler: Starting job streaming job 1501527316000 ms.0 from job set of time 1501527316000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:16 INFO scheduler.JobScheduler: Finished job streaming job 1501527316000 ms.0 from job set of time 1501527316000 ms
17/07/31 18:55:16 INFO python.PythonRDD: Removing RDD 81 from persistence list
17/07/31 18:55:16 INFO scheduler.JobScheduler: Total delay: 0.030 s for time 1501527316000 ms (execution: 0.014 s)
17/07/31 18:55:16 INFO storage.BlockManager: Removing RDD 81
17/07/31 18:55:16 INFO rdd.BlockRDD: Removing RDD 80 from persistence list
17/07/31 18:55:16 INFO storage.BlockManager: Removing RDD 80
17/07/31 18:55:16 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[80] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527316000 ms
17/07/31 18:55:16 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527314000 ms
17/07/31 18:55:16 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527314000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Added jobs for time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Starting job streaming job 1501527317000 ms.0 from job set of time 1501527317000 ms
------------------------LISTENING KAFKA TOPIC:SPOT-INGEST-proxy-18_54_13------------------------
17/07/31 18:55:17 INFO scheduler.JobScheduler: Finished job streaming job 1501527317000 ms.0 from job set of time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.JobScheduler: Total delay: 0.029 s for time 1501527317000 ms (execution: 0.011 s)
17/07/31 18:55:17 INFO python.PythonRDD: Removing RDD 83 from persistence list
17/07/31 18:55:17 INFO storage.BlockManager: Removing RDD 83
17/07/31 18:55:17 INFO rdd.BlockRDD: Removing RDD 82 from persistence list
17/07/31 18:55:17 INFO storage.BlockManager: Removing RDD 82
17/07/31 18:55:17 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[82] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527317000 ms
17/07/31 18:55:17 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527315000 ms
17/07/31 18:55:17 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527315000 ms
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1059.6 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1059.6 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.4 KB, free: 1057.9 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.4 KB, free: 1057.9 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 880.3 KB, free: 1057.0 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 880.3 KB, free: 1057.1 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.0 KB, free: 1056.2 MB)
17/07/31 18:55:17 INFO storage.BlockManagerInfo: Added input-0-1501527317600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.0 KB, free: 1056.2 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527317800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.3 KB, free: 1054.5 MB)
17/07/31 18:55:18 INFO scheduler.JobScheduler: Added jobs for time 1501527318000 ms
17/07/31 18:55:18 INFO scheduler.JobScheduler: Starting job streaming job 1501527318000 ms.0 from job set of time 1501527318000 ms
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527317800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.3 KB, free: 1054.5 MB)
17/07/31 18:55:18 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Got job 2 (runJob at PythonRDD.scala:441) with 1 output partitions
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Final stage: ResultStage 3 (runJob at PythonRDD.scala:441)
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Submitting ResultStage 3 (PythonRDD[88] at RDD at PythonRDD.scala:48), which has no missing parents
17/07/31 18:55:18 INFO memory.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/07/31 18:55:18 INFO memory.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.0.0.199:59442 (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:18 INFO spark.SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:996
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (PythonRDD[88] at RDD at PythonRDD.scala:48)
17/07/31 18:55:18 INFO cluster.YarnScheduler: Adding task set 3.0 with 1 tasks
17/07/31 18:55:18 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 3.0 (TID 71, ip-10-0-0-151.ec2.internal, executor 2, partition 0, NODE_LOCAL, 5800 bytes)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 5.3 KB, free: 1053.6 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.3 KB, free: 1052.7 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.3 KB, free: 1052.8 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.1 KB, free: 1050.2 MB)
17/07/31 18:55:18 INFO storage.BlockManagerInfo: Added input-0-1501527318600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.1 KB, free: 1050.2 MB)
17/07/31 18:55:18 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 3.0 (TID 71) in 893 ms on ip-10-0-0-151.ec2.internal (executor 2) (1/1)
17/07/31 18:55:18 INFO cluster.YarnScheduler: Removed TaskSet 3.0, whose tasks have all completed, from pool
17/07/31 18:55:18 INFO scheduler.DAGScheduler: ResultStage 3 (runJob at PythonRDD.scala:441) finished in 0.896 s
17/07/31 18:55:18 INFO scheduler.DAGScheduler: Job 2 finished: runJob at PythonRDD.scala:441, took 0.913501 s
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527318800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.5 KB, free: 1049.3 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527318800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.5 KB, free: 1049.3 MB)
17/07/31 18:55:19 INFO scheduler.JobScheduler: Added jobs for time 1501527319000 ms
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 887.9 KB, free: 1048.4 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 887.9 KB, free: 1048.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1047.6 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1047.6 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319400 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.2 KB, free: 1046.7 MB)
17/07/31 18:55:19 INFO spark.ContextCleaner: Cleaned shuffle 0
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319400 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.2 KB, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 10.0.0.199:59442 in memory (size: 1988.0 B, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1988.0 B, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on 10.0.0.199:59442 in memory (size: 1956.0 B, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1956.0 B, free: 1046.7 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on 10.0.0.199:59442 in memory (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 5.3 KB, free: 1046.8 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319600 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 1758.2 KB, free: 1045.0 MB)
17/07/31 18:55:19 INFO storage.BlockManagerInfo: Added input-0-1501527319600 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 1758.2 KB, free: 1045.0 MB)
17/07/31 18:55:19 INFO execution.SparkSqlParser: Parsing command: spotdb.proxy
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527319800 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527319800 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
17/07/31 18:55:20 INFO metastore.ObjectStore: ObjectStore, initialize called
17/07/31 18:55:20 INFO scheduler.JobScheduler: Added jobs for time 1501527320000 ms
17/07/31 18:55:20 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing
17/07/31 18:55:20 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
17/07/31 18:55:20 INFO metastore.ObjectStore: Initialized ObjectStore
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_table start=1501527320011 end=1501527320059 duration=48 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_table : db=spotdb tbl=proxy
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_table start=1501527320177 end=1501527320179 duration=2 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320000 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 879.1 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320000 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 879.1 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO log.PerfLogger: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
17/07/31 18:55:20 INFO metastore.HiveMetaStore: 1: get_database: spotdb
17/07/31 18:55:20 INFO HiveMetaStore.audit: ugi=spot-user ip=unknown-ip-addr cmd=get_database: spotdb
17/07/31 18:55:20 WARN metastore.ObjectStore: Failed to get database spotdb, returning NoSuchObjectException
17/07/31 18:55:20 INFO log.PerfLogger: </PERFLOG method=get_database start=1501527320239 end=1501527320242 duration=3 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=1 retryCount=-1 error=true>
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527318000 ms.0 from job set of time 1501527318000 ms
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 2.266 s for time 1501527318000 ms (execution: 2.249 s)
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 85 from persistence list
17/07/31 18:55:20 INFO scheduler.JobScheduler: Starting job streaming job 1501527319000 ms.0 from job set of time 1501527319000 ms
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 84 from persistence list
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527318000 ms.0
org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 71, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u"Database 'spotdb' not found;"
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[84] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527318000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 85
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527316000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527316000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 84
17/07/31 18:55:20 INFO spark.SparkContext: Starting job: runJob at PythonRDD.scala:441
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Got job 3 (runJob at PythonRDD.scala:441) with 1 output partitions
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Final stage: ResultStage 4 (runJob at PythonRDD.scala:441)
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Missing parents: List()
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Submitting ResultStage 4 (PythonRDD[97] at RDD at PythonRDD.scala:48), which has no missing parents
Traceback (most recent call last):
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 148, in <module>
main()
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 77, in main
bluecoat_parse(args.zk,args.topic,args.db,args.db_table,args.num_of_workers,args.batch_size)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 144, in bluecoat_parse
ssc.awaitTermination()
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/context.py", line 206, in awaitTermination
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o48.awaitTermination.
: org.apache.spark.SparkException: An exception was raised by Python:
Traceback (most recent call last):
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/util.py", line 65, in call
r = self.func(t, *rdds)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/streaming/dstream.py", line 159, in <lambda>
func = lambda t, rdd: old_func(rdd)
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 142, in <lambda>
saved_data = proxy_data.foreachRDD(lambda row: save_data(row,sqc,db,db_table,topic))
File "/home/spot-user/spot-ingest/pipelines/proxy/bluecoat.py", line 124, in save_data
df.write.saveAsTable(hive_table,format="parquet",mode="append",partitionBy=('y','m','d','h'))
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 588, in saveAsTable
self._jwrite.saveAsTable(name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 71, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
AnalysisException: u"Database 'spotdb' not found;"
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:95)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/07/31 18:55:20 INFO memory.MemoryStore: Block broadcast_4 stored as values in memory (estimated size 8.9 KB, free 1060.4 MB)
17/07/31 18:55:20 INFO memory.MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 5.3 KB, free 1060.4 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on 10.0.0.199:59442 (size: 5.3 KB, free: 1060.5 MB)
17/07/31 18:55:20 INFO spark.SparkContext: Created broadcast 4 from broadcast at DAGScheduler.scala:996
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 4 (PythonRDD[97] at RDD at PythonRDD.scala:48)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Adding task set 4.0 with 1 tasks
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 4.0 (TID 72, ip-10-0-0-151.ec2.internal, executor 2, partition 0, NODE_LOCAL, 5800 bytes)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 5.3 KB, free: 1043.3 MB)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527319000 ms.0 from job set of time 1501527319000 ms
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 1.329 s for time 1501527319000 ms (execution: 0.051 s)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Starting job streaming job 1501527320000 ms.0 from job set of time 1501527320000 ms
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527319000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy36.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 87 from persistence list
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 86 from persistence list
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 87
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[86] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527319000 ms
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 86
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527317000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527317000 ms
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317000 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.2 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317400 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 880.3 KB, free: 1045.0 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317200 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.4 KB, free: 1046.7 MB)
17/07/31 18:55:20 INFO streaming.StreamingContext: Invoking stop(stopGracefully=false) from shutdown hook
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317600 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.0 KB, free: 1047.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317000 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.2 KB, free: 1044.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317200 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.4 KB, free: 1045.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317400 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 880.3 KB, free: 1046.7 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317600 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.0 KB, free: 1047.6 MB)
17/07/31 18:55:20 INFO scheduler.JobScheduler: Finished job streaming job 1501527320000 ms.0 from job set of time 1501527320000 ms
17/07/31 18:55:20 INFO python.PythonRDD: Removing RDD 90 from persistence list
17/07/31 18:55:20 INFO scheduler.JobScheduler: Total delay: 0.333 s for time 1501527320000 ms (execution: 0.003 s)
17/07/31 18:55:20 ERROR scheduler.JobScheduler: Error running job streaming job 1501527320000 ms.0
py4j.Py4JException: Error while sending a command.
at py4j.CallbackClient.sendCommand(CallbackClient.java:357)
at py4j.CallbackClient.sendCommand(CallbackClient.java:316)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:103)
at com.sun.proxy.$Proxy36.call(Unknown Source)
at org.apache.spark.streaming.api.python.TransformFunction.callPythonTransformFunction(PythonDStream.scala:92)
at org.apache.spark.streaming.api.python.TransformFunction.apply(PythonDStream.scala:78)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.api.python.PythonDStream$$anonfun$callForeachRDD$1.apply(PythonDStream.scala:179)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: py4j.Py4JNetworkException
at py4j.CallbackConnection.sendCommand(CallbackConnection.java:138)
at py4j.CallbackClient.sendCommand(CallbackClient.java:344)
... 24 more
17/07/31 18:55:20 INFO rdd.BlockRDD: Removing RDD 89 from persistence list
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 90
17/07/31 18:55:20 INFO storage.BlockManager: Removing RDD 89
17/07/31 18:55:20 INFO kafka.KafkaInputDStream: Removing blocks of RDD BlockRDD[89] at createStream at NativeMethodAccessorImpl.java:0 of time 1501527320000 ms
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: Sent stop signal to all 1 receivers
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317800 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.3 KB, free: 1049.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318000 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.1 KB, free: 1050.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318200 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.3 KB, free: 1051.0 MB)
17/07/31 18:55:20 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1501527318000 ms
17/07/31 18:55:20 INFO scheduler.InputInfoTracker: remove old batch metadata: 1501527318000 ms
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318400 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318600 on ip-10-0-0-229.ec2.internal:35659 in memory (size: 1758.1 KB, free: 1053.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527317800 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.3 KB, free: 1049.3 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318000 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.1 KB, free: 1050.2 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318200 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.3 KB, free: 1051.0 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318400 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 879.2 KB, free: 1051.9 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Removed input-0-1501527318600 on ip-10-0-0-151.ec2.internal:54365 in memory (size: 1758.1 KB, free: 1053.6 MB)
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320200 in memory on ip-10-0-0-229.ec2.internal:35659 (size: 880.6 KB, free: 1052.7 MB)
17/07/31 18:55:20 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver
17/07/31 18:55:20 INFO storage.BlockManagerInfo: Added input-0-1501527320200 in memory on ip-10-0-0-151.ec2.internal:54365 (size: 880.6 KB, free: 1052.8 MB)
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 4.0 (TID 72) in 289 ms on ip-10-0-0-151.ec2.internal (executor 2) (1/1)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Removed TaskSet 4.0, whose tasks have all completed, from pool
17/07/31 18:55:20 ERROR scheduler.DAGScheduler: Failed to update accumulators for task 0
org.apache.spark.SparkException: EOF reached before Python server acknowledged
at org.apache.spark.api.python.PythonAccumulatorV2.merge(PythonRDD.scala:919)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$updateAccumulators$1.apply(DAGScheduler.scala:1088)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$updateAccumulators$1.apply(DAGScheduler.scala:1080)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.DAGScheduler.updateAccumulators(DAGScheduler.scala:1080)
at org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:1156)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1647)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/07/31 18:55:20 INFO scheduler.DAGScheduler: ResultStage 4 (runJob at PythonRDD.scala:441) finished in 0.292 s
17/07/31 18:55:20 INFO scheduler.DAGScheduler: Job 3 finished: runJob at PythonRDD.scala:441, took 0.309062 s
17/07/31 18:55:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 2.0 (TID 70) in 43848 ms on ip-10-0-0-229.ec2.internal (executor 1) (1/1)
17/07/31 18:55:20 INFO cluster.YarnScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool
17/07/31 18:55:20 INFO scheduler.DAGScheduler: ResultStage 2 (start at NativeMethodAccessorImpl.java:0) finished in 43.849 s
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: All of the receivers have deregistered successfully
17/07/31 18:55:20 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
17/07/31 18:55:20 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
17/07/31 18:55:20 INFO util.RecurringTimer: Stopped timer for JobGenerator after time 1501527320000
17/07/31 18:55:20 INFO scheduler.JobGenerator: Stopped JobGenerator
17/07/31 18:55:20 INFO scheduler.JobScheduler: Stopped JobScheduler
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19fd4723{/streaming,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@22116fc3{/streaming/batch,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@70d301d6{/static/streaming,null,UNAVAILABLE}
17/07/31 18:55:20 INFO streaming.StreamingContext: StreamingContext stopped successfully
17/07/31 18:55:20 INFO spark.SparkContext: Invoking stop() from shutdown hook
17/07/31 18:55:20 INFO server.ServerConnector: Stopped ServerConnector@188e6c7f{HTTP/1.1}{0.0.0.0:4040}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4f734944{/stages/stage/kill,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@b7d4da0{/jobs/job/kill,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ab71bb9{/api,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5d37ce06{/,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@27f55b38{/static,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1159f15e{/executors/threadDump/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3ff970c2{/executors/threadDump,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@615b86f5{/executors/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@19bb7285{/executors,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3b9882ec{/environment/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7b3691b6{/environment,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@423a3429{/storage/rdd/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@13016b86{/storage/rdd,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7ea732f0{/storage/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2d4aa15a{/storage,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5cbdbf0f{/stages/pool/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14c62558{/stages/pool,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5c61bd1a{/stages/stage/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4eb72ecd{/stages/stage,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1bf090df{/stages/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@14b58fc0{/stages,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@30e71b5d{/jobs/job/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@232864a3{/jobs/job,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1f4da763{/jobs/json,null,UNAVAILABLE}
17/07/31 18:55:20 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1a3b1e79{/jobs,null,UNAVAILABLE}
17/07/31 18:55:20 INFO ui.SparkUI: Stopped Spark web UI at http://10.0.0.199:4040
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors
17/07/31 18:55:20 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
17/07/31 18:55:20 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
services=List(),
started=false)
17/07/31 18:55:20 INFO cluster.YarnClientSchedulerBackend: Stopped
17/07/31 18:55:20 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/07/31 18:55:20 INFO memory.MemoryStore: MemoryStore cleared
17/07/31 18:55:20 INFO storage.BlockManager: BlockManager stopped
17/07/31 18:55:20 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
17/07/31 18:55:20 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/07/31 18:55:20 INFO spark.SparkContext: Successfully stopped SparkContext
17/07/31 18:55:20 INFO util.ShutdownHookManager: Shutdown hook called
17/07/31 18:55:20 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40
17/07/31 18:55:20 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-b3920a3b-947f-4886-8afb-ed6f62d8ba40/pyspark-506939c9-a852-404b-adea-6ad40ad0d73f
Closing worker...
Thanks
Deon Griessel