You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Shaofeng SHI (JIRA)" <ji...@apache.org> on 2018/11/18 06:49:00 UTC
[jira] [Commented] (KYLIN-3494) build cube with spark
ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/KYLIN-3494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16690816#comment-16690816 ]
Shaofeng SHI commented on KYLIN-3494:
-------------------------------------
Could you please provide the cube json?
> build cube with spark ArrayIndexOutOfBoundsException
> -----------------------------------------------------
>
> Key: KYLIN-3494
> URL: https://issues.apache.org/jira/browse/KYLIN-3494
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Affects Versions: v2.4.0
> Reporter: jilai liu
> Priority: Major
> Labels: patch
>
> Logged in as: dr.who
> Application
> About
> Jobs
> Tools
> Log Type: stderr
> Log Upload Time: Mon Aug 13 15:50:10 +0800 2018
> Log Length: 74544
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/data3/test/data/hadoop/hdfs/data/usercache/hadoop/filecache/17809/__spark_libs__6649521663189541594.zip/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/data1/test/soft/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 18/08/13 15:49:38 INFO util.SignalUtils: Registered signal handler for TERM
> 18/08/13 15:49:38 INFO util.SignalUtils: Registered signal handler for HUP
> 18/08/13 15:49:38 INFO util.SignalUtils: Registered signal handler for INT
> 18/08/13 15:49:38 INFO yarn.ApplicationMaster: Preparing Local resources
> 18/08/13 15:49:39 INFO yarn.ApplicationMaster: ApplicationAttemptId: appattempt_1533616206085_5657_000001
> 18/08/13 15:49:39 INFO spark.SecurityManager: Changing view acls to: hadoop
> 18/08/13 15:49:39 INFO spark.SecurityManager: Changing modify acls to: hadoop
> 18/08/13 15:49:39 INFO spark.SecurityManager: Changing view acls groups to:
> 18/08/13 15:49:39 INFO spark.SecurityManager: Changing modify acls groups to:
> 18/08/13 15:49:39 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(hadoop); groups with view permissions: Set(); users with modify permissions: Set(hadoop); groups with modify permissions: Set()
> 18/08/13 15:49:39 INFO yarn.ApplicationMaster: Starting the user application in a separate Thread
> 18/08/13 15:49:39 INFO yarn.ApplicationMaster: Waiting for spark context initialization...
> 18/08/13 15:49:39 INFO spark.SparkContext: Running Spark version 2.1.2
> 18/08/13 15:49:39 INFO spark.SecurityManager: Changing view acls to: hadoop
> 18/08/13 15:49:39 INFO spark.SecurityManager: Changing modify acls to: hadoop
> 18/08/13 15:49:39 INFO spark.SecurityManager: Changing view acls groups to:
> 18/08/13 15:49:39 INFO spark.SecurityManager: Changing modify acls groups to:
> 18/08/13 15:49:39 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(hadoop); groups with view permissions: Set(); users with modify permissions: Set(hadoop); groups with modify permissions: Set()
> 18/08/13 15:49:40 INFO util.Utils: Successfully started service 'sparkDriver' on port 40358.
> 18/08/13 15:49:40 INFO spark.SparkEnv: Registering MapOutputTracker
> 18/08/13 15:49:40 INFO spark.SparkEnv: Registering BlockManagerMaster
> 18/08/13 15:49:40 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
> 18/08/13 15:49:40 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
> 18/08/13 15:49:40 INFO storage.DiskBlockManager: Created local directory at /data1/test/data/hadoop/hdfs/data/usercache/hadoop/appcache/application_1533616206085_5657/blockmgr-4cd0fed9-78ae-4e2c-826d-b42a8d6364d2
> 18/08/13 15:49:40 INFO storage.DiskBlockManager: Created local directory at /data2/test/data/hadoop/hdfs/data/usercache/hadoop/appcache/application_1533616206085_5657/blockmgr-8cca22e9-ece0-469b-b7fa-3cd9567504d9
> 18/08/13 15:49:40 INFO storage.DiskBlockManager: Created local directory at /data3/test/data/hadoop/hdfs/data/usercache/hadoop/appcache/application_1533616206085_5657/blockmgr-15f5073d-5ea2-4766-ab22-b8c68834fb80
> 18/08/13 15:49:40 INFO memory.MemoryStore: MemoryStore started with capacity 305.3 MB
> 18/08/13 15:49:40 INFO spark.SparkEnv: Registering OutputCommitCoordinator
> 18/08/13 15:49:40 INFO util.log: Logging initialized @2958ms
> 18/08/13 15:49:40 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
> 18/08/13 15:49:40 INFO server.Server: jetty-9.2.z-SNAPSHOT
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2506206a\{/jobs,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7f1b8616\{/jobs/json,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5001120\{/jobs/job,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4a662152\{/jobs/job/json,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5ef75d04\{/stages,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7dff5bfa\{/stages/json,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@78f3dc74\{/stages/stage,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3e40e89\{/stages/stage/json,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@679f6c8c\{/stages/pool,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@60b8cb0e\{/stages/pool/json,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@64eab11\{/storage,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@25fd6d17\{/storage/json,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@399a8e28\{/storage/rdd,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1fdcd2ae\{/storage/rdd/json,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6c39f467\{/environment,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@22e2b922\{/environment/json,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4fe49898\{/executors,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@36d46a68\{/executors/json,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@15ed2a19\{/executors/threadDump,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@22f1aa2f\{/executors/threadDump/json,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@493ad6b1\{/static,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5ca862d4\{/,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@206926ba\{/api,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@f66404a\{/jobs/job/kill,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@10f69049\{/stages/stage/kill,null,AVAILABLE,@Spark}
> 18/08/13 15:49:40 INFO server.ServerConnector: Started Spark@6e45bafc\{HTTP/1.1}{0.0.0.0:36502}
> 18/08/13 15:49:40 INFO server.Server: Started @3103ms
> 18/08/13 15:49:40 INFO util.Utils: Successfully started service 'SparkUI' on port 36502.
> 18/08/13 15:49:40 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://172.16.19.79:36502
> 18/08/13 15:49:40 INFO cluster.YarnClusterScheduler: Created YarnClusterScheduler
> 18/08/13 15:49:40 INFO cluster.SchedulerExtensionServices: Starting Yarn extension services with app application_1533616206085_5657 and attemptId Some(appattempt_1533616206085_5657_000001)
> 18/08/13 15:49:40 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 46273.
> 18/08/13 15:49:40 INFO netty.NettyBlockTransferService: Server created on 172.16.19.79:46273
> 18/08/13 15:49:40 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
> 18/08/13 15:49:40 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 172.16.19.79, 46273, None)
> 18/08/13 15:49:40 INFO storage.BlockManagerMasterEndpoint: Registering block manager 172.16.19.79:46273 with 305.3 MB RAM, BlockManagerId(driver, 172.16.19.79, 46273, None)
> 18/08/13 15:49:40 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 172.16.19.79, 46273, None)
> 18/08/13 15:49:40 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 172.16.19.79, 46273, None)
> 18/08/13 15:49:40 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@636d84f8\{/metrics/json,null,AVAILABLE,@Spark}
> 18/08/13 15:49:41 INFO scheduler.EventLoggingListener: Logging events to hdfs:///kylin/spark-history/application_1533616206085_5657_1
> 18/08/13 15:49:41 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(spark://YarnAM@172.16.19.79:40358)
> 18/08/13 15:49:41 INFO yarn.ApplicationMaster:
> ===============================================================================
> YARN executor launch context:
> env:
> CLASSPATH -> \{{PWD}}<CPS>\{{PWD}}/__spark_conf__<CPS>\{{PWD}}/__spark_libs__/*<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*
> SPARK_YARN_STAGING_DIR -> hdfs://test-online/user/hadoop/.sparkStaging/application_1533616206085_5657
> SPARK_USER -> hadoop
> SPARK_YARN_MODE -> true
> command:
> \{{JAVA_HOME}}/bin/java \
> -server \
> -Xmx8192m \
> '-Dhdp.version=current' \
> -Djava.io.tmpdir=\{{PWD}}/tmp \
> -Dspark.yarn.app.container.log.dir=<LOG_DIR> \
> -XX:OnOutOfMemoryError='kill %p' \
> org.apache.spark.executor.CoarseGrainedExecutorBackend \
> --driver-url \
> spark://CoarseGrainedScheduler@172.16.19.79:40358 \
> --executor-id \
> <executorId> \
> --hostname \
> <hostname> \
> --cores \
> 2 \
> --app-id \
> application_1533616206085_5657 \
> --user-class-path \
> file:$PWD/__app__.jar \
> 1><LOG_DIR>/stdout \
> 2><LOG_DIR>/stderr
> resources:
> __app__.jar -> resource \{ scheme: "hdfs" host: "test-online" port: -1 file: "/user/hadoop/.sparkStaging/application_1533616206085_5657/kylin-job-2.4.0.jar" } size: 34310788 timestamp: 1534146575706 type: FILE visibility: PRIVATE
> __spark_libs__ -> resource \{ scheme: "hdfs" host: "test-online" port: -1 file: "/user/hadoop/.sparkStaging/application_1533616206085_5657/__spark_libs__6649521663189541594.zip" } size: 200815831 timestamp: 1534146575510 type: ARCHIVE visibility: PRIVATE
> __spark_conf__ -> resource \{ scheme: "hdfs" host: "test-online" port: -1 file: "/user/hadoop/.sparkStaging/application_1533616206085_5657/__spark_conf__.zip" } size: 142316 timestamp: 1534146575796 type: ARCHIVE visibility: PRIVATE
> ===============================================================================
> 18/08/13 15:49:41 INFO yarn.YarnRMClient: Registering the ApplicationMaster
> 18/08/13 15:49:41 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
> 18/08/13 15:49:41 INFO yarn.YarnAllocator: Will request 10 executor container(s), each with 2 core(s) and 9011 MB memory (including 819 MB of overhead)
> 18/08/13 15:49:41 INFO yarn.YarnAllocator: Submitted 10 unlocalized container requests.
> 18/08/13 15:49:41 INFO yarn.ApplicationMaster: Started progress reporter thread with (heartbeat : 3000, initial allocation : 200) intervals
> 18/08/13 15:49:41 INFO impl.AMRMClientImpl: Received new token for : hadoop073:46083
> 18/08/13 15:49:41 INFO impl.AMRMClientImpl: Received new token for : hadoop057:36259
> 18/08/13 15:49:41 INFO impl.AMRMClientImpl: Received new token for : hadoop062:33335
> 18/08/13 15:49:41 INFO impl.AMRMClientImpl: Received new token for : hadoop067:44632
> 18/08/13 15:49:41 INFO impl.AMRMClientImpl: Received new token for : hadoop075:39257
> 18/08/13 15:49:41 INFO impl.AMRMClientImpl: Received new token for : hadoop051:45512
> 18/08/13 15:49:41 INFO yarn.YarnAllocator: Launching container container_e79_1533616206085_5657_01_000004 on host hadoop073
> 18/08/13 15:49:41 INFO yarn.YarnAllocator: Launching container container_e79_1533616206085_5657_01_000005 on host hadoop057
> 18/08/13 15:49:41 INFO yarn.YarnAllocator: Launching container container_e79_1533616206085_5657_01_000007 on host hadoop062
> 18/08/13 15:49:41 INFO yarn.YarnAllocator: Launching container container_e79_1533616206085_5657_01_000009 on host hadoop067
> 18/08/13 15:49:41 INFO yarn.YarnAllocator: Launching container container_e79_1533616206085_5657_01_000010 on host hadoop075
> 18/08/13 15:49:41 INFO yarn.YarnAllocator: Launching container container_e79_1533616206085_5657_01_000011 on host hadoop051
> 18/08/13 15:49:41 INFO yarn.YarnAllocator: Received 6 containers from YARN, launching executors on 6 of them.
> 18/08/13 15:49:41 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
> 18/08/13 15:49:41 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
> 18/08/13 15:49:41 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
> 18/08/13 15:49:41 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
> 18/08/13 15:49:41 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
> 18/08/13 15:49:41 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
> 18/08/13 15:49:41 INFO impl.ContainerManagementProtocolProxy: Opening proxy : hadoop057:36259
> 18/08/13 15:49:41 INFO impl.ContainerManagementProtocolProxy: Opening proxy : hadoop067:44632
> 18/08/13 15:49:41 INFO impl.ContainerManagementProtocolProxy: Opening proxy : hadoop073:46083
> 18/08/13 15:49:41 INFO impl.ContainerManagementProtocolProxy: Opening proxy : hadoop075:39257
> 18/08/13 15:49:41 INFO impl.ContainerManagementProtocolProxy: Opening proxy : hadoop051:45512
> 18/08/13 15:49:41 INFO impl.ContainerManagementProtocolProxy: Opening proxy : hadoop062:33335
> 18/08/13 15:49:42 INFO impl.AMRMClientImpl: Received new token for : hadoop070:42023
> 18/08/13 15:49:42 INFO impl.AMRMClientImpl: Received new token for : hadoop063:34154
> 18/08/13 15:49:42 INFO impl.AMRMClientImpl: Received new token for : hadoop053:33601
> 18/08/13 15:49:42 INFO impl.AMRMClientImpl: Received new token for : hadoop079:40497
> 18/08/13 15:49:42 INFO yarn.YarnAllocator: Launching container container_e79_1533616206085_5657_01_000013 on host hadoop070
> 18/08/13 15:49:42 INFO yarn.YarnAllocator: Launching container container_e79_1533616206085_5657_01_000017 on host hadoop063
> 18/08/13 15:49:42 INFO yarn.YarnAllocator: Launching container container_e79_1533616206085_5657_01_000018 on host hadoop053
> 18/08/13 15:49:42 INFO yarn.YarnAllocator: Launching container container_e79_1533616206085_5657_01_000020 on host hadoop079
> 18/08/13 15:49:42 INFO yarn.YarnAllocator: Received 4 containers from YARN, launching executors on 4 of them.
> 18/08/13 15:49:42 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
> 18/08/13 15:49:42 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
> 18/08/13 15:49:42 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
> 18/08/13 15:49:42 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
> 18/08/13 15:49:42 INFO impl.ContainerManagementProtocolProxy: Opening proxy : hadoop079:40497
> 18/08/13 15:49:42 INFO impl.ContainerManagementProtocolProxy: Opening proxy : hadoop070:42023
> 18/08/13 15:49:42 INFO impl.ContainerManagementProtocolProxy: Opening proxy : hadoop063:34154
> 18/08/13 15:49:42 INFO impl.ContainerManagementProtocolProxy: Opening proxy : hadoop053:33601
> 18/08/13 15:49:44 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (172.16.19.79:39942) with ID 10
> 18/08/13 15:49:44 INFO storage.BlockManagerMasterEndpoint: Registering block manager hadoop079:44235 with 3.4 GB RAM, BlockManagerId(10, hadoop079, 44235, None)
> 18/08/13 15:49:45 INFO impl.AMRMClientImpl: Received new token for : hadoop060:39607
> 18/08/13 15:49:45 INFO impl.AMRMClientImpl: Received new token for : hadoop084:38240
> 18/08/13 15:49:45 INFO impl.AMRMClientImpl: Received new token for : hadoop061:40680
> 18/08/13 15:49:45 INFO impl.AMRMClientImpl: Received new token for : hadoop074:38414
> 18/08/13 15:49:45 INFO yarn.YarnAllocator: Received 4 containers from YARN, launching executors on 0 of them.
> 18/08/13 15:49:45 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (172.16.19.57:35082) with ID 2
> 18/08/13 15:49:45 INFO storage.BlockManagerMasterEndpoint: Registering block manager hadoop057:40961 with 3.4 GB RAM, BlockManagerId(2, hadoop057, 40961, None)
> 18/08/13 15:49:45 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (172.16.19.51:35812) with ID 6
> 18/08/13 15:49:45 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (172.16.19.75:45130) with ID 5
> 18/08/13 15:49:45 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (172.16.19.63:38512) with ID 8
> 18/08/13 15:49:45 INFO storage.BlockManagerMasterEndpoint: Registering block manager hadoop075:33510 with 3.4 GB RAM, BlockManagerId(5, hadoop075, 33510, None)
> 18/08/13 15:49:45 INFO storage.BlockManagerMasterEndpoint: Registering block manager hadoop063:46329 with 3.4 GB RAM, BlockManagerId(8, hadoop063, 46329, None)
> 18/08/13 15:49:45 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (172.16.19.73:33060) with ID 1
> 18/08/13 15:49:45 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (172.16.19.53:38988) with ID 9
> 18/08/13 15:49:45 INFO storage.BlockManagerMasterEndpoint: Registering block manager hadoop051:43539 with 3.4 GB RAM, BlockManagerId(6, hadoop051, 43539, None)
> 18/08/13 15:49:45 INFO storage.BlockManagerMasterEndpoint: Registering block manager hadoop053:40239 with 3.4 GB RAM, BlockManagerId(9, hadoop053, 40239, None)
> 18/08/13 15:49:45 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (172.16.19.67:33364) with ID 4
> 18/08/13 15:49:45 INFO storage.BlockManagerMasterEndpoint: Registering block manager hadoop073:39426 with 3.4 GB RAM, BlockManagerId(1, hadoop073, 39426, None)
> 18/08/13 15:49:45 INFO storage.BlockManagerMasterEndpoint: Registering block manager hadoop067:37637 with 3.4 GB RAM, BlockManagerId(4, hadoop067, 37637, None)
> 18/08/13 15:49:45 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (172.16.19.62:37576) with ID 3
> 18/08/13 15:49:45 INFO cluster.YarnClusterSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
> 18/08/13 15:49:45 INFO cluster.YarnClusterScheduler: YarnClusterScheduler.postStartHook done
> 18/08/13 15:49:45 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (172.16.19.70:59750) with ID 7
> 18/08/13 15:49:45 INFO common.AbstractHadoopJob: Ready to load KylinConfig from uri: kylin_metadata@hdfs,path=hdfs://test-online/kylin/kylin_metadata/kylin-ed7d5d0d-1007-404d-8512-73e946cfaa73/commonlog_andriod_cube7/metadata
> 18/08/13 15:49:45 INFO storage.BlockManagerMasterEndpoint: Registering block manager hadoop062:34791 with 3.4 GB RAM, BlockManagerId(3, hadoop062, 34791, None)
> 18/08/13 15:49:45 INFO storage.BlockManagerMasterEndpoint: Registering block manager hadoop070:34116 with 3.4 GB RAM, BlockManagerId(7, hadoop070, 34116, None)
> 18/08/13 15:49:46 INFO common.KylinConfig: Creating new manager instance of class org.apache.kylin.cube.CubeManager
> 18/08/13 15:49:46 INFO cube.CubeManager: Initializing CubeManager with config null
> 18/08/13 15:49:46 INFO persistence.ResourceStore: Using metadata url kylin_metadata@hdfs,path=hdfs://test-online/kylin/kylin_metadata/kylin-ed7d5d0d-1007-404d-8512-73e946cfaa73/commonlog_andriod_cube7/metadata for resource store
> 18/08/13 15:49:46 INFO persistence.HDFSResourceStore: hdfs meta path : hdfs://test-online/kylin/kylin_metadata/kylin-ed7d5d0d-1007-404d-8512-73e946cfaa73/commonlog_andriod_cube7/metadata
> 18/08/13 15:49:46 INFO common.KylinConfig: Creating new manager instance of class org.apache.kylin.cube.CubeDescManager
> 18/08/13 15:49:46 INFO cube.CubeDescManager: Initializing CubeDescManager with config kylin_metadata@hdfs,path=hdfs://test-online/kylin/kylin_metadata/kylin-ed7d5d0d-1007-404d-8512-73e946cfaa73/commonlog_andriod_cube7/metadata
> 18/08/13 15:49:46 INFO common.KylinConfig: Creating new manager instance of class org.apache.kylin.metadata.project.ProjectManager
> 18/08/13 15:49:46 INFO project.ProjectManager: Initializing ProjectManager with metadata url kylin_metadata@hdfs,path=hdfs://test-online/kylin/kylin_metadata/kylin-ed7d5d0d-1007-404d-8512-73e946cfaa73/commonlog_andriod_cube7/metadata
> 18/08/13 15:49:46 INFO common.KylinConfig: Creating new manager instance of class org.apache.kylin.metadata.cachesync.Broadcaster
> 18/08/13 15:49:46 INFO common.KylinConfig: Creating new manager instance of class org.apache.kylin.metadata.model.DataModelManager
> 18/08/13 15:49:46 INFO common.KylinConfig: Creating new manager instance of class org.apache.kylin.metadata.TableMetadataManager
> 18/08/13 15:49:46 INFO measure.MeasureTypeFactory: Checking custom measure types from kylin config
> 18/08/13 15:49:46 INFO measure.MeasureTypeFactory: registering COUNT_DISTINCT(hllc), class org.apache.kylin.measure.hllc.HLLCMeasureType$Factory
> 18/08/13 15:49:46 INFO measure.MeasureTypeFactory: registering COUNT_DISTINCT(bitmap), class org.apache.kylin.measure.bitmap.BitmapMeasureType$Factory
> 18/08/13 15:49:46 INFO measure.MeasureTypeFactory: registering TOP_N(topn), class org.apache.kylin.measure.topn.TopNMeasureType$Factory
> 18/08/13 15:49:46 INFO measure.MeasureTypeFactory: registering RAW(raw), class org.apache.kylin.measure.raw.RawMeasureType$Factory
> 18/08/13 15:49:46 INFO measure.MeasureTypeFactory: registering EXTENDED_COLUMN(extendedcolumn), class org.apache.kylin.measure.extendedcolumn.ExtendedColumnMeasureType$Factory
> 18/08/13 15:49:46 INFO measure.MeasureTypeFactory: registering PERCENTILE_APPROX(percentile), class org.apache.kylin.measure.percentile.PercentileMeasureType$Factory
> 18/08/13 15:49:46 INFO measure.MeasureTypeFactory: registering COUNT_DISTINCT(dim_dc), class org.apache.kylin.measure.dim.DimCountDistinctMeasureType$Factory
> 18/08/13 15:49:46 INFO model.DataModelManager: Model commonlog_iphone_model is missing or unloaded yet
> 18/08/13 15:49:46 INFO model.DataModelManager: Model commonlog_andriod_model is missing or unloaded yet
> 18/08/13 15:49:46 INFO spark.SparkCubingByLayer: RDD input path: hdfs://test-online/kylin/kylin_metadata/kylin-ed7d5d0d-1007-404d-8512-73e946cfaa73/kylin_intermediate_commonlog_andriod_cube7_d1db5d22_cf49_48fc_b9c0_54e8e70cf362
> 18/08/13 15:49:46 INFO spark.SparkCubingByLayer: RDD Output path: hdfs://test-online/kylin/kylin_metadata/kylin-ed7d5d0d-1007-404d-8512-73e946cfaa73/commonlog_andriod_cube7/cuboid/
> 18/08/13 15:49:46 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
> 18/08/13 15:49:46 INFO compress.CodecPool: Got brand-new decompressor [.deflate]
> 18/08/13 15:49:46 INFO spark.SparkCubingByLayer: All measure are normal (agg on all cuboids) ? : true
> 18/08/13 15:49:47 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 297.1 KB, free 305.0 MB)
> 18/08/13 15:49:47 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 25.5 KB, free 304.9 MB)
> 18/08/13 15:49:47 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 172.16.19.79:46273 (size: 25.5 KB, free: 305.2 MB)
> 18/08/13 15:49:47 INFO spark.SparkContext: Created broadcast 0 from sequenceFile at SparkCubingByLayer.java:182
> 18/08/13 15:49:47 INFO common.KylinConfig: Creating new manager instance of class org.apache.kylin.cube.cuboid.CuboidManager
> 18/08/13 15:49:47 INFO common.KylinConfig: Creating new manager instance of class org.apache.kylin.dict.DictionaryManager
> 18/08/13 15:49:47 INFO dict.DictionaryManager: DictionaryManager(258090034) loading DictionaryInfo(loadDictObj:true) at /dict/ODS.COMMONLOG_ANDROID/APP_VERSION/f0950bcb-7ec1-40b2-b7aa-394989976397.dict
> 18/08/13 15:49:47 INFO dict.DictionaryManager: DictionaryManager(258090034) loading DictionaryInfo(loadDictObj:true) at /dict/ODS.COMMONLOG_ANDROID/PID/b91eeb9d-a818-4648-816d-8ec209a97641.dict
> 18/08/13 15:49:47 INFO dict.DictionaryManager: DictionaryManager(258090034) loading DictionaryInfo(loadDictObj:true) at /dict/ODS.COMMONLOG_ANDROID/LANGUAGE/f63e6df9-73b8-47c1-afa4-2d5bd43926c3.dict
> 18/08/13 15:49:47 INFO dict.DictionaryManager: DictionaryManager(258090034) loading DictionaryInfo(loadDictObj:true) at /dict/ODS.COMMONLOG_ANDROID/OS_VERSION/2c3fa118-448e-45b4-b8c7-c8e2c5d3090c.dict
> 18/08/13 15:49:47 INFO dict.DictionaryManager: DictionaryManager(258090034) loading DictionaryInfo(loadDictObj:true) at /dict/ODS.COMMONLOG_ANDROID/VALUE/c1e34ccc-68a2-4acb-a643-8d17c3067e74.dict
> 18/08/13 15:49:47 INFO dict.DictionaryManager: DictionaryManager(258090034) loading DictionaryInfo(loadDictObj:true) at /dict/ODS.COMMONLOG_ANDROID/DU/e125650e-2cb8-4b51-ba99-739476b08d33.dict
> 18/08/13 15:49:47 INFO dict.DictionaryManager: DictionaryManager(258090034) loading DictionaryInfo(loadDictObj:true) at /dict/ODS.COMMONLOG_ANDROID/KEY/b779f6d9-23a6-45e2-8dbf-83f898670ec2.dict
> 18/08/13 15:49:47 INFO dict.DictionaryManager: DictionaryManager(258090034) loading DictionaryInfo(loadDictObj:true) at /dict/ODS.COMMONLOG_ANDROID/NET/dba93801-f8cc-4958-97ba-e2cdeec7abd4.dict
> 18/08/13 15:49:47 INFO dict.DictionaryManager: DictionaryManager(258090034) loading DictionaryInfo(loadDictObj:true) at /dict/ODS.COMMONLOG_ANDROID/CITYID/ebadaeae-c3dd-45d6-9cb8-4c0ecef82cee.dict
> 18/08/13 15:49:47 INFO common.CubeStatsReader: Estimating size for layer 0, all cuboids are 511, total size is 449.9567413330078
> 18/08/13 15:49:47 INFO spark.SparkCubingByLayer: Partition for spark cubing: 44
> 18/08/13 15:49:47 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
> 18/08/13 15:49:47 INFO spark.SparkContext: Starting job: saveAsNewAPIHadoopDataset at SparkCubingByLayer.java:277
> 18/08/13 15:49:47 INFO mapred.FileInputFormat: Total input paths to process : 44
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /dc1/rack6/172.16.19.86:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /default/rack/172.16.19.99:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /default/rack/172.16.19.106:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /dc1/rack4/172.16.19.73:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /dc1/rack4/172.16.19.74:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /dc1/rack5/172.16.19.84:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /default/rack/172.16.19.103:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /dc1/rack4/172.16.19.71:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /dc1/rack4/172.16.19.72:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /dc1/rack2/172.16.19.59:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /dc1/rack1/172.16.19.51:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /dc1/rack1/172.16.19.50:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /dc1/rack5/172.16.19.81:50010
> 18/08/13 15:49:48 INFO net.NetworkTopology: Adding a new node: /dc1/rack5/172.16.19.83:50010
> 18/08/13 15:49:48 INFO scheduler.DAGScheduler: Registering RDD 3 (mapToPair at SparkCubingByLayer.java:182)
> 18/08/13 15:49:48 INFO scheduler.DAGScheduler: Got job 0 (saveAsNewAPIHadoopDataset at SparkCubingByLayer.java:277) with 44 output partitions
> 18/08/13 15:49:48 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (saveAsNewAPIHadoopDataset at SparkCubingByLayer.java:277)
> 18/08/13 15:49:48 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
> 18/08/13 15:49:48 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
> 18/08/13 15:49:48 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[3] at mapToPair at SparkCubingByLayer.java:182), which has no missing parents
> 18/08/13 15:49:48 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 79.7 KB, free 304.9 MB)
> 18/08/13 15:49:48 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 30.0 KB, free 304.8 MB)
> 18/08/13 15:49:48 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 172.16.19.79:46273 (size: 30.0 KB, free: 305.2 MB)
> 18/08/13 15:49:48 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:996
> 18/08/13 15:49:48 INFO scheduler.DAGScheduler: Submitting 105 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[3] at mapToPair at SparkCubingByLayer.java:182)
> 18/08/13 15:49:48 INFO cluster.YarnClusterScheduler: Adding task set 0.0 with 105 tasks
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 0.0 (TID 0, hadoop067, executor 4, partition 14, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 0.0 (TID 1, hadoop075, executor 5, partition 4, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 0.0 (TID 2, hadoop057, executor 2, partition 10, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 0.0 (TID 3, hadoop053, executor 9, partition 13, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 0.0 (TID 4, hadoop070, executor 7, partition 11, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 0.0 (TID 5, hadoop062, executor 3, partition 16, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 0.0 (TID 6, hadoop079, executor 10, partition 18, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 0.0 (TID 7, hadoop063, executor 8, partition 31, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 0.0 (TID 8, hadoop073, executor 1, partition 25, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 0.0 (TID 9, hadoop075, executor 5, partition 5, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 0.0 (TID 10, hadoop057, executor 2, partition 45, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 0.0 (TID 11, hadoop053, executor 9, partition 22, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 0.0 (TID 12, hadoop070, executor 7, partition 19, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 0.0 (TID 13, hadoop062, executor 3, partition 42, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 0.0 (TID 14, hadoop079, executor 10, partition 21, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 0.0 (TID 15, hadoop063, executor 8, partition 33, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 0.0 (TID 16, hadoop073, executor 1, partition 26, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:48 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on hadoop079:44235 (size: 30.0 KB, free: 3.4 GB)
> 18/08/13 15:49:48 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on hadoop053:40239 (size: 30.0 KB, free: 3.4 GB)
> 18/08/13 15:49:48 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on hadoop073:39426 (size: 30.0 KB, free: 3.4 GB)
> 18/08/13 15:49:48 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on hadoop067:37637 (size: 30.0 KB, free: 3.4 GB)
> 18/08/13 15:49:48 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on hadoop070:34116 (size: 30.0 KB, free: 3.4 GB)
> 18/08/13 15:49:48 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on hadoop063:46329 (size: 30.0 KB, free: 3.4 GB)
> 18/08/13 15:49:48 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on hadoop075:33510 (size: 30.0 KB, free: 3.4 GB)
> 18/08/13 15:49:48 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on hadoop062:34791 (size: 30.0 KB, free: 3.4 GB)
> 18/08/13 15:49:48 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on hadoop057:40961 (size: 30.0 KB, free: 3.4 GB)
> 18/08/13 15:49:49 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on hadoop062:34791 (size: 25.5 KB, free: 3.4 GB)
> 18/08/13 15:49:49 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on hadoop075:33510 (size: 25.5 KB, free: 3.4 GB)
> 18/08/13 15:49:49 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on hadoop070:34116 (size: 25.5 KB, free: 3.4 GB)
> 18/08/13 15:49:49 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on hadoop067:37637 (size: 25.5 KB, free: 3.4 GB)
> 18/08/13 15:49:49 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on hadoop073:39426 (size: 25.5 KB, free: 3.4 GB)
> 18/08/13 15:49:49 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on hadoop063:46329 (size: 25.5 KB, free: 3.4 GB)
> 18/08/13 15:49:49 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on hadoop057:40961 (size: 25.5 KB, free: 3.4 GB)
> 18/08/13 15:49:49 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on hadoop053:40239 (size: 25.5 KB, free: 3.4 GB)
> 18/08/13 15:49:49 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on hadoop079:44235 (size: 25.5 KB, free: 3.4 GB)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 0.0 (TID 17, hadoop067, executor 4, partition 15, RACK_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 18, hadoop051, executor 6, partition 1, RACK_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 0.0 (TID 19, hadoop051, executor 6, partition 12, RACK_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 0.0 (TID 20, hadoop075, executor 5, partition 6, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 0.0 (TID 21, hadoop075, executor 5, partition 7, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 WARN scheduler.TaskSetManager: Lost task 4.0 in stage 0.0 (TID 1, hadoop075, executor 5): java.lang.ArrayIndexOutOfBoundsException: 8
> at org.apache.kylin.engine.mr.common.BaseCuboidBuilder.getCell(BaseCuboidBuilder.java:167)
> at org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildKey(BaseCuboidBuilder.java:116)
> at org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:343)
> at org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:307)
> at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043)
> at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043)
> at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
> at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:193)
> at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:325)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Lost task 5.0 in stage 0.0 (TID 9) on hadoop075, executor 5: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 1]
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 0.0 (TID 22, hadoop070, executor 7, partition 20, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Lost task 19.0 in stage 0.0 (TID 12) on hadoop070, executor 7: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 2]
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 19.1 in stage 0.0 (TID 23, hadoop070, executor 7, partition 19, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Lost task 11.0 in stage 0.0 (TID 4) on hadoop070, executor 7: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 3]
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Lost task 14.0 in stage 0.0 (TID 0) on hadoop067, executor 4: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 4]
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 14.1 in stage 0.0 (TID 24, hadoop067, executor 4, partition 14, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 50.0 in stage 0.0 (TID 25, hadoop079, executor 10, partition 50, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Lost task 21.0 in stage 0.0 (TID 14) on hadoop079, executor 10: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 5]
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 52.0 in stage 0.0 (TID 26, hadoop057, executor 2, partition 52, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Lost task 45.0 in stage 0.0 (TID 10) on hadoop057, executor 2: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 6]
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 45.1 in stage 0.0 (TID 27, hadoop057, executor 2, partition 45, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Lost task 10.0 in stage 0.0 (TID 2) on hadoop057, executor 2: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 7]
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Lost task 15.0 in stage 0.0 (TID 17) on hadoop067, executor 4: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 8]
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 10.1 in stage 0.0 (TID 28, hadoop062, executor 3, partition 10, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Lost task 42.0 in stage 0.0 (TID 13) on hadoop062, executor 3: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 9]
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Starting task 42.1 in stage 0.0 (TID 29, hadoop062, executor 3, partition 42, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:51 INFO scheduler.TaskSetManager: Lost task 16.0 in stage 0.0 (TID 5) on hadoop062, executor 3: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 10]
> 18/08/13 15:49:52 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on hadoop051:43539 (size: 30.0 KB, free: 3.4 GB)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 14.1 in stage 0.0 (TID 24) on hadoop067, executor 4: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 11]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 14.2 in stage 0.0 (TID 30, hadoop067, executor 4, partition 14, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 11.1 in stage 0.0 (TID 31, hadoop075, executor 5, partition 11, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 7.0 in stage 0.0 (TID 21) on hadoop075, executor 5: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 12]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 7.1 in stage 0.0 (TID 32, hadoop075, executor 5, partition 7, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 6.0 in stage 0.0 (TID 20) on hadoop075, executor 5: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 13]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 0.0 (TID 33, hadoop073, executor 1, partition 27, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 26.0 in stage 0.0 (TID 16) on hadoop073, executor 1: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 14]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 26.1 in stage 0.0 (TID 34, hadoop073, executor 1, partition 26, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 90.0 in stage 0.0 (TID 35, hadoop063, executor 8, partition 90, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 25.0 in stage 0.0 (TID 8) on hadoop073, executor 1: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 15]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 33.0 in stage 0.0 (TID 15) on hadoop063, executor 8: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 16]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 21.1 in stage 0.0 (TID 36, hadoop070, executor 7, partition 21, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 20.0 in stage 0.0 (TID 22) on hadoop070, executor 7: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 17]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 75.0 in stage 0.0 (TID 37, hadoop057, executor 2, partition 75, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 52.0 in stage 0.0 (TID 26) on hadoop057, executor 2: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 18]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 20.1 in stage 0.0 (TID 38, hadoop070, executor 7, partition 20, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 19.1 in stage 0.0 (TID 23) on hadoop070, executor 7: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 19]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 52.1 in stage 0.0 (TID 39, hadoop057, executor 2, partition 52, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 45.1 in stage 0.0 (TID 27) on hadoop057, executor 2: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 20]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 50.0 in stage 0.0 (TID 25) on hadoop079, executor 10: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 21]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 50.1 in stage 0.0 (TID 40, hadoop079, executor 10, partition 50, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 33.1 in stage 0.0 (TID 41, hadoop063, executor 8, partition 33, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 31.0 in stage 0.0 (TID 7) on hadoop063, executor 8: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 22]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 18.0 in stage 0.0 (TID 6) on hadoop079, executor 10: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 23]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 18.1 in stage 0.0 (TID 42, hadoop079, executor 10, partition 18, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 19.2 in stage 0.0 (TID 43, hadoop070, executor 7, partition 19, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 21.1 in stage 0.0 (TID 36) on hadoop070, executor 7: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 24]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 14.2 in stage 0.0 (TID 30) on hadoop067, executor 4: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 25]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 14.3 in stage 0.0 (TID 44, hadoop067, executor 4, partition 14, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 31.1 in stage 0.0 (TID 45, hadoop063, executor 8, partition 31, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 90.0 in stage 0.0 (TID 35) on hadoop063, executor 8: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 26]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 16.1 in stage 0.0 (TID 46, hadoop062, executor 3, partition 16, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 42.1 in stage 0.0 (TID 29) on hadoop062, executor 3: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 27]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 45.2 in stage 0.0 (TID 47, hadoop057, executor 2, partition 45, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 52.1 in stage 0.0 (TID 39) on hadoop057, executor 2: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 28]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 42.2 in stage 0.0 (TID 48, hadoop062, executor 3, partition 42, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 10.1 in stage 0.0 (TID 28) on hadoop062, executor 3: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 29]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 10.2 in stage 0.0 (TID 49, hadoop075, executor 5, partition 10, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 7.1 in stage 0.0 (TID 32) on hadoop075, executor 5: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 30]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 21.2 in stage 0.0 (TID 50, hadoop070, executor 7, partition 21, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 20.1 in stage 0.0 (TID 38) on hadoop070, executor 7: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 31]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 7.2 in stage 0.0 (TID 51, hadoop075, executor 5, partition 7, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 11.1 in stage 0.0 (TID 31) on hadoop075, executor 5: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 32]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 52.2 in stage 0.0 (TID 52, hadoop057, executor 2, partition 52, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 75.0 in stage 0.0 (TID 37) on hadoop057, executor 2: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 33]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 0.0 (TID 53, hadoop053, executor 9, partition 35, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 22.0 in stage 0.0 (TID 11) on hadoop053, executor 9: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 34]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 22.1 in stage 0.0 (TID 54, hadoop053, executor 9, partition 22, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 13.0 in stage 0.0 (TID 3) on hadoop053, executor 9: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 35]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 90.1 in stage 0.0 (TID 55, hadoop073, executor 1, partition 90, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 26.1 in stage 0.0 (TID 34) on hadoop073, executor 1: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 36]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Starting task 26.2 in stage 0.0 (TID 56, hadoop073, executor 1, partition 26, NODE_LOCAL, 6146 bytes)
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 27.0 in stage 0.0 (TID 33) on hadoop073, executor 1: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 37]
> 18/08/13 15:49:52 INFO scheduler.TaskSetManager: Lost task 14.3 in stage 0.0 (TID 44) on hadoop067, executor 4: java.lang.ArrayIndexOutOfBoundsException (8) [duplicate 38]
> 18/08/13 15:49:52 ERROR scheduler.TaskSetManager: Task 14 in stage 0.0 failed 4 times; aborting job
> 18/08/13 15:49:52 INFO cluster.YarnClusterScheduler: Cancelling stage 0
> 18/08/13 15:49:52 INFO cluster.YarnClusterScheduler: Stage 0 was cancelled
> 18/08/13 15:49:52 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (mapToPair at SparkCubingByLayer.java:182) failed in 4.198 s due to Job aborted due to stage failure: Task 14 in stage 0.0 failed 4 times, most recent failure: Lost task 14.3 in stage 0.0 (TID 44, hadoop067, executor 4): java.lang.ArrayIndexOutOfBoundsException: 8
> at org.apache.kylin.engine.mr.common.BaseCuboidBuilder.getCell(BaseCuboidBuilder.java:167)
> at org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildKey(BaseCuboidBuilder.java:116)
> at org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:343)
> at org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:307)
> at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043)
> at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043)
> at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
> at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:193)
> at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:325)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Driver stacktrace:
> 18/08/13 15:49:52 INFO scheduler.DAGScheduler: Job 0 failed: saveAsNewAPIHadoopDataset at SparkCubingByLayer.java:277, took 4.768935 s
> 18/08/13 15:49:52 ERROR yarn.ApplicationMaster: User class threw exception: java.lang.RuntimeException: error execute org.apache.kylin.engine.spark.SparkCubingByLayer
> java.lang.RuntimeException: error execute org.apache.kylin.engine.spark.SparkCubingByLayer
> at org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
> at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:636)
> Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 14 in stage 0.0 failed 4 times, most recent failure: Lost task 14.3 in stage 0.0 (TID 44, hadoop067, executor 4): java.lang.ArrayIndexOutOfBoundsException: 8
> at org.apache.kylin.engine.mr.common.BaseCuboidBuilder.getCell(BaseCuboidBuilder.java:167)
> at org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildKey(BaseCuboidBuilder.java:116)
> at org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:343)
> at org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:307)
> at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043)
> at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043)
> at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
> at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:193)
> at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:325)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Driver stacktrace:
> at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1435)
> at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1423)
> at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1422)
> at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1422)
> at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)
> at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)
> at scala.Option.foreach(Option.scala:257)
> at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:802)
> at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1650)
> at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)
> at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)
> at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
> at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:628)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1928)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1941)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1961)
> at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1158)
> at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:1085)
> at org.apache.spark.api.java.JavaPairRDD.saveAsNewAPIHadoopDataset(JavaPairRDD.scala:831)
> at org.apache.kylin.engine.spark.SparkCubingByLayer.saveToHDFS(SparkCubingByLayer.java:277)
> at org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:230)
> at org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 6 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 8
> at org.apache.kylin.engine.mr.common.BaseCuboidBuilder.getCell(BaseCuboidBuilder.java:167)
> at org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildKey(BaseCuboidBuilder.java:116)
> at org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:343)
> at org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:307)
> at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043)
> at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043)
> at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
> at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:193)
> at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:325)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> 18/08/13 15:49:52 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.lang.RuntimeException: error execute org.apache.kylin.engine.spark.SparkCubingByLayer)
> 18/08/13 15:49:52 INFO spark.SparkContext: Invoking stop() from shutdown hook
> 18/08/13 15:49:52 INFO server.ServerConnector: Stopped Spark@6e45bafc\{HTTP/1.1}{0.0.0.0:0}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@10f69049\{/stages/stage/kill,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@f66404a\{/jobs/job/kill,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@206926ba\{/api,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5ca862d4\{/,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@493ad6b1\{/static,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@22f1aa2f\{/executors/threadDump/json,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@15ed2a19\{/executors/threadDump,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@36d46a68\{/executors/json,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4fe49898\{/executors,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@22e2b922\{/environment/json,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@6c39f467\{/environment,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1fdcd2ae\{/storage/rdd/json,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@399a8e28\{/storage/rdd,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@25fd6d17\{/storage/json,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@64eab11\{/storage,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@60b8cb0e\{/stages/pool/json,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@679f6c8c\{/stages/pool,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3e40e89\{/stages/stage/json,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@78f3dc74\{/stages/stage,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7dff5bfa\{/stages/json,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5ef75d04\{/stages,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4a662152\{/jobs/job/json,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5001120\{/jobs/job,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7f1b8616\{/jobs/json,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2506206a\{/jobs,null,UNAVAILABLE,@Spark}
> 18/08/13 15:49:52 INFO ui.SparkUI: Stopped Spark web UI at http://172.16.19.79:36502
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 16.1 in stage 0.0 (TID 46, hadoop062, executor 3): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@5725003b,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 45.2 in stage 0.0 (TID 47, hadoop057, executor 2): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@18373e72,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 22.1 in stage 0.0 (TID 54, hadoop053, executor 9): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@3b7b08b6,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 90.1 in stage 0.0 (TID 55, hadoop073, executor 1): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@1707c5a7,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 35.0 in stage 0.0 (TID 53, hadoop053, executor 9): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@68dc3ba4,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 18.1 in stage 0.0 (TID 42, hadoop079, executor 10): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@49394c63,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 7.2 in stage 0.0 (TID 51, hadoop075, executor 5): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@2a64d34a,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 26.2 in stage 0.0 (TID 56, hadoop073, executor 1): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@367115c,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 21.2 in stage 0.0 (TID 50, hadoop070, executor 7): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@a6f1b8f,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 31.1 in stage 0.0 (TID 45, hadoop063, executor 8): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@93604f5,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 50.1 in stage 0.0 (TID 40, hadoop079, executor 10): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@544681a,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 42.2 in stage 0.0 (TID 48, hadoop062, executor 3): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@3e12741f,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 19.2 in stage 0.0 (TID 43, hadoop070, executor 7): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@4c53b331,null)
> 18/08/13 15:49:52 WARN scheduler.TaskSetManager: Lost task 52.2 in stage 0.0 (TID 52, hadoop057, executor 2): TaskKilled (killed intentionally)
> 18/08/13 15:49:52 ERROR scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerTaskEnd(0,0,ShuffleMapTask,TaskKilled,org.apache.spark.scheduler.TaskInfo@757b07e2,null)
> 18/08/13 15:49:52 INFO yarn.YarnAllocator: Driver requested a total number of 0 executor(s).
> 18/08/13 15:49:52 INFO cluster.YarnClusterSchedulerBackend: Shutting down all executors
> 18/08/13 15:49:52 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
> 18/08/13 15:49:52 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices
> (serviceOption=None,
> services=List(),
> started=false)
> 18/08/13 15:49:52 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
> 18/08/13 15:49:52 ERROR server.TransportRequestHandler: Error while invoking RpcHandler#receive() for one-way message.
> org.apache.spark.SparkException: Could not find CoarseGrainedScheduler.
> at org.apache.spark.rpc.netty.Dispatcher.postMessage(Dispatcher.scala:154)
> at org.apache.spark.rpc.netty.Dispatcher.postOneWayMessage(Dispatcher.scala:134)
> at org.apache.spark.rpc.netty.NettyRpcHandler.receive(NettyRpcEnv.scala:570)
> at org.apache.spark.network.server.TransportRequestHandler.processOneWayMessage(TransportRequestHandler.java:180)
> at org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:109)
> at org.apache.spark.network.server.TransportChannelHandler.channelRead(TransportChannelHandler.java:118)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:336)
> at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:336)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:336)
> at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:336)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:643)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:566)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:480)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:442)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
> at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
> at java.lang.Thread.run(Thread.java:748)
> 18/08/13 15:49:52 ERROR server.TransportRequestHandler: Error while invoking RpcHandler#receive() for one-way message.
> org.apache.spark.SparkException: Could not find CoarseGrainedScheduler.
> at org.apache.spark.rpc.netty.Dispatcher.postMessage(Dispatcher.scala:154)
> at org.apache.spark.rpc.netty.Dispatcher.postOneWayMessage(Dispatcher.scala:134)
> at org.apache.spark.rpc.netty.NettyRpcHandler.receive(NettyRpcEnv.scala:570)
> at org.apache.spark.network.server.TransportRequestHandler.processOneWayMessage(TransportRequestHandler.java:180)
> at org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:109)
> at org.apache.spark.network.server.TransportChannelHandler.channelRead(TransportChannelHandler.java:118)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:336)
> at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:336)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:336)
> at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:336)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:643)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:566)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:480)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:442)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
> at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
> at java.lang.Thread.run(Thread.java:748)
> 18/08/13 15:49:52 INFO memory.MemoryStore: MemoryStore cleared
> 18/08/13 15:49:52 INFO storage.BlockManager: BlockManager stopped
> 18/08/13 15:49:52 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
> 18/08/13 15:49:52 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
> 18/08/13 15:49:52 INFO spark.SparkContext: Successfully stopped SparkContext
> 18/08/13 15:49:52 INFO util.ShutdownHookManager: Shutdown hook called
> 18/08/13 15:49:52 INFO util.ShutdownHookManager: Deleting directory /data3/test/data/hadoop/hdfs/data/usercache/hadoop/appcache/application_1533616206085_5657/spark-69de25a9-16d2-4eaa-be70-d23ec191776a
> 18/08/13 15:49:52 INFO util.ShutdownHookManager: Deleting directory /data2/test/data/hadoop/hdfs/data/usercache/hadoop/appcache/application_1533616206085_5657/spark-a4b4251c-feaf-426a-a363-4159cdc092f6
> 18/08/13 15:49:52 INFO util.ShutdownHookManager: Deleting directory /data1/test/data/hadoop/hdfs/data/usercache/hadoop/appcache/application_1533616206085_5657/spark-751dcea0-e9c4-4160-9112-a6fe081ec4fb
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)