You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by qiumingming <cs...@gmail.com> on 2017/06/03 10:36:26 UTC
Failed to find metadata store when cubing by spark
Hi All:
I got a error when cubing by spark which make me confused. When I start
kylin I got these log:
2017-06-03 18:09:19,647 INFO [Thread-8-EventThread]
state.ConnectionStateManager:228 : State
change: CONNECTED
2017-06-03 18:09:19,680 INFO [Thread-8] execution.ExecutableManager:53
: Using metadata url:
kylin_v2_metadata@hbase
2017-06-03 18:09:19,682 INFO [Thread-8] dao.ExecutableDao:75 : Using
metadata url:
kylin_v2_metadata@hbase
2017-06-03 18:09:19,687 INFO [Thread-8] persistence.ResourceStore:88 :
Using metadata url
kylin_v2_metadata@hbase for resource store
2017-06-03 18:09:19,708 DEBUG [Thread-8] hbase.HBaseConnection:274 :
HTable
'kylin_v2_metadata' already exists
2017-06-03 18:09:19,710 DEBUG [Thread-8] metadata.MetadataManager:424 :
Reloading SourceTable
from folder kylin_v2_metadata(key='/table')@kylin_v2_metadata@hbase
It seems the metadata store works well. But when I start cubing by spark
, the log file shows these info:
17/06/03 18:14:16 WARN scheduler.TaskSetManager: Lost task 0.0 in stage
0.0 (TID 0, n6-129-021.byted.org): java.lang.IllegalArgumentException:
Failed to find metadata store by url: kylin_v2_metadata@hbase
at
org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:99)
at
org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:110)
at
org.apache.kylin.cube.CubeDescManager.getStore(CubeDescManager.java:370)
at
org.apache.kylin.cube.CubeDescManager.reloadAllCubeDesc(CubeDescManager.java:298)
at
org.apache.kylin.cube.CubeDescManager.<init>(CubeDescManager.java:109)
at
org.apache.kylin.cube.CubeDescManager.getInstance(CubeDescManager.java:81)
at
org.apache.kylin.cube.CubeInstance.getDescriptor(CubeInstance.java:109)
at
org.apache.kylin.cube.CubeSegment.getCubeDesc(CubeSegment.java:119)
at
org.apache.kylin.cube.CubeSegment.isEnableSharding(CubeSegment.java:467)
at
org.apache.kylin.cube.kv.RowKeyEncoder.<init>(RowKeyEncoder.java:48)
at
org.apache.kylin.cube.kv.AbstractRowKeyEncoder.createInstance(AbstractRowKeyEncoder.java:48)
at
org.apache.kylin.engine.spark.SparkCubingByLayer$2.call(SparkCubingByLayer.java:209)
at
org.apache.kylin.engine.spark.SparkCubingByLayer$2.call(SparkCubingByLayer.java:197)
at
org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1025)
at
org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1025)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at
org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:191)
at
org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:64)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:218)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
I don't know why this happened, I need your help. Thanks.
--
View this message in context: http://apache-kylin.74782.x6.nabble.com/Failed-to-find-metadata-store-when-cubing-by-spark-tp8167.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
Re: Failed to find metadata store when cubing by spark
Posted by qiumingming <cs...@gmail.com>.
(1) the correct hbase-site.xml does in the HADOOP_CONF_DIR.
(2) All dependent jars have been submitted.
Below is the source code of ResourceStore.createResourceStore:
private static ResourceStore createResourceStore(KylinConfig
kylinConfig) {
List<Throwable> es = new ArrayList<Throwable>();
logger.info("Using metadata url " + kylinConfig.getMetadataUrl() + "
for resource store");
for (Class<? extends ResourceStore> cls : getKnownImpl(kylinConfig))
{
try {
return
cls.getConstructor(KylinConfig.class).newInstance(kylinConfig);
} catch (Throwable e) {
es.add(e);
}
}
for (Throwable exceptionOrError : es) {
logger.error("Create new store instance failed ",
exceptionOrError);
}
throw new IllegalArgumentException("Failed to find metadata store by
url: " + kylinConfig.getMetadataUrl());
}
This method throw a IllegalArgumentException, and I can see log info
like "Failed to find metadata store by url...." in the log file, but I can
not search anything which match "Create new store instance failed". This
makes me very confused. If it throw a IllegalArgumentException, why it did
not print "Create new store instance failed" in the log file?
--
View this message in context: http://apache-kylin.74782.x6.nabble.com/Failed-to-find-metadata-store-when-cubing-by-spark-tp8167p8177.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
Re: Failed to find metadata store when cubing by spark
Posted by ShaoFeng Shi <sh...@apache.org>.
Spark couldn't connect with the HBase cluster for metadata. Some possible
causes I can think of:
1) the correct hbase-site.xml wasn't in the HADOOP_CONF_DIR
2) some HBase dependent jar wasn't submitted with the application
You may need check more logs, including the Spark executor logs. Start
Spark history sever and then check that job's detail log.
2017-06-03 18:36 GMT+08:00 qiumingming <cs...@gmail.com>:
> Hi All:
> I got a error when cubing by spark which make me confused. When I start
> kylin I got these log:
> 2017-06-03 18:09:19,647 INFO [Thread-8-EventThread]
> state.ConnectionStateManager:228 : State
> change: CONNECTED
> 2017-06-03 18:09:19,680 INFO [Thread-8] execution.ExecutableManager:53
> : Using metadata url:
> kylin_v2_metadata@hbase
> 2017-06-03 18:09:19,682 INFO [Thread-8] dao.ExecutableDao:75 : Using
> metadata url:
> kylin_v2_metadata@hbase
> 2017-06-03 18:09:19,687 INFO [Thread-8] persistence.ResourceStore:88 :
> Using metadata url
> kylin_v2_metadata@hbase for resource store
> 2017-06-03 18:09:19,708 DEBUG [Thread-8] hbase.HBaseConnection:274 :
> HTable
> 'kylin_v2_metadata' already exists
> 2017-06-03 18:09:19,710 DEBUG [Thread-8] metadata.MetadataManager:424 :
> Reloading SourceTable
> from folder kylin_v2_metadata(key='/table')@kylin_v2_metadata@hbase
>
> It seems the metadata store works well. But when I start cubing by spark
> , the log file shows these info:
> 17/06/03 18:14:16 WARN scheduler.TaskSetManager: Lost task 0.0 in stage
> 0.0 (TID 0, n6-129-021.byted.org): java.lang.IllegalArgumentException:
> Failed to find metadata store by url: kylin_v2_metadata@hbase
> at
> org.apache.kylin.common.persistence.ResourceStore.createResourceStore(
> ResourceStore.java:99)
> at
> org.apache.kylin.common.persistence.ResourceStore.
> getStore(ResourceStore.java:110)
> at
> org.apache.kylin.cube.CubeDescManager.getStore(CubeDescManager.java:370)
> at
> org.apache.kylin.cube.CubeDescManager.reloadAllCubeDesc(
> CubeDescManager.java:298)
> at
> org.apache.kylin.cube.CubeDescManager.<init>(CubeDescManager.java:109)
> at
> org.apache.kylin.cube.CubeDescManager.getInstance(CubeDescManager.java:81)
> at
> org.apache.kylin.cube.CubeInstance.getDescriptor(CubeInstance.java:109)
> at
> org.apache.kylin.cube.CubeSegment.getCubeDesc(CubeSegment.java:119)
> at
> org.apache.kylin.cube.CubeSegment.isEnableSharding(CubeSegment.java:467)
> at
> org.apache.kylin.cube.kv.RowKeyEncoder.<init>(RowKeyEncoder.java:48)
> at
> org.apache.kylin.cube.kv.AbstractRowKeyEncoder.createInstance(
> AbstractRowKeyEncoder.java:48)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer$2.call(
> SparkCubingByLayer.java:209)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer$2.call(
> SparkCubingByLayer.java:197)
> at
> org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(
> JavaPairRDD.scala:1025)
> at
> org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(
> JavaPairRDD.scala:1025)
> at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
> at
> org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.
> scala:191)
> at
> org.apache.spark.shuffle.sort.SortShuffleWriter.write(
> SortShuffleWriter.scala:64)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:218)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> I don't know why this happened, I need your help. Thanks.
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/Failed-to-find-metadata-store-when-cubing-by-spark-tp8167.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>
--
Best regards,
Shaofeng Shi 史少锋