You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by WSJian <52...@qq.com> on 2021/01/25 08:14:03 UTC

cube can't building in flink 1.11.3 / 1.10.0

ENV: Kylin 3.1.1&nbsp; Flink 1.10.0 / 1.11.3&nbsp; CDH 5.7

/etc/profile --&gt; FLINK_HOME、KYLIN_HOME、HADOOP_CONF_DIR&nbsp; all down.


Only Spark Env no pass(Because there isn't) when I start kylin.sh. Then build `kylin_sales_cube` cube. But throw ERROR during Build Cube With Flink process.&nbsp; 


java.lang.NoSuchFieldError: MANAGED_MEMORY_PRE_ALLOCATE
&nbsp; &nbsp; &nbsp; at org.apache.kylin.engine.flink.FlinkOnYarnConfigMapping.<clinit&gt;(FlinkOnYarnConfigMapping.java:69)
&nbsp; &nbsp; &nbsp; ......


I looked at the source code. Found 
```java
    ConfigOption<Boolean&gt; tmMemoryPreallocate = TaskManagerOptions.MANAGED_MEMORY_PRE_ALLOCATE;
```
So I think flink not start in Yarn success, and after start flink in Yarn.
Throw new Exception.


DefaultClassNotFound: FlinkOnYarnConfigMapping............................


This is mean the class not load? Or classpath error. Need moving dependent JAR to where?

Re: cube can't building in flink 1.11.3 / 1.10.0

Posted by Yaqian Zhang <Ya...@126.com>.
Hi:

Kylin version 3.1.1 supports Flink 1.9.x.

If you want to use Flink 1.10.0 or 1.11, someone in the Kylin community recently proposed the PR  https://github.com/apache/kylin/pull/1462 to upgrade Flink.

You can type this patch and try again.

> 在 2021年1月25日，16:14，WSJian <52...@qq.com> 写道：
> 
> ENV: Kylin 3.1.1&nbsp; Flink 1.10.0 / 1.11.3&nbsp; CDH 5.7
> 
> /etc/profile --&gt; FLINK_HOME、KYLIN_HOME、HADOOP_CONF_DIR&nbsp; all down.
> 
> 
> Only Spark Env no pass(Because there isn't) when I start kylin.sh. Then build `kylin_sales_cube` cube. But throw ERROR during Build Cube With Flink process.&nbsp; 
> 
> 
> java.lang.NoSuchFieldError: MANAGED_MEMORY_PRE_ALLOCATE
> &nbsp; &nbsp; &nbsp; at org.apache.kylin.engine.flink.FlinkOnYarnConfigMapping.<clinit&gt;(FlinkOnYarnConfigMapping.java:69)
> &nbsp; &nbsp; &nbsp; ......
> 
> 
> I looked at the source code. Found 
> ```java
>    ConfigOption<Boolean&gt; tmMemoryPreallocate = TaskManagerOptions.MANAGED_MEMORY_PRE_ALLOCATE;
> ```
> So I think flink not start in Yarn success, and after start flink in Yarn.
> Throw new Exception.
> 
> 
> DefaultClassNotFound: FlinkOnYarnConfigMapping............................
> 
> 
> This is mean the class not load? Or classpath error. Need moving dependent JAR to where?