回复： mismatch hdfs addr error when using kylin2.1 on two hadoop cluster

[朱真龙 <18...@qq.com>]

thank you for your response. I am really glad that you are patient enough to read my email which maybe have many solecisms on it .


I have configured right ,beause some model works well between two hadoop cluster .  I used  core-site.xml copied from hive cluster's hdfs ,and in kylin.properties
I configured kylin.hbase.cluster.hdfs.config.file to hbase's hdfs-site.xml, and configured kylin.storage.hbase.cluster-fs to hbase's hdfs cluster addr. 







I  used kylin 1.6 between two hadoop cluster for about one month already.   and  changed to kylin2.1 last week.



------------------ 原始邮件 ------------------
发件人: "ShaoFeng Shi";<sh...@apache.org>;
发送时间: 2017年12月22日(星期五) 晚上9:15
收件人: "user"<us...@kylin.apache.org>;

主题: Re: mismatch hdfs addr error when using kylin2.1 on two hadoop cluster



When configure Kylin with two cluster (one for hive/mapreduce, the other for hbase), need:

1. in kylin node, the core-site.xml uses Hive cluster's HDFS as default FS.
2. in kylin.properties, configure "kylin.storage.hbase.cluster-fs" to the HDFS of HBase cluster


In your case, I guess you didn't configure 2); please double check


2017-12-22 16:10 GMT+08:00 jxs <jx...@126.com>:
I also deploy kylin on two small EMR clusters for resource isolation, I used MySQL for Hive metastore and shared the same hive configuration between the two clusters.
You may try this.








在2017年12月22 13时08分, "朱真龙"<18...@qq.com>写道:

    thank you for your attention first,I am a chinses kylin user that always read english documents but write little, and i know most of kylin developer are chinese 。so ，if you don't know what i mean，i will describe again in chinese。


     I am using kylin2.1.0 between two hadoop cluster （all configured HA） and hadoop have same version(2.7.1).  by now runing well with sample model  , but  not good in my own model which has many  columns encoding with dict。like this：



when  building cube , got error like this:



after looked into source code that the error metioned, find that the dictionary got hdfs path from hive table desc , and when checking before load these hfiles into hbase ,found hfiles and hbase table are on different hdfs cluster .


apache-kylin-2.1.0-src\apache-kylin-2.1.0\core-cube\src\main\java\org\apache\kylin\cube\CubeManager





so, could you tell me how could i do on this case ?
















-- 
Best regards,

Shaofeng Shi 史少锋