You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Xiaoyu Wang (JIRA)" <ji...@apache.org> on 2015/11/03 11:09:27 UTC
[jira] [Updated] (KYLIN-1120) MapReduce job read local meta issue
[ https://issues.apache.org/jira/browse/KYLIN-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiaoyu Wang updated KYLIN-1120:
-------------------------------
Attachment: 0001-KYLIN-1120-MapReduce-job-read-local-meta-issue.patch
this is the patch for 1.x-staging
> MapReduce job read local meta issue
> -----------------------------------
>
> Key: KYLIN-1120
> URL: https://issues.apache.org/jira/browse/KYLIN-1120
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
> Affects Versions: 1.2
> Environment: Kylin-1.x-staging branch
> Reporter: Xiaoyu Wang
> Assignee: liyang
> Attachments: 0001-KYLIN-1120-MapReduce-job-read-local-meta-issue.patch
>
>
> set the property "kylin.metadata.url" in kylin.properties like this:
> {code}
> kylin.metadata.url=kylin_metadata_test1.2@hbase
> {code}
> 1.Create a test cube and build.
> 2.In the step 2:"Fact_Distinct_Columns" got exception:
> {noformat}
> 2015-11-02 17:23:50,354 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.NullPointerException
> at org.apache.kylin.job.hadoop.cube.FactDistinctColumnsMapper.setup(FactDistinctColumnsMapper.java:73)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> {noformat}
> reason:
> In the MapReduce job Kylin set the local dir to metadataUrl for read metadata.
> {code:title=FactDistinctColumnsMapper.java|borderStyle=solid}
> KylinConfig config = AbstractHadoopJob.loadKylinPropsAndMetadata(conf);
> {code}
> But the ResourceStore.java return a "HBaseResourceStore" instance from "getStore" method.
> Because the HBaseResourceStore's constructor will not throw exception when the metadataUrl is local address not "xxx@hbase"
> "HBaseResourceStore" will read the default HBase table "kylin_metadata" to get the cube metadata and My metadata is in "kylin_metadata_test1.2" table,so got the NPE.
> I modify the class add order in ResourceStore's static code to fixed this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)