You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by Mars Xu <xu...@gmail.com> on 2016/12/12 12:14:21 UTC
synchronize Hive metastore Error
Hello All,
I notice that Kylin 1.6 has a feature that more robust approach for hive schema changes. I have test it , just alter the column name of cube’s fact table ,it throws an error like this:
java.lang.NullPointerException
at org.apache.kylin.cube.model.CubeDesc.getFactTableDesc(CubeDesc.java:342)
at org.apache.kylin.source.hive.SchemaChecker$1.apply(SchemaChecker.java:110)
at org.apache.kylin.source.hive.SchemaChecker$1.apply(SchemaChecker.java:101)
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:702)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:263)
at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:223)
at org.apache.kylin.source.hive.SchemaChecker.findCubeByTable(SchemaChecker.java:119)
at org.apache.kylin.source.hive.SchemaChecker.allowReload(SchemaChecker.java:212)
at org.apache.kylin.source.hive.HiveSourceTableLoader.reloadHiveTables(HiveSourceTableLoader.java:66)
at org.apache.kylin.rest.service.CubeService.reloadHiveTable(CubeService.java:538)
at org.apache.kylin.rest.service.CubeService$$FastClassBySpringCGLIB$$17a07c0e.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:700)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
……
What I do is to load hive table in the datasource tab after I change the hive table using hive cli. Is any solution to this problem?
Re: synchronize Hive metastore Error
Posted by Li Yang <li...@apache.org>.
No perfect solution at the moment. From a design perspective, we can cope
with column addition. But column rename/deletion are very difficult to
handle. Usually requires rebuild the cube again.
Any better idea?
Yang
On Tue, Dec 13, 2016 at 8:50 AM, Mars Xu <xu...@gmail.com> wrote:
>
>
> > 在 2016年12月12日，下午8:14，Mars Xu <xu...@gmail.com> 写道：
> >
> > Hello All,
> >
> > I notice that Kylin 1.6 has a feature that more robust approach for
> hive schema changes. I have test it , just alter the column name of cube’s
> fact table ,it throws an error like this:
> >
> > java.lang.NullPointerException
> > at org.apache.kylin.cube.model.CubeDesc.getFactTableDesc(
> CubeDesc.java:342)
> > at org.apache.kylin.source.hive.SchemaChecker$1.apply(
> SchemaChecker.java:110)
> > at org.apache.kylin.source.hive.SchemaChecker$1.apply(
> SchemaChecker.java:101)
> > at com.google.common.collect.Iterators$7.computeNext(
> Iterators.java:702)
> > at com.google.common.collect.AbstractIterator.tryToComputeNext(
> AbstractIterator.java:143)
> > at com.google.common.collect.AbstractIterator.hasNext(
> AbstractIterator.java:138)
> > at com.google.common.collect.ImmutableList.copyOf(
> ImmutableList.java:263)
> > at com.google.common.collect.ImmutableList.copyOf(
> ImmutableList.java:223)
> > at org.apache.kylin.source.hive.SchemaChecker.findCubeByTable(
> SchemaChecker.java:119)
> > at org.apache.kylin.source.hive.SchemaChecker.allowReload(
> SchemaChecker.java:212)
> > at org.apache.kylin.source.hive.HiveSourceTableLoader.
> reloadHiveTables(HiveSourceTableLoader.java:66)
> > at org.apache.kylin.rest.service.CubeService.reloadHiveTable(
> CubeService.java:538)
> > at org.apache.kylin.rest.service.CubeService$$
> FastClassBySpringCGLIB$$17a07c0e.invoke(<generated>)
> > at org.springframework.cglib.proxy.MethodProxy.invoke(
> MethodProxy.java:204)
> > at org.springframework.aop.framework.CglibAopProxy$
> CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:700)
> > at org.springframework.aop.framework.ReflectiveMethodInvocation.
> proceed(ReflectiveMethodInvocation.java:150)
> > ……
> >
> > What I do is to load hive table in the datasource tab after I change the
> hive table using hive cli. Is any solution to this problem?
>
>
>
> Mars
Re: synchronize Hive metastore Error
Posted by Mars Xu <xu...@gmail.com>.
> 在 2016年12月12日，下午8:14，Mars Xu <xu...@gmail.com> 写道：
>
> Hello All,
>
> I notice that Kylin 1.6 has a feature that more robust approach for hive schema changes. I have test it , just alter the column name of cube’s fact table ,it throws an error like this:
>
> java.lang.NullPointerException
> at org.apache.kylin.cube.model.CubeDesc.getFactTableDesc(CubeDesc.java:342)
> at org.apache.kylin.source.hive.SchemaChecker$1.apply(SchemaChecker.java:110)
> at org.apache.kylin.source.hive.SchemaChecker$1.apply(SchemaChecker.java:101)
> at com.google.common.collect.Iterators$7.computeNext(Iterators.java:702)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:263)
> at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:223)
> at org.apache.kylin.source.hive.SchemaChecker.findCubeByTable(SchemaChecker.java:119)
> at org.apache.kylin.source.hive.SchemaChecker.allowReload(SchemaChecker.java:212)
> at org.apache.kylin.source.hive.HiveSourceTableLoader.reloadHiveTables(HiveSourceTableLoader.java:66)
> at org.apache.kylin.rest.service.CubeService.reloadHiveTable(CubeService.java:538)
> at org.apache.kylin.rest.service.CubeService$$FastClassBySpringCGLIB$$17a07c0e.invoke(<generated>)
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:700)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> ……
>
> What I do is to load hive table in the datasource tab after I change the hive table using hive cli. Is any solution to this problem?
Mars