You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by alex schufo <al...@gmail.com> on 2015/06/05 10:58:11 UTC
Cube built but not possible to query
Hi all,
I built a cube on a Hive fact table of about 300 millions rows, all the
steps are FINISHED with success but it seems I cannot do any query.
select * from myFactTable; (or any other query)
returns From line 1, column 15 to line 1, column 43: Table 'MYFACTTABLE'
not found while executing SQL: "select * from myFactTable LIMIT 50000"
I tried rebuilding completely the cube after a purge, created a new
project, etc. but it doesn't solve the issue.
It seems like a new KYLIN_ table has been created and contains data. How
can I know which HBase table corresponds to which cube?
The "Convert Cuboid Data to HFile" step took a while, the MapReduce job
stayed in Undefined status for a long time but finally Succeeded. When I
click on the link from the Kylin UI it send me to 7070/kylin/N/A instead of
the MapReduce job info, is that a bad sign?
In the logs I don't see any error in the Build N-Dimension Cuboid Data,
Calculate HTable Region Splits, Create HTable
I am not sure if it's event related but in the logs can this:
2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job: map 1%
reduce 0%
2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job: Task Id :
attempt_1432057232815_49758_m_000003_2, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
at
org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobContextImpl.java:174)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1867)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
... 8 more
but then it seems to go on anyway:
2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job: map 2%
reduce 0%
2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job: map 3%
reduce 0%
2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job: map 4%
reduce 0%
2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job: map 5%
reduce 0%
2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job: map 6%
reduce 0%
...
2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job: map 100%
reduce 100%
2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job: Job
job_1432057232815_49758 completed successfully
This comes after
Hive Column Cardinality calculation for table
and a step called :
+------------------------------------------------------------------------------------------------------+
| null
|
+------------------------------------------------------------------------------------------------------+
Thanks for your help, I would really like to make this PoC work!
Re: Cube built but not possible to query
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
okay, please go ahead to open a JIRA for tracking the issue, with the
necessary information like problem description, log messages, screenshots,
etc; We will check then. Thanks;
On 6/10/15, 11:41 PM, "alex schufo" <al...@gmail.com> wrote:
>The description of the problem seems similar (Kylin doesn't show up the
>table/columns in "Query" tab after cube be built) but I don't see the
>exception mentioned in my log:
>
>$ grep OLAPSchemaFactory tomcat/logs/*
>
>tomcat/logs/kylin.log:[http-bio-7070-exec-10]:[2015-06-10
>01:55:06,407][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDat
>aSource(BasicService.java:109)]
>- "factory":
>"org.apache.kylin.query.schema.OLAPSchemaFactory",
>
>tomcat/logs/kylin.log:[http-bio-7070-exec-2]:[2015-06-10
>01:59:21,381][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDat
>aSource(BasicService.java:109)]
>- "factory":
>"org.apache.kylin.query.schema.OLAPSchemaFactory",
>
>tomcat/logs/kylin.log.2015-06-05:[http-bio-7070-exec-3]:[2015-06-05
>05:36:33,311][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDat
>aSource(BasicService.java:109)]
>- "factory":
>"org.apache.kylin.query.schema.OLAPSchemaFactory",
>
>tomcat/logs/kylin.log.2015-06-05:[http-bio-7070-exec-6]:[2015-06-09
>08:18:56,643][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDat
>aSource(BasicService.java:109)]
>- "factory":
>"org.apache.kylin.query.schema.OLAPSchemaFactory",
>
>tomcat/logs/kylin.log.2015-06-09:[http-bio-7070-exec-6]:[2015-06-09
>08:18:56,643][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDat
>aSource(BasicService.java:109)]
>- "factory":
>"org.apache.kylin.query.schema.OLAPSchemaFactory",
>
>(no Error / Exception related to that)
>
>the bin/find-hive-dependency.sh I use already includes this fix:
>
>hive_lib=`find "$hdp_home/hive/lib" -name '*.jar' ! -name '*calcite*'
>-printf '%p:' | sed 's/:$//'`
>
>hive_dependency=${hive_conf_path}:${hive_lib}:${hcatalog}
>
>
>On Wed, Jun 10, 2015 at 3:18 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
>
>> Please check whether this matches your problem:
>> https://issues.apache.org/jira/browse/KYLIN-761
>>
>> If yes then check whether your Kylin installation includes this fix (in
>> bin/find-hive-dependency.sh):
>>
>>
>>https://github.com/apache/incubator-kylin/commit/2439b9b5289d6c1cefe6dd49
>>fc
>> 3403b2d1644ff6
>>
>>
>> On 6/10/15, 7:21 PM, "alex schufo" <al...@gmail.com> wrote:
>>
>> >So like I said in my previous email after copying the jars and
>>restarting
>> >Kylin I could see the tables and query them.
>> >
>> >But yesterday I built a new cube and the tables disappeared for both
>> >projects. I restart Kylin and then I could see the tables for both
>> >projects
>> >and query them.
>> >
>> >Is there a Jira for this?
>> >
>> >On Fri, Jun 5, 2015 at 2:34 PM, divye sheth <di...@gmail.com>
>>wrote:
>> >
>> >> My bad just realised it when compared it with my working copy. This
>>has
>> >>to
>> >> do something with the loading of jars though they are available in
>>the
>> >>lib
>> >> but somehow are not loaded. I believe there is already a jira for
>>this.
>> >> Will send the link once I get it.
>> >> On Jun 5, 2015 18:49, "alex schufo" <al...@gmail.com> wrote:
>> >>
>> >> > Awesome!
>> >> >
>> >> > I tried this and it solved my issue. I now the the table / database
>> >>and
>> >> can
>> >> > query the cube with second latency, thanks a lot!
>> >> >
>> >> > Just one thing, the source and destination are actually reversed,
>>here
>> >> are
>> >> > the commands:
>> >> >
>> >> > cp tomcat/webapps/kylin/WEB-INF/lib/calcite-core-${version}.jar
>> >> tomcat/lib/
>> >> >
>> >> > cp tomcat/webapps/kylin/WEB-INF/lib/kylin-query-${version}.jar
>> >> tomcat/lib/
>> >> >
>> >> > On Fri, Jun 5, 2015 at 11:47 AM, divye sheth <di...@gmail.com>
>> >> wrote:
>> >> >
>> >> > > Hi Alex,
>> >> > >
>> >> > > I had faced the same issue and I believe it was addressed by one
>>of
>> >>the
>> >> > > developers of Kylin, cant get hold of the link though. Here is
>>the
>> >> > solution
>> >> > > that was posted there and I was able to get this working.
>> >> > >
>> >> > > Copy the kylin-query-${version}.jar and
>>calcite-core-${version}.jar
>> >>the
>> >> > > source and destination are given below:
>> >> > > sourcce location: $KYLIN_HOME/tomcat/lib
>> >> > > destination: $KYLIN_HOME/tomcat/webapps/kylin/WEB-INF/lib
>> >> > >
>> >> > > after you have copied please restart the kylin. Let me know if
>>this
>> >> > solves
>> >> > > your issue.
>> >> > >
>> >> > > Thanks
>> >> > > Divye Sheth
>> >> > >
>> >> > > On Fri, Jun 5, 2015 at 4:04 PM, alex schufo
>><al...@gmail.com>
>> >> > wrote:
>> >> > >
>> >> > > > I checked that but there isn't any dependency error, see below
>>the
>> >> > > output:
>> >> > > >
>> >> > > > $ bin/check-env.sh
>> >> > > >
>> >> > > > KYLIN_HOME is set to
>> >> > /home/username/kylin/kylin-0.7.1-incubating-SNAPSHOT
>> >> > > >
>> >> > > > On Fri, Jun 5, 2015 at 11:16 AM, Shi, Shaofeng
>><sh...@ebay.com>
>> >> > wrote:
>> >> > > >
>> >> > > > > Please run $KYLIN_HOME/bin/find-hive-dependency.sh and then
>>copy
>> >> the
>> >> > > > > output here; Kylin need add the Hcatalog to MR class path, if
>> >>the
>> >> > > > Hcatalog
>> >> > > > > jar wasn¹t found, the "HCatInputFormat not found² error will
>>be
>> >> > > reported;
>> >> > > > >
>> >> > > > > On 6/5/15, 4:58 PM, "alex schufo" <al...@gmail.com>
>>wrote:
>> >> > > > >
>> >> > > > > >Hi all,
>> >> > > > > >
>> >> > > > > >I built a cube on a Hive fact table of about 300 millions
>>rows,
>> >> all
>> >> > > the
>> >> > > > > >steps are FINISHED with success but it seems I cannot do any
>> >> query.
>> >> > > > > >
>> >> > > > > >select * from myFactTable; (or any other query)
>> >> > > > > >
>> >> > > > > >returns From line 1, column 15 to line 1, column 43: Table
>> >> > > 'MYFACTTABLE'
>> >> > > > > >not found while executing SQL: "select * from myFactTable
>>LIMIT
>> >> > 50000"
>> >> > > > > >
>> >> > > > > >I tried rebuilding completely the cube after a purge,
>>created a
>> >> new
>> >> > > > > >project, etc. but it doesn't solve the issue.
>> >> > > > > >
>> >> > > > > >It seems like a new KYLIN_ table has been created and
>>contains
>> >> data.
>> >> > > How
>> >> > > > > >can I know which HBase table corresponds to which cube?
>> >> > > > > >
>> >> > > > > >The "Convert Cuboid Data to HFile" step took a while, the
>> >> MapReduce
>> >> > > job
>> >> > > > > >stayed in Undefined status for a long time but finally
>> >>Succeeded.
>> >> > > When I
>> >> > > > > >click on the link from the Kylin UI it send me to
>> >>7070/kylin/N/A
>> >> > > instead
>> >> > > > > >of
>> >> > > > > >the MapReduce job info, is that a bad sign?
>> >> > > > > >
>> >> > > > > >In the logs I don't see any error in the Build N-Dimension
>> >>Cuboid
>> >> > > Data,
>> >> > > > > >Calculate HTable Region Splits, Create HTable
>> >> > > > > >
>> >> > > > > >I am not sure if it's event related but in the logs can
>>this:
>> >> > > > > >
>> >> > > > > >2015-06-04 10:17:10,910 INFO [pool-7-thread-10]
>>mapreduce.Job:
>> >> map
>> >> > > 1%
>> >> > > > > >reduce 0%
>> >> > > > > >
>> >> > > > > >2015-06-04 10:17:11,936 INFO [pool-7-thread-10]
>>mapreduce.Job:
>> >> Task
>> >> > > Id
>> >> > > > :
>> >> > > > > >attempt_1432057232815_49758_m_000003_2, Status : FAILED
>> >> > > > > >
>> >> > > > > >Error: java.lang.RuntimeException:
>> >> java.lang.ClassNotFoundException:
>> >> > > > Class
>> >> > > > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
>> >> > > > > >
>> >> > > > > > at
>> >> > > > >
>> >> >
>>
>>>>>org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
>> >> > > > > >
>> >> > > > > > at
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>>
>>>>>org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(Jo
>>>>>bC
>> >>>on
>> >> > > > > >textImpl.java:174)
>> >> > > > > >
>> >> > > > > > at
>> >> > > > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>> >> > > > > >
>> >> > > > > > at
>> >>org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>> >> > > > > >
>> >> > > > > > at
>> >> > > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
>> >> > > > > >
>> >> > > > > > at
>>java.security.AccessController.doPrivileged(Native
>> >> > Method)
>> >> > > > > >
>> >> > > > > > at
>>javax.security.auth.Subject.doAs(Subject.java:415)
>> >> > > > > >
>> >> > > > > > at
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>>
>>>>>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat
>>>>>io
>> >>>n.
>> >> > > > > >java:1594)
>> >> > > > > >
>> >> > > > > > at
>> >> > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
>> >> > > > > >
>> >> > > > > >Caused by: java.lang.ClassNotFoundException: Class
>> >> > > > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
>> >> > > > > >
>> >> > > > > > at
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>>
>>>>>org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java
>>>>>:1
>> >>>86
>> >> > > > > >7)
>> >> > > > > >
>> >> > > > > > at
>> >> > > > >
>> >> >
>>
>>>>>org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
>> >> > > > > >
>> >> > > > > > ... 8 more
>> >> > > > > >
>> >> > > > > >
>> >> > > > > >
>> >> > > > > >but then it seems to go on anyway:
>> >> > > > > >
>> >> > > > > >2015-06-04 10:17:15,127 INFO [pool-7-thread-10]
>>mapreduce.Job:
>> >> map
>> >> > > 2%
>> >> > > > > >reduce 0%
>> >> > > > > >
>> >> > > > > >2015-06-04 10:17:18,174 INFO [pool-7-thread-10]
>>mapreduce.Job:
>> >> map
>> >> > > 3%
>> >> > > > > >reduce 0%
>> >> > > > > >
>> >> > > > > >2015-06-04 10:17:20,202 INFO [pool-7-thread-10]
>>mapreduce.Job:
>> >> map
>> >> > > 4%
>> >> > > > > >reduce 0%
>> >> > > > > >
>> >> > > > > >2015-06-04 10:17:22,230 INFO [pool-7-thread-10]
>>mapreduce.Job:
>> >> map
>> >> > > 5%
>> >> > > > > >reduce 0%
>> >> > > > > >
>> >> > > > > >2015-06-04 10:17:23,244 INFO [pool-7-thread-10]
>>mapreduce.Job:
>> >> map
>> >> > > 6%
>> >> > > > > >reduce 0%
>> >> > > > > >
>> >> > > > > >...
>> >> > > > > >
>> >> > > > > >2015-06-04 10:19:33,458 INFO [pool-7-thread-10]
>>mapreduce.Job:
>> >> map
>> >> > > > 100%
>> >> > > > > >reduce 100%
>> >> > > > > >
>> >> > > > > >2015-06-04 10:19:33,481 INFO [pool-7-thread-10]
>>mapreduce.Job:
>> >> Job
>> >> > > > > >job_1432057232815_49758 completed successfully
>> >> > > > > >
>> >> > > > > >
>> >> > > > > >This comes after
>> >> > > > > >
>> >> > > > > >
>> >> > > > > >Hive Column Cardinality calculation for table
>> >> > > > > >
>> >> > > > > >and a step called :
>> >> > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>>
>>>>>+---------------------------------------------------------------------
>>>>>--
>> >>>--
>> >> > > > > >-----------------------------+
>> >> > > > > >
>> >> > > > > >| null
>> >> > > > > > |
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>>
>>>>>+---------------------------------------------------------------------
>>>>>--
>> >>>--
>> >> > > > > >-----------------------------+
>> >> > > > > >
>> >> > > > > >
>> >> > > > > >Thanks for your help, I would really like to make this PoC
>> >>work!
>> >> > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>>
>>
Re: Cube built but not possible to query
Posted by alex schufo <al...@gmail.com>.
The description of the problem seems similar (Kylin doesn't show up the
table/columns in "Query" tab after cube be built) but I don't see the
exception mentioned in my log:
$ grep OLAPSchemaFactory tomcat/logs/*
tomcat/logs/kylin.log:[http-bio-7070-exec-10]:[2015-06-10
01:55:06,407][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDataSource(BasicService.java:109)]
- "factory": "org.apache.kylin.query.schema.OLAPSchemaFactory",
tomcat/logs/kylin.log:[http-bio-7070-exec-2]:[2015-06-10
01:59:21,381][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDataSource(BasicService.java:109)]
- "factory": "org.apache.kylin.query.schema.OLAPSchemaFactory",
tomcat/logs/kylin.log.2015-06-05:[http-bio-7070-exec-3]:[2015-06-05
05:36:33,311][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDataSource(BasicService.java:109)]
- "factory": "org.apache.kylin.query.schema.OLAPSchemaFactory",
tomcat/logs/kylin.log.2015-06-05:[http-bio-7070-exec-6]:[2015-06-09
08:18:56,643][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDataSource(BasicService.java:109)]
- "factory": "org.apache.kylin.query.schema.OLAPSchemaFactory",
tomcat/logs/kylin.log.2015-06-09:[http-bio-7070-exec-6]:[2015-06-09
08:18:56,643][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDataSource(BasicService.java:109)]
- "factory": "org.apache.kylin.query.schema.OLAPSchemaFactory",
(no Error / Exception related to that)
the bin/find-hive-dependency.sh I use already includes this fix:
hive_lib=`find "$hdp_home/hive/lib" -name '*.jar' ! -name '*calcite*'
-printf '%p:' | sed 's/:$//'`
hive_dependency=${hive_conf_path}:${hive_lib}:${hcatalog}
On Wed, Jun 10, 2015 at 3:18 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> Please check whether this matches your problem:
> https://issues.apache.org/jira/browse/KYLIN-761
>
> If yes then check whether your Kylin installation includes this fix (in
> bin/find-hive-dependency.sh):
>
> https://github.com/apache/incubator-kylin/commit/2439b9b5289d6c1cefe6dd49fc
> 3403b2d1644ff6
>
>
> On 6/10/15, 7:21 PM, "alex schufo" <al...@gmail.com> wrote:
>
> >So like I said in my previous email after copying the jars and restarting
> >Kylin I could see the tables and query them.
> >
> >But yesterday I built a new cube and the tables disappeared for both
> >projects. I restart Kylin and then I could see the tables for both
> >projects
> >and query them.
> >
> >Is there a Jira for this?
> >
> >On Fri, Jun 5, 2015 at 2:34 PM, divye sheth <di...@gmail.com> wrote:
> >
> >> My bad just realised it when compared it with my working copy. This has
> >>to
> >> do something with the loading of jars though they are available in the
> >>lib
> >> but somehow are not loaded. I believe there is already a jira for this.
> >> Will send the link once I get it.
> >> On Jun 5, 2015 18:49, "alex schufo" <al...@gmail.com> wrote:
> >>
> >> > Awesome!
> >> >
> >> > I tried this and it solved my issue. I now the the table / database
> >>and
> >> can
> >> > query the cube with second latency, thanks a lot!
> >> >
> >> > Just one thing, the source and destination are actually reversed, here
> >> are
> >> > the commands:
> >> >
> >> > cp tomcat/webapps/kylin/WEB-INF/lib/calcite-core-${version}.jar
> >> tomcat/lib/
> >> >
> >> > cp tomcat/webapps/kylin/WEB-INF/lib/kylin-query-${version}.jar
> >> tomcat/lib/
> >> >
> >> > On Fri, Jun 5, 2015 at 11:47 AM, divye sheth <di...@gmail.com>
> >> wrote:
> >> >
> >> > > Hi Alex,
> >> > >
> >> > > I had faced the same issue and I believe it was addressed by one of
> >>the
> >> > > developers of Kylin, cant get hold of the link though. Here is the
> >> > solution
> >> > > that was posted there and I was able to get this working.
> >> > >
> >> > > Copy the kylin-query-${version}.jar and calcite-core-${version}.jar
> >>the
> >> > > source and destination are given below:
> >> > > sourcce location: $KYLIN_HOME/tomcat/lib
> >> > > destination: $KYLIN_HOME/tomcat/webapps/kylin/WEB-INF/lib
> >> > >
> >> > > after you have copied please restart the kylin. Let me know if this
> >> > solves
> >> > > your issue.
> >> > >
> >> > > Thanks
> >> > > Divye Sheth
> >> > >
> >> > > On Fri, Jun 5, 2015 at 4:04 PM, alex schufo <al...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > I checked that but there isn't any dependency error, see below the
> >> > > output:
> >> > > >
> >> > > > $ bin/check-env.sh
> >> > > >
> >> > > > KYLIN_HOME is set to
> >> > /home/username/kylin/kylin-0.7.1-incubating-SNAPSHOT
> >> > > >
> >> > > > On Fri, Jun 5, 2015 at 11:16 AM, Shi, Shaofeng <sh...@ebay.com>
> >> > wrote:
> >> > > >
> >> > > > > Please run $KYLIN_HOME/bin/find-hive-dependency.sh and then copy
> >> the
> >> > > > > output here; Kylin need add the Hcatalog to MR class path, if
> >>the
> >> > > > Hcatalog
> >> > > > > jar wasn¹t found, the "HCatInputFormat not found² error will be
> >> > > reported;
> >> > > > >
> >> > > > > On 6/5/15, 4:58 PM, "alex schufo" <al...@gmail.com> wrote:
> >> > > > >
> >> > > > > >Hi all,
> >> > > > > >
> >> > > > > >I built a cube on a Hive fact table of about 300 millions rows,
> >> all
> >> > > the
> >> > > > > >steps are FINISHED with success but it seems I cannot do any
> >> query.
> >> > > > > >
> >> > > > > >select * from myFactTable; (or any other query)
> >> > > > > >
> >> > > > > >returns From line 1, column 15 to line 1, column 43: Table
> >> > > 'MYFACTTABLE'
> >> > > > > >not found while executing SQL: "select * from myFactTable LIMIT
> >> > 50000"
> >> > > > > >
> >> > > > > >I tried rebuilding completely the cube after a purge, created a
> >> new
> >> > > > > >project, etc. but it doesn't solve the issue.
> >> > > > > >
> >> > > > > >It seems like a new KYLIN_ table has been created and contains
> >> data.
> >> > > How
> >> > > > > >can I know which HBase table corresponds to which cube?
> >> > > > > >
> >> > > > > >The "Convert Cuboid Data to HFile" step took a while, the
> >> MapReduce
> >> > > job
> >> > > > > >stayed in Undefined status for a long time but finally
> >>Succeeded.
> >> > > When I
> >> > > > > >click on the link from the Kylin UI it send me to
> >>7070/kylin/N/A
> >> > > instead
> >> > > > > >of
> >> > > > > >the MapReduce job info, is that a bad sign?
> >> > > > > >
> >> > > > > >In the logs I don't see any error in the Build N-Dimension
> >>Cuboid
> >> > > Data,
> >> > > > > >Calculate HTable Region Splits, Create HTable
> >> > > > > >
> >> > > > > >I am not sure if it's event related but in the logs can this:
> >> > > > > >
> >> > > > > >2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > 1%
> >> > > > > >reduce 0%
> >> > > > > >
> >> > > > > >2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job:
> >> Task
> >> > > Id
> >> > > > :
> >> > > > > >attempt_1432057232815_49758_m_000003_2, Status : FAILED
> >> > > > > >
> >> > > > > >Error: java.lang.RuntimeException:
> >> java.lang.ClassNotFoundException:
> >> > > > Class
> >> > > > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> >> > > > > >
> >> > > > > > at
> >> > > > >
> >> >
> >>>org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
> >> > > > > >
> >> > > > > > at
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>>org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobC
> >>>on
> >> > > > > >textImpl.java:174)
> >> > > > > >
> >> > > > > > at
> >> > > > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
> >> > > > > >
> >> > > > > > at
> >>org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> >> > > > > >
> >> > > > > > at
> >> > > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> >> > > > > >
> >> > > > > > at java.security.AccessController.doPrivileged(Native
> >> > Method)
> >> > > > > >
> >> > > > > > at javax.security.auth.Subject.doAs(Subject.java:415)
> >> > > > > >
> >> > > > > > at
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformatio
> >>>n.
> >> > > > > >java:1594)
> >> > > > > >
> >> > > > > > at
> >> > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> >> > > > > >
> >> > > > > >Caused by: java.lang.ClassNotFoundException: Class
> >> > > > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> >> > > > > >
> >> > > > > > at
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>>org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1
> >>>86
> >> > > > > >7)
> >> > > > > >
> >> > > > > > at
> >> > > > >
> >> >
> >>>org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
> >> > > > > >
> >> > > > > > ... 8 more
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >but then it seems to go on anyway:
> >> > > > > >
> >> > > > > >2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > 2%
> >> > > > > >reduce 0%
> >> > > > > >
> >> > > > > >2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > 3%
> >> > > > > >reduce 0%
> >> > > > > >
> >> > > > > >2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > 4%
> >> > > > > >reduce 0%
> >> > > > > >
> >> > > > > >2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > 5%
> >> > > > > >reduce 0%
> >> > > > > >
> >> > > > > >2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > 6%
> >> > > > > >reduce 0%
> >> > > > > >
> >> > > > > >...
> >> > > > > >
> >> > > > > >2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > > 100%
> >> > > > > >reduce 100%
> >> > > > > >
> >> > > > > >2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job:
> >> Job
> >> > > > > >job_1432057232815_49758 completed successfully
> >> > > > > >
> >> > > > > >
> >> > > > > >This comes after
> >> > > > > >
> >> > > > > >
> >> > > > > >Hive Column Cardinality calculation for table
> >> > > > > >
> >> > > > > >and a step called :
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>>+-----------------------------------------------------------------------
> >>>--
> >> > > > > >-----------------------------+
> >> > > > > >
> >> > > > > >| null
> >> > > > > > |
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>>+-----------------------------------------------------------------------
> >>>--
> >> > > > > >-----------------------------+
> >> > > > > >
> >> > > > > >
> >> > > > > >Thanks for your help, I would really like to make this PoC
> >>work!
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
>
Re: Cube built but not possible to query
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
Please check whether this matches your problem:
https://issues.apache.org/jira/browse/KYLIN-761
If yes then check whether your Kylin installation includes this fix (in
bin/find-hive-dependency.sh):
https://github.com/apache/incubator-kylin/commit/2439b9b5289d6c1cefe6dd49fc
3403b2d1644ff6
On 6/10/15, 7:21 PM, "alex schufo" <al...@gmail.com> wrote:
>So like I said in my previous email after copying the jars and restarting
>Kylin I could see the tables and query them.
>
>But yesterday I built a new cube and the tables disappeared for both
>projects. I restart Kylin and then I could see the tables for both
>projects
>and query them.
>
>Is there a Jira for this?
>
>On Fri, Jun 5, 2015 at 2:34 PM, divye sheth <di...@gmail.com> wrote:
>
>> My bad just realised it when compared it with my working copy. This has
>>to
>> do something with the loading of jars though they are available in the
>>lib
>> but somehow are not loaded. I believe there is already a jira for this.
>> Will send the link once I get it.
>> On Jun 5, 2015 18:49, "alex schufo" <al...@gmail.com> wrote:
>>
>> > Awesome!
>> >
>> > I tried this and it solved my issue. I now the the table / database
>>and
>> can
>> > query the cube with second latency, thanks a lot!
>> >
>> > Just one thing, the source and destination are actually reversed, here
>> are
>> > the commands:
>> >
>> > cp tomcat/webapps/kylin/WEB-INF/lib/calcite-core-${version}.jar
>> tomcat/lib/
>> >
>> > cp tomcat/webapps/kylin/WEB-INF/lib/kylin-query-${version}.jar
>> tomcat/lib/
>> >
>> > On Fri, Jun 5, 2015 at 11:47 AM, divye sheth <di...@gmail.com>
>> wrote:
>> >
>> > > Hi Alex,
>> > >
>> > > I had faced the same issue and I believe it was addressed by one of
>>the
>> > > developers of Kylin, cant get hold of the link though. Here is the
>> > solution
>> > > that was posted there and I was able to get this working.
>> > >
>> > > Copy the kylin-query-${version}.jar and calcite-core-${version}.jar
>>the
>> > > source and destination are given below:
>> > > sourcce location: $KYLIN_HOME/tomcat/lib
>> > > destination: $KYLIN_HOME/tomcat/webapps/kylin/WEB-INF/lib
>> > >
>> > > after you have copied please restart the kylin. Let me know if this
>> > solves
>> > > your issue.
>> > >
>> > > Thanks
>> > > Divye Sheth
>> > >
>> > > On Fri, Jun 5, 2015 at 4:04 PM, alex schufo <al...@gmail.com>
>> > wrote:
>> > >
>> > > > I checked that but there isn't any dependency error, see below the
>> > > output:
>> > > >
>> > > > $ bin/check-env.sh
>> > > >
>> > > > KYLIN_HOME is set to
>> > /home/username/kylin/kylin-0.7.1-incubating-SNAPSHOT
>> > > >
>> > > > On Fri, Jun 5, 2015 at 11:16 AM, Shi, Shaofeng <sh...@ebay.com>
>> > wrote:
>> > > >
>> > > > > Please run $KYLIN_HOME/bin/find-hive-dependency.sh and then copy
>> the
>> > > > > output here; Kylin need add the Hcatalog to MR class path, if
>>the
>> > > > Hcatalog
>> > > > > jar wasn¹t found, the "HCatInputFormat not found² error will be
>> > > reported;
>> > > > >
>> > > > > On 6/5/15, 4:58 PM, "alex schufo" <al...@gmail.com> wrote:
>> > > > >
>> > > > > >Hi all,
>> > > > > >
>> > > > > >I built a cube on a Hive fact table of about 300 millions rows,
>> all
>> > > the
>> > > > > >steps are FINISHED with success but it seems I cannot do any
>> query.
>> > > > > >
>> > > > > >select * from myFactTable; (or any other query)
>> > > > > >
>> > > > > >returns From line 1, column 15 to line 1, column 43: Table
>> > > 'MYFACTTABLE'
>> > > > > >not found while executing SQL: "select * from myFactTable LIMIT
>> > 50000"
>> > > > > >
>> > > > > >I tried rebuilding completely the cube after a purge, created a
>> new
>> > > > > >project, etc. but it doesn't solve the issue.
>> > > > > >
>> > > > > >It seems like a new KYLIN_ table has been created and contains
>> data.
>> > > How
>> > > > > >can I know which HBase table corresponds to which cube?
>> > > > > >
>> > > > > >The "Convert Cuboid Data to HFile" step took a while, the
>> MapReduce
>> > > job
>> > > > > >stayed in Undefined status for a long time but finally
>>Succeeded.
>> > > When I
>> > > > > >click on the link from the Kylin UI it send me to
>>7070/kylin/N/A
>> > > instead
>> > > > > >of
>> > > > > >the MapReduce job info, is that a bad sign?
>> > > > > >
>> > > > > >In the logs I don't see any error in the Build N-Dimension
>>Cuboid
>> > > Data,
>> > > > > >Calculate HTable Region Splits, Create HTable
>> > > > > >
>> > > > > >I am not sure if it's event related but in the logs can this:
>> > > > > >
>> > > > > >2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job:
>> map
>> > > 1%
>> > > > > >reduce 0%
>> > > > > >
>> > > > > >2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job:
>> Task
>> > > Id
>> > > > :
>> > > > > >attempt_1432057232815_49758_m_000003_2, Status : FAILED
>> > > > > >
>> > > > > >Error: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException:
>> > > > Class
>> > > > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
>> > > > > >
>> > > > > > at
>> > > > >
>> >
>>>org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
>> > > > > >
>> > > > > > at
>> > > > >
>> > > >
>> > >
>> >
>>
>>>org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobC
>>>on
>> > > > > >textImpl.java:174)
>> > > > > >
>> > > > > > at
>> > > > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>> > > > > >
>> > > > > > at
>>org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>> > > > > >
>> > > > > > at
>> > > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
>> > > > > >
>> > > > > > at java.security.AccessController.doPrivileged(Native
>> > Method)
>> > > > > >
>> > > > > > at javax.security.auth.Subject.doAs(Subject.java:415)
>> > > > > >
>> > > > > > at
>> > > > >
>> > > >
>> > >
>> >
>>
>>>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformatio
>>>n.
>> > > > > >java:1594)
>> > > > > >
>> > > > > > at
>> > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
>> > > > > >
>> > > > > >Caused by: java.lang.ClassNotFoundException: Class
>> > > > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
>> > > > > >
>> > > > > > at
>> > > > >
>> > > >
>> > >
>> >
>>
>>>org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1
>>>86
>> > > > > >7)
>> > > > > >
>> > > > > > at
>> > > > >
>> >
>>>org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
>> > > > > >
>> > > > > > ... 8 more
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >but then it seems to go on anyway:
>> > > > > >
>> > > > > >2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job:
>> map
>> > > 2%
>> > > > > >reduce 0%
>> > > > > >
>> > > > > >2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job:
>> map
>> > > 3%
>> > > > > >reduce 0%
>> > > > > >
>> > > > > >2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job:
>> map
>> > > 4%
>> > > > > >reduce 0%
>> > > > > >
>> > > > > >2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job:
>> map
>> > > 5%
>> > > > > >reduce 0%
>> > > > > >
>> > > > > >2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job:
>> map
>> > > 6%
>> > > > > >reduce 0%
>> > > > > >
>> > > > > >...
>> > > > > >
>> > > > > >2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job:
>> map
>> > > > 100%
>> > > > > >reduce 100%
>> > > > > >
>> > > > > >2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job:
>> Job
>> > > > > >job_1432057232815_49758 completed successfully
>> > > > > >
>> > > > > >
>> > > > > >This comes after
>> > > > > >
>> > > > > >
>> > > > > >Hive Column Cardinality calculation for table
>> > > > > >
>> > > > > >and a step called :
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>>+-----------------------------------------------------------------------
>>>--
>> > > > > >-----------------------------+
>> > > > > >
>> > > > > >| null
>> > > > > > |
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>>+-----------------------------------------------------------------------
>>>--
>> > > > > >-----------------------------+
>> > > > > >
>> > > > > >
>> > > > > >Thanks for your help, I would really like to make this PoC
>>work!
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
Re: Cube built but not possible to query
Posted by alex schufo <al...@gmail.com>.
So like I said in my previous email after copying the jars and restarting
Kylin I could see the tables and query them.
But yesterday I built a new cube and the tables disappeared for both
projects. I restart Kylin and then I could see the tables for both projects
and query them.
Is there a Jira for this?
On Fri, Jun 5, 2015 at 2:34 PM, divye sheth <di...@gmail.com> wrote:
> My bad just realised it when compared it with my working copy. This has to
> do something with the loading of jars though they are available in the lib
> but somehow are not loaded. I believe there is already a jira for this.
> Will send the link once I get it.
> On Jun 5, 2015 18:49, "alex schufo" <al...@gmail.com> wrote:
>
> > Awesome!
> >
> > I tried this and it solved my issue. I now the the table / database and
> can
> > query the cube with second latency, thanks a lot!
> >
> > Just one thing, the source and destination are actually reversed, here
> are
> > the commands:
> >
> > cp tomcat/webapps/kylin/WEB-INF/lib/calcite-core-${version}.jar
> tomcat/lib/
> >
> > cp tomcat/webapps/kylin/WEB-INF/lib/kylin-query-${version}.jar
> tomcat/lib/
> >
> > On Fri, Jun 5, 2015 at 11:47 AM, divye sheth <di...@gmail.com>
> wrote:
> >
> > > Hi Alex,
> > >
> > > I had faced the same issue and I believe it was addressed by one of the
> > > developers of Kylin, cant get hold of the link though. Here is the
> > solution
> > > that was posted there and I was able to get this working.
> > >
> > > Copy the kylin-query-${version}.jar and calcite-core-${version}.jar the
> > > source and destination are given below:
> > > sourcce location: $KYLIN_HOME/tomcat/lib
> > > destination: $KYLIN_HOME/tomcat/webapps/kylin/WEB-INF/lib
> > >
> > > after you have copied please restart the kylin. Let me know if this
> > solves
> > > your issue.
> > >
> > > Thanks
> > > Divye Sheth
> > >
> > > On Fri, Jun 5, 2015 at 4:04 PM, alex schufo <al...@gmail.com>
> > wrote:
> > >
> > > > I checked that but there isn't any dependency error, see below the
> > > output:
> > > >
> > > > $ bin/check-env.sh
> > > >
> > > > KYLIN_HOME is set to
> > /home/username/kylin/kylin-0.7.1-incubating-SNAPSHOT
> > > >
> > > > On Fri, Jun 5, 2015 at 11:16 AM, Shi, Shaofeng <sh...@ebay.com>
> > wrote:
> > > >
> > > > > Please run $KYLIN_HOME/bin/find-hive-dependency.sh and then copy
> the
> > > > > output here; Kylin need add the Hcatalog to MR class path, if the
> > > > Hcatalog
> > > > > jar wasn¹t found, the "HCatInputFormat not found² error will be
> > > reported;
> > > > >
> > > > > On 6/5/15, 4:58 PM, "alex schufo" <al...@gmail.com> wrote:
> > > > >
> > > > > >Hi all,
> > > > > >
> > > > > >I built a cube on a Hive fact table of about 300 millions rows,
> all
> > > the
> > > > > >steps are FINISHED with success but it seems I cannot do any
> query.
> > > > > >
> > > > > >select * from myFactTable; (or any other query)
> > > > > >
> > > > > >returns From line 1, column 15 to line 1, column 43: Table
> > > 'MYFACTTABLE'
> > > > > >not found while executing SQL: "select * from myFactTable LIMIT
> > 50000"
> > > > > >
> > > > > >I tried rebuilding completely the cube after a purge, created a
> new
> > > > > >project, etc. but it doesn't solve the issue.
> > > > > >
> > > > > >It seems like a new KYLIN_ table has been created and contains
> data.
> > > How
> > > > > >can I know which HBase table corresponds to which cube?
> > > > > >
> > > > > >The "Convert Cuboid Data to HFile" step took a while, the
> MapReduce
> > > job
> > > > > >stayed in Undefined status for a long time but finally Succeeded.
> > > When I
> > > > > >click on the link from the Kylin UI it send me to 7070/kylin/N/A
> > > instead
> > > > > >of
> > > > > >the MapReduce job info, is that a bad sign?
> > > > > >
> > > > > >In the logs I don't see any error in the Build N-Dimension Cuboid
> > > Data,
> > > > > >Calculate HTable Region Splits, Create HTable
> > > > > >
> > > > > >I am not sure if it's event related but in the logs can this:
> > > > > >
> > > > > >2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job:
> map
> > > 1%
> > > > > >reduce 0%
> > > > > >
> > > > > >2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job:
> Task
> > > Id
> > > > :
> > > > > >attempt_1432057232815_49758_m_000003_2, Status : FAILED
> > > > > >
> > > > > >Error: java.lang.RuntimeException:
> java.lang.ClassNotFoundException:
> > > > Class
> > > > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> > > > > >
> > > > > > at
> > > > >
> > >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
> > > > > >
> > > > > > at
> > > > >
> > > >
> > >
> >
> >org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobCon
> > > > > >textImpl.java:174)
> > > > > >
> > > > > > at
> > > > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
> > > > > >
> > > > > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> > > > > >
> > > > > > at
> > > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> > > > > >
> > > > > > at java.security.AccessController.doPrivileged(Native
> > Method)
> > > > > >
> > > > > > at javax.security.auth.Subject.doAs(Subject.java:415)
> > > > > >
> > > > > > at
> > > > >
> > > >
> > >
> >
> >org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.
> > > > > >java:1594)
> > > > > >
> > > > > > at
> > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> > > > > >
> > > > > >Caused by: java.lang.ClassNotFoundException: Class
> > > > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> > > > > >
> > > > > > at
> > > > >
> > > >
> > >
> >
> >org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:186
> > > > > >7)
> > > > > >
> > > > > > at
> > > > >
> > >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
> > > > > >
> > > > > > ... 8 more
> > > > > >
> > > > > >
> > > > > >
> > > > > >but then it seems to go on anyway:
> > > > > >
> > > > > >2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job:
> map
> > > 2%
> > > > > >reduce 0%
> > > > > >
> > > > > >2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job:
> map
> > > 3%
> > > > > >reduce 0%
> > > > > >
> > > > > >2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job:
> map
> > > 4%
> > > > > >reduce 0%
> > > > > >
> > > > > >2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job:
> map
> > > 5%
> > > > > >reduce 0%
> > > > > >
> > > > > >2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job:
> map
> > > 6%
> > > > > >reduce 0%
> > > > > >
> > > > > >...
> > > > > >
> > > > > >2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job:
> map
> > > > 100%
> > > > > >reduce 100%
> > > > > >
> > > > > >2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job:
> Job
> > > > > >job_1432057232815_49758 completed successfully
> > > > > >
> > > > > >
> > > > > >This comes after
> > > > > >
> > > > > >
> > > > > >Hive Column Cardinality calculation for table
> > > > > >
> > > > > >and a step called :
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >+-------------------------------------------------------------------------
> > > > > >-----------------------------+
> > > > > >
> > > > > >| null
> > > > > > |
> > > > > >
> > > > >
> > > >
> > >
> >
> >+-------------------------------------------------------------------------
> > > > > >-----------------------------+
> > > > > >
> > > > > >
> > > > > >Thanks for your help, I would really like to make this PoC work!
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Cube built but not possible to query
Posted by divye sheth <di...@gmail.com>.
My bad just realised it when compared it with my working copy. This has to
do something with the loading of jars though they are available in the lib
but somehow are not loaded. I believe there is already a jira for this.
Will send the link once I get it.
On Jun 5, 2015 18:49, "alex schufo" <al...@gmail.com> wrote:
> Awesome!
>
> I tried this and it solved my issue. I now the the table / database and can
> query the cube with second latency, thanks a lot!
>
> Just one thing, the source and destination are actually reversed, here are
> the commands:
>
> cp tomcat/webapps/kylin/WEB-INF/lib/calcite-core-${version}.jar tomcat/lib/
>
> cp tomcat/webapps/kylin/WEB-INF/lib/kylin-query-${version}.jar tomcat/lib/
>
> On Fri, Jun 5, 2015 at 11:47 AM, divye sheth <di...@gmail.com> wrote:
>
> > Hi Alex,
> >
> > I had faced the same issue and I believe it was addressed by one of the
> > developers of Kylin, cant get hold of the link though. Here is the
> solution
> > that was posted there and I was able to get this working.
> >
> > Copy the kylin-query-${version}.jar and calcite-core-${version}.jar the
> > source and destination are given below:
> > sourcce location: $KYLIN_HOME/tomcat/lib
> > destination: $KYLIN_HOME/tomcat/webapps/kylin/WEB-INF/lib
> >
> > after you have copied please restart the kylin. Let me know if this
> solves
> > your issue.
> >
> > Thanks
> > Divye Sheth
> >
> > On Fri, Jun 5, 2015 at 4:04 PM, alex schufo <al...@gmail.com>
> wrote:
> >
> > > I checked that but there isn't any dependency error, see below the
> > output:
> > >
> > > $ bin/check-env.sh
> > >
> > > KYLIN_HOME is set to
> /home/username/kylin/kylin-0.7.1-incubating-SNAPSHOT
> > >
> > > On Fri, Jun 5, 2015 at 11:16 AM, Shi, Shaofeng <sh...@ebay.com>
> wrote:
> > >
> > > > Please run $KYLIN_HOME/bin/find-hive-dependency.sh and then copy the
> > > > output here; Kylin need add the Hcatalog to MR class path, if the
> > > Hcatalog
> > > > jar wasn¹t found, the "HCatInputFormat not found² error will be
> > reported;
> > > >
> > > > On 6/5/15, 4:58 PM, "alex schufo" <al...@gmail.com> wrote:
> > > >
> > > > >Hi all,
> > > > >
> > > > >I built a cube on a Hive fact table of about 300 millions rows, all
> > the
> > > > >steps are FINISHED with success but it seems I cannot do any query.
> > > > >
> > > > >select * from myFactTable; (or any other query)
> > > > >
> > > > >returns From line 1, column 15 to line 1, column 43: Table
> > 'MYFACTTABLE'
> > > > >not found while executing SQL: "select * from myFactTable LIMIT
> 50000"
> > > > >
> > > > >I tried rebuilding completely the cube after a purge, created a new
> > > > >project, etc. but it doesn't solve the issue.
> > > > >
> > > > >It seems like a new KYLIN_ table has been created and contains data.
> > How
> > > > >can I know which HBase table corresponds to which cube?
> > > > >
> > > > >The "Convert Cuboid Data to HFile" step took a while, the MapReduce
> > job
> > > > >stayed in Undefined status for a long time but finally Succeeded.
> > When I
> > > > >click on the link from the Kylin UI it send me to 7070/kylin/N/A
> > instead
> > > > >of
> > > > >the MapReduce job info, is that a bad sign?
> > > > >
> > > > >In the logs I don't see any error in the Build N-Dimension Cuboid
> > Data,
> > > > >Calculate HTable Region Splits, Create HTable
> > > > >
> > > > >I am not sure if it's event related but in the logs can this:
> > > > >
> > > > >2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job: map
> > 1%
> > > > >reduce 0%
> > > > >
> > > > >2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job: Task
> > Id
> > > :
> > > > >attempt_1432057232815_49758_m_000003_2, Status : FAILED
> > > > >
> > > > >Error: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> > > Class
> > > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> > > > >
> > > > > at
> > > >
> >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
> > > > >
> > > > > at
> > > >
> > >
> >
> >org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobCon
> > > > >textImpl.java:174)
> > > > >
> > > > > at
> > > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
> > > > >
> > > > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> > > > >
> > > > > at
> > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> > > > >
> > > > > at java.security.AccessController.doPrivileged(Native
> Method)
> > > > >
> > > > > at javax.security.auth.Subject.doAs(Subject.java:415)
> > > > >
> > > > > at
> > > >
> > >
> >
> >org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.
> > > > >java:1594)
> > > > >
> > > > > at
> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> > > > >
> > > > >Caused by: java.lang.ClassNotFoundException: Class
> > > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> > > > >
> > > > > at
> > > >
> > >
> >
> >org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:186
> > > > >7)
> > > > >
> > > > > at
> > > >
> >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
> > > > >
> > > > > ... 8 more
> > > > >
> > > > >
> > > > >
> > > > >but then it seems to go on anyway:
> > > > >
> > > > >2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job: map
> > 2%
> > > > >reduce 0%
> > > > >
> > > > >2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job: map
> > 3%
> > > > >reduce 0%
> > > > >
> > > > >2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job: map
> > 4%
> > > > >reduce 0%
> > > > >
> > > > >2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job: map
> > 5%
> > > > >reduce 0%
> > > > >
> > > > >2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job: map
> > 6%
> > > > >reduce 0%
> > > > >
> > > > >...
> > > > >
> > > > >2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job: map
> > > 100%
> > > > >reduce 100%
> > > > >
> > > > >2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job: Job
> > > > >job_1432057232815_49758 completed successfully
> > > > >
> > > > >
> > > > >This comes after
> > > > >
> > > > >
> > > > >Hive Column Cardinality calculation for table
> > > > >
> > > > >and a step called :
> > > > >
> > > > >
> > > >
> > >
> >
> >+-------------------------------------------------------------------------
> > > > >-----------------------------+
> > > > >
> > > > >| null
> > > > > |
> > > > >
> > > >
> > >
> >
> >+-------------------------------------------------------------------------
> > > > >-----------------------------+
> > > > >
> > > > >
> > > > >Thanks for your help, I would really like to make this PoC work!
> > > >
> > > >
> > >
> >
>
Re: Cube built but not possible to query
Posted by alex schufo <al...@gmail.com>.
Awesome!
I tried this and it solved my issue. I now the the table / database and can
query the cube with second latency, thanks a lot!
Just one thing, the source and destination are actually reversed, here are
the commands:
cp tomcat/webapps/kylin/WEB-INF/lib/calcite-core-${version}.jar tomcat/lib/
cp tomcat/webapps/kylin/WEB-INF/lib/kylin-query-${version}.jar tomcat/lib/
On Fri, Jun 5, 2015 at 11:47 AM, divye sheth <di...@gmail.com> wrote:
> Hi Alex,
>
> I had faced the same issue and I believe it was addressed by one of the
> developers of Kylin, cant get hold of the link though. Here is the solution
> that was posted there and I was able to get this working.
>
> Copy the kylin-query-${version}.jar and calcite-core-${version}.jar the
> source and destination are given below:
> sourcce location: $KYLIN_HOME/tomcat/lib
> destination: $KYLIN_HOME/tomcat/webapps/kylin/WEB-INF/lib
>
> after you have copied please restart the kylin. Let me know if this solves
> your issue.
>
> Thanks
> Divye Sheth
>
> On Fri, Jun 5, 2015 at 4:04 PM, alex schufo <al...@gmail.com> wrote:
>
> > I checked that but there isn't any dependency error, see below the
> output:
> >
> > $ bin/check-env.sh
> >
> > KYLIN_HOME is set to /home/username/kylin/kylin-0.7.1-incubating-SNAPSHOT
> >
> > On Fri, Jun 5, 2015 at 11:16 AM, Shi, Shaofeng <sh...@ebay.com> wrote:
> >
> > > Please run $KYLIN_HOME/bin/find-hive-dependency.sh and then copy the
> > > output here; Kylin need add the Hcatalog to MR class path, if the
> > Hcatalog
> > > jar wasn¹t found, the "HCatInputFormat not found² error will be
> reported;
> > >
> > > On 6/5/15, 4:58 PM, "alex schufo" <al...@gmail.com> wrote:
> > >
> > > >Hi all,
> > > >
> > > >I built a cube on a Hive fact table of about 300 millions rows, all
> the
> > > >steps are FINISHED with success but it seems I cannot do any query.
> > > >
> > > >select * from myFactTable; (or any other query)
> > > >
> > > >returns From line 1, column 15 to line 1, column 43: Table
> 'MYFACTTABLE'
> > > >not found while executing SQL: "select * from myFactTable LIMIT 50000"
> > > >
> > > >I tried rebuilding completely the cube after a purge, created a new
> > > >project, etc. but it doesn't solve the issue.
> > > >
> > > >It seems like a new KYLIN_ table has been created and contains data.
> How
> > > >can I know which HBase table corresponds to which cube?
> > > >
> > > >The "Convert Cuboid Data to HFile" step took a while, the MapReduce
> job
> > > >stayed in Undefined status for a long time but finally Succeeded.
> When I
> > > >click on the link from the Kylin UI it send me to 7070/kylin/N/A
> instead
> > > >of
> > > >the MapReduce job info, is that a bad sign?
> > > >
> > > >In the logs I don't see any error in the Build N-Dimension Cuboid
> Data,
> > > >Calculate HTable Region Splits, Create HTable
> > > >
> > > >I am not sure if it's event related but in the logs can this:
> > > >
> > > >2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job: map
> 1%
> > > >reduce 0%
> > > >
> > > >2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job: Task
> Id
> > :
> > > >attempt_1432057232815_49758_m_000003_2, Status : FAILED
> > > >
> > > >Error: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> > Class
> > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> > > >
> > > > at
> > > >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
> > > >
> > > > at
> > >
> >
> >org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobCon
> > > >textImpl.java:174)
> > > >
> > > > at
> > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
> > > >
> > > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> > > >
> > > > at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> > > >
> > > > at java.security.AccessController.doPrivileged(Native Method)
> > > >
> > > > at javax.security.auth.Subject.doAs(Subject.java:415)
> > > >
> > > > at
> > >
> >
> >org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.
> > > >java:1594)
> > > >
> > > > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> > > >
> > > >Caused by: java.lang.ClassNotFoundException: Class
> > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> > > >
> > > > at
> > >
> >
> >org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:186
> > > >7)
> > > >
> > > > at
> > > >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
> > > >
> > > > ... 8 more
> > > >
> > > >
> > > >
> > > >but then it seems to go on anyway:
> > > >
> > > >2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job: map
> 2%
> > > >reduce 0%
> > > >
> > > >2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job: map
> 3%
> > > >reduce 0%
> > > >
> > > >2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job: map
> 4%
> > > >reduce 0%
> > > >
> > > >2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job: map
> 5%
> > > >reduce 0%
> > > >
> > > >2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job: map
> 6%
> > > >reduce 0%
> > > >
> > > >...
> > > >
> > > >2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job: map
> > 100%
> > > >reduce 100%
> > > >
> > > >2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job: Job
> > > >job_1432057232815_49758 completed successfully
> > > >
> > > >
> > > >This comes after
> > > >
> > > >
> > > >Hive Column Cardinality calculation for table
> > > >
> > > >and a step called :
> > > >
> > > >
> > >
> >
> >+-------------------------------------------------------------------------
> > > >-----------------------------+
> > > >
> > > >| null
> > > > |
> > > >
> > >
> >
> >+-------------------------------------------------------------------------
> > > >-----------------------------+
> > > >
> > > >
> > > >Thanks for your help, I would really like to make this PoC work!
> > >
> > >
> >
>
Re: Cube built but not possible to query
Posted by divye sheth <di...@gmail.com>.
Hi Alex,
I had faced the same issue and I believe it was addressed by one of the
developers of Kylin, cant get hold of the link though. Here is the solution
that was posted there and I was able to get this working.
Copy the kylin-query-${version}.jar and calcite-core-${version}.jar the
source and destination are given below:
sourcce location: $KYLIN_HOME/tomcat/lib
destination: $KYLIN_HOME/tomcat/webapps/kylin/WEB-INF/lib
after you have copied please restart the kylin. Let me know if this solves
your issue.
Thanks
Divye Sheth
On Fri, Jun 5, 2015 at 4:04 PM, alex schufo <al...@gmail.com> wrote:
> I checked that but there isn't any dependency error, see below the output:
>
> $ bin/check-env.sh
>
> KYLIN_HOME is set to /home/username/kylin/kylin-0.7.1-incubating-SNAPSHOT
>
> On Fri, Jun 5, 2015 at 11:16 AM, Shi, Shaofeng <sh...@ebay.com> wrote:
>
> > Please run $KYLIN_HOME/bin/find-hive-dependency.sh and then copy the
> > output here; Kylin need add the Hcatalog to MR class path, if the
> Hcatalog
> > jar wasn¹t found, the "HCatInputFormat not found² error will be reported;
> >
> > On 6/5/15, 4:58 PM, "alex schufo" <al...@gmail.com> wrote:
> >
> > >Hi all,
> > >
> > >I built a cube on a Hive fact table of about 300 millions rows, all the
> > >steps are FINISHED with success but it seems I cannot do any query.
> > >
> > >select * from myFactTable; (or any other query)
> > >
> > >returns From line 1, column 15 to line 1, column 43: Table 'MYFACTTABLE'
> > >not found while executing SQL: "select * from myFactTable LIMIT 50000"
> > >
> > >I tried rebuilding completely the cube after a purge, created a new
> > >project, etc. but it doesn't solve the issue.
> > >
> > >It seems like a new KYLIN_ table has been created and contains data. How
> > >can I know which HBase table corresponds to which cube?
> > >
> > >The "Convert Cuboid Data to HFile" step took a while, the MapReduce job
> > >stayed in Undefined status for a long time but finally Succeeded. When I
> > >click on the link from the Kylin UI it send me to 7070/kylin/N/A instead
> > >of
> > >the MapReduce job info, is that a bad sign?
> > >
> > >In the logs I don't see any error in the Build N-Dimension Cuboid Data,
> > >Calculate HTable Region Splits, Create HTable
> > >
> > >I am not sure if it's event related but in the logs can this:
> > >
> > >2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job: map 1%
> > >reduce 0%
> > >
> > >2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job: Task Id
> :
> > >attempt_1432057232815_49758_m_000003_2, Status : FAILED
> > >
> > >Error: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class
> > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> > >
> > > at
> > >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
> > >
> > > at
> >
> >org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobCon
> > >textImpl.java:174)
> > >
> > > at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
> > >
> > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> > >
> > > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> > >
> > > at java.security.AccessController.doPrivileged(Native Method)
> > >
> > > at javax.security.auth.Subject.doAs(Subject.java:415)
> > >
> > > at
> >
> >org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.
> > >java:1594)
> > >
> > > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> > >
> > >Caused by: java.lang.ClassNotFoundException: Class
> > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> > >
> > > at
> >
> >org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:186
> > >7)
> > >
> > > at
> > >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
> > >
> > > ... 8 more
> > >
> > >
> > >
> > >but then it seems to go on anyway:
> > >
> > >2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job: map 2%
> > >reduce 0%
> > >
> > >2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job: map 3%
> > >reduce 0%
> > >
> > >2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job: map 4%
> > >reduce 0%
> > >
> > >2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job: map 5%
> > >reduce 0%
> > >
> > >2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job: map 6%
> > >reduce 0%
> > >
> > >...
> > >
> > >2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job: map
> 100%
> > >reduce 100%
> > >
> > >2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job: Job
> > >job_1432057232815_49758 completed successfully
> > >
> > >
> > >This comes after
> > >
> > >
> > >Hive Column Cardinality calculation for table
> > >
> > >and a step called :
> > >
> > >
> >
> >+-------------------------------------------------------------------------
> > >-----------------------------+
> > >
> > >| null
> > > |
> > >
> >
> >+-------------------------------------------------------------------------
> > >-----------------------------+
> > >
> > >
> > >Thanks for your help, I would really like to make this PoC work!
> >
> >
>
Re: Cube built but not possible to query
Posted by alex schufo <al...@gmail.com>.
Sorry wrong output indeed, here is the correct one:
$ bin/find-hive-dependency.sh
Logging initialized using configuration in
file:/etc/hive/conf.dist/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/edw/hive/auxlib/hive-udfs.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
hive dependency:
/etc/hive/conf:/usr/lib/hive/lib/ant-1.9.1.jar:/usr/lib/hive/lib/hive-common-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/hive-service.jar:/usr/lib/hive/lib/hive-shims.jar:/usr/lib/hive/lib/hive-shims-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/optiq-avatica-0.5.jar:/usr/lib/hive/lib/servlet-api-2.5.jar:/usr/lib/hive/lib/paranamer-2.3.jar:/usr/lib/hive/lib/commons-pool-1.5.4.jar:/usr/lib/hive/lib/activation-1.1.jar:/usr/lib/hive/lib/hive-shims-common-secure.jar:/usr/lib/hive/lib/datanucleus-core-3.2.10.jar:/usr/lib/hive/lib/hive-beeline.jar:/usr/lib/hive/lib/hamcrest-core-1.1.jar:/usr/lib/hive/lib/jetty-6.1.26.jar:/usr/lib/hive/lib/jline-0.9.94.jar:/usr/lib/hive/lib/commons-httpclient-3.0.1.jar:/usr/lib/hive/lib/hive-metastore.jar:/usr/lib/hive/lib/avro-1.7.5.jar:/usr/lib/hive/lib/hive-common.jar:/usr/lib/hive/lib/guava-11.0.2.jar:/usr/lib/hive/lib/log4j-1.2.16.jar:/usr/lib/hive/lib/commons-compress-1.4.1.jar:/usr/lib/hive/lib/ant-launcher-1.9.1.jar:/usr/lib/hive/lib/hive-shims-common-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/bonecp-0.8.0.RELEASE.jar:/usr/lib/hive/lib/hive-cli.jar:/usr/lib/hive/lib/commons-codec-1.4.jar:/usr/lib/hive/lib/stax-api-1.0.1.jar:/usr/lib/hive/lib/hive-service-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/hive-testutils.jar:/usr/lib/hive/lib/oro-2.0.8.jar:/usr/lib/hive/lib/optiq-core-0.5.jar:/usr/lib/hive/lib/hive-shims-0.23-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/hive-cli-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/hive-beeline-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/commons-logging-1.1.3.jar:/usr/lib/hive/lib/hive-exec-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/hive-jdbc-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/hive-contrib-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/libthrift-0.9.0.jar:/usr/lib/hive/lib/linq4j-0.1.13.jar:/usr/lib/hive/lib/hive-custom-hooks-0.0.1-SNAPSHOT.jar:/usr/lib/hive/lib/jdo-api-3.0.1.jar:/usr/lib/hive/lib/hive-shims-0.20S-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/commons-collections-3.1.jar:/usr/lib/hive/lib/hive-shims-common.jar:/usr/lib/hive/lib/hive-hwi.jar:/usr/lib/hive/lib/xz-1.0.jar:/usr/lib/hive/lib/jetty-util-6.1.26.jar:/usr/lib/hive/lib/hive-metastore-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/hive-jdbc.jar:/usr/lib/hive/lib/servlet-api-2.5-20081211.jar:/usr/lib/hive/lib/commons-compiler-2.7.3.jar:/usr/lib/hive/lib/tempus-fugit-1.1.jar:/usr/lib/hive/lib/httpcore-4.2.5.jar:/usr/lib/hive/lib/hive-serde.jar:/usr/lib/hive/lib/hive-shims-0.20-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/mail-1.4.1.jar:/usr/lib/hive/lib/janino-2.7.3.jar:/usr/lib/hive/lib/hive-ant-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/asm-tree-3.1.jar:/usr/lib/hive/lib/eigenbase-properties-1.1.4.jar:/usr/lib/hive/lib/datanucleus-api-jdo-3.2.6.jar:/usr/lib/hive/lib/derby-10.10.1.1.jar:/usr/lib/hive/lib/commons-cli-1.2.jar:/usr/lib/hive/lib/stringtemplate-3.2.1.jar:/usr/lib/hive/lib/hive-hbase-handler-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/jta-1.1.jar:/usr/lib/hive/lib/geronimo-jaspic_1.0_spec-1.0.jar:/usr/lib/hive/lib/libfb303-0.9.0.jar:/usr/lib/hive/lib/geronimo-jta_1.1_spec-1.1.1.jar:/usr/lib/hive/lib/hive-hbase-handler.jar:/usr/lib/hive/lib/commons-lang-2.4.jar:/usr/lib/hive/lib/commons-dbcp-1.4.jar:/usr/lib/hive/lib/hive-ant.jar:/usr/lib/hive/lib/hive-testutils-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/derbynet-10.10.1.1.jar:/usr/lib/hive/lib/httpclient-4.2.5.jar:/usr/lib/hive/lib/antlr-2.7.7.jar:/usr/lib/hive/lib/hive-hwi-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/junit-4.10.jar:/usr/lib/hive/lib/derbyclient-10.10.1.1.jar:/usr/lib/hive/lib/hive-exec.jar:/usr/lib/hive/lib/datanucleus-rdbms-3.2.9.jar:/usr/lib/hive/lib/groovy-all-2.1.6.jar:/usr/lib/hive/lib/snappy-java-1.0.5.jar:/usr/lib/hive/lib/antlr-runtime-3.4.jar:/usr/lib/hive/lib/hive-contrib.jar:/usr/lib/hive/lib/ST4-4.0.4.jar:/usr/lib/hive/lib/asm-commons-3.1.jar:/usr/lib/hive/lib/jetty-all-7.6.0.v20120127.jar:/usr/lib/hive/lib/jpam-1.1.jar:/usr/lib/hive/lib/eigenbase-xom-1.3.4.jar:/usr/lib/hive/lib/velocity-1.5.jar:/usr/lib/hive/lib/hive-shims-common-secure-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/zookeeper-3.4.5.2.1.7.0-784.jar:/usr/lib/hive/lib/hive-serde-0.13.0.2.1.7.0-784.jar:/usr/lib/hive/lib/jsr305-1.3.9.jar:/usr/lib/hive/lib/geronimo-annotation_1.0_spec-1.1.1.jar:/usr/lib/hive/lib/commons-io-2.4.jar:/usr/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core-0.13.0.2.1.7.0-784.jar
hive conf: /etc/hive/conf
On Fri, Jun 5, 2015 at 11:48 AM, Shi, Shaofeng <sh...@ebay.com> wrote:
> Not check-env.sh, try find-hive-dependency.sh; it will output something
> like "hive dependency: xxxx”;
>
> On 6/5/15, 6:34 PM, "alex schufo" <al...@gmail.com> wrote:
>
> >I checked that but there isn't any dependency error, see below the output:
> >
> >$ bin/check-env.sh
> >
> >KYLIN_HOME is set to /home/username/kylin/kylin-0.7.1-incubating-SNAPSHOT
> >
> >On Fri, Jun 5, 2015 at 11:16 AM, Shi, Shaofeng <sh...@ebay.com> wrote:
> >
> >> Please run $KYLIN_HOME/bin/find-hive-dependency.sh and then copy the
> >> output here; Kylin need add the Hcatalog to MR class path, if the
> >>Hcatalog
> >> jar wasn¹t found, the "HCatInputFormat not found² error will be
> >>reported;
> >>
> >> On 6/5/15, 4:58 PM, "alex schufo" <al...@gmail.com> wrote:
> >>
> >> >Hi all,
> >> >
> >> >I built a cube on a Hive fact table of about 300 millions rows, all the
> >> >steps are FINISHED with success but it seems I cannot do any query.
> >> >
> >> >select * from myFactTable; (or any other query)
> >> >
> >> >returns From line 1, column 15 to line 1, column 43: Table
> >>'MYFACTTABLE'
> >> >not found while executing SQL: "select * from myFactTable LIMIT 50000"
> >> >
> >> >I tried rebuilding completely the cube after a purge, created a new
> >> >project, etc. but it doesn't solve the issue.
> >> >
> >> >It seems like a new KYLIN_ table has been created and contains data.
> >>How
> >> >can I know which HBase table corresponds to which cube?
> >> >
> >> >The "Convert Cuboid Data to HFile" step took a while, the MapReduce job
> >> >stayed in Undefined status for a long time but finally Succeeded. When
> >>I
> >> >click on the link from the Kylin UI it send me to 7070/kylin/N/A
> >>instead
> >> >of
> >> >the MapReduce job info, is that a bad sign?
> >> >
> >> >In the logs I don't see any error in the Build N-Dimension Cuboid Data,
> >> >Calculate HTable Region Splits, Create HTable
> >> >
> >> >I am not sure if it's event related but in the logs can this:
> >> >
> >> >2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job: map 1%
> >> >reduce 0%
> >> >
> >> >2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job: Task
> >>Id :
> >> >attempt_1432057232815_49758_m_000003_2, Status : FAILED
> >> >
> >> >Error: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> >>Class
> >> >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> >> >
> >> > at
> >> >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
> >> >
> >> > at
> >>
> >>>org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobC
> >>>on
> >> >textImpl.java:174)
> >> >
> >> > at
> >>org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
> >> >
> >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> >> >
> >> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> >> >
> >> > at java.security.AccessController.doPrivileged(Native Method)
> >> >
> >> > at javax.security.auth.Subject.doAs(Subject.java:415)
> >> >
> >> > at
> >>
> >>>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformatio
> >>>n.
> >> >java:1594)
> >> >
> >> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> >> >
> >> >Caused by: java.lang.ClassNotFoundException: Class
> >> >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> >> >
> >> > at
> >>
> >>>org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1
> >>>86
> >> >7)
> >> >
> >> > at
> >> >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
> >> >
> >> > ... 8 more
> >> >
> >> >
> >> >
> >> >but then it seems to go on anyway:
> >> >
> >> >2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job: map 2%
> >> >reduce 0%
> >> >
> >> >2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job: map 3%
> >> >reduce 0%
> >> >
> >> >2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job: map 4%
> >> >reduce 0%
> >> >
> >> >2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job: map 5%
> >> >reduce 0%
> >> >
> >> >2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job: map 6%
> >> >reduce 0%
> >> >
> >> >...
> >> >
> >> >2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job: map
> >>100%
> >> >reduce 100%
> >> >
> >> >2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job: Job
> >> >job_1432057232815_49758 completed successfully
> >> >
> >> >
> >> >This comes after
> >> >
> >> >
> >> >Hive Column Cardinality calculation for table
> >> >
> >> >and a step called :
> >> >
> >> >
> >>
> >>>+-----------------------------------------------------------------------
> >>>--
> >> >-----------------------------+
> >> >
> >> >| null
> >> > |
> >> >
> >>
> >>>+-----------------------------------------------------------------------
> >>>--
> >> >-----------------------------+
> >> >
> >> >
> >> >Thanks for your help, I would really like to make this PoC work!
> >>
> >>
>
>
Re: Cube built but not possible to query
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
Not check-env.sh, try find-hive-dependency.sh; it will output something
like "hive dependency: xxxx”;
On 6/5/15, 6:34 PM, "alex schufo" <al...@gmail.com> wrote:
>I checked that but there isn't any dependency error, see below the output:
>
>$ bin/check-env.sh
>
>KYLIN_HOME is set to /home/username/kylin/kylin-0.7.1-incubating-SNAPSHOT
>
>On Fri, Jun 5, 2015 at 11:16 AM, Shi, Shaofeng <sh...@ebay.com> wrote:
>
>> Please run $KYLIN_HOME/bin/find-hive-dependency.sh and then copy the
>> output here; Kylin need add the Hcatalog to MR class path, if the
>>Hcatalog
>> jar wasn¹t found, the "HCatInputFormat not found² error will be
>>reported;
>>
>> On 6/5/15, 4:58 PM, "alex schufo" <al...@gmail.com> wrote:
>>
>> >Hi all,
>> >
>> >I built a cube on a Hive fact table of about 300 millions rows, all the
>> >steps are FINISHED with success but it seems I cannot do any query.
>> >
>> >select * from myFactTable; (or any other query)
>> >
>> >returns From line 1, column 15 to line 1, column 43: Table
>>'MYFACTTABLE'
>> >not found while executing SQL: "select * from myFactTable LIMIT 50000"
>> >
>> >I tried rebuilding completely the cube after a purge, created a new
>> >project, etc. but it doesn't solve the issue.
>> >
>> >It seems like a new KYLIN_ table has been created and contains data.
>>How
>> >can I know which HBase table corresponds to which cube?
>> >
>> >The "Convert Cuboid Data to HFile" step took a while, the MapReduce job
>> >stayed in Undefined status for a long time but finally Succeeded. When
>>I
>> >click on the link from the Kylin UI it send me to 7070/kylin/N/A
>>instead
>> >of
>> >the MapReduce job info, is that a bad sign?
>> >
>> >In the logs I don't see any error in the Build N-Dimension Cuboid Data,
>> >Calculate HTable Region Splits, Create HTable
>> >
>> >I am not sure if it's event related but in the logs can this:
>> >
>> >2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job: map 1%
>> >reduce 0%
>> >
>> >2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job: Task
>>Id :
>> >attempt_1432057232815_49758_m_000003_2, Status : FAILED
>> >
>> >Error: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>>Class
>> >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
>> >
>> > at
>> >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
>> >
>> > at
>>
>>>org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobC
>>>on
>> >textImpl.java:174)
>> >
>> > at
>>org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>> >
>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>> >
>> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
>> >
>> > at java.security.AccessController.doPrivileged(Native Method)
>> >
>> > at javax.security.auth.Subject.doAs(Subject.java:415)
>> >
>> > at
>>
>>>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformatio
>>>n.
>> >java:1594)
>> >
>> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
>> >
>> >Caused by: java.lang.ClassNotFoundException: Class
>> >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
>> >
>> > at
>>
>>>org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1
>>>86
>> >7)
>> >
>> > at
>> >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
>> >
>> > ... 8 more
>> >
>> >
>> >
>> >but then it seems to go on anyway:
>> >
>> >2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job: map 2%
>> >reduce 0%
>> >
>> >2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job: map 3%
>> >reduce 0%
>> >
>> >2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job: map 4%
>> >reduce 0%
>> >
>> >2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job: map 5%
>> >reduce 0%
>> >
>> >2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job: map 6%
>> >reduce 0%
>> >
>> >...
>> >
>> >2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job: map
>>100%
>> >reduce 100%
>> >
>> >2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job: Job
>> >job_1432057232815_49758 completed successfully
>> >
>> >
>> >This comes after
>> >
>> >
>> >Hive Column Cardinality calculation for table
>> >
>> >and a step called :
>> >
>> >
>>
>>>+-----------------------------------------------------------------------
>>>--
>> >-----------------------------+
>> >
>> >| null
>> > |
>> >
>>
>>>+-----------------------------------------------------------------------
>>>--
>> >-----------------------------+
>> >
>> >
>> >Thanks for your help, I would really like to make this PoC work!
>>
>>
Re: Cube built but not possible to query
Posted by alex schufo <al...@gmail.com>.
I checked that but there isn't any dependency error, see below the output:
$ bin/check-env.sh
KYLIN_HOME is set to /home/username/kylin/kylin-0.7.1-incubating-SNAPSHOT
On Fri, Jun 5, 2015 at 11:16 AM, Shi, Shaofeng <sh...@ebay.com> wrote:
> Please run $KYLIN_HOME/bin/find-hive-dependency.sh and then copy the
> output here; Kylin need add the Hcatalog to MR class path, if the Hcatalog
> jar wasn¹t found, the "HCatInputFormat not found² error will be reported;
>
> On 6/5/15, 4:58 PM, "alex schufo" <al...@gmail.com> wrote:
>
> >Hi all,
> >
> >I built a cube on a Hive fact table of about 300 millions rows, all the
> >steps are FINISHED with success but it seems I cannot do any query.
> >
> >select * from myFactTable; (or any other query)
> >
> >returns From line 1, column 15 to line 1, column 43: Table 'MYFACTTABLE'
> >not found while executing SQL: "select * from myFactTable LIMIT 50000"
> >
> >I tried rebuilding completely the cube after a purge, created a new
> >project, etc. but it doesn't solve the issue.
> >
> >It seems like a new KYLIN_ table has been created and contains data. How
> >can I know which HBase table corresponds to which cube?
> >
> >The "Convert Cuboid Data to HFile" step took a while, the MapReduce job
> >stayed in Undefined status for a long time but finally Succeeded. When I
> >click on the link from the Kylin UI it send me to 7070/kylin/N/A instead
> >of
> >the MapReduce job info, is that a bad sign?
> >
> >In the logs I don't see any error in the Build N-Dimension Cuboid Data,
> >Calculate HTable Region Splits, Create HTable
> >
> >I am not sure if it's event related but in the logs can this:
> >
> >2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job: map 1%
> >reduce 0%
> >
> >2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job: Task Id :
> >attempt_1432057232815_49758_m_000003_2, Status : FAILED
> >
> >Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
> >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> >
> > at
> >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
> >
> > at
> >org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobCon
> >textImpl.java:174)
> >
> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
> >
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> >
> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> >
> > at java.security.AccessController.doPrivileged(Native Method)
> >
> > at javax.security.auth.Subject.doAs(Subject.java:415)
> >
> > at
> >org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.
> >java:1594)
> >
> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> >
> >Caused by: java.lang.ClassNotFoundException: Class
> >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> >
> > at
> >org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:186
> >7)
> >
> > at
> >org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
> >
> > ... 8 more
> >
> >
> >
> >but then it seems to go on anyway:
> >
> >2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job: map 2%
> >reduce 0%
> >
> >2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job: map 3%
> >reduce 0%
> >
> >2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job: map 4%
> >reduce 0%
> >
> >2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job: map 5%
> >reduce 0%
> >
> >2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job: map 6%
> >reduce 0%
> >
> >...
> >
> >2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job: map 100%
> >reduce 100%
> >
> >2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job: Job
> >job_1432057232815_49758 completed successfully
> >
> >
> >This comes after
> >
> >
> >Hive Column Cardinality calculation for table
> >
> >and a step called :
> >
> >
> >+-------------------------------------------------------------------------
> >-----------------------------+
> >
> >| null
> > |
> >
> >+-------------------------------------------------------------------------
> >-----------------------------+
> >
> >
> >Thanks for your help, I would really like to make this PoC work!
>
>
Re: Cube built but not possible to query
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
Please run $KYLIN_HOME/bin/find-hive-dependency.sh and then copy the
output here; Kylin need add the Hcatalog to MR class path, if the Hcatalog
jar wasn¹t found, the "HCatInputFormat not found² error will be reported;
On 6/5/15, 4:58 PM, "alex schufo" <al...@gmail.com> wrote:
>Hi all,
>
>I built a cube on a Hive fact table of about 300 millions rows, all the
>steps are FINISHED with success but it seems I cannot do any query.
>
>select * from myFactTable; (or any other query)
>
>returns From line 1, column 15 to line 1, column 43: Table 'MYFACTTABLE'
>not found while executing SQL: "select * from myFactTable LIMIT 50000"
>
>I tried rebuilding completely the cube after a purge, created a new
>project, etc. but it doesn't solve the issue.
>
>It seems like a new KYLIN_ table has been created and contains data. How
>can I know which HBase table corresponds to which cube?
>
>The "Convert Cuboid Data to HFile" step took a while, the MapReduce job
>stayed in Undefined status for a long time but finally Succeeded. When I
>click on the link from the Kylin UI it send me to 7070/kylin/N/A instead
>of
>the MapReduce job info, is that a bad sign?
>
>In the logs I don't see any error in the Build N-Dimension Cuboid Data,
>Calculate HTable Region Splits, Create HTable
>
>I am not sure if it's event related but in the logs can this:
>
>2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job: map 1%
>reduce 0%
>
>2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job: Task Id :
>attempt_1432057232815_49758_m_000003_2, Status : FAILED
>
>Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
>org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
>
> at
>org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
>
> at
>org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobCon
>textImpl.java:174)
>
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:415)
>
> at
>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.
>java:1594)
>
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
>
>Caused by: java.lang.ClassNotFoundException: Class
>org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
>
> at
>org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:186
>7)
>
> at
>org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
>
> ... 8 more
>
>
>
>but then it seems to go on anyway:
>
>2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job: map 2%
>reduce 0%
>
>2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job: map 3%
>reduce 0%
>
>2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job: map 4%
>reduce 0%
>
>2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job: map 5%
>reduce 0%
>
>2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job: map 6%
>reduce 0%
>
>...
>
>2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job: map 100%
>reduce 100%
>
>2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job: Job
>job_1432057232815_49758 completed successfully
>
>
>This comes after
>
>
>Hive Column Cardinality calculation for table
>
>and a step called :
>
>
>+-------------------------------------------------------------------------
>-----------------------------+
>
>| null
> |
>
>+-------------------------------------------------------------------------
>-----------------------------+
>
>
>Thanks for your help, I would really like to make this PoC work!
Re: Cube built but not possible to query
Posted by alex schufo <al...@gmail.com>.
I use kylin-0.7.1 and no there is no table / database available in the
query window.
On Fri, Jun 5, 2015 at 10:45 AM, divye sheth <di...@gmail.com> wrote:
> What is the version of kylin being used? Are you able to see the
> tables/database name in the left pane of the query window?
> On Jun 5, 2015 14:28, "alex schufo" <al...@gmail.com> wrote:
>
> > Hi all,
> >
> > I built a cube on a Hive fact table of about 300 millions rows, all the
> > steps are FINISHED with success but it seems I cannot do any query.
> >
> > select * from myFactTable; (or any other query)
> >
> > returns From line 1, column 15 to line 1, column 43: Table 'MYFACTTABLE'
> > not found while executing SQL: "select * from myFactTable LIMIT 50000"
> >
> > I tried rebuilding completely the cube after a purge, created a new
> > project, etc. but it doesn't solve the issue.
> >
> > It seems like a new KYLIN_ table has been created and contains data. How
> > can I know which HBase table corresponds to which cube?
> >
> > The "Convert Cuboid Data to HFile" step took a while, the MapReduce job
> > stayed in Undefined status for a long time but finally Succeeded. When I
> > click on the link from the Kylin UI it send me to 7070/kylin/N/A instead
> of
> > the MapReduce job info, is that a bad sign?
> >
> > In the logs I don't see any error in the Build N-Dimension Cuboid Data,
> > Calculate HTable Region Splits, Create HTable
> >
> > I am not sure if it's event related but in the logs can this:
> >
> > 2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job: map 1%
> > reduce 0%
> >
> > 2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job: Task Id :
> > attempt_1432057232815_49758_m_000003_2, Status : FAILED
> >
> > Error: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class
> > org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> >
> > at
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
> >
> > at
> >
> >
> org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobContextImpl.java:174)
> >
> > at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
> >
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> >
> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> >
> > at java.security.AccessController.doPrivileged(Native Method)
> >
> > at javax.security.auth.Subject.doAs(Subject.java:415)
> >
> > at
> >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
> >
> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> >
> > Caused by: java.lang.ClassNotFoundException: Class
> > org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> >
> > at
> >
> >
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1867)
> >
> > at
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
> >
> > ... 8 more
> >
> >
> >
> > but then it seems to go on anyway:
> >
> > 2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job: map 2%
> > reduce 0%
> >
> > 2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job: map 3%
> > reduce 0%
> >
> > 2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job: map 4%
> > reduce 0%
> >
> > 2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job: map 5%
> > reduce 0%
> >
> > 2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job: map 6%
> > reduce 0%
> >
> > ...
> >
> > 2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job: map 100%
> > reduce 100%
> >
> > 2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job: Job
> > job_1432057232815_49758 completed successfully
> >
> >
> > This comes after
> >
> >
> > Hive Column Cardinality calculation for table
> >
> > and a step called :
> >
> >
> >
> >
> +------------------------------------------------------------------------------------------------------+
> >
> > | null
> > |
> >
> >
> >
> +------------------------------------------------------------------------------------------------------+
> >
> >
> > Thanks for your help, I would really like to make this PoC work!
> >
>
Re: Cube built but not possible to query
Posted by divye sheth <di...@gmail.com>.
What is the version of kylin being used? Are you able to see the
tables/database name in the left pane of the query window?
On Jun 5, 2015 14:28, "alex schufo" <al...@gmail.com> wrote:
> Hi all,
>
> I built a cube on a Hive fact table of about 300 millions rows, all the
> steps are FINISHED with success but it seems I cannot do any query.
>
> select * from myFactTable; (or any other query)
>
> returns From line 1, column 15 to line 1, column 43: Table 'MYFACTTABLE'
> not found while executing SQL: "select * from myFactTable LIMIT 50000"
>
> I tried rebuilding completely the cube after a purge, created a new
> project, etc. but it doesn't solve the issue.
>
> It seems like a new KYLIN_ table has been created and contains data. How
> can I know which HBase table corresponds to which cube?
>
> The "Convert Cuboid Data to HFile" step took a while, the MapReduce job
> stayed in Undefined status for a long time but finally Succeeded. When I
> click on the link from the Kylin UI it send me to 7070/kylin/N/A instead of
> the MapReduce job info, is that a bad sign?
>
> In the logs I don't see any error in the Build N-Dimension Cuboid Data,
> Calculate HTable Region Splits, Create HTable
>
> I am not sure if it's event related but in the logs can this:
>
> 2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job: map 1%
> reduce 0%
>
> 2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job: Task Id :
> attempt_1432057232815_49758_m_000003_2, Status : FAILED
>
> Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
> org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
>
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
>
> at
>
> org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobContextImpl.java:174)
>
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:415)
>
> at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
>
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
>
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
>
> at
>
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1867)
>
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
>
> ... 8 more
>
>
>
> but then it seems to go on anyway:
>
> 2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job: map 2%
> reduce 0%
>
> 2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job: map 3%
> reduce 0%
>
> 2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job: map 4%
> reduce 0%
>
> 2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job: map 5%
> reduce 0%
>
> 2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job: map 6%
> reduce 0%
>
> ...
>
> 2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job: map 100%
> reduce 100%
>
> 2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job: Job
> job_1432057232815_49758 completed successfully
>
>
> This comes after
>
>
> Hive Column Cardinality calculation for table
>
> and a step called :
>
>
>
> +------------------------------------------------------------------------------------------------------+
>
> | null
> |
>
>
> +------------------------------------------------------------------------------------------------------+
>
>
> Thanks for your help, I would really like to make this PoC work!
>