You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by david <da...@qunb.com> on 2015/07/02 18:02:20 UTC
Error at Step 8: Convert Cuboid Data to HFile
Hi,
I installed Kylin 0.7.1 on top on the above cluster configuration:
* hadoop version 2.6.0
* hive version 1.0.0
* hbase version 0.98.13
I try to build a cube using webserver logs (client ip, timestamp, url
basically) without metadata. The 7 first steps (from 'Create Intermediate
Flat Hive Table' to 'Create HTable') work well however it stops on the step
'Convert Cuboid Data to HFile' with the following error :
java.lang.IllegalArgumentException: Can not create a Path from a null string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)
at org.apache.hadoop.fs.Path.<init>(Path.java:135)
at org.apache.hadoop.fs.Path.<init>(Path.java:89)
at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:545)
at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:394)
at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:88)
at org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:89)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:112)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
After digging into tomcat logs, I saw another error:
java.lang.IllegalArgumentException: No enum constant
org.apache.hadoop.mapreduce.JobCounter.VCORES_MILLIS_REDUCES
at java.lang.Enum.valueOf(Enum.java:238)
at
org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.valueOf(FrameworkCounterGroup.java:148)
at
org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.findCounter(FrameworkCounterGroup.java:182)
at
org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
at
org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:240)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:370)
at org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:511)
at org.apache.hadoop.mapreduce.Job$7.run(Job.java:756)
at org.apache.hadoop.mapreduce.Job$7.run(Job.java:753)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:753)
at
org.apache.kylin.job.common.HadoopCmdOutput.updateJobCounter(HadoopCmdOutput.java:86)
at
org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:144)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
However, I'm pretty sure that the jar contaning this class is loaded into
the classpath.
I would appreciate if someone already raised those issues, any hint would be
helpful.
Thanks,
David
--
View this message in context: http://apache-kylin-incubating.74782.x6.nabble.com/Error-at-Step-8-Convert-Cuboid-Data-to-HFile-tp233.html
Sent from the Apache Kylin (Incubating) mailing list archive at Nabble.com.
Re: Error at Step 8: Convert Cuboid Data to HFile
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
Seems some old hadoop jar was appeared in the class path; Try to search
³Hadoop job classpath is² in $KYLIN_HOME/tomcat/logs/kylin.log, which are
the class paths that Kylin runs hadoop job; and then check if multiple
hadoop-mapreduce-client*.jar there; Just have a try;
On 7/3/15, 12:02 AM, "david" <da...@qunb.com> wrote:
>Hi,
>
>I installed Kylin 0.7.1 on top on the above cluster configuration:
>* hadoop version 2.6.0
>* hive version 1.0.0
>* hbase version 0.98.13
>
>I try to build a cube using webserver logs (client ip, timestamp, url
>basically) without metadata. The 7 first steps (from 'Create Intermediate
>Flat Hive Table' to 'Create HTable') work well however it stops on the
>step
>'Convert Cuboid Data to HFile' with the following error :
>
>java.lang.IllegalArgumentException: Can not create a Path from a null
>string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)
> at org.apache.hadoop.fs.Path.<init>(Path.java:135)
> at org.apache.hadoop.fs.Path.<init>(Path.java:89)
> at
>org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(
>HFileOutputFormat2.java:545)
> at
>org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalL
>oad(HFileOutputFormat2.java:394)
> at
>org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLo
>ad(HFileOutputFormat.java:88)
> at
>org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:89)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at
>org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable
>.java:112)
> at
>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutab
>le.java:107)
> at
>org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChai
>nedExecutable.java:50)
> at
>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutab
>le.java:107)
> at
>org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(Defaul
>tScheduler.java:132)
> at
>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
>1142)
> at
>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
>After digging into tomcat logs, I saw another error:
>
>java.lang.IllegalArgumentException: No enum constant
>org.apache.hadoop.mapreduce.JobCounter.VCORES_MILLIS_REDUCES
> at java.lang.Enum.valueOf(Enum.java:238)
> at
>org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.valueOf(Framewo
>rkCounterGroup.java:148)
> at
>org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.findCounter(Fra
>meworkCounterGroup.java:182)
> at
>org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(Abstract
>Counters.java:154)
> at
>org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:240)
> at
>org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServic
>eDelegate.java:370)
> at
>org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:511)
> at org.apache.hadoop.mapreduce.Job$7.run(Job.java:756)
> at org.apache.hadoop.mapreduce.Job$7.run(Job.java:753)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.
>java:1491)
> at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:753)
> at
>org.apache.kylin.job.common.HadoopCmdOutput.updateJobCounter(HadoopCmdOutp
>ut.java:86)
> at
>org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable
>.java:144)
> at
>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutab
>le.java:107)
> at
>org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChai
>nedExecutable.java:50)
> at
>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutab
>le.java:107)
> at
>org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(Defaul
>tScheduler.java:132)
> at
>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
>1142)
> at
>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>:617)
> at java.lang.Thread.run(Thread.java:745)
>
>However, I'm pretty sure that the jar contaning this class is loaded into
>the classpath.
>I would appreciate if someone already raised those issues, any hint would
>be
>helpful.
>Thanks,
>
>David
>
>
>
>
>
>--
>View this message in context:
>http://apache-kylin-incubating.74782.x6.nabble.com/Error-at-Step-8-Convert
>-Cuboid-Data-to-HFile-tp233.html
>Sent from the Apache Kylin (Incubating) mailing list archive at
>Nabble.com.