You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by ShaoFeng Shi <sh...@apache.org> on 2017/04/02 14:39:48 UTC
Re: NoClassDefFoundError:org/apache/hive/hcatalog/mapreduce/HCatInputFormat
See the first item in Kylin FAQ:
https://kylin.apache.org/docs16/gettingstarted/faq.html
2017-03-25 6:58 GMT+08:00 Li Yang <li...@apache.org>:
> Check bin/find-hive-dependency.sh and see if find the correct hive jars in
> your env.
>
> On Wed, Mar 22, 2017 at 1:36 PM, LIGHRE <li...@foxmail.com> wrote:
>
> > Hello team,
> >
> >
> >
> >
> > Recently I was trying to install kylin 1.6.0-hbase1.x in my mac ox system
> > . But I encounted this error below when I try to build the sample cube. I
> > checked solutions online , the most common one is to upload the dependent
> > jar into a hdfs folder and set the kylin.job.mr.lib.dir in
> kylin.properties
> > . But it’s not working . Can you help to look into it ?
> >
> >
> >
> >
> > Env info is :
> >
> > Hadoop 2.5.2 (standalone )
> >
> > Hive 1.0.1
> >
> > Hbase 1.2.3
> >
> >
> >
> >
> > Detail log is attached below .
> >
> >
> >
> >
> > Thans a lot !
> >
> >
> >
> >
> > 2017-03-22 10:32:25,969 INFO [pool-9-thread-1]
> > steps.FactDistinctColumnsJob:86 : Starting: Kylin_Fact_Distinct_Columns_
> > kylin_sales_cube_Step
> >
> > 2017-03-22 10:32:25,970 INFO [pool-9-thread-1]
> > common.AbstractHadoopJob:163 : append job jar: /Users/lighre/Downloads/
> > apache-kylin-1.6.0-hbase1.x-bin/lib/kylin-job-1.6.0.jar
> >
> > 2017-03-22 10:32:25,971 INFO [pool-9-thread-1]
> > common.AbstractHadoopJob:171 : append kylin.hbase.dependency:
> > /Users/lighre/Downloads/hbase-1.2.3/bin/../lib/hbase-common-1.2.3.jar to
> > mapreduce.application.classpath
> >
> > 2017-03-22 10:32:25,971 INFO [pool-9-thread-1]
> > common.AbstractHadoopJob:176 : Didn't find mapreduce.application.
> classpath
> > in job configuration, will run 'mapred classpath' to get the default
> value.
> >
> > 2017-03-22 10:32:26,069 INFO [pool-9-thread-1]
> > common.AbstractHadoopJob:178 : The default mapred classpath is:
> > /Users/lighre/Downloads/hadoop-2.5.2/etc/hadoop,/
> > Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/common/lib/
> > *,/Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/
> > common/*,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/hdfs,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/hdfs/lib/*,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/hdfs/*,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/yarn/lib/*,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/yarn/*,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/mapreduce/lib/*,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/mapreduce/*,/contrib/capacity-
> > scheduler/*.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/mapreduce/hadoop-mapreduce-client-app-2.5.2.
> > jar,/Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/
> > mapreduce/hadoop-mapreduce-client-common-2.5.2.jar,/
> > Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/
> > hadoop-mapreduce-client-core-2.5.2.jar,/Users/lighre/
> > Downloads/hadoop-2.5.2/share/hadoop/mapreduce/hadoop-
> > mapreduce-client-hs-2.5.2.jar,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/mapreduce/hadoop-mapreduce-
> > client-hs-plugins-2.5.2.jar,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/mapreduce/hadoop-mapreduce-client-
> > jobclient-2.5.2-tests.jar,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/mapreduce/hadoop-mapreduce-client-
> > jobclient-2.5.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.5.
> > 2.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/
> > mapreduce/hadoop-mapreduce-examples-2.5.2.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/yarn/hadoop-yarn-
> > api-2.5.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/yarn/hadoop-yarn-applications-distributedshell-
> > 2.5.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/yarn/hadoop-yarn-applications-unmanaged-am-
> > launcher-2.5.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/yarn/hadoop-yarn-client-2.5.2.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/yarn/hadoop-yarn-
> > common-2.5.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.5.
> > 2.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/
> > yarn/hadoop-yarn-server-common-2.5.2.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/yarn/hadoop-yarn-
> > server-nodemanager-2.5.2.jar,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.5.2.jar,/
> > Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/yarn/
> > hadoop-yarn-server-tests-2.5.2.jar,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/yarn/hadoop-yarn-server-web-
> > proxy-2.5.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/mapreduce/lib/aopalliance-1.0.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/lib/
> > asm-3.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/mapreduce/lib/avro-1.7.4.jar,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/mapreduce/lib/commons-
> compress-1.4.1.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/lib/
> > commons-io-2.4.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/mapreduce/lib/guice-3.0.jar,/Users/lighre/
> > Downloads/hadoop-2.5.2/share/hadoop/mapreduce/lib/guice-
> > servlet-3.0.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/mapreduce/lib/hadoop-annotations-2.5.2.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/lib/
> > hamcrest-core-1.3.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar,/
> > Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/
> > lib/jackson-mapper-asl-1.9.13.jar,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/mapreduce/lib/javax.inject-1.
> > jar,/Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/
> > mapreduce/lib/jersey-core-1.9.jar,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/mapreduce/lib/jersey-guice-1.
> > 9.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/
> > mapreduce/lib/jersey-server-1.9.jar,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/mapreduce/lib/junit-4.11.jar,/
> > Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/
> > lib/leveldbjni-all-1.8.jar,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/mapreduce/lib/log4j-1.2.17.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/lib/
> > netty-3.6.2.Final.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/mapreduce/lib/paranamer-2.3.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/lib/
> > protobuf-java-2.5.0.jar,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar,/
> > Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/
> > lib/xz-1.0.jar,/Users/lighre/Downloads/hadoop-2.5.2/modules/*.jar
> >
> > 2017-03-22 10:32:26,070 INFO [pool-9-thread-1]
> > common.AbstractHadoopJob:188 : Hadoop job classpath is:
> > /Users/lighre/Downloads/hadoop-2.5.2/etc/hadoop,/
> > Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/common/lib/
> > *,/Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/
> > common/*,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/hdfs,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/hdfs/lib/*,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/hdfs/*,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/yarn/lib/*,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/yarn/*,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/mapreduce/lib/*,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/mapreduce/*,/contrib/capacity-
> > scheduler/*.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/mapreduce/hadoop-mapreduce-client-app-2.5.2.
> > jar,/Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/
> > mapreduce/hadoop-mapreduce-client-common-2.5.2.jar,/
> > Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/
> > hadoop-mapreduce-client-core-2.5.2.jar,/Users/lighre/
> > Downloads/hadoop-2.5.2/share/hadoop/mapreduce/hadoop-
> > mapreduce-client-hs-2.5.2.jar,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/mapreduce/hadoop-mapreduce-
> > client-hs-plugins-2.5.2.jar,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/mapreduce/hadoop-mapreduce-client-
> > jobclient-2.5.2-tests.jar,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/mapreduce/hadoop-mapreduce-client-
> > jobclient-2.5.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.5.
> > 2.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/
> > mapreduce/hadoop-mapreduce-examples-2.5.2.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/yarn/hadoop-yarn-
> > api-2.5.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/yarn/hadoop-yarn-applications-distributedshell-
> > 2.5.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/yarn/hadoop-yarn-applications-unmanaged-am-
> > launcher-2.5.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/yarn/hadoop-yarn-client-2.5.2.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/yarn/hadoop-yarn-
> > common-2.5.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.5.
> > 2.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/
> > yarn/hadoop-yarn-server-common-2.5.2.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/yarn/hadoop-yarn-
> > server-nodemanager-2.5.2.jar,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.5.2.jar,/
> > Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/yarn/
> > hadoop-yarn-server-tests-2.5.2.jar,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/yarn/hadoop-yarn-server-web-
> > proxy-2.5.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/mapreduce/lib/aopalliance-1.0.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/lib/
> > asm-3.2.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/mapreduce/lib/avro-1.7.4.jar,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/mapreduce/lib/commons-
> compress-1.4.1.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/lib/
> > commons-io-2.4.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/mapreduce/lib/guice-3.0.jar,/Users/lighre/
> > Downloads/hadoop-2.5.2/share/hadoop/mapreduce/lib/guice-
> > servlet-3.0.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/
> > hadoop/mapreduce/lib/hadoop-annotations-2.5.2.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/lib/
> > hamcrest-core-1.3.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar,/
> > Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/
> > lib/jackson-mapper-asl-1.9.13.jar,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/mapreduce/lib/javax.inject-1.
> > jar,/Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/
> > mapreduce/lib/jersey-core-1.9.jar,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/mapreduce/lib/jersey-guice-1.
> > 9.jar,/Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/
> > mapreduce/lib/jersey-server-1.9.jar,/Users/lighre/Downloads/
> > hadoop-2.5.2/share/hadoop/mapreduce/lib/junit-4.11.jar,/
> > Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/
> > lib/leveldbjni-all-1.8.jar,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/mapreduce/lib/log4j-1.2.17.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/lib/
> > netty-3.6.2.Final.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > share/hadoop/mapreduce/lib/paranamer-2.3.jar,/Users/
> > lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/lib/
> > protobuf-java-2.5.0.jar,/Users/lighre/Downloads/hadoop-
> > 2.5.2/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar,/
> > Users/lighre/Downloads/hadoop-2.5.2/share/hadoop/mapreduce/
> > lib/xz-1.0.jar,/Users/lighre/Downloads/hadoop-2.5.2/
> > modules/*.jar,/Users/lighre/Downloads/hbase-1.2.3/bin/../
> > lib/hbase-common-1.2.3.jar
> >
> > 2017-03-22 10:32:26,072 INFO [pool-9-thread-1]
> > common.AbstractHadoopJob:200 : Hive Dependencies Before Filtered:
> > /Users/lighre/Downloads/apache-hive-1.0.1-bin/conf,,/
> > Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/accumulo-
> > core-1.6.0.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/accumulo-fate-1.6.0.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/accumulo-start-1.6.0.jar,/
> > Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/accumulo-
> > trace-1.6.0.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/activation-1.1.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/ant-1.9.1.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/ant-launcher-1.9.1.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/antlr-2.7.7.jar,
> > /Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > antlr-runtime-3.4.jar,/Users/lighre/Downloads/apache-hive-
> > 1.0.1-bin/lib/asm-commons-3.1.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/asm-tree-3.1.jar,/Users/lighre/
> > Downloads/apache-hive-1.0.1-bin/lib/avro-1.7.5.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/bonecp-0.8.0.
> > RELEASE.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/commons-beanutils-1.7.0.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/commons-beanutils-core-1.8.0.
> > jar,/Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > commons-cli-1.2.jar,/Users/lighre/Downloads/apache-hive-
> > 1.0.1-bin/lib/commons-codec-1.4.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/commons-collections-3.2.1.jar,
> > /Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > commons-compiler-2.7.6.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/commons-compress-1.4.1.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/commons-
> > configuration-1.6.jar,/Users/lighre/Downloads/apache-hive-
> > 1.0.1-bin/lib/commons-dbcp-1.4.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/commons-digester-1.8.jar,/
> > Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/commons-
> > httpclient-3.0.1.jar,/Users/lighre/Downloads/apache-hive-
> > 1.0.1-bin/lib/commons-io-2.4.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/commons-lang-2.6.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/commons-logging-
> > 1.1.3.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/commons-math-2.1.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/commons-pool-1.5.4.jar,/Users/lighre/
> > Downloads/apache-hive-1.0.1-bin/lib/commons-vfs2-2.0.jar,/
> > Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/curator-
> > client-2.6.0.jar,/Users/lighre/Downloads/apache-hive-
> > 1.0.1-bin/lib/curator-framework-2.6.0.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/datanucleus-api-
> > jdo-3.2.6.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/datanucleus-core-3.2.10.jar,/Users/lighre/
> > Downloads/apache-hive-1.0.1-bin/lib/datanucleus-rdbms-3.2.
> > 9.jar,/Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > derby-10.10.1.1.jar,/Users/lighre/Downloads/apache-hive-
> > 1.0.1-bin/lib/eigenbase-properties-1.1.4.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/geronimo-
> > annotation_1.0_spec-1.1.1.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/geronimo-jaspic_1.0_spec-1.0.
> > jar,/Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > geronimo-jta_1.1_spec-1.1.1.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/groovy-all-2.1.6.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/guava-11.0.2.
> > jar,/Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > hamcrest-core-1.1.jar,/Users/lighre/Downloads/apache-hive-
> > 1.0.1-bin/lib/hive-accumulo-handler-1.0.1.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/hive-ant-1.0.1.
> > jar,/Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > hive-beeline-1.0.1.jar,/Users/lighre/Downloads/apache-hive-
> > 1.0.1-bin/lib/hive-cli-1.0.1.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/hive-common-1.0.1.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/hive-contrib-1.
> > 0.1.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/hive-exec-1.0.1.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/hive-hbase-handler-1.0.1.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/hive-hwi-1.0.1.
> > jar,/Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > hive-jdbc-1.0.1-standalone.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/hive-jdbc-1.0.1.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/hive-metastore-
> > 1.0.1.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/hive-serde-1.0.1.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/hive-service-1.0.1.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/hive-shims-0.20-
> > 1.0.1.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/hive-shims-0.20S-1.0.1.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/hive-shims-0.23-1.0.1.jar,/
> > Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/hive-shims-
> > 1.0.1.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/hive-shims-common-1.0.1.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/hive-shims-common-secure-1.0.
> > 1.jar,/Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > hive-testutils-1.0.1.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/httpclient-4.2.5.jar,/Users/lighre/
> > Downloads/apache-hive-1.0.1-bin/lib/httpcore-4.2.5.jar,/
> > Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/janino-2.7.
> > 6.jar,/Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > jansi-1.11.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/jcommander-1.32.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/jdo-api-3.0.1.jar,/Users/lighre/
> > Downloads/apache-hive-1.0.1-bin/lib/jetty-all-7.6.0.
> > v20120127.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/jetty-all-server-7.6.0.v20120127.jar,/Users/lighre/
> > Downloads/apache-hive-1.0.1-bin/lib/jline-0.9.94.jar,/
> > Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/jpam-1.1.
> > jar,/Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > jsr305-1.3.9.jar,/Users/lighre/Downloads/apache-hive-
> > 1.0.1-bin/lib/jta-1.1.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/junit-4.11.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/libfb303-0.9.0.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/libthrift-0.9.0.
> > jar,/Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > linq4j-0.4.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/log4j-1.2.16.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/mail-1.4.1.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/maven-scm-api-1.4.jar,/Users/
> > lighre/Downloads/apache-hive-1.0.1-bin/lib/maven-scm-
> > provider-svn-commons-1.4.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/maven-scm-provider-svnexe-1.4.jar,/
> > Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/mysql-
> > connector-java-5.1.31.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/opencsv-2.3.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/oro-2.0.8.jar,/Users/lighre/
> > Downloads/apache-hive-1.0.1-bin/lib/paranamer-2.3.jar,/
> > Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/pentaho-
> > aggdesigner-algorithm-5.1.3-jhyde.jar,/Users/lighre/
> > Downloads/apache-hive-1.0.1-bin/lib/plexus-utils-1.5.6.
> > jar,/Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > quidem-0.1.1.jar,/Users/lighre/Downloads/apache-hive-
> > 1.0.1-bin/lib/regexp-1.3.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/servlet-api-2.5.jar,/Users/lighre/
> > Downloads/apache-hive-1.0.1-bin/lib/snappy-java-1.0.5.jar,
> > /Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/ST4-
> > 4.0.4.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/stax-api-1.0.1.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/stringtemplate-3.2.1.jar,/
> > Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/super-csv-
> > 2.2.0.jar,/Users/lighre/Downloads/apache-hive-1.0.1-
> > bin/lib/tempus-fugit-1.1.jar,/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/velocity-1.5.jar,/Users/lighre/Downloads/
> > apache-hive-1.0.1-bin/lib/xz-1.0.jar,/Users/lighre/
> > Downloads/apache-hive-1.0.1-bin/lib/zookeeper-3.4.5.jar,/
> > Users/lighre/Downloads/apache-hive-1.0.1-bin/hcatalog/share/
> > hcatalog/hive-hcatalog-core-1.0.1.jar
> >
> > 2017-03-22 10:32:26,114 INFO [pool-9-thread-1]
> > common.AbstractHadoopJob:202 : Hive Dependencies After Filtered:
> > /Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > hive-exec-1.0.1.jar,/Users/lighre/Downloads/apache-hive-
> > 1.0.1-bin/lib/hive-metastore-1.0.1.jar,/Users/lighre/
> > Downloads/apache-hive-1.0.1-bin/hcatalog/share/hcatalog/
> > hive-hcatalog-core-1.0.1.jar
> >
> > 2017-03-22 10:32:26,114 INFO [pool-9-thread-1]
> > common.AbstractHadoopJob:230 : Kafka Dependencies:
> >
> > 2017-03-22 10:32:26,281 INFO [pool-9-thread-1]
> > common.AbstractHadoopJob:358 : Job 'tmpjars' updated --
> > file:/Users/lighre/Downloads/apache-hive-1.0.1-bin/lib/
> > hive-exec-1.0.1.jar,file:/Users/lighre/Downloads/apache-
> > hive-1.0.1-bin/lib/hive-metastore-1.0.1.jar,file:/
> > Users/lighre/Downloads/apache-hive-1.0.1-bin/hcatalog/share/
> > hcatalog/hive-hcatalog-core-1.0.1.jar,hdfs://localhost:8020/
> > kylin_dep/hive-hcatalog-core-1.0.1.jar
> >
> > 2017-03-22 10:32:26,282 INFO [pool-9-thread-1]
> > steps.FactDistinctColumnsJob:100 : Found segment: kylin_sales_cube[
> > 20170331093000_20170331235500]
> >
> > 2017-03-22 10:32:26,294 INFO [pool-9-thread-1]
> > common.AbstractHadoopJob:506 : tempMetaFileString is : null
> >
> > 2017-03-22 10:32:26,297 ERROR [pool-9-thread-1]
> > execution.AbstractExecutable:115 : error running Executable:
> > MapReduceExecutable{id=abef03dd-e459-4745-bd08-cac4d1b05ef7-02,
> > name=Extract Fact Table Distinct Columns, state=RUNNING}
> >
> > 2017-03-22 10:32:26,301 DEBUG [pool-9-thread-1] dao.ExecutableDao:210 :
> > updating job output, id: abef03dd-e459-4745-bd08-cac4d1b05ef7-02
> >
> > 2017-03-22 10:32:26,305 DEBUG [pool-9-thread-1]
> > hbase.HBaseResourceStore:262 : Update row /execute_output/abef03dd-e459-
> 4745-bd08-cac4d1b05ef7-02
> > from oldTs: 1490149945566, to newTs: 1490149946301, operation result:
> true
> >
> > 2017-03-22 10:32:26,309 DEBUG [pool-9-thread-1] dao.ExecutableDao:210 :
> > updating job output, id: abef03dd-e459-4745-bd08-cac4d1b05ef7-02
> >
> > 2017-03-22 10:32:26,319 DEBUG [pool-9-thread-1]
> > hbase.HBaseResourceStore:262 : Update row /execute_output/abef03dd-e459-
> 4745-bd08-cac4d1b05ef7-02
> > from oldTs: 1490149946301, to newTs: 1490149946309, operation result:
> true
> >
> > 2017-03-22 10:32:26,320 INFO [pool-9-thread-1]
> > manager.ExecutableManager:292 : job id:abef03dd-e459-4745-bd08-
> cac4d1b05ef7-02
> > from RUNNING to ERROR
> >
> > 2017-03-22 10:32:26,340 ERROR [pool-9-thread-1]
> > execution.AbstractExecutable:115 : error running Executable:
> > CubingJob{id=abef03dd-e459-4745-bd08-cac4d1b05ef7, name=kylin_sales_cube
> > - 20170331093000_20170331235500 - BUILD - GMT+08:00 2017-03-22 09:24:16,
> > state=RUNNING}
> >
> > 2017-03-22 10:32:26,344 DEBUG [pool-9-thread-1] dao.ExecutableDao:210 :
> > updating job output, id: abef03dd-e459-4745-bd08-cac4d1b05ef7
> >
> > 2017-03-22 10:32:26,350 DEBUG [pool-9-thread-1]
> > hbase.HBaseResourceStore:262 : Update row /execute_output/abef03dd-e459-
> 4745-bd08-cac4d1b05ef7
> > from oldTs: 1490149945548, to newTs: 1490149946344, operation result:
> true
> >
> > 2017-03-22 10:32:26,353 DEBUG [pool-9-thread-1] dao.ExecutableDao:210 :
> > updating job output, id: abef03dd-e459-4745-bd08-cac4d1b05ef7
> >
> > 2017-03-22 10:32:26,357 DEBUG [pool-9-thread-1]
> > hbase.HBaseResourceStore:262 : Update row /execute_output/abef03dd-e459-
> 4745-bd08-cac4d1b05ef7
> > from oldTs: 1490149946344, to newTs: 1490149946354, operation result:
> true
> >
> > 2017-03-22 10:32:26,357 INFO [pool-9-thread-1]
> > manager.ExecutableManager:292 : job id:abef03dd-e459-4745-bd08-
> cac4d1b05ef7
> > from RUNNING to ERROR
> >
> > 2017-03-22 10:32:26,358 WARN [pool-9-thread-1]
> > execution.AbstractExecutable:247 : no need to send email, user list is
> > empty
> >
> > 2017-03-22 10:32:26,358 ERROR [pool-9-thread-1]
> > threadpool.DefaultScheduler:140 : ExecuteException
> > job:abef03dd-e459-4745-bd08-cac4d1b05ef7
> >
> > org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.job.
> exception.ExecuteException:
> > java.lang.NoClassDefFoundError: org/apache/hive/hcatalog/
> > mapreduce/HCatInputFormat
> >
> > at org.apache.kylin.job.execution.AbstractExecutable.
> > execute(AbstractExecutable.java:123)
> >
> > at org.apache.kylin.job.impl.threadpool.DefaultScheduler$
> > JobRunner.run(DefaultScheduler.java:136)
> >
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > ThreadPoolExecutor.java:1142)
> >
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > ThreadPoolExecutor.java:617)
> >
> > at java.lang.Thread.run(Thread.java:745)
> >
> > Caused by: org.apache.kylin.job.exception.ExecuteException: java.lang.
> NoClassDefFoundError:
> > org/apache/hive/hcatalog/mapreduce/HCatInputFormat
> >
> > at org.apache.kylin.job.execution.AbstractExecutable.
> > execute(AbstractExecutable.java:123)
> >
> > at org.apache.kylin.job.execution.DefaultChainedExecutable.
> doWork(
> > DefaultChainedExecutable.java:57)
> >
> > at org.apache.kylin.job.execution.AbstractExecutable.
> > execute(AbstractExecutable.java:113)
> >
> > ... 4 more
> >
> > Caused by: java.lang.NoClassDefFoundError: org/apache/hive/hcatalog/
> > mapreduce/HCatInputFormat
> >
> > at org.apache.kylin.source.hive.HiveMRInput$
> HiveTableInputFormat.
> > configureJob(HiveMRInput.java:105)
> >
> > at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.
> > setupMapper(FactDistinctColumnsJob.java:119)
> >
> > at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.
> > run(FactDistinctColumnsJob.java:103)
> >
> > at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
> >
> > at org.apache.kylin.engine.mr.common.MapReduceExecutable.
> > doWork(MapReduceExecutable.java:120)
> >
> > at org.apache.kylin.job.execution.AbstractExecutable.
> > execute(AbstractExecutable.java:113)
> >
> > ... 6 more
> >
> > Caused by: java.lang.ClassNotFoundException: org.apache.hive.hcatalog.
> > mapreduce.HCatInputFormat
> >
> > at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(
> > WebappClassLoaderBase.java:1858)
> >
> > at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(
> > WebappClassLoaderBase.java:1701)
> >
> >
> >
> >
> > ... 12 more
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Best regards,
> > Larry
>
--
Best regards,
Shaofeng Shi 史少锋