You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by "wangxianbin1987@gmail.com" <wa...@gmail.com> on 2016/01/28 05:14:07 UTC
Re: [jira] [Commented] (KYLIN-1082) Hive dependencies should be add to tmpjars
hi yanghong!
do you mean hadoop-common jar?
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0</version>
haven't found these class in it.
wangxianbin1987@gmail.com
From: Zhong Yanghong (JIRA)
Date: 2016-01-27 22:15
To: issues
Subject: [jira] [Commented] (KYLIN-1082) Hive dependencies should be add to tmpjars
[ https://issues.apache.org/jira/browse/KYLIN-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15119280#comment-15119280 ]
Zhong Yanghong commented on KYLIN-1082:
---------------------------------------
Maybe the previous clause is not good. For example, in the Class "CubeHFileJob", we imported some classes related to hbase,
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.mapreduce.HFileOutputFormat;
import org.apache.hadoop.hbase.mapreduce.KeyValueSortReducer;
They are responsible for convert Cuboid to HBase. However, they have been included in the hadoop-common jars.
> Hive dependencies should be add to tmpjars
> ------------------------------------------
>
> Key: KYLIN-1082
> URL: https://issues.apache.org/jira/browse/KYLIN-1082
> Project: Kylin
> Issue Type: Bug
> Components: Environment , Job Engine
> Reporter: liyang
> Assignee: Zhong Yanghong
> Labels: newbie
> Fix For: v2.1, v1.3
>
> Attachments: auto_hive_tmpjars_1_x_staging.patch, auto_hive_tmpjars_2_x_staging.patch
>
>
> Currently kylin assume all data nodes have hive deployment at exact same FS location. However, a better position is to think hive as a client side app. Then we need to ship hive jar with MR job every time.
> This make deploy kylin a lot easier in cluster that does not have hive on all data nodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)