You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2014/11/20 09:22:33 UTC
[jira] [Commented] (TAJO-1053) ADD PARTITIONS for HCatalogStore
[ https://issues.apache.org/jira/browse/TAJO-1053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14219131#comment-14219131 ]
ASF GitHub Bot commented on TAJO-1053:
--------------------------------------
GitHub user blrunner opened a pull request:
https://github.com/apache/tajo/pull/263
TAJO-1053: ADD PARTITIONS for HCatalogStore.
I tried to resolve this issue. But unfortunately, current tajo doesn't provide ALTER PARTITION command. In addition, hive doesn't support a api for repair all partitions at a time. So, there is a one way which scan all directories of a tajo and run ALTER PARITION api for each all partitions. But it is very inefficient way and it will be a cause of HiveMetaStore low performance. Thus, we need to guide our users to run msck command on hive.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/blrunner/tajo TAJO-1053
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/263.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #263
----
commit e90a003c1d86ef4bbe484906cd4f329c84d6bdb5
Author: JaeHwa Jung <bl...@apache.org>
Date: 2014-11-20T08:15:17Z
TAJO-1053: ADD PARTITIONS for HCatalogStore.
----
> ADD PARTITIONS for HCatalogStore
> --------------------------------
>
> Key: TAJO-1053
> URL: https://issues.apache.org/jira/browse/TAJO-1053
> Project: Tajo
> Issue Type: Improvement
> Components: catalog
> Reporter: Jaehwa Jung
> Assignee: Jaehwa Jung
> Fix For: 0.9.1
>
>
> Currently, tajo provides column partition for partitioned table. But there is a lack in HCatalogStore. Tajo doesn't store a list of partitions for partitioned table in CatalogStore. But Hive store a list of partitions in HiveMetaStore. So, if you wrote partitioned table on Tajo, you must run msck repair function or add partition on Hive. I think that it is a very inefficient work. Thus, tajo need to repair partitioned directory list after writing partitioned table in HCatalgStore as follows:
> {code:xml}
> INSERT OVERWRITE INTO t1 SELECT l_orderkey FROM lineitem;
> ALTER TABLE lineitem ADD PARTITIONS;
> {code}
> For reference, "ADD PARTITIONS" just run on HCatalgStore.It doesn't run on another CatalogStore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)