You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Olga Natkovich (JIRA)" <ji...@apache.org> on 2011/03/03 02:27:37 UTC
[jira] Resolved: (PIG-1411) [Zebra] Can Zebra use HAR to reduce
file/block count for namenode
[ https://issues.apache.org/jira/browse/PIG-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich resolved PIG-1411.
---------------------------------
Resolution: Won't Fix
There is no active work on zebra
> [Zebra] Can Zebra use HAR to reduce file/block count for namenode
> -----------------------------------------------------------------
>
> Key: PIG-1411
> URL: https://issues.apache.org/jira/browse/PIG-1411
> Project: Pig
> Issue Type: New Feature
> Components: impl
> Affects Versions: 0.8.0
> Reporter: Gaurav Jain
> Assignee: Gaurav Jain
> Priority: Minor
>
> Due to column group structure, Zebra can create extra files for namenode to remember. That means namenode taking more memory for Zebra related files.
> The goal is to reduce the no of files/blocks
> The idea among various options is to use HAR ( Hadoop Archive ). Hadoop Archive reduces the block and file count by copying data from small files ( 1M, 2M ...) into a hdfs-block of larger size. Thus, reducing the total no. of blocks and files.
>
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira