You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Jerome Boulon (JIRA)" <ji...@apache.org> on 2008/10/20 18:25:44 UTC
[jira] Created: (HADOOP-4462) Rolling mechanism for demux output
Rolling mechanism for demux output
----------------------------------
Key: HADOOP-4462
URL: https://issues.apache.org/jira/browse/HADOOP-4462
Project: Hadoop Core
Issue Type: Improvement
Components: contrib/chukwa
Reporter: Jerome Boulon
Assignee: Jerome Boulon
In order to reduce the number of file on HDFS we need to have a rolling mechanism for the demux output
- avoid immediate merging if there's already file for the same time range, create a spill file instead
- merge all raw files every hours
- merge all hourly files every days
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-4462) Rolling mechanism for demux output
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon resolved HADOOP-4462.
-----------------------------------
Resolution: Fixed
> Rolling mechanism for demux output
> ----------------------------------
>
> Key: HADOOP-4462
> URL: https://issues.apache.org/jira/browse/HADOOP-4462
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
>
> In order to reduce the number of file on HDFS we need to have a rolling mechanism for the demux output
> - avoid immediate merging if there's already file for the same time range, create a spill file instead
> - merge all raw files every hours
> - merge all hourly files every days
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.