You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flume.apache.org by "Hari (JIRA)" <ji...@apache.org> on 2015/06/16 15:41:00 UTC

[jira] [Commented] (FLUME-2720) HDFS Sink: Autogenerate LZO indexes while creating LZO files

    [ https://issues.apache.org/jira/browse/FLUME-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588043#comment-14588043 ] 

Hari commented on FLUME-2720:
-----------------------------

The patch attached does not include any unit test. Because it requires significant refactoring of current Unit tests to use MiniDFSCluster instead of MockHDFSWriter/MockFileSystem to really assert the index file generation. 

Any thoughts if the tests are absolutely mandatory ?

> HDFS Sink: Autogenerate LZO indexes while creating LZO files
> ------------------------------------------------------------
>
>                 Key: FLUME-2720
>                 URL: https://issues.apache.org/jira/browse/FLUME-2720
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.7.0
>            Reporter: Hari
>            Priority: Minor
>         Attachments: 0001-FLUME-2720-Autogenerate-LZO-indexes-while-creating-L.patch
>
>
> The LZO indexes are now generated offline using DistributedLZOIndexer. 
> It will be nice to auto generate these index files during the ingestion itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)