You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@datafu.apache.org by "Matthew Hayes (JIRA)" <ji...@apache.org> on 2014/11/04 17:31:34 UTC

[jira] [Created] (DATAFU-71) Create IncrementalAvroStorage UDF for incrementally processing date partitioned data

Matthew Hayes created DATAFU-71:
-----------------------------------

             Summary: Create IncrementalAvroStorage UDF for incrementally processing date partitioned data
                 Key: DATAFU-71
                 URL: https://issues.apache.org/jira/browse/DATAFU-71
             Project: DataFu
          Issue Type: New Feature
            Reporter: Matthew Hayes
            Assignee: Matthew Hayes


Data can sometimes be stored in HDFS in a time-partitioned manner, e.g. /some/input/yyyy/mm/dd.  You may want to process this data incrementally, where the output has a format like /some/output/yyyy/mm/dd.  It be useful if there is a UDF that handles the incremental processing for you.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)