You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@datafu.apache.org by "Matthew Hayes (JIRA)" <ji...@apache.org> on 2014/11/04 17:31:34 UTC
[jira] [Created] (DATAFU-71) Create IncrementalAvroStorage UDF for
incrementally processing date partitioned data
Matthew Hayes created DATAFU-71:
-----------------------------------
Summary: Create IncrementalAvroStorage UDF for incrementally processing date partitioned data
Key: DATAFU-71
URL: https://issues.apache.org/jira/browse/DATAFU-71
Project: DataFu
Issue Type: New Feature
Reporter: Matthew Hayes
Assignee: Matthew Hayes
Data can sometimes be stored in HDFS in a time-partitioned manner, e.g. /some/input/yyyy/mm/dd. You may want to process this data incrementally, where the output has a format like /some/output/yyyy/mm/dd. It be useful if there is a UDF that handles the incremental processing for you.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)