[jira] [Created] (TEZ-1260) Allow KeyValueWriter to support writing list of values also

["Rohini Palaniswamy (JIRA)" <ji...@apache.org>]

Rohini Palaniswamy created TEZ-1260:
---------------------------------------

             Summary: Allow KeyValueWriter to support writing list of values also
                 Key: TEZ-1260
                 URL: https://issues.apache.org/jira/browse/TEZ-1260
             Project: Apache Tez
          Issue Type: Improvement
            Reporter: Rohini Palaniswamy


TEZ-1228 adds support to IFile for storing K,L<V>.  Currently KeyValueWriter allows write of K,V

public void write(Object key, Object value) throws IOException;

We should add support for 

public void write(Object key, Iterable<Object> values) throws IOException;

taking advantage of TEZ-1228. In few cases, pig unwraps key, list<values> and writes them as separate K,V pairs. This can avoid that overhead. That may enable us to even add something similar to hash based partial aggregation for join like what we do for groupby.



--
This message was sent by Atlassian JIRA
(v6.2#6252)