[jira] [Commented] (TEZ-1260) Allow KeyValueWriter to support writing list of values also

["Bikas Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054201#comment-14054201 ] 

Bikas Saha commented on TEZ-1260:
---------------------------------

bq. add something similar to hash based partial aggregation for join like what we do for groupby
Can you please elaborate on that?

> Allow KeyValueWriter to support writing list of values also
> -----------------------------------------------------------
>
>                 Key: TEZ-1260
>                 URL: https://issues.apache.org/jira/browse/TEZ-1260
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Rohini Palaniswamy
>
> TEZ-1228 adds support to IFile for storing K,L<V>.  Currently KeyValueWriter allows write of K,V
> public void write(Object key, Object value) throws IOException;
> We should add support for 
> public void write(Object key, Iterable<Object> values) throws IOException;
> taking advantage of TEZ-1228. In few cases, pig unwraps key, list<values> and writes them as separate K,V pairs. This can avoid that overhead. That may enable us to even add something similar to hash based partial aggregation for join like what we do for groupby.



--
This message was sent by Atlassian JIRA
(v6.2#6252)