[jira] [Commented] (HBASE-14833) HFileOutputFormat2 should allow for custom reducer logic

["Nick Dimiduk (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-14833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15012314#comment-15012314 ] 

Nick Dimiduk commented on HBASE-14833:
--------------------------------------

The better solution is to remove the use of custom reducers entirely. The {{SortReducer}} implementations are a code smell and scalability bottleneck. See also: HBASE-7743

> HFileOutputFormat2 should allow for custom reducer logic
> --------------------------------------------------------
>
>                 Key: HBASE-14833
>                 URL: https://issues.apache.org/jira/browse/HBASE-14833
>             Project: HBase
>          Issue Type: Improvement
>          Components: API, mapreduce
>            Reporter: Gowtam Lal
>            Priority: Minor
>
> Right now, HFileOutputFormat2.configureIncrementalLoad() will configure a ReducerClass which takes all input and passes it through to be written to an HFile. Unfortunately, this prevents a user from plugging in his or her own logic for the Reduce phase. 
> TableMapReduceUtil.initTableReducerJob() accepts a custom ReducerClass, allowing users to operate on the tuples coming out of the MapperClass before they're written to output. Could HFileOutputFormat2.configureIncrementalLoad() possibly do something similar?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)