You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Bob Cummins (JIRA)" <ji...@apache.org> on 2011/03/14 13:37:30 UTC
[jira] Created: (MAPREDUCE-2382) Key/Value ordering within a single
key/value set when multiple values exist for a key
Key/Value ordering within a single key/value set when multiple values exist for a key
-------------------------------------------------------------------------------------
Key: MAPREDUCE-2382
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2382
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: client
Affects Versions: 0.20.2
Reporter: Bob Cummins
Priority: Minor
>>The context of this issue is entirely within one key/value(s) pair/set, NOT between key/value sets as they are funneled to a reducer by mappers.<<
When mapper writes multiple values for a key, the underlying collection class maps each of the values to the key, but not always in chronological order. If chronological order were guaranteed each of the values mapped to the key, each of the values could be understood as specific and different parameters between the mapper and the reducer.
I've done little tricks like having the mapper flag one a the values by making it a negative number, which the reducer recognizes and can write it to hbase as a unique column value.This is a kluge workaround which it would be nice to not have to do.
Used to formulate this suggestion:
TableMapper<ImmutableBytesWritable,IntWritable>
TableReducer<ImmutableBytesWritable,IntWritable, ImmutableBytesWritable>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2382) Key/Value ordering within a
single key/value set when multiple values exist for a key
Posted by "Bob Cummins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13006888#comment-13006888 ]
Bob Cummins commented on MAPREDUCE-2382:
----------------------------------------
You know better than I where it belongs--I will submit it as a feature request if that makes the most sense. Yes this is 2369 communicated more clearly. Thanks for your time.
> Key/Value ordering within a single key/value set when multiple values exist for a key
> -------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2382
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2382
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.20.2
> Reporter: Bob Cummins
> Priority: Minor
>
> >>The context of this issue is entirely within one key/value(s) pair/set, NOT between key/value sets as they are funneled to a reducer by mappers.<<
> When mapper writes multiple values for a key, the underlying collection class maps each of the values to the key, but not always in chronological order. If chronological order were guaranteed each of the values mapped to the key, each of the values could be understood as specific and different parameters between the mapper and the reducer.
> I've done little tricks like having the mapper flag one a the values by making it a negative number, which the reducer recognizes and can write it to hbase as a unique column value.This is a kluge workaround which it would be nice to not have to do.
> Used to formulate this suggestion:
> TableMapper<ImmutableBytesWritable,IntWritable>
> TableReducer<ImmutableBytesWritable,IntWritable, ImmutableBytesWritable>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (MAPREDUCE-2382) Key/Value ordering within a
single key/value set when multiple values exist for a key
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon resolved MAPREDUCE-2382.
------------------------------------
Resolution: Not A Problem
> Key/Value ordering within a single key/value set when multiple values exist for a key
> -------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2382
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2382
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.20.2
> Reporter: Bob Cummins
> Priority: Minor
>
> >>The context of this issue is entirely within one key/value(s) pair/set, NOT between key/value sets as they are funneled to a reducer by mappers.<<
> When mapper writes multiple values for a key, the underlying collection class maps each of the values to the key, but not always in chronological order. If chronological order were guaranteed each of the values mapped to the key, each of the values could be understood as specific and different parameters between the mapper and the reducer.
> I've done little tricks like having the mapper flag one a the values by making it a negative number, which the reducer recognizes and can write it to hbase as a unique column value.This is a kluge workaround which it would be nice to not have to do.
> Used to formulate this suggestion:
> TableMapper<ImmutableBytesWritable,IntWritable>
> TableReducer<ImmutableBytesWritable,IntWritable, ImmutableBytesWritable>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2382) Key/Value ordering within a
single key/value set when multiple values exist for a key
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13006516#comment-13006516 ]
Todd Lipcon commented on MAPREDUCE-2382:
----------------------------------------
Bob, it seems like this is a feature request for a new API for HBase, and probably doesn't belong on the MR JIRA. Am I misunderstanding?
> Key/Value ordering within a single key/value set when multiple values exist for a key
> -------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2382
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2382
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.20.2
> Reporter: Bob Cummins
> Priority: Minor
>
> >>The context of this issue is entirely within one key/value(s) pair/set, NOT between key/value sets as they are funneled to a reducer by mappers.<<
> When mapper writes multiple values for a key, the underlying collection class maps each of the values to the key, but not always in chronological order. If chronological order were guaranteed each of the values mapped to the key, each of the values could be understood as specific and different parameters between the mapper and the reducer.
> I've done little tricks like having the mapper flag one a the values by making it a negative number, which the reducer recognizes and can write it to hbase as a unique column value.This is a kluge workaround which it would be nice to not have to do.
> Used to formulate this suggestion:
> TableMapper<ImmutableBytesWritable,IntWritable>
> TableReducer<ImmutableBytesWritable,IntWritable, ImmutableBytesWritable>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2382) Key/Value ordering within a
single key/value set when multiple values exist for a key
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13006539#comment-13006539 ]
Allen Wittenauer commented on MAPREDUCE-2382:
---------------------------------------------
Is this the same thing being talked about in MAPREDUCE-2369?
> Key/Value ordering within a single key/value set when multiple values exist for a key
> -------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2382
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2382
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.20.2
> Reporter: Bob Cummins
> Priority: Minor
>
> >>The context of this issue is entirely within one key/value(s) pair/set, NOT between key/value sets as they are funneled to a reducer by mappers.<<
> When mapper writes multiple values for a key, the underlying collection class maps each of the values to the key, but not always in chronological order. If chronological order were guaranteed each of the values mapped to the key, each of the values could be understood as specific and different parameters between the mapper and the reducer.
> I've done little tricks like having the mapper flag one a the values by making it a negative number, which the reducer recognizes and can write it to hbase as a unique column value.This is a kluge workaround which it would be nice to not have to do.
> Used to formulate this suggestion:
> TableMapper<ImmutableBytesWritable,IntWritable>
> TableReducer<ImmutableBytesWritable,IntWritable, ImmutableBytesWritable>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2382) Key/Value ordering within a
single key/value set when multiple values exist for a key
Posted by "Bob Cummins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13006895#comment-13006895 ]
Bob Cummins commented on MAPREDUCE-2382:
----------------------------------------
This issue is now under https://issues.apache.org/jira/browse/HBASE-3646
> Key/Value ordering within a single key/value set when multiple values exist for a key
> -------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2382
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2382
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.20.2
> Reporter: Bob Cummins
> Priority: Minor
>
> >>The context of this issue is entirely within one key/value(s) pair/set, NOT between key/value sets as they are funneled to a reducer by mappers.<<
> When mapper writes multiple values for a key, the underlying collection class maps each of the values to the key, but not always in chronological order. If chronological order were guaranteed each of the values mapped to the key, each of the values could be understood as specific and different parameters between the mapper and the reducer.
> I've done little tricks like having the mapper flag one a the values by making it a negative number, which the reducer recognizes and can write it to hbase as a unique column value.This is a kluge workaround which it would be nice to not have to do.
> Used to formulate this suggestion:
> TableMapper<ImmutableBytesWritable,IntWritable>
> TableReducer<ImmutableBytesWritable,IntWritable, ImmutableBytesWritable>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira