You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by jingguo yao <ya...@gmail.com> on 2013/03/26 06:25:18 UTC
Is the input values for reduce method sorted in any order?
Is the Iterable values associated with a key sorted in any order? Are
there are any configuration options controlling how the input values are
sorted?
I know that the secondary sort way can be used to achieve the same
effect. I am not asking for a workaround.
--
Jingguo
Re: Is the input values for reduce method sorted in any order?
Posted by jingguo yao <ya...@gmail.com>.
Harsh, thanks.
On Tue, Mar 26, 2013 at 2:28 PM, Harsh J <ha...@cloudera.com> wrote:
> MR will partition and sort inputs by keys by the key comparator, and
> then group them together when reading back via a grouping comparator
> (which is usually the same as the key comparator). It will not re-sort
> the values nor look at any of the value's fields during this process.
>
> If you want your values to be sorted, implementing secondary sort
> would need to happen wherein you extract the value-sort attributes
> onto your keys and have them grouped differently.
>
> On Tue, Mar 26, 2013 at 10:55 AM, jingguo yao <ya...@gmail.com> wrote:
>> Is the Iterable values associated with a key sorted in any order? Are
>> there are any configuration options controlling how the input values are
>> sorted?
>>
>> I know that the secondary sort way can be used to achieve the same
>> effect. I am not asking for a workaround.
>>
>> --
>> Jingguo
>
>
>
> --
> Harsh J
--
Jingguo
Re: Is the input values for reduce method sorted in any order?
Posted by jingguo yao <ya...@gmail.com>.
Harsh, thanks.
On Tue, Mar 26, 2013 at 2:28 PM, Harsh J <ha...@cloudera.com> wrote:
> MR will partition and sort inputs by keys by the key comparator, and
> then group them together when reading back via a grouping comparator
> (which is usually the same as the key comparator). It will not re-sort
> the values nor look at any of the value's fields during this process.
>
> If you want your values to be sorted, implementing secondary sort
> would need to happen wherein you extract the value-sort attributes
> onto your keys and have them grouped differently.
>
> On Tue, Mar 26, 2013 at 10:55 AM, jingguo yao <ya...@gmail.com> wrote:
>> Is the Iterable values associated with a key sorted in any order? Are
>> there are any configuration options controlling how the input values are
>> sorted?
>>
>> I know that the secondary sort way can be used to achieve the same
>> effect. I am not asking for a workaround.
>>
>> --
>> Jingguo
>
>
>
> --
> Harsh J
--
Jingguo
Re: Is the input values for reduce method sorted in any order?
Posted by jingguo yao <ya...@gmail.com>.
Harsh, thanks.
On Tue, Mar 26, 2013 at 2:28 PM, Harsh J <ha...@cloudera.com> wrote:
> MR will partition and sort inputs by keys by the key comparator, and
> then group them together when reading back via a grouping comparator
> (which is usually the same as the key comparator). It will not re-sort
> the values nor look at any of the value's fields during this process.
>
> If you want your values to be sorted, implementing secondary sort
> would need to happen wherein you extract the value-sort attributes
> onto your keys and have them grouped differently.
>
> On Tue, Mar 26, 2013 at 10:55 AM, jingguo yao <ya...@gmail.com> wrote:
>> Is the Iterable values associated with a key sorted in any order? Are
>> there are any configuration options controlling how the input values are
>> sorted?
>>
>> I know that the secondary sort way can be used to achieve the same
>> effect. I am not asking for a workaround.
>>
>> --
>> Jingguo
>
>
>
> --
> Harsh J
--
Jingguo
Re: Is the input values for reduce method sorted in any order?
Posted by jingguo yao <ya...@gmail.com>.
Harsh, thanks.
On Tue, Mar 26, 2013 at 2:28 PM, Harsh J <ha...@cloudera.com> wrote:
> MR will partition and sort inputs by keys by the key comparator, and
> then group them together when reading back via a grouping comparator
> (which is usually the same as the key comparator). It will not re-sort
> the values nor look at any of the value's fields during this process.
>
> If you want your values to be sorted, implementing secondary sort
> would need to happen wherein you extract the value-sort attributes
> onto your keys and have them grouped differently.
>
> On Tue, Mar 26, 2013 at 10:55 AM, jingguo yao <ya...@gmail.com> wrote:
>> Is the Iterable values associated with a key sorted in any order? Are
>> there are any configuration options controlling how the input values are
>> sorted?
>>
>> I know that the secondary sort way can be used to achieve the same
>> effect. I am not asking for a workaround.
>>
>> --
>> Jingguo
>
>
>
> --
> Harsh J
--
Jingguo
Re: Is the input values for reduce method sorted in any order?
Posted by Harsh J <ha...@cloudera.com>.
MR will partition and sort inputs by keys by the key comparator, and
then group them together when reading back via a grouping comparator
(which is usually the same as the key comparator). It will not re-sort
the values nor look at any of the value's fields during this process.
If you want your values to be sorted, implementing secondary sort
would need to happen wherein you extract the value-sort attributes
onto your keys and have them grouped differently.
On Tue, Mar 26, 2013 at 10:55 AM, jingguo yao <ya...@gmail.com> wrote:
> Is the Iterable values associated with a key sorted in any order? Are
> there are any configuration options controlling how the input values are
> sorted?
>
> I know that the secondary sort way can be used to achieve the same
> effect. I am not asking for a workaround.
>
> --
> Jingguo
--
Harsh J
Re: Is the input values for reduce method sorted in any order?
Posted by Harsh J <ha...@cloudera.com>.
MR will partition and sort inputs by keys by the key comparator, and
then group them together when reading back via a grouping comparator
(which is usually the same as the key comparator). It will not re-sort
the values nor look at any of the value's fields during this process.
If you want your values to be sorted, implementing secondary sort
would need to happen wherein you extract the value-sort attributes
onto your keys and have them grouped differently.
On Tue, Mar 26, 2013 at 10:55 AM, jingguo yao <ya...@gmail.com> wrote:
> Is the Iterable values associated with a key sorted in any order? Are
> there are any configuration options controlling how the input values are
> sorted?
>
> I know that the secondary sort way can be used to achieve the same
> effect. I am not asking for a workaround.
>
> --
> Jingguo
--
Harsh J
Re: Is the input values for reduce method sorted in any order?
Posted by Harsh J <ha...@cloudera.com>.
MR will partition and sort inputs by keys by the key comparator, and
then group them together when reading back via a grouping comparator
(which is usually the same as the key comparator). It will not re-sort
the values nor look at any of the value's fields during this process.
If you want your values to be sorted, implementing secondary sort
would need to happen wherein you extract the value-sort attributes
onto your keys and have them grouped differently.
On Tue, Mar 26, 2013 at 10:55 AM, jingguo yao <ya...@gmail.com> wrote:
> Is the Iterable values associated with a key sorted in any order? Are
> there are any configuration options controlling how the input values are
> sorted?
>
> I know that the secondary sort way can be used to achieve the same
> effect. I am not asking for a workaround.
>
> --
> Jingguo
--
Harsh J
Re: Is the input values for reduce method sorted in any order?
Posted by Harsh J <ha...@cloudera.com>.
MR will partition and sort inputs by keys by the key comparator, and
then group them together when reading back via a grouping comparator
(which is usually the same as the key comparator). It will not re-sort
the values nor look at any of the value's fields during this process.
If you want your values to be sorted, implementing secondary sort
would need to happen wherein you extract the value-sort attributes
onto your keys and have them grouped differently.
On Tue, Mar 26, 2013 at 10:55 AM, jingguo yao <ya...@gmail.com> wrote:
> Is the Iterable values associated with a key sorted in any order? Are
> there are any configuration options controlling how the input values are
> sorted?
>
> I know that the secondary sort way can be used to achieve the same
> effect. I am not asking for a workaround.
>
> --
> Jingguo
--
Harsh J