You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Balaji Rao <sb...@gmail.com> on 2012/05/18 20:57:28 UTC
Reduce operation stuck
I have a simple query running on HIVE using Amazon EMR where the
reduce operation seems to be stuck. The table is small with just
300,000+ rows (data_analysis).
I intend to work on a 30,000,000 row table once I'm able to fix this.
The query (modified to hide some information) is as follows:
INSERT OVERWRITE TABLE data_analysis
SELECT
a.clientid, a.isi, a.os, a.bro, a.engaged, a.iframed,
hover, hower1, hower5, hower60,
Visible, a.x1, a.x2, a.x3,
NoViewMeasure, a.x4, a.x5, a.x6,
a.alerttype, a.x7, a.x9, a.x11, a.x13, a.x17, a.x21, a.x27, a.x28,
COUNT(a.auction_id_64), SUM(CASE WHEN b.auction_id_64 IS NULL THEN
1 ELSE 0 END)
FROM
data_subset_2 a
LEFT OUTER JOIN
OrderFields b on a.auction_id_64=b.auction_id_64
GROUP BY
a.clientid, a.isi, a.os, a.bro, a.engaged, a.iframed,
hover, hower1, hower5, hower60,
Visible, a.x1, a.x2, a.x3,
NoViewMeasure, a.x4, a.x5, a.x6,
a.alerttype, a.x7, a.x9, a.x11, a.x13, a.x17, a.x21, a.x27, a.x28;
2012-05-18 17:49:32,440 Stage-1 map = 100%, reduce = 97%
2012-05-18 17:50:32,813 Stage-1 map = 100%, reduce = 97%
2012-05-18 17:51:33,176 Stage-1 map = 100%, reduce = 97%
2012-05-18 17:52:33,538 Stage-1 map = 100%, reduce = 97%
2012-05-18 17:53:33,906 Stage-1 map = 100%, reduce = 97%
2012-05-18 17:54:34,270 Stage-1 map = 100%, reduce = 97%
2012-05-18 17:55:34,644 Stage-1 map = 100%, reduce = 97%
2012-05-18 17:56:35,017 Stage-1 map = 100%, reduce = 97%
2012-05-18 17:57:35,387 Stage-1 map = 100%, reduce = 97%
2012-05-18 17:58:35,754 Stage-1 map = 100%, reduce = 97%
2012-05-18 17:59:36,125 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:00:36,503 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:01:36,873 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:02:36,990 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:03:37,355 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:04:37,719 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:05:38,081 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:06:38,432 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:07:38,777 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:08:39,135 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:09:39,489 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:10:39,852 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:11:40,221 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:12:40,591 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:13:40,962 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:14:41,336 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:15:41,700 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:16:42,070 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:17:42,440 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:18:42,801 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:19:43,156 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:20:43,531 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:21:43,899 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:22:44,255 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:23:44,619 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:24:44,978 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:25:45,347 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:26:45,720 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:27:46,088 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:28:46,446 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:29:46,814 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:30:47,183 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:31:47,546 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:32:47,918 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:33:48,282 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:34:48,408 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:35:48,766 Stage-1 map = 100%, reduce = 97%
2012-05-18 18:36:49,118 Stage-1 map = 100%, reduce = 97%
Thanks,
Balaji
Re: Reduce operation stuck
Posted by Bejoy KS <be...@yahoo.com>.
Hi Balaji
How many reduce tasks are being triggered?
If more than 1, how many of them are completed and and how many are still running?
Can you try increasing the number of reduce tasks and post in the behavior you are seeing.
I believe skew is already enabled for joins in your hive configuration.
Regards
Bejoy KS
Sent from handheld, please excuse typos.
-----Original Message-----
From: Ranjith <ra...@gmail.com>
Date: Fri, 18 May 2012 22:50:47
To: user@hive.apache.org<us...@hive.apache.org>
Reply-To: user@hive.apache.org
Cc: user@hive.apache.org<us...@hive.apache.org>
Subject: Re: Reduce operation stuck
This could be a result of the data being skewed. When you look at the job tracker page are you noticing just a few lurking around?
Thanks,
Ranjith
On May 18, 2012, at 1:57 PM, Balaji Rao <sb...@gmail.com> wrote:
> I have a simple query running on HIVE using Amazon EMR where the
> reduce operation seems to be stuck. The table is small with just
> 300,000+ rows (data_analysis).
> I intend to work on a 30,000,000 row table once I'm able to fix this.
> The query (modified to hide some information) is as follows:
>
> INSERT OVERWRITE TABLE data_analysis
> SELECT
> a.clientid, a.isi, a.os, a.bro, a.engaged, a.iframed,
> hover, hower1, hower5, hower60,
> Visible, a.x1, a.x2, a.x3,
> NoViewMeasure, a.x4, a.x5, a.x6,
> a.alerttype, a.x7, a.x9, a.x11, a.x13, a.x17, a.x21, a.x27, a.x28,
> COUNT(a.auction_id_64), SUM(CASE WHEN b.auction_id_64 IS NULL THEN
> 1 ELSE 0 END)
> FROM
> data_subset_2 a
> LEFT OUTER JOIN
> OrderFields b on a.auction_id_64=b.auction_id_64
> GROUP BY
> a.clientid, a.isi, a.os, a.bro, a.engaged, a.iframed,
> hover, hower1, hower5, hower60,
> Visible, a.x1, a.x2, a.x3,
> NoViewMeasure, a.x4, a.x5, a.x6,
> a.alerttype, a.x7, a.x9, a.x11, a.x13, a.x17, a.x21, a.x27, a.x28;
>
>
>
>
> 2012-05-18 17:49:32,440 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:50:32,813 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:51:33,176 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:52:33,538 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:53:33,906 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:54:34,270 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:55:34,644 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:56:35,017 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:57:35,387 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:58:35,754 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:59:36,125 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:00:36,503 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:01:36,873 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:02:36,990 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:03:37,355 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:04:37,719 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:05:38,081 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:06:38,432 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:07:38,777 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:08:39,135 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:09:39,489 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:10:39,852 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:11:40,221 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:12:40,591 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:13:40,962 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:14:41,336 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:15:41,700 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:16:42,070 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:17:42,440 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:18:42,801 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:19:43,156 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:20:43,531 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:21:43,899 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:22:44,255 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:23:44,619 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:24:44,978 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:25:45,347 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:26:45,720 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:27:46,088 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:28:46,446 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:29:46,814 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:30:47,183 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:31:47,546 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:32:47,918 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:33:48,282 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:34:48,408 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:35:48,766 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:36:49,118 Stage-1 map = 100%, reduce = 97%
>
> Thanks,
> Balaji
Re: Reduce operation stuck
Posted by Abhishek Pratap Singh <ma...@gmail.com>.
Can you paste the logs here?
Abhishek
On Sun, May 20, 2012 at 9:36 AM, Balaji Rao <sb...@gmail.com> wrote:
> There is only one task attempt that is lurking around and that never
> completes. It probably times out or fails after a long time.
>
> -Balaji
>
> On Fri, May 18, 2012 at 11:50 PM, Ranjith <ra...@gmail.com>
> wrote:
> > This could be a result of the data being skewed. When you look at the
> job tracker page are you noticing just a few lurking around?
> >
> > Thanks,
> > Ranjith
> >
> > On May 18, 2012, at 1:57 PM, Balaji Rao <sb...@gmail.com> wrote:
> >
> >> I have a simple query running on HIVE using Amazon EMR where the
> >> reduce operation seems to be stuck. The table is small with just
> >> 300,000+ rows (data_analysis).
> >> I intend to work on a 30,000,000 row table once I'm able to fix this.
> >> The query (modified to hide some information) is as follows:
> >>
> >> INSERT OVERWRITE TABLE data_analysis
> >> SELECT
> >> a.clientid, a.isi, a.os, a.bro, a.engaged, a.iframed,
> >> hover, hower1, hower5, hower60,
> >> Visible, a.x1, a.x2, a.x3,
> >> NoViewMeasure, a.x4, a.x5, a.x6,
> >> a.alerttype, a.x7, a.x9, a.x11, a.x13, a.x17, a.x21, a.x27,
> a.x28,
> >> COUNT(a.auction_id_64), SUM(CASE WHEN b.auction_id_64 IS NULL THEN
> >> 1 ELSE 0 END)
> >> FROM
> >> data_subset_2 a
> >> LEFT OUTER JOIN
> >> OrderFields b on a.auction_id_64=b.auction_id_64
> >> GROUP BY
> >> a.clientid, a.isi, a.os, a.bro, a.engaged, a.iframed,
> >> hover, hower1, hower5, hower60,
> >> Visible, a.x1, a.x2, a.x3,
> >> NoViewMeasure, a.x4, a.x5, a.x6,
> >> a.alerttype, a.x7, a.x9, a.x11, a.x13, a.x17, a.x21, a.x27,
> a.x28;
> >>
> >>
> >>
> >>
> >> 2012-05-18 17:49:32,440 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 17:50:32,813 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 17:51:33,176 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 17:52:33,538 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 17:53:33,906 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 17:54:34,270 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 17:55:34,644 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 17:56:35,017 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 17:57:35,387 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 17:58:35,754 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 17:59:36,125 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:00:36,503 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:01:36,873 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:02:36,990 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:03:37,355 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:04:37,719 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:05:38,081 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:06:38,432 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:07:38,777 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:08:39,135 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:09:39,489 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:10:39,852 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:11:40,221 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:12:40,591 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:13:40,962 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:14:41,336 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:15:41,700 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:16:42,070 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:17:42,440 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:18:42,801 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:19:43,156 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:20:43,531 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:21:43,899 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:22:44,255 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:23:44,619 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:24:44,978 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:25:45,347 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:26:45,720 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:27:46,088 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:28:46,446 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:29:46,814 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:30:47,183 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:31:47,546 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:32:47,918 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:33:48,282 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:34:48,408 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:35:48,766 Stage-1 map = 100%, reduce = 97%
> >> 2012-05-18 18:36:49,118 Stage-1 map = 100%, reduce = 97%
> >>
> >> Thanks,
> >> Balaji
>
Re: Reduce operation stuck
Posted by Balaji Rao <sb...@gmail.com>.
There is only one task attempt that is lurking around and that never
completes. It probably times out or fails after a long time.
-Balaji
On Fri, May 18, 2012 at 11:50 PM, Ranjith <ra...@gmail.com> wrote:
> This could be a result of the data being skewed. When you look at the job tracker page are you noticing just a few lurking around?
>
> Thanks,
> Ranjith
>
> On May 18, 2012, at 1:57 PM, Balaji Rao <sb...@gmail.com> wrote:
>
>> I have a simple query running on HIVE using Amazon EMR where the
>> reduce operation seems to be stuck. The table is small with just
>> 300,000+ rows (data_analysis).
>> I intend to work on a 30,000,000 row table once I'm able to fix this.
>> The query (modified to hide some information) is as follows:
>>
>> INSERT OVERWRITE TABLE data_analysis
>> SELECT
>> a.clientid, a.isi, a.os, a.bro, a.engaged, a.iframed,
>> hover, hower1, hower5, hower60,
>> Visible, a.x1, a.x2, a.x3,
>> NoViewMeasure, a.x4, a.x5, a.x6,
>> a.alerttype, a.x7, a.x9, a.x11, a.x13, a.x17, a.x21, a.x27, a.x28,
>> COUNT(a.auction_id_64), SUM(CASE WHEN b.auction_id_64 IS NULL THEN
>> 1 ELSE 0 END)
>> FROM
>> data_subset_2 a
>> LEFT OUTER JOIN
>> OrderFields b on a.auction_id_64=b.auction_id_64
>> GROUP BY
>> a.clientid, a.isi, a.os, a.bro, a.engaged, a.iframed,
>> hover, hower1, hower5, hower60,
>> Visible, a.x1, a.x2, a.x3,
>> NoViewMeasure, a.x4, a.x5, a.x6,
>> a.alerttype, a.x7, a.x9, a.x11, a.x13, a.x17, a.x21, a.x27, a.x28;
>>
>>
>>
>>
>> 2012-05-18 17:49:32,440 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 17:50:32,813 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 17:51:33,176 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 17:52:33,538 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 17:53:33,906 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 17:54:34,270 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 17:55:34,644 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 17:56:35,017 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 17:57:35,387 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 17:58:35,754 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 17:59:36,125 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:00:36,503 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:01:36,873 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:02:36,990 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:03:37,355 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:04:37,719 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:05:38,081 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:06:38,432 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:07:38,777 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:08:39,135 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:09:39,489 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:10:39,852 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:11:40,221 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:12:40,591 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:13:40,962 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:14:41,336 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:15:41,700 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:16:42,070 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:17:42,440 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:18:42,801 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:19:43,156 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:20:43,531 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:21:43,899 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:22:44,255 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:23:44,619 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:24:44,978 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:25:45,347 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:26:45,720 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:27:46,088 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:28:46,446 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:29:46,814 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:30:47,183 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:31:47,546 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:32:47,918 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:33:48,282 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:34:48,408 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:35:48,766 Stage-1 map = 100%, reduce = 97%
>> 2012-05-18 18:36:49,118 Stage-1 map = 100%, reduce = 97%
>>
>> Thanks,
>> Balaji
Re: Reduce operation stuck
Posted by Ranjith <ra...@gmail.com>.
This could be a result of the data being skewed. When you look at the job tracker page are you noticing just a few lurking around?
Thanks,
Ranjith
On May 18, 2012, at 1:57 PM, Balaji Rao <sb...@gmail.com> wrote:
> I have a simple query running on HIVE using Amazon EMR where the
> reduce operation seems to be stuck. The table is small with just
> 300,000+ rows (data_analysis).
> I intend to work on a 30,000,000 row table once I'm able to fix this.
> The query (modified to hide some information) is as follows:
>
> INSERT OVERWRITE TABLE data_analysis
> SELECT
> a.clientid, a.isi, a.os, a.bro, a.engaged, a.iframed,
> hover, hower1, hower5, hower60,
> Visible, a.x1, a.x2, a.x3,
> NoViewMeasure, a.x4, a.x5, a.x6,
> a.alerttype, a.x7, a.x9, a.x11, a.x13, a.x17, a.x21, a.x27, a.x28,
> COUNT(a.auction_id_64), SUM(CASE WHEN b.auction_id_64 IS NULL THEN
> 1 ELSE 0 END)
> FROM
> data_subset_2 a
> LEFT OUTER JOIN
> OrderFields b on a.auction_id_64=b.auction_id_64
> GROUP BY
> a.clientid, a.isi, a.os, a.bro, a.engaged, a.iframed,
> hover, hower1, hower5, hower60,
> Visible, a.x1, a.x2, a.x3,
> NoViewMeasure, a.x4, a.x5, a.x6,
> a.alerttype, a.x7, a.x9, a.x11, a.x13, a.x17, a.x21, a.x27, a.x28;
>
>
>
>
> 2012-05-18 17:49:32,440 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:50:32,813 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:51:33,176 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:52:33,538 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:53:33,906 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:54:34,270 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:55:34,644 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:56:35,017 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:57:35,387 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:58:35,754 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 17:59:36,125 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:00:36,503 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:01:36,873 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:02:36,990 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:03:37,355 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:04:37,719 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:05:38,081 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:06:38,432 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:07:38,777 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:08:39,135 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:09:39,489 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:10:39,852 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:11:40,221 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:12:40,591 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:13:40,962 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:14:41,336 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:15:41,700 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:16:42,070 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:17:42,440 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:18:42,801 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:19:43,156 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:20:43,531 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:21:43,899 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:22:44,255 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:23:44,619 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:24:44,978 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:25:45,347 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:26:45,720 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:27:46,088 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:28:46,446 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:29:46,814 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:30:47,183 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:31:47,546 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:32:47,918 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:33:48,282 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:34:48,408 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:35:48,766 Stage-1 map = 100%, reduce = 97%
> 2012-05-18 18:36:49,118 Stage-1 map = 100%, reduce = 97%
>
> Thanks,
> Balaji