You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Hive QA (JIRA)" <ji...@apache.org> on 2014/03/01 05:59:19 UTC
[jira] [Commented] (HIVE-6521) WebHCat cannot fetch correct
percentComplete for Hive jobs
[ https://issues.apache.org/jira/browse/HIVE-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13916931#comment-13916931 ]
Hive QA commented on HIVE-6521:
-------------------------------
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12631797/HIVE-6521.2.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5185 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
{noformat}
Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1560/testReport
Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1560/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12631797
> WebHCat cannot fetch correct percentComplete for Hive jobs
> ----------------------------------------------------------
>
> Key: HIVE-6521
> URL: https://issues.apache.org/jira/browse/HIVE-6521
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
> Affects Versions: 0.13.0
> Reporter: Deepesh Khandelwal
> Assignee: Deepesh Khandelwal
> Attachments: HIVE-6521.2.patch, HIVE-6521.patch
>
>
> WebHCat E2E test TestHive_7 failed because percentComplete wasn't returned as expected.
> {noformat}
> check_job_percent_complete failed. got percentComplete "map 0% reduce 0%", expected "map 100% reduce 100%"
> {noformat}
> So, there are two problems here.
> # The log parsing is broken for status of percentComplete. In the stderr of the job we see:
> {noformat}
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_1393486488858_0691, Tracking URL = http://ambari-sec-1393480847-others-2-4.cs1cloud.internal:8088/proxy/application_1393486488858_0691/
> Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1393486488858_0691
> Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
> 2014-02-27 18:40:50,166 Stage-1 map = 0%, reduce = 0%
> 2014-02-27 18:40:56,599 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.87 sec
> 2014-02-27 18:40:57,656 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.87 sec
> 2014-02-27 18:40:58,706 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.87 sec
> MapReduce Total cumulative CPU time: 870 msec
> Ended Job = job_1393486488858_0691
> MapReduce Jobs Launched:
> Job 0: Map: 1 Cumulative CPU: 0.87 sec HDFS Read: 305 HDFS Write: 0 SUCCESS
> Total MapReduce CPU Time Spent: 870 msec
> {noformat}
> The assumption in the code is that the line containing the percent status will end after "reduce = \d+%" but that fails with the above.
> # The last status from Hive job is "map = 100%, reduce = 0%" instead of expected "map = 100%, reduce = 100%".
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)