[jira] [Commented] (MAPREDUCE-6251) JobClient needs additional retries at a higher level to address not-immediately-consistent dfs corner cases

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529910#comment-14529910 ] 

Hadoop QA commented on MAPREDUCE-6251:
--------------------------------------

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |  14m 32s | Pre-patch trunk compilation is healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any @author tags. |
| {color:green}+1{color} | tests included |   0m  0s | The patch appears to include 1 new or modified test files. |
| {color:green}+1{color} | javac |   7m 28s | There were no new javac warning messages. |
| {color:green}+1{color} | javadoc |   9m 32s | There were no new javadoc warning messages. |
| {color:green}+1{color} | release audit |   0m 23s | The applied patch does not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle |   1m 10s | The applied patch generated  21 new checkstyle issues (total was 554, now 575). |
| {color:red}-1{color} | whitespace |   0m  1s | The patch has 2  line(s) that end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install |   1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse |   0m 32s | The patch built with eclipse:eclipse. |
| {color:green}+1{color} | findbugs |   1m 56s | The patch does not introduce any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | mapreduce tests |   1m 35s | Tests passed in hadoop-mapreduce-client-core. |
| {color:green}+1{color} | mapreduce tests | 107m 45s | Tests passed in hadoop-mapreduce-client-jobclient. |
| | | 146m 51s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | http://issues.apache.org/jira/secure/attachment/12730673/MAPREDUCE-6251.4.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 90b3845 |
| checkstyle |  https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5649/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt |
| whitespace | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5649/artifact/patchprocess/whitespace.txt |
| hadoop-mapreduce-client-core test log | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5649/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt |
| hadoop-mapreduce-client-jobclient test log | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5649/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt |
| Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5649/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5649/console |


This message was automatically generated.

> JobClient needs additional retries at a higher level to address not-immediately-consistent dfs corner cases
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6251
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6251
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobhistoryserver, mrv2
>    Affects Versions: 2.6.0
>            Reporter: Craig Welch
>            Assignee: Craig Welch
>              Labels: BB2015-05-TBR
>         Attachments: MAPREDUCE-6251.0.patch, MAPREDUCE-6251.1.patch, MAPREDUCE-6251.2.patch, MAPREDUCE-6251.3.patch, MAPREDUCE-6251.4.patch
>
>
> The JobClient is used to get job status information for running and completed jobs.  Final state and history for a job is communicated from the application master to the job history server via a distributed file system - where the history is uploaded by the application master to the dfs and then scanned/loaded by the jobhistory server.  While HDFS has strong consistency guarantees not all Hadoop DFS's do.  When used in conjunction with a distributed file system which does not have this guarantee there will be cases where the history server may not see an uploaded file, resulting in the dreaded "no such job" and a null value for the RunningJob in the client.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)