[jira] [Created] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly

["Roman Gavryliuk (JIRA)" <ji...@apache.org>]

Roman Gavryliuk created MAPREDUCE-6737:
------------------------------------------

             Summary: HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
                 Key: MAPREDUCE-6737
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: jobhistoryserver
    Affects Versions: 2.5.1, 2.7.0
            Reporter: Roman Gavryliuk


The problem shows itself while recovering old apps information:
2016-07-18 16:08:35,031 WARN
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
start time from job history file
job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
: java.lang.NumberFormatException: For input string: 
""

The problem is in JobHistoryEventHandler.java class in the
following part of code:

//initialize the launchTime in the JobIndexInfo of MetaInfo
      if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
        JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
        mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());

Because of job was not initialized properly, the 'if' statement takes value
'false' and .setJobStartTime() is not called.

In JobIndexInfo constructor, we have a default value for jobStartTime:
this.jobStartTime = -1;

When history server recovers any application's info, it passes all parameters
to array of strings jobDetails:
String[] jobDetails = fileName.split(DELIMITER);

Please note, DELIMETER is initialized in the following way:
static final String DELIMITER = "-";

So, jobDetails array has 10 elements - job ID, submit time, username, job name,
finish time, number of maps, number of reducers, job status, queue, and start
time).
If jobStartTime = -1, the minus symbol is considered as delimeter and the code
will assign an empty string "" as a value for 9-th element in jobDetails array.

In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
NumberFormatException will appear while trying to parse empty string to long
type.
Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));

The most simple fix is to change the value this.jobStartTime to 0 in
JobIndexInfo constructor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org