You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Jay Vyas <ja...@gmail.com> on 2014/05/22 19:36:08 UTC
getCounters NPE
Hi hadoop ... Is there a reason why line 220, below, should ever return
null when
being called through the code path of job.getCounters() ?
It appears that there could be some indirection involving RPCs, etc, so im
thinking its best to ask before I attempt to trace the call.
217 @Override
218 public GetCountersResponse getCounters(GetCountersRequest request)
219 throws IOException {
220 JobId jobId = request.getJobId();
221 Job job = verifyAndGetJob(jobId);
222 GetCountersResponse response =
recordFactory.newRecordInstance(GetCountersResponse.class);
223 response.setCounters(TypeConverter.toYarn(job.getAllCounters()));
224 return response;
225 }
--
Jay Vyas
http://jayunit100.blogspot.com
Re: getCounters NPE
Posted by Jay Vyas <ja...@gmail.com>.
To add some more detail on this: after turning logging to debug, it's clear that the job history server is skipping the copying of a completed jhist file into the done/ directory.
Is there any reason why some jhist files never get recognized, even though they are indeed in a directory which contains other jhist files (which *are* recognized)?
Also I've created a jira for finer grained logging during the directoryScan(..) operation: https://issues.apache.org/jira/browse/MAPREDUCE-5902
> On May 22, 2014, at 1:37 PM, Jay Vyas <ja...@gmail.com> wrote:
>
> (sorry, i meant THROW a NPE, not " return a null). Big difference of course !
>
>
>> On Thu, May 22, 2014 at 1:36 PM, Jay Vyas <ja...@gmail.com> wrote:
>> Hi hadoop ... Is there a reason why line 220, below, should ever return null when
>> being called through the code path of job.getCounters() ?
>>
>> It appears that there could be some indirection involving RPCs, etc, so im thinking its best to ask before I attempt to trace the call.
>>
>> 217 @Override
>> 218 public GetCountersResponse getCounters(GetCountersRequest request)
>> 219 throws IOException {
>> 220 JobId jobId = request.getJobId();
>> 221 Job job = verifyAndGetJob(jobId);
>> 222 GetCountersResponse response = recordFactory.newRecordInstance(GetCountersResponse.class);
>> 223 response.setCounters(TypeConverter.toYarn(job.getAllCounters()));
>> 224 return response;
>> 225 }
>>
>>
>> --
>> Jay Vyas
>> http://jayunit100.blogspot.com
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
Re: getCounters NPE
Posted by Jay Vyas <ja...@gmail.com>.
(sorry, i meant THROW a NPE, not " return a null). Big difference of
course !
On Thu, May 22, 2014 at 1:36 PM, Jay Vyas <ja...@gmail.com> wrote:
> Hi hadoop ... Is there a reason why line 220, below, should ever return
> null when
> being called through the code path of job.getCounters() ?
>
> It appears that there could be some indirection involving RPCs, etc, so im
> thinking its best to ask before I attempt to trace the call.
>
> 217 @Override
> 218 public GetCountersResponse getCounters(GetCountersRequest request)
> 219 throws IOException {
> 220 JobId jobId = request.getJobId();
> 221 Job job = verifyAndGetJob(jobId);
> 222 GetCountersResponse response =
> recordFactory.newRecordInstance(GetCountersResponse.class);
> 223 response.setCounters(TypeConverter.toYarn(job.getAllCounters()));
> 224 return response;
> 225 }
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
Jay Vyas
http://jayunit100.blogspot.com