You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by aaron morton <aa...@thelastpickle.com> on 2011/08/15 04:04:55 UTC
NPE in TaskLogAppender
I'm running the Cassandra Brisk server with Haddop core 20.203 on OSX, everything is local.
I keep running into this problem for Hive jobs
INFO 13:52:39,923 Error from attempt_201108151342_0001_m_000001_1: java.lang.NullPointerException
at org.apache.hadoop.mapred.TaskLogAppender.flush(TaskLogAppender.java:67)
at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:264)
at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:253)
The only info I've found online was http://www.mail-archive.com/common-user@hadoop.apache.org/msg12829.html
Just for fun I tried…
* setting mapred.acls.enabled to true
* setting mapred.queue.default.acl-submit-job and mapred.queue.default.acl-administer-jobs to *
There was no discernible increase in joy though.
Any thoughts ?
Cheers
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
Re: NPE in TaskLogAppender
Posted by aaron morton <aa...@thelastpickle.com>.
Thanks but we were already on log4j 1.2.16.
The underlying problem is trying to use two different log4j configs it the same process. The cassandra one is causing all the appenders in the Hadoop config to be closed when it starts watching it's own config. And as the hadoop config is not in the cassandra config those appenders are not re-activated.
I'll put up a patch to make the TaskLogAppender a little safer by checking if if it's closed before flush. But we need to keep the diff configs away from each other.
Cheers
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 19/08/2011, at 4:26 AM, Aaron Baff wrote:
> Yes, we had this issue, we simply grabbed the latest log4j jar file, and copied it over on top of the existing one. See https://issues.apache.org/jira/browse/MAPREDUCE-2378
>
>
> -----Original Message-----
> From: Isaac Dooley [mailto:Isaac.Dooley@twosigma.com]
> Sent: Thursday, August 18, 2011 8:00 AM
> To: common-user@hadoop.apache.org
> Subject: RE: NPE in TaskLogAppender
>
> I've had this problem on linux with 20.203. It occured when a map or reduce task would mess around with log4j configurations. Basically, I had to create a modified TaskManager that would set some environment variables that would keep my code's static blocks from reconfiguring log4j.
>
>
>
> -----Original Message-----
> From: aaron morton [mailto:aaron@thelastpickle.com]
> Sent: Thursday, August 18, 2011 9:30 AM
> To: common-user@hadoop.apache.org
> Subject: Re: NPE in TaskLogAppender
>
> An update incase anyone else has this problem. It looks like an issue with competing log4j configurations that could be resolved by using Logger Repositories. I've updated a thread in the brisk user group http://groups.google.com/group/brisk-users/msg/619b5ebba2fb0e89
>
> Has anyone else had this problem ?
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 15/08/2011, at 2:04 PM, aaron morton wrote:
>
>> I'm running the Cassandra Brisk server with Haddop core 20.203 on OSX, everything is local.
>>
>> I keep running into this problem for Hive jobs
>>
>> INFO 13:52:39,923 Error from attempt_201108151342_0001_m_000001_1: java.lang.NullPointerException
>> at org.apache.hadoop.mapred.TaskLogAppender.flush(TaskLogAppender.java:67)
>> at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:264)
>> at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:396)
>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>> at org.apache.hadoop.mapred.Child.main(Child.java:253)
>>
>> The only info I've found online was http://www.mail-archive.com/common-user@hadoop.apache.org/msg12829.html
>>
>> Just for fun I tried...
>> * setting mapred.acls.enabled to true
>> * setting mapred.queue.default.acl-submit-job and mapred.queue.default.acl-administer-jobs to *
>>
>> There was no discernible increase in joy though.
>>
>> Any thoughts ?
>>
>> Cheers
>>
>> -----------------
>> Aaron Morton
>> Freelance Cassandra Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>
RE: NPE in TaskLogAppender
Posted by Aaron Baff <Aa...@telescope.tv>.
Yes, we had this issue, we simply grabbed the latest log4j jar file, and copied it over on top of the existing one. See https://issues.apache.org/jira/browse/MAPREDUCE-2378
-----Original Message-----
From: Isaac Dooley [mailto:Isaac.Dooley@twosigma.com]
Sent: Thursday, August 18, 2011 8:00 AM
To: common-user@hadoop.apache.org
Subject: RE: NPE in TaskLogAppender
I've had this problem on linux with 20.203. It occured when a map or reduce task would mess around with log4j configurations. Basically, I had to create a modified TaskManager that would set some environment variables that would keep my code's static blocks from reconfiguring log4j.
-----Original Message-----
From: aaron morton [mailto:aaron@thelastpickle.com]
Sent: Thursday, August 18, 2011 9:30 AM
To: common-user@hadoop.apache.org
Subject: Re: NPE in TaskLogAppender
An update incase anyone else has this problem. It looks like an issue with competing log4j configurations that could be resolved by using Logger Repositories. I've updated a thread in the brisk user group http://groups.google.com/group/brisk-users/msg/619b5ebba2fb0e89
Has anyone else had this problem ?
Cheers
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 15/08/2011, at 2:04 PM, aaron morton wrote:
> I'm running the Cassandra Brisk server with Haddop core 20.203 on OSX, everything is local.
>
> I keep running into this problem for Hive jobs
>
> INFO 13:52:39,923 Error from attempt_201108151342_0001_m_000001_1: java.lang.NullPointerException
> at org.apache.hadoop.mapred.TaskLogAppender.flush(TaskLogAppender.java:67)
> at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:264)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.mapred.Child.main(Child.java:253)
>
> The only info I've found online was http://www.mail-archive.com/common-user@hadoop.apache.org/msg12829.html
>
> Just for fun I tried...
> * setting mapred.acls.enabled to true
> * setting mapred.queue.default.acl-submit-job and mapred.queue.default.acl-administer-jobs to *
>
> There was no discernible increase in joy though.
>
> Any thoughts ?
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
RE: NPE in TaskLogAppender
Posted by Isaac Dooley <Is...@twosigma.com>.
I've had this problem on linux with 20.203. It occured when a map or reduce task would mess around with log4j configurations. Basically, I had to create a modified TaskManager that would set some environment variables that would keep my code's static blocks from reconfiguring log4j.
-----Original Message-----
From: aaron morton [mailto:aaron@thelastpickle.com]
Sent: Thursday, August 18, 2011 9:30 AM
To: common-user@hadoop.apache.org
Subject: Re: NPE in TaskLogAppender
An update incase anyone else has this problem. It looks like an issue with competing log4j configurations that could be resolved by using Logger Repositories. I've updated a thread in the brisk user group http://groups.google.com/group/brisk-users/msg/619b5ebba2fb0e89
Has anyone else had this problem ?
Cheers
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 15/08/2011, at 2:04 PM, aaron morton wrote:
> I'm running the Cassandra Brisk server with Haddop core 20.203 on OSX, everything is local.
>
> I keep running into this problem for Hive jobs
>
> INFO 13:52:39,923 Error from attempt_201108151342_0001_m_000001_1: java.lang.NullPointerException
> at org.apache.hadoop.mapred.TaskLogAppender.flush(TaskLogAppender.java:67)
> at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:264)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.mapred.Child.main(Child.java:253)
>
> The only info I've found online was http://www.mail-archive.com/common-user@hadoop.apache.org/msg12829.html
>
> Just for fun I tried...
> * setting mapred.acls.enabled to true
> * setting mapred.queue.default.acl-submit-job and mapred.queue.default.acl-administer-jobs to *
>
> There was no discernible increase in joy though.
>
> Any thoughts ?
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
Re: NPE in TaskLogAppender
Posted by aaron morton <aa...@thelastpickle.com>.
An update incase anyone else has this problem. It looks like an issue with competing log4j configurations that could be resolved by using Logger Repositories. I've updated a thread in the brisk user group http://groups.google.com/group/brisk-users/msg/619b5ebba2fb0e89
Has anyone else had this problem ?
Cheers
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 15/08/2011, at 2:04 PM, aaron morton wrote:
> I'm running the Cassandra Brisk server with Haddop core 20.203 on OSX, everything is local.
>
> I keep running into this problem for Hive jobs
>
> INFO 13:52:39,923 Error from attempt_201108151342_0001_m_000001_1: java.lang.NullPointerException
> at org.apache.hadoop.mapred.TaskLogAppender.flush(TaskLogAppender.java:67)
> at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:264)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.mapred.Child.main(Child.java:253)
>
> The only info I've found online was http://www.mail-archive.com/common-user@hadoop.apache.org/msg12829.html
>
> Just for fun I tried…
> * setting mapred.acls.enabled to true
> * setting mapred.queue.default.acl-submit-job and mapred.queue.default.acl-administer-jobs to *
>
> There was no discernible increase in joy though.
>
> Any thoughts ?
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>