You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Jeff Zhang <zj...@gmail.com> on 2015/02/06 04:35:01 UTC

Localizer failed but it works after I restart yarn

I intermittently meet this issue of Localizer failed, but after I restart
the yarn, then the problem is gone. Is this a known issue ?

Here's log in node manager


2015-02-06 11:31:24,898 INFO
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
Localizer CWD set to
/tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
=
file:/tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
2015-02-06 11:31:24,906 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
Localizer failed
java.lang.NullPointerException
    at java.util.zip.ZipFile.getZipEntry(ZipFile.java:531)
    at java.util.zip.ZipFile.access$900(ZipFile.java:56)
    at java.util.zip.ZipFile$1.nextElement(ZipFile.java:513)
    at java.util.zip.ZipFile$1.nextElement(ZipFile.java:483)
    at java.util.jar.JarFile$1.nextElement(JarFile.java:244)
    at java.util.jar.JarFile$1.nextElement(JarFile.java:239)
    at sun.misc.URLClassPath$JarLoader.validIndex(URLClassPath.java:803)
    at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:937)
    at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:848)
    at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:818)
    at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
    at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
    at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
    at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
    at java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
    at org.apache.xerces.parsers.SecuritySupport$6.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at
org.apache.xerces.parsers.SecuritySupport.getResourceAsStream(Unknown
Source)
    at
org.apache.xerces.parsers.ObjectFactory.findJarServiceProvider(Unknown
Source)
    at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
    at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
    at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
    at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
    at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source)
    at
org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
Source)
    at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2402)
    at
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2364)
    at
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2281)
    at org.apache.hadoop.conf.Configuration.set(Configuration.java:1019)
    at org.apache.hadoop.conf.Configuration.set(Configuration.java:991)
    at
org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1791)
    at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:407)
    at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:129)
    at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:129)
    at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)

-- 
Best Regards

Jeff Zhang

Re: Localizer failed but it works after I restart yarn

Posted by daemeon reiydelle <da...@gmail.com>.
My first concern is that temp is a virtual file system and cannot exceed
real memory plus swap space. You may not see file system full errors flow
up as the root cause of the stack trace, but you will see null pointers. So
move your temp directory onto your file system (~/Documents/HDFSTMP or
something) and retry.



*.......*






*“Life should not be a journey to the grave with the intention of arriving
safely in apretty and well preserved body, but rather to skid in broadside
in a cloud of smoke,thoroughly used up, totally worn out, and loudly
proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
(+1) 415.501.0198London (+44) (0) 20 8144 9872*

On Thu, Feb 5, 2015 at 7:36 PM, Jeff Zhang <zj...@gmail.com> wrote:

> BTW, I am using hadoop 2.6 and it's a single node cluster on mac.
>
>
>
> On Fri, Feb 6, 2015 at 11:35 AM, Jeff Zhang <zj...@gmail.com> wrote:
>
>> I intermittently meet this issue of Localizer failed, but after I restart
>> the yarn, then the problem is gone. Is this a known issue ?
>>
>> Here's log in node manager
>>
>>
>> 2015-02-06 11:31:24,898 INFO
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
>> Localizer CWD set to
>> /tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
>> =
>> file:/tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
>> 2015-02-06 11:31:24,906 INFO
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
>> Localizer failed
>> java.lang.NullPointerException
>>     at java.util.zip.ZipFile.getZipEntry(ZipFile.java:531)
>>     at java.util.zip.ZipFile.access$900(ZipFile.java:56)
>>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:513)
>>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:483)
>>     at java.util.jar.JarFile$1.nextElement(JarFile.java:244)
>>     at java.util.jar.JarFile$1.nextElement(JarFile.java:239)
>>     at sun.misc.URLClassPath$JarLoader.validIndex(URLClassPath.java:803)
>>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:937)
>>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:848)
>>     at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:818)
>>     at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
>>     at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
>>     at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
>>     at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
>>     at
>> java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
>>     at org.apache.xerces.parsers.SecuritySupport$6.run(Unknown Source)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at
>> org.apache.xerces.parsers.SecuritySupport.getResourceAsStream(Unknown
>> Source)
>>     at
>> org.apache.xerces.parsers.ObjectFactory.findJarServiceProvider(Unknown
>> Source)
>>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown
>> Source)
>>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown
>> Source)
>>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>>     at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source)
>>     at
>> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
>> Source)
>>     at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2402)
>>     at
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2364)
>>     at
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2281)
>>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:1019)
>>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:991)
>>     at
>> org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1791)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:407)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:129)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:129)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: Localizer failed but it works after I restart yarn

Posted by daemeon reiydelle <da...@gmail.com>.
My first concern is that temp is a virtual file system and cannot exceed
real memory plus swap space. You may not see file system full errors flow
up as the root cause of the stack trace, but you will see null pointers. So
move your temp directory onto your file system (~/Documents/HDFSTMP or
something) and retry.



*.......*






*“Life should not be a journey to the grave with the intention of arriving
safely in apretty and well preserved body, but rather to skid in broadside
in a cloud of smoke,thoroughly used up, totally worn out, and loudly
proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
(+1) 415.501.0198London (+44) (0) 20 8144 9872*

On Thu, Feb 5, 2015 at 7:36 PM, Jeff Zhang <zj...@gmail.com> wrote:

> BTW, I am using hadoop 2.6 and it's a single node cluster on mac.
>
>
>
> On Fri, Feb 6, 2015 at 11:35 AM, Jeff Zhang <zj...@gmail.com> wrote:
>
>> I intermittently meet this issue of Localizer failed, but after I restart
>> the yarn, then the problem is gone. Is this a known issue ?
>>
>> Here's log in node manager
>>
>>
>> 2015-02-06 11:31:24,898 INFO
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
>> Localizer CWD set to
>> /tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
>> =
>> file:/tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
>> 2015-02-06 11:31:24,906 INFO
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
>> Localizer failed
>> java.lang.NullPointerException
>>     at java.util.zip.ZipFile.getZipEntry(ZipFile.java:531)
>>     at java.util.zip.ZipFile.access$900(ZipFile.java:56)
>>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:513)
>>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:483)
>>     at java.util.jar.JarFile$1.nextElement(JarFile.java:244)
>>     at java.util.jar.JarFile$1.nextElement(JarFile.java:239)
>>     at sun.misc.URLClassPath$JarLoader.validIndex(URLClassPath.java:803)
>>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:937)
>>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:848)
>>     at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:818)
>>     at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
>>     at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
>>     at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
>>     at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
>>     at
>> java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
>>     at org.apache.xerces.parsers.SecuritySupport$6.run(Unknown Source)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at
>> org.apache.xerces.parsers.SecuritySupport.getResourceAsStream(Unknown
>> Source)
>>     at
>> org.apache.xerces.parsers.ObjectFactory.findJarServiceProvider(Unknown
>> Source)
>>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown
>> Source)
>>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown
>> Source)
>>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>>     at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source)
>>     at
>> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
>> Source)
>>     at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2402)
>>     at
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2364)
>>     at
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2281)
>>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:1019)
>>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:991)
>>     at
>> org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1791)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:407)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:129)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:129)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: Localizer failed but it works after I restart yarn

Posted by daemeon reiydelle <da...@gmail.com>.
My first concern is that temp is a virtual file system and cannot exceed
real memory plus swap space. You may not see file system full errors flow
up as the root cause of the stack trace, but you will see null pointers. So
move your temp directory onto your file system (~/Documents/HDFSTMP or
something) and retry.



*.......*






*“Life should not be a journey to the grave with the intention of arriving
safely in apretty and well preserved body, but rather to skid in broadside
in a cloud of smoke,thoroughly used up, totally worn out, and loudly
proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
(+1) 415.501.0198London (+44) (0) 20 8144 9872*

On Thu, Feb 5, 2015 at 7:36 PM, Jeff Zhang <zj...@gmail.com> wrote:

> BTW, I am using hadoop 2.6 and it's a single node cluster on mac.
>
>
>
> On Fri, Feb 6, 2015 at 11:35 AM, Jeff Zhang <zj...@gmail.com> wrote:
>
>> I intermittently meet this issue of Localizer failed, but after I restart
>> the yarn, then the problem is gone. Is this a known issue ?
>>
>> Here's log in node manager
>>
>>
>> 2015-02-06 11:31:24,898 INFO
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
>> Localizer CWD set to
>> /tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
>> =
>> file:/tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
>> 2015-02-06 11:31:24,906 INFO
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
>> Localizer failed
>> java.lang.NullPointerException
>>     at java.util.zip.ZipFile.getZipEntry(ZipFile.java:531)
>>     at java.util.zip.ZipFile.access$900(ZipFile.java:56)
>>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:513)
>>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:483)
>>     at java.util.jar.JarFile$1.nextElement(JarFile.java:244)
>>     at java.util.jar.JarFile$1.nextElement(JarFile.java:239)
>>     at sun.misc.URLClassPath$JarLoader.validIndex(URLClassPath.java:803)
>>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:937)
>>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:848)
>>     at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:818)
>>     at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
>>     at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
>>     at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
>>     at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
>>     at
>> java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
>>     at org.apache.xerces.parsers.SecuritySupport$6.run(Unknown Source)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at
>> org.apache.xerces.parsers.SecuritySupport.getResourceAsStream(Unknown
>> Source)
>>     at
>> org.apache.xerces.parsers.ObjectFactory.findJarServiceProvider(Unknown
>> Source)
>>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown
>> Source)
>>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown
>> Source)
>>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>>     at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source)
>>     at
>> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
>> Source)
>>     at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2402)
>>     at
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2364)
>>     at
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2281)
>>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:1019)
>>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:991)
>>     at
>> org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1791)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:407)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:129)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:129)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: Localizer failed but it works after I restart yarn

Posted by daemeon reiydelle <da...@gmail.com>.
My first concern is that temp is a virtual file system and cannot exceed
real memory plus swap space. You may not see file system full errors flow
up as the root cause of the stack trace, but you will see null pointers. So
move your temp directory onto your file system (~/Documents/HDFSTMP or
something) and retry.



*.......*






*“Life should not be a journey to the grave with the intention of arriving
safely in apretty and well preserved body, but rather to skid in broadside
in a cloud of smoke,thoroughly used up, totally worn out, and loudly
proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
(+1) 415.501.0198London (+44) (0) 20 8144 9872*

On Thu, Feb 5, 2015 at 7:36 PM, Jeff Zhang <zj...@gmail.com> wrote:

> BTW, I am using hadoop 2.6 and it's a single node cluster on mac.
>
>
>
> On Fri, Feb 6, 2015 at 11:35 AM, Jeff Zhang <zj...@gmail.com> wrote:
>
>> I intermittently meet this issue of Localizer failed, but after I restart
>> the yarn, then the problem is gone. Is this a known issue ?
>>
>> Here's log in node manager
>>
>>
>> 2015-02-06 11:31:24,898 INFO
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
>> Localizer CWD set to
>> /tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
>> =
>> file:/tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
>> 2015-02-06 11:31:24,906 INFO
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
>> Localizer failed
>> java.lang.NullPointerException
>>     at java.util.zip.ZipFile.getZipEntry(ZipFile.java:531)
>>     at java.util.zip.ZipFile.access$900(ZipFile.java:56)
>>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:513)
>>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:483)
>>     at java.util.jar.JarFile$1.nextElement(JarFile.java:244)
>>     at java.util.jar.JarFile$1.nextElement(JarFile.java:239)
>>     at sun.misc.URLClassPath$JarLoader.validIndex(URLClassPath.java:803)
>>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:937)
>>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:848)
>>     at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:818)
>>     at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
>>     at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
>>     at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
>>     at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
>>     at
>> java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
>>     at org.apache.xerces.parsers.SecuritySupport$6.run(Unknown Source)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at
>> org.apache.xerces.parsers.SecuritySupport.getResourceAsStream(Unknown
>> Source)
>>     at
>> org.apache.xerces.parsers.ObjectFactory.findJarServiceProvider(Unknown
>> Source)
>>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown
>> Source)
>>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown
>> Source)
>>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>>     at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source)
>>     at
>> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
>> Source)
>>     at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2402)
>>     at
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2364)
>>     at
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2281)
>>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:1019)
>>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:991)
>>     at
>> org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1791)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:407)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:129)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:129)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: Localizer failed but it works after I restart yarn

Posted by Jeff Zhang <zj...@gmail.com>.
BTW, I am using hadoop 2.6 and it's a single node cluster on mac.



On Fri, Feb 6, 2015 at 11:35 AM, Jeff Zhang <zj...@gmail.com> wrote:

> I intermittently meet this issue of Localizer failed, but after I restart
> the yarn, then the problem is gone. Is this a known issue ?
>
> Here's log in node manager
>
>
> 2015-02-06 11:31:24,898 INFO
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
> Localizer CWD set to
> /tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
> =
> file:/tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
> 2015-02-06 11:31:24,906 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
> Localizer failed
> java.lang.NullPointerException
>     at java.util.zip.ZipFile.getZipEntry(ZipFile.java:531)
>     at java.util.zip.ZipFile.access$900(ZipFile.java:56)
>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:513)
>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:483)
>     at java.util.jar.JarFile$1.nextElement(JarFile.java:244)
>     at java.util.jar.JarFile$1.nextElement(JarFile.java:239)
>     at sun.misc.URLClassPath$JarLoader.validIndex(URLClassPath.java:803)
>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:937)
>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:848)
>     at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:818)
>     at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
>     at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
>     at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
>     at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
>     at java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
>     at org.apache.xerces.parsers.SecuritySupport$6.run(Unknown Source)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at
> org.apache.xerces.parsers.SecuritySupport.getResourceAsStream(Unknown
> Source)
>     at
> org.apache.xerces.parsers.ObjectFactory.findJarServiceProvider(Unknown
> Source)
>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>     at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source)
>     at
> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
> Source)
>     at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2402)
>     at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2364)
>     at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2281)
>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:1019)
>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:991)
>     at
> org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1791)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:407)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:129)
>     at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:129)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
Best Regards

Jeff Zhang

Re: Localizer failed but it works after I restart yarn

Posted by Jeff Zhang <zj...@gmail.com>.
BTW, I am using hadoop 2.6 and it's a single node cluster on mac.



On Fri, Feb 6, 2015 at 11:35 AM, Jeff Zhang <zj...@gmail.com> wrote:

> I intermittently meet this issue of Localizer failed, but after I restart
> the yarn, then the problem is gone. Is this a known issue ?
>
> Here's log in node manager
>
>
> 2015-02-06 11:31:24,898 INFO
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
> Localizer CWD set to
> /tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
> =
> file:/tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
> 2015-02-06 11:31:24,906 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
> Localizer failed
> java.lang.NullPointerException
>     at java.util.zip.ZipFile.getZipEntry(ZipFile.java:531)
>     at java.util.zip.ZipFile.access$900(ZipFile.java:56)
>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:513)
>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:483)
>     at java.util.jar.JarFile$1.nextElement(JarFile.java:244)
>     at java.util.jar.JarFile$1.nextElement(JarFile.java:239)
>     at sun.misc.URLClassPath$JarLoader.validIndex(URLClassPath.java:803)
>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:937)
>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:848)
>     at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:818)
>     at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
>     at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
>     at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
>     at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
>     at java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
>     at org.apache.xerces.parsers.SecuritySupport$6.run(Unknown Source)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at
> org.apache.xerces.parsers.SecuritySupport.getResourceAsStream(Unknown
> Source)
>     at
> org.apache.xerces.parsers.ObjectFactory.findJarServiceProvider(Unknown
> Source)
>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>     at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source)
>     at
> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
> Source)
>     at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2402)
>     at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2364)
>     at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2281)
>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:1019)
>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:991)
>     at
> org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1791)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:407)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:129)
>     at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:129)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
Best Regards

Jeff Zhang

Re: Localizer failed but it works after I restart yarn

Posted by Jeff Zhang <zj...@gmail.com>.
BTW, I am using hadoop 2.6 and it's a single node cluster on mac.



On Fri, Feb 6, 2015 at 11:35 AM, Jeff Zhang <zj...@gmail.com> wrote:

> I intermittently meet this issue of Localizer failed, but after I restart
> the yarn, then the problem is gone. Is this a known issue ?
>
> Here's log in node manager
>
>
> 2015-02-06 11:31:24,898 INFO
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
> Localizer CWD set to
> /tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
> =
> file:/tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
> 2015-02-06 11:31:24,906 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
> Localizer failed
> java.lang.NullPointerException
>     at java.util.zip.ZipFile.getZipEntry(ZipFile.java:531)
>     at java.util.zip.ZipFile.access$900(ZipFile.java:56)
>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:513)
>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:483)
>     at java.util.jar.JarFile$1.nextElement(JarFile.java:244)
>     at java.util.jar.JarFile$1.nextElement(JarFile.java:239)
>     at sun.misc.URLClassPath$JarLoader.validIndex(URLClassPath.java:803)
>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:937)
>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:848)
>     at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:818)
>     at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
>     at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
>     at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
>     at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
>     at java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
>     at org.apache.xerces.parsers.SecuritySupport$6.run(Unknown Source)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at
> org.apache.xerces.parsers.SecuritySupport.getResourceAsStream(Unknown
> Source)
>     at
> org.apache.xerces.parsers.ObjectFactory.findJarServiceProvider(Unknown
> Source)
>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>     at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source)
>     at
> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
> Source)
>     at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2402)
>     at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2364)
>     at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2281)
>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:1019)
>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:991)
>     at
> org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1791)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:407)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:129)
>     at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:129)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
Best Regards

Jeff Zhang

Re: Localizer failed but it works after I restart yarn

Posted by Jeff Zhang <zj...@gmail.com>.
BTW, I am using hadoop 2.6 and it's a single node cluster on mac.



On Fri, Feb 6, 2015 at 11:35 AM, Jeff Zhang <zj...@gmail.com> wrote:

> I intermittently meet this issue of Localizer failed, but after I restart
> the yarn, then the problem is gone. Is this a known issue ?
>
> Here's log in node manager
>
>
> 2015-02-06 11:31:24,898 INFO
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
> Localizer CWD set to
> /tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
> =
> file:/tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
> 2015-02-06 11:31:24,906 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
> Localizer failed
> java.lang.NullPointerException
>     at java.util.zip.ZipFile.getZipEntry(ZipFile.java:531)
>     at java.util.zip.ZipFile.access$900(ZipFile.java:56)
>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:513)
>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:483)
>     at java.util.jar.JarFile$1.nextElement(JarFile.java:244)
>     at java.util.jar.JarFile$1.nextElement(JarFile.java:239)
>     at sun.misc.URLClassPath$JarLoader.validIndex(URLClassPath.java:803)
>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:937)
>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:848)
>     at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:818)
>     at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
>     at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
>     at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
>     at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
>     at java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
>     at org.apache.xerces.parsers.SecuritySupport$6.run(Unknown Source)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at
> org.apache.xerces.parsers.SecuritySupport.getResourceAsStream(Unknown
> Source)
>     at
> org.apache.xerces.parsers.ObjectFactory.findJarServiceProvider(Unknown
> Source)
>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>     at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source)
>     at
> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
> Source)
>     at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2402)
>     at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2364)
>     at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2281)
>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:1019)
>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:991)
>     at
> org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1791)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:407)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:129)
>     at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:129)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
Best Regards

Jeff Zhang