You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by ch huang <ju...@gmail.com> on 2014/01/03 08:44:08 UTC
issue about running hive MR job in hadoop
hi, i submit a MR job through hive ,but when it run stage-2 ,it failed why?
it seems permission problem ,but i do not know which dir cause the problem
Application application_1388730279827_0035 failed 1 times due to AM
Container for appattempt_1388730279827_0035_000001 exited with exitCode:
-1000 due to: EPERM: Operation not permitted at
org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:581)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:388)
at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1041) at
org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:150)
at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:190) at
org.apache.hadoop.fs.FileContext$4.next(FileContext.java:698) at
org.apache.hadoop.fs.FileContext$4.next(FileContext.java:695) at
org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2325)
at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:695) at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:385)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:130)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:861)
.Failing this attempt.. Failing the application.
Re: issue about running hive MR job in hadoop
Posted by ch huang <ju...@gmail.com>.
yes i checked the code ,and find the Exception from
lfs.mkdir(userFileCacheDir, null, false);
also find the AM located in CHBM224 ,all will failed but,AM located in
CHBM223,all success
in CHBM224
# ls -l /data/mrlocal/1/yarn/
total 8
drwxrwxrwx 5 yarn yarn 4096 Nov 5 20:50 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:57 logs
# ls -l /data/mrlocal/2/yarn/
total 8
drwxrwxrwx 5 yarn yarn 4096 Nov 5 20:50 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:57 logs
in CHBM223
# ls /data/mrlocal/1/yarn/ -l
total 8
drwxr-xr-x 5 yarn yarn 4096 Nov 5 20:51 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:46 logs
# ls /data/mrlocal/2/yarn/ -l
total 8
drwxr-xr-x 5 yarn yarn 4096 Nov 5 20:51 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:46 logs
i also find if i let abnormal node (CHBM224) run and shutdown the other
normal node ,when i submit a MR job use hive ,and the dir
"/data/mrlocal/2/yarn/local/usercache/hive/filecache" ,it's mode will flush
to 710 ,even i change the file to 755,but i test on a normal node (open one
normal node ,and shutdown others) ,the dir mode will not changed
# ls -l /data/mrlocal/2/yarn/local/usercache/hive/
total 16
drwx--x--- 7 yarn yarn 4096 Jan 3 16:30 appcache
drwx--x--- 148 yarn yarn 12288 Jan 3 10:03 filecache
On Fri, Jan 3, 2014 at 3:52 PM, Bing Jiang <ji...@gmail.com> wrote:
> Could you check your yarn-local directory authority? From the diagnosis,
> error occurs at mkdir in local directory.
> I guess something wrong with local direcotry which is set as yarn local
> dir.
>
>
>
> 2014/1/3 ch huang <ju...@gmail.com>
>
>> hi, i submit a MR job through hive ,but when it run stage-2 ,it failed
>> why?
>>
>> it seems permission problem ,but i do not know which dir cause the
>> problem
>>
>> Application application_1388730279827_0035 failed 1 times due to AM
>> Container for appattempt_1388730279827_0035_000001 exited with exitCode:
>> -1000 due to: EPERM: Operation not permitted at
>> org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at
>> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:581)
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:388)
>> at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1041) at
>> org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:150)
>> at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:190) at
>> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:698) at
>> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:695) at
>> org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2325)
>> at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:695) at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:385)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:130)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:861)
>> .Failing this attempt.. Failing the application.
>>
>
>
>
> --
> Bing Jiang
> Tel：(86)134-2619-1361
> weibo: http://weibo.com/jiangbinglover
> BLOG: www.binospace.com
> BLOG: http://blog.sina.com.cn/jiangbinglover
> Focus on distributed computing, HDFS/HBase
>
Re: issue about running hive MR job in hadoop
Posted by ch huang <ju...@gmail.com>.
yes i checked the code ,and find the Exception from
lfs.mkdir(userFileCacheDir, null, false);
also find the AM located in CHBM224 ,all will failed but,AM located in
CHBM223,all success
in CHBM224
# ls -l /data/mrlocal/1/yarn/
total 8
drwxrwxrwx 5 yarn yarn 4096 Nov 5 20:50 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:57 logs
# ls -l /data/mrlocal/2/yarn/
total 8
drwxrwxrwx 5 yarn yarn 4096 Nov 5 20:50 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:57 logs
in CHBM223
# ls /data/mrlocal/1/yarn/ -l
total 8
drwxr-xr-x 5 yarn yarn 4096 Nov 5 20:51 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:46 logs
# ls /data/mrlocal/2/yarn/ -l
total 8
drwxr-xr-x 5 yarn yarn 4096 Nov 5 20:51 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:46 logs
i also find if i let abnormal node (CHBM224) run and shutdown the other
normal node ,when i submit a MR job use hive ,and the dir
"/data/mrlocal/2/yarn/local/usercache/hive/filecache" ,it's mode will flush
to 710 ,even i change the file to 755,but i test on a normal node (open one
normal node ,and shutdown others) ,the dir mode will not changed
# ls -l /data/mrlocal/2/yarn/local/usercache/hive/
total 16
drwx--x--- 7 yarn yarn 4096 Jan 3 16:30 appcache
drwx--x--- 148 yarn yarn 12288 Jan 3 10:03 filecache
On Fri, Jan 3, 2014 at 3:52 PM, Bing Jiang <ji...@gmail.com> wrote:
> Could you check your yarn-local directory authority? From the diagnosis,
> error occurs at mkdir in local directory.
> I guess something wrong with local direcotry which is set as yarn local
> dir.
>
>
>
> 2014/1/3 ch huang <ju...@gmail.com>
>
>> hi, i submit a MR job through hive ,but when it run stage-2 ,it failed
>> why?
>>
>> it seems permission problem ,but i do not know which dir cause the
>> problem
>>
>> Application application_1388730279827_0035 failed 1 times due to AM
>> Container for appattempt_1388730279827_0035_000001 exited with exitCode:
>> -1000 due to: EPERM: Operation not permitted at
>> org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at
>> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:581)
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:388)
>> at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1041) at
>> org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:150)
>> at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:190) at
>> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:698) at
>> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:695) at
>> org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2325)
>> at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:695) at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:385)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:130)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:861)
>> .Failing this attempt.. Failing the application.
>>
>
>
>
> --
> Bing Jiang
> Tel：(86)134-2619-1361
> weibo: http://weibo.com/jiangbinglover
> BLOG: www.binospace.com
> BLOG: http://blog.sina.com.cn/jiangbinglover
> Focus on distributed computing, HDFS/HBase
>
Re: issue about running hive MR job in hadoop
Posted by ch huang <ju...@gmail.com>.
yes i checked the code ,and find the Exception from
lfs.mkdir(userFileCacheDir, null, false);
also find the AM located in CHBM224 ,all will failed but,AM located in
CHBM223,all success
in CHBM224
# ls -l /data/mrlocal/1/yarn/
total 8
drwxrwxrwx 5 yarn yarn 4096 Nov 5 20:50 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:57 logs
# ls -l /data/mrlocal/2/yarn/
total 8
drwxrwxrwx 5 yarn yarn 4096 Nov 5 20:50 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:57 logs
in CHBM223
# ls /data/mrlocal/1/yarn/ -l
total 8
drwxr-xr-x 5 yarn yarn 4096 Nov 5 20:51 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:46 logs
# ls /data/mrlocal/2/yarn/ -l
total 8
drwxr-xr-x 5 yarn yarn 4096 Nov 5 20:51 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:46 logs
i also find if i let abnormal node (CHBM224) run and shutdown the other
normal node ,when i submit a MR job use hive ,and the dir
"/data/mrlocal/2/yarn/local/usercache/hive/filecache" ,it's mode will flush
to 710 ,even i change the file to 755,but i test on a normal node (open one
normal node ,and shutdown others) ,the dir mode will not changed
# ls -l /data/mrlocal/2/yarn/local/usercache/hive/
total 16
drwx--x--- 7 yarn yarn 4096 Jan 3 16:30 appcache
drwx--x--- 148 yarn yarn 12288 Jan 3 10:03 filecache
On Fri, Jan 3, 2014 at 3:52 PM, Bing Jiang <ji...@gmail.com> wrote:
> Could you check your yarn-local directory authority? From the diagnosis,
> error occurs at mkdir in local directory.
> I guess something wrong with local direcotry which is set as yarn local
> dir.
>
>
>
> 2014/1/3 ch huang <ju...@gmail.com>
>
>> hi, i submit a MR job through hive ,but when it run stage-2 ,it failed
>> why?
>>
>> it seems permission problem ,but i do not know which dir cause the
>> problem
>>
>> Application application_1388730279827_0035 failed 1 times due to AM
>> Container for appattempt_1388730279827_0035_000001 exited with exitCode:
>> -1000 due to: EPERM: Operation not permitted at
>> org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at
>> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:581)
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:388)
>> at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1041) at
>> org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:150)
>> at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:190) at
>> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:698) at
>> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:695) at
>> org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2325)
>> at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:695) at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:385)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:130)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:861)
>> .Failing this attempt.. Failing the application.
>>
>
>
>
> --
> Bing Jiang
> Tel：(86)134-2619-1361
> weibo: http://weibo.com/jiangbinglover
> BLOG: www.binospace.com
> BLOG: http://blog.sina.com.cn/jiangbinglover
> Focus on distributed computing, HDFS/HBase
>
Re: issue about running hive MR job in hadoop
Posted by ch huang <ju...@gmail.com>.
yes i checked the code ,and find the Exception from
lfs.mkdir(userFileCacheDir, null, false);
also find the AM located in CHBM224 ,all will failed but,AM located in
CHBM223,all success
in CHBM224
# ls -l /data/mrlocal/1/yarn/
total 8
drwxrwxrwx 5 yarn yarn 4096 Nov 5 20:50 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:57 logs
# ls -l /data/mrlocal/2/yarn/
total 8
drwxrwxrwx 5 yarn yarn 4096 Nov 5 20:50 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:57 logs
in CHBM223
# ls /data/mrlocal/1/yarn/ -l
total 8
drwxr-xr-x 5 yarn yarn 4096 Nov 5 20:51 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:46 logs
# ls /data/mrlocal/2/yarn/ -l
total 8
drwxr-xr-x 5 yarn yarn 4096 Nov 5 20:51 local
drwxr-xr-x 3 yarn yarn 4096 Jan 3 15:46 logs
i also find if i let abnormal node (CHBM224) run and shutdown the other
normal node ,when i submit a MR job use hive ,and the dir
"/data/mrlocal/2/yarn/local/usercache/hive/filecache" ,it's mode will flush
to 710 ,even i change the file to 755,but i test on a normal node (open one
normal node ,and shutdown others) ,the dir mode will not changed
# ls -l /data/mrlocal/2/yarn/local/usercache/hive/
total 16
drwx--x--- 7 yarn yarn 4096 Jan 3 16:30 appcache
drwx--x--- 148 yarn yarn 12288 Jan 3 10:03 filecache
On Fri, Jan 3, 2014 at 3:52 PM, Bing Jiang <ji...@gmail.com> wrote:
> Could you check your yarn-local directory authority? From the diagnosis,
> error occurs at mkdir in local directory.
> I guess something wrong with local direcotry which is set as yarn local
> dir.
>
>
>
> 2014/1/3 ch huang <ju...@gmail.com>
>
>> hi, i submit a MR job through hive ,but when it run stage-2 ,it failed
>> why?
>>
>> it seems permission problem ,but i do not know which dir cause the
>> problem
>>
>> Application application_1388730279827_0035 failed 1 times due to AM
>> Container for appattempt_1388730279827_0035_000001 exited with exitCode:
>> -1000 due to: EPERM: Operation not permitted at
>> org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at
>> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:581)
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:388)
>> at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1041) at
>> org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:150)
>> at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:190) at
>> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:698) at
>> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:695) at
>> org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2325)
>> at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:695) at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:385)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:130)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:861)
>> .Failing this attempt.. Failing the application.
>>
>
>
>
> --
> Bing Jiang
> Tel：(86)134-2619-1361
> weibo: http://weibo.com/jiangbinglover
> BLOG: www.binospace.com
> BLOG: http://blog.sina.com.cn/jiangbinglover
> Focus on distributed computing, HDFS/HBase
>
Re: issue about running hive MR job in hadoop
Posted by Bing Jiang <ji...@gmail.com>.
Could you check your yarn-local directory authority? From the diagnosis,
error occurs at mkdir in local directory.
I guess something wrong with local direcotry which is set as yarn local
dir.
2014/1/3 ch huang <ju...@gmail.com>
> hi, i submit a MR job through hive ,but when it run stage-2 ,it failed why?
>
> it seems permission problem ,but i do not know which dir cause the problem
>
> Application application_1388730279827_0035 failed 1 times due to AM
> Container for appattempt_1388730279827_0035_000001 exited with exitCode:
> -1000 due to: EPERM: Operation not permitted at
> org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:581)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:388)
> at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1041) at
> org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:150)
> at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:190) at
> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:698) at
> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:695) at
> org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2325)
> at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:695) at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:130)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:861)
> .Failing this attempt.. Failing the application.
>
--
Bing Jiang
Tel：(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase
Re: issue about running hive MR job in hadoop
Posted by Bing Jiang <ji...@gmail.com>.
Could you check your yarn-local directory authority? From the diagnosis,
error occurs at mkdir in local directory.
I guess something wrong with local direcotry which is set as yarn local
dir.
2014/1/3 ch huang <ju...@gmail.com>
> hi, i submit a MR job through hive ,but when it run stage-2 ,it failed why?
>
> it seems permission problem ,but i do not know which dir cause the problem
>
> Application application_1388730279827_0035 failed 1 times due to AM
> Container for appattempt_1388730279827_0035_000001 exited with exitCode:
> -1000 due to: EPERM: Operation not permitted at
> org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:581)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:388)
> at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1041) at
> org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:150)
> at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:190) at
> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:698) at
> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:695) at
> org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2325)
> at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:695) at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:130)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:861)
> .Failing this attempt.. Failing the application.
>
--
Bing Jiang
Tel：(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase
Re: issue about running hive MR job in hadoop
Posted by Bing Jiang <ji...@gmail.com>.
Could you check your yarn-local directory authority? From the diagnosis,
error occurs at mkdir in local directory.
I guess something wrong with local direcotry which is set as yarn local
dir.
2014/1/3 ch huang <ju...@gmail.com>
> hi, i submit a MR job through hive ,but when it run stage-2 ,it failed why?
>
> it seems permission problem ,but i do not know which dir cause the problem
>
> Application application_1388730279827_0035 failed 1 times due to AM
> Container for appattempt_1388730279827_0035_000001 exited with exitCode:
> -1000 due to: EPERM: Operation not permitted at
> org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:581)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:388)
> at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1041) at
> org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:150)
> at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:190) at
> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:698) at
> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:695) at
> org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2325)
> at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:695) at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:130)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:861)
> .Failing this attempt.. Failing the application.
>
--
Bing Jiang
Tel：(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase
Re: issue about running hive MR job in hadoop
Posted by Bing Jiang <ji...@gmail.com>.
Could you check your yarn-local directory authority? From the diagnosis,
error occurs at mkdir in local directory.
I guess something wrong with local direcotry which is set as yarn local
dir.
2014/1/3 ch huang <ju...@gmail.com>
> hi, i submit a MR job through hive ,but when it run stage-2 ,it failed why?
>
> it seems permission problem ,but i do not know which dir cause the problem
>
> Application application_1388730279827_0035 failed 1 times due to AM
> Container for appattempt_1388730279827_0035_000001 exited with exitCode:
> -1000 due to: EPERM: Operation not permitted at
> org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:581)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:388)
> at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1041) at
> org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:150)
> at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:190) at
> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:698) at
> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:695) at
> org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2325)
> at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:695) at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:130)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:861)
> .Failing this attempt.. Failing the application.
>
--
Bing Jiang
Tel：(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase