You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Amit <mr...@gmail.com> on 2013/02/20 14:18:28 UTC
Not able to start JobTracker in cygwin environment
I have configured cygwin with Windows 7 64bit and trying to run hadoop on
it.
Was successful to run namenode, datanode, secondarynamenode and jobtracker,
but facing problem with tasktracker.
Here is the log -
13/02/20 18:37:04 INFO mapred.TaskTracker: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting TaskTracker
STARTUP_MSG: host = Franklin/192.168.33.241
STARTUP_MSG: args = []
STARTUP_MSG: version = 1.0.4
STARTUP_MSG: build =
https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r
1393290; compiled by 'hortonfo' on Wed Oct 3 05:13:58 UTC 2012
************************************************************/
13/02/20 18:37:04 INFO impl.MetricsConfig: loaded properties from
hadoop-metrics2.properties
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source
MetricsSystem,sub=Stats registered.
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: Scheduled snapshot period at
10 second(s).
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: TaskTracker metrics system
started
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source ugi
registered.
13/02/20 18:37:04 WARN impl.MetricsSystemImpl: Source name ugi already
exists!
13/02/20 18:37:05 INFO mortbay.log: Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
org.mortbay.log.Slf4jLog
13/02/20 18:37:05 INFO http.HttpServer: Added global filtersafety
(class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
13/02/20 18:37:05 INFO mapred.TaskLogsTruncater: Initializing logs'
truncater with mapRetainSize=-1 and reduceRetainSize=-1
13/02/20 18:37:05 INFO mapred.TaskTracker: Starting tasktracker with owner
as Admin
13/02/20 18:37:05 INFO mapred.TaskTracker: Good mapred local directories
are: /app/hadoop/tmp/mapred/local
13/02/20 18:37:05 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
13/02/20 18:37:05 ERROR mapred.TaskTracker: Can not start task tracker
because java.io.IOException: Failed to set permissions of path:
\app\hadoop\tmp\mapred\local\ttprivate to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at
org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at
org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:728)
at
org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1459)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742)
13/02/20 18:37:05 INFO mapred.TaskTracker: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down TaskTracker at Franklin/192.168.33.241
************************************************************/
Tried after setting permission of \app\hadoop\tmp\mapred\local\ttprivate to
0700 using cygwin shell but getting the same exception. Can you help me?
--
Thanks,
Amit
RE: Not able to start JobTracker in cygwin environment
Posted by Brad Sarsfield <br...@bing.com>.
I'd recommend picking up branch-trunk-win; (a call for vote for merge into trunk is going to happen soon after the work is complete and precommit built is clean; then these changes will be in trunk (!) ).
This removes the Cygwin dependency for running Hadoop on Windows. This work is being done under https://issues.apache.org/jira/browse/HADOOP-8562
It includes
1. Necessary changes in Hadoop to run natively on Windows. These changes handle differences in platforms related to path names, process/task management etc.
2. Addition of winutils tools for managing file permissions and ownership, user group mapping, hardlinks, symbolic links, chmod, disk utilization, and process/task management.
3. Added cmd scripts equivalent to existing shell scripts hadoop-daemon.sh, start and stop scripts.
4. Addition of block placement policy implementation to support cloud environment, more specifically Azure.
~Brad
From: Amit [mailto:mramitkrde@gmail.com]
Sent: Wednesday, February 20, 2013 5:18 AM
To: user@hadoop.apache.org
Subject: Not able to start JobTracker in cygwin environment
I have configured cygwin with Windows 7 64bit and trying to run hadoop on it.
Was successful to run namenode, datanode, secondarynamenode and jobtracker, but facing problem with tasktracker.
Here is the log -
13/02/20 18:37:04 INFO mapred.TaskTracker: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting TaskTracker
STARTUP_MSG: host = Franklin/192.168.33.241
STARTUP_MSG: args = []
STARTUP_MSG: version = 1.0.4
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct 3 05:13:58 UTC 2012
************************************************************/
13/02/20 18:37:04 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: TaskTracker metrics system started
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source ugi registered.
13/02/20 18:37:04 WARN impl.MetricsSystemImpl: Source name ugi already exists!
13/02/20 18:37:05 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
13/02/20 18:37:05 INFO http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
13/02/20 18:37:05 INFO mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
13/02/20 18:37:05 INFO mapred.TaskTracker: Starting tasktracker with owner as Admin
13/02/20 18:37:05 INFO mapred.TaskTracker: Good mapred local directories are: /app/hadoop/tmp/mapred/local
13/02/20 18:37:05 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/02/20 18:37:05 ERROR mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \app\hadoop\tmp\mapred\local\ttprivate to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:728)
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1459)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742)
13/02/20 18:37:05 INFO mapred.TaskTracker: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down TaskTracker at Franklin/192.168.33.241
************************************************************/
Tried after setting permission of \app\hadoop\tmp\mapred\local\ttprivate to 0700 using cygwin shell but getting the same exception. Can you help me?
--
Thanks,
Amit
RE: Not able to start JobTracker in cygwin environment
Posted by Brad Sarsfield <br...@bing.com>.
I'd recommend picking up branch-trunk-win; (a call for vote for merge into trunk is going to happen soon after the work is complete and precommit built is clean; then these changes will be in trunk (!) ).
This removes the Cygwin dependency for running Hadoop on Windows. This work is being done under https://issues.apache.org/jira/browse/HADOOP-8562
It includes
1. Necessary changes in Hadoop to run natively on Windows. These changes handle differences in platforms related to path names, process/task management etc.
2. Addition of winutils tools for managing file permissions and ownership, user group mapping, hardlinks, symbolic links, chmod, disk utilization, and process/task management.
3. Added cmd scripts equivalent to existing shell scripts hadoop-daemon.sh, start and stop scripts.
4. Addition of block placement policy implementation to support cloud environment, more specifically Azure.
~Brad
From: Amit [mailto:mramitkrde@gmail.com]
Sent: Wednesday, February 20, 2013 5:18 AM
To: user@hadoop.apache.org
Subject: Not able to start JobTracker in cygwin environment
I have configured cygwin with Windows 7 64bit and trying to run hadoop on it.
Was successful to run namenode, datanode, secondarynamenode and jobtracker, but facing problem with tasktracker.
Here is the log -
13/02/20 18:37:04 INFO mapred.TaskTracker: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting TaskTracker
STARTUP_MSG: host = Franklin/192.168.33.241
STARTUP_MSG: args = []
STARTUP_MSG: version = 1.0.4
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct 3 05:13:58 UTC 2012
************************************************************/
13/02/20 18:37:04 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: TaskTracker metrics system started
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source ugi registered.
13/02/20 18:37:04 WARN impl.MetricsSystemImpl: Source name ugi already exists!
13/02/20 18:37:05 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
13/02/20 18:37:05 INFO http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
13/02/20 18:37:05 INFO mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
13/02/20 18:37:05 INFO mapred.TaskTracker: Starting tasktracker with owner as Admin
13/02/20 18:37:05 INFO mapred.TaskTracker: Good mapred local directories are: /app/hadoop/tmp/mapred/local
13/02/20 18:37:05 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/02/20 18:37:05 ERROR mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \app\hadoop\tmp\mapred\local\ttprivate to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:728)
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1459)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742)
13/02/20 18:37:05 INFO mapred.TaskTracker: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down TaskTracker at Franklin/192.168.33.241
************************************************************/
Tried after setting permission of \app\hadoop\tmp\mapred\local\ttprivate to 0700 using cygwin shell but getting the same exception. Can you help me?
--
Thanks,
Amit
RE: Not able to start JobTracker in cygwin environment
Posted by Brad Sarsfield <br...@bing.com>.
I'd recommend picking up branch-trunk-win; (a call for vote for merge into trunk is going to happen soon after the work is complete and precommit built is clean; then these changes will be in trunk (!) ).
This removes the Cygwin dependency for running Hadoop on Windows. This work is being done under https://issues.apache.org/jira/browse/HADOOP-8562
It includes
1. Necessary changes in Hadoop to run natively on Windows. These changes handle differences in platforms related to path names, process/task management etc.
2. Addition of winutils tools for managing file permissions and ownership, user group mapping, hardlinks, symbolic links, chmod, disk utilization, and process/task management.
3. Added cmd scripts equivalent to existing shell scripts hadoop-daemon.sh, start and stop scripts.
4. Addition of block placement policy implementation to support cloud environment, more specifically Azure.
~Brad
From: Amit [mailto:mramitkrde@gmail.com]
Sent: Wednesday, February 20, 2013 5:18 AM
To: user@hadoop.apache.org
Subject: Not able to start JobTracker in cygwin environment
I have configured cygwin with Windows 7 64bit and trying to run hadoop on it.
Was successful to run namenode, datanode, secondarynamenode and jobtracker, but facing problem with tasktracker.
Here is the log -
13/02/20 18:37:04 INFO mapred.TaskTracker: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting TaskTracker
STARTUP_MSG: host = Franklin/192.168.33.241
STARTUP_MSG: args = []
STARTUP_MSG: version = 1.0.4
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct 3 05:13:58 UTC 2012
************************************************************/
13/02/20 18:37:04 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: TaskTracker metrics system started
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source ugi registered.
13/02/20 18:37:04 WARN impl.MetricsSystemImpl: Source name ugi already exists!
13/02/20 18:37:05 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
13/02/20 18:37:05 INFO http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
13/02/20 18:37:05 INFO mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
13/02/20 18:37:05 INFO mapred.TaskTracker: Starting tasktracker with owner as Admin
13/02/20 18:37:05 INFO mapred.TaskTracker: Good mapred local directories are: /app/hadoop/tmp/mapred/local
13/02/20 18:37:05 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/02/20 18:37:05 ERROR mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \app\hadoop\tmp\mapred\local\ttprivate to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:728)
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1459)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742)
13/02/20 18:37:05 INFO mapred.TaskTracker: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down TaskTracker at Franklin/192.168.33.241
************************************************************/
Tried after setting permission of \app\hadoop\tmp\mapred\local\ttprivate to 0700 using cygwin shell but getting the same exception. Can you help me?
--
Thanks,
Amit
RE: Not able to start JobTracker in cygwin environment
Posted by Brad Sarsfield <br...@bing.com>.
I'd recommend picking up branch-trunk-win; (a call for vote for merge into trunk is going to happen soon after the work is complete and precommit built is clean; then these changes will be in trunk (!) ).
This removes the Cygwin dependency for running Hadoop on Windows. This work is being done under https://issues.apache.org/jira/browse/HADOOP-8562
It includes
1. Necessary changes in Hadoop to run natively on Windows. These changes handle differences in platforms related to path names, process/task management etc.
2. Addition of winutils tools for managing file permissions and ownership, user group mapping, hardlinks, symbolic links, chmod, disk utilization, and process/task management.
3. Added cmd scripts equivalent to existing shell scripts hadoop-daemon.sh, start and stop scripts.
4. Addition of block placement policy implementation to support cloud environment, more specifically Azure.
~Brad
From: Amit [mailto:mramitkrde@gmail.com]
Sent: Wednesday, February 20, 2013 5:18 AM
To: user@hadoop.apache.org
Subject: Not able to start JobTracker in cygwin environment
I have configured cygwin with Windows 7 64bit and trying to run hadoop on it.
Was successful to run namenode, datanode, secondarynamenode and jobtracker, but facing problem with tasktracker.
Here is the log -
13/02/20 18:37:04 INFO mapred.TaskTracker: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting TaskTracker
STARTUP_MSG: host = Franklin/192.168.33.241
STARTUP_MSG: args = []
STARTUP_MSG: version = 1.0.4
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct 3 05:13:58 UTC 2012
************************************************************/
13/02/20 18:37:04 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: TaskTracker metrics system started
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source ugi registered.
13/02/20 18:37:04 WARN impl.MetricsSystemImpl: Source name ugi already exists!
13/02/20 18:37:05 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
13/02/20 18:37:05 INFO http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
13/02/20 18:37:05 INFO mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
13/02/20 18:37:05 INFO mapred.TaskTracker: Starting tasktracker with owner as Admin
13/02/20 18:37:05 INFO mapred.TaskTracker: Good mapred local directories are: /app/hadoop/tmp/mapred/local
13/02/20 18:37:05 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/02/20 18:37:05 ERROR mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \app\hadoop\tmp\mapred\local\ttprivate to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:728)
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1459)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742)
13/02/20 18:37:05 INFO mapred.TaskTracker: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down TaskTracker at Franklin/192.168.33.241
************************************************************/
Tried after setting permission of \app\hadoop\tmp\mapred\local\ttprivate to 0700 using cygwin shell but getting the same exception. Can you help me?
--
Thanks,
Amit