You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Carlo Sartiani <sa...@gmail.com> on 2014/03/13 12:34:23 UTC
Problems when running Giraph1.0 on Hadoop 2.0.5alpha.gphd2.1.0
Hi all.
I am encountering a strange problem when running a Giraph application on
a cluster with Hadoop 2.0.5alpha.gphd2.1.0 (Pivotal Hadoop distribution).
Our application is based on Giraph 1.0 and is executed in mapreduce
compatibility mode rather than in yarn mode. The application works
smoothly on a standard Hadoop 2.0.5alpha distribution in our local
cluster. However, when moved to an external cluster with Pivotal Hadoop
2.0.5alpha, it returns the error described by the following log:
2014-03-12 17:33:21,489 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for
application appattempt_1388715708455_1597_000001
2014-03-1217:33:21,722 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-03-12 17:33:21,723 WARN [main]
org.apache.hadoop.conf.Configuration: job.xml:an attempt to override
final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-03-12 17:33:21,989 WARN [main]
org.apache.hadoop.conf.Configuration: job.xml:an attempt to override
final parameter: mapreduce.job.end-notification.max.retry.interval;
Ignoring.
2014-03-12 17:33:21,989 WARN [main]
org.apache.hadoop.conf.Configuration: job.xml:an attempt to override
final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-03-12 17:33:22,005 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster:
jobSubmitDir=file:/data5/phd/yarn/nm-local-dir/usercache/csartiani/appcache/application_1388715708455_1597/container_1388715708455_1597_01_000001/jobSubmitDir
jobTokenFile=file:/data5/phd/yarn/nm-local-dir/usercache/csartiani/appcache/application_1388715708455_1597/container_1388715708455_1597_01_000001/jobSubmitDir/appTokens
2014-03-12 17:33:22,005 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: AM Retries: 1 attempt
num: 1 is last retry: true
2014-03-12 17:33:22,013 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred
newApiCommitter.
2014-03-12 17:33:22,315 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in
config null
2014-03-12 17:33:22,377 WARN [main]
org.apache.giraph.bsp.BspOutputFormat: getOutputCommitter: Returning
ImmutableOutputCommiter (does nothing).
2014-03-12 17:33:22,377 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is
org.apache.giraph.bsp.ImmutableOutputCommitter
2014-03-12 17:33:22,377 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Not starting
RecoveryService: recoveryEnabled: true recoverySupportedByCommitter:
false ApplicationAttemptID: 1
2014-03-1217:33:22,394 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.jobhistory.EventType for class
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2014-03-12 17:33:22,395 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
2014-03-12 17:33:22,396 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
2014-03-12 17:33:22,396 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for
class
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
2014-03-12 17:33:22,397 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2014-03-12 17:33:22,401 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for
class
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
2014-03-12 17:33:22,402 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for
class
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2014-03-12 17:33:22,403 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType
for class
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
2014-03-12 17:33:22,403 INFO [main]
org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is
inited.
2014-03-12 17:33:22,403 INFO [main]
org.apache.hadoop.yarn.service.AbstractService: Service:MRClientService
is inited.
2014-03-12 17:33:22,403 INFO [main]
org.apache.hadoop.yarn.service.AbstractService:
Service:CommitterEventHandler is inited.
2014-03-12 17:33:22,405 INFO [main]
org.apache.hadoop.yarn.service.AbstractService:
Service:TaskHeartbeatHandler is inited.
2014-03-12 17:33:22,405 INFO [main]
org.apache.hadoop.yarn.service.AbstractService:
Service:org.apache.hadoop.mapred.TaskAttemptListenerImpl is inited.
2014-03-12 17:33:22,405 INFO [main]
org.apache.hadoop.yarn.service.AbstractService:
Service:org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator
is inited.
2014-03-12 17:33:22,405 INFO [main]
org.apache.hadoop.yarn.service.AbstractService:
Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
is inited.
2014-03-12 17:33:22,405 INFO [main]
org.apache.hadoop.yarn.service.AbstractService:
Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
is inited.
2014-03-12 17:33:22,405 INFO [main]
org.apache.hadoop.yarn.service.AbstractService:
Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$StagingDirCleaningService
is inited.
2014-03-12 17:33:22,411 INFO [main]
org.apache.hadoop.yarn.service.AbstractService:
Service:JobHistoryEventHandler is inited.
2014-03-12 17:33:22,411 INFO [main]
org.apache.hadoop.yarn.service.AbstractService:
Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster is inited.
2014-03-12 17:33:22,460 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for
class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2014-03-12 17:33:22,653 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
hadoop-metrics2.properties
2014-03-12 17:33:22,696 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
period at 10 second(s).
2014-03-12 17:33:22,696 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics
system started
2014-03-12 17:33:22,704 INFO [main]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token
for job_1388715708455_1597 to jobTokenSecretManager
2014-03-12 17:33:22,785 INFO [main]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing
job_1388715708455_1597 because: not enabled; too much RAM;
2014-03-12 17:33:22,799 INFO [main]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job
job_1388715708455_1597 = 0. Number of splits = 1
2014-03-1217:33:22,799 INFO [main]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces
for job job_1388715708455_1597 = 0
2014-03-12 17:33:22,799 INFO [main]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
job_1388715708455_1597Job Transitioned from NEW to INITED
2014-03-12 17:33:22,799 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching
normal, non-uberized, multi-container job job_1388715708455_1597.
2014-03-12 17:33:22,799 INFO [main]
org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is
started.
2014-03-12 17:33:22,801 INFO [AsyncDispatcher event handler]
org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator:
JOB_CREATE job_1388715708455_1597
2014-03-12 17:33:22,833 INFO [Socket Reader #1 for port 43578]
org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 43578
2014-03-12 17:33:22,846 INFO [main]
org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding
protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the
server
2014-03-12 17:33:22,847 INFO [IPC Server Responder]
org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2014-03-12 17:33:22,847 INFO [IPC Server listener on 43578]
org.apache.hadoop.ipc.Server: IPC Server listener on 43578: starting
2014-03-12 17:33:22,847 INFO [main]
org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated
MRClientService at node0801/10.1.41.1:43578
2014-03-12 17:33:22,867 ERROR [main]
org.apache.hadoop.yarn.service.CompositeService: Error starting services
org.apache.hadoop.mapreduce.v2.app.MRAppMaster
java.lang.NoSuchMethodError:
org.apache.hadoop.http.HttpServer.<init>(Ljava/lang/String;Ljava/lang/String;IZLorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/security/authorize/AccessControlList;Lorg/mortbay/jetty/Connector;[Ljava/lang/String;)V
at
org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:196)
at
org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:149)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1022)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1276)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1272)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1228)
2014-03-12 17:33:22,868 INFO [main] org.apache.hadoop.ipc.Server:
Stopping server on 43578
2014-03-12 17:33:22,869 INFO [IPC Server listener on 43578]
org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 43578
2014-03-12 17:33:22,869 INFO [main]
org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is
stopped.
2014-03-12 17:33:22,869 INFO [IPC Server Responder]
org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2014-03-12 17:33:22,869 FATAL [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
org.apache.hadoop.yarn.YarnException: Failed to Start
org.apache.hadoop.mapreduce.v2.app.MRAppMaster
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1022)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1276)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1272)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1228)
Caused by: java.lang.NoSuchMethodError:
org.apache.hadoop.http.HttpServer.<init>(Ljava/lang/String;Ljava/lang/String;IZLorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/security/authorize/AccessControlList;Lorg/mortbay/jetty/Connector;[Ljava/lang/String;)V
at
org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:196)
at
org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:149)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
... 7 more
2014-03-12 17:33:22,870 INFO [Thread-1]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster received a
signal. Signaling RMCommunicator and JobHistoryEventHandler.
2014-03-12 17:33:22,871 WARN [Thread-1]
org.apache.hadoop.util.ShutdownHookManager: ShutdownHook
'MRAppMasterShutdownHook' failed, java.lang.NullPointerException
java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.setSignalled(MRAppMaster.java:811)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1249)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
The error seems quite strange, as the system is reporting the lack of a
given constructor for org.apache.hadoop.http.HttpServer.
Other applications that we are running on this cluster and that do not
use Giraph work perfectly. We have no control on this cluster, so we
cannot modify any cluster setting.
Do you any idea on how to solve this issue?
Thanks in advance.
Best Regards
Carlo Sartiani
--
"Humanity is overrated", Gregory House, MD
Re: Problems when running Giraph1.0 on Hadoop 2.0.5alpha.gphd2.1.0
Posted by Eli Reisman <ap...@gmail.com>.
Hadoop 2.0.5 and in fact all of Hadoop 2.0.x-alpha versions are no longer
supported in Giraph's Hadoop YARN profile. The decision was made last fall
to move our YARN support to a minimum Hadoop version of 2.2.0 GA. This is
because the API was not stabilized until this version of Hadoop.
tl;dr you will probably only be able to run Giraph on 2.0.5 at this point
by altering the original Giraph YARN implementation on Hadoop 2.0.3 in the
POM profile to run on 2.0.5 instead (I had put up a patch to add this
support before the 2.2.0 migration was chosen) or to run only on MRv1.
On Mon, Mar 17, 2014 at 5:29 AM, Carlo Sartiani <sa...@gmail.com> wrote:
> The cluster uses Pivotal own version of Hadoop 2.0.5alpha. We are
> submitting the Giraph job by using the standard GIRAPH api. The cluster
> tries to activate a mapreducev2 job, but it seems failing in launching the
> web server. The strange thing is that, if we use a "standard" version of
> Hadoop 2.0.5alpha, everything works fine.
>
> This is the piece of code we use to configure and launch the job:
>
> GiraphConfiguration conf = new GiraphConfiguration();
>
>
> conf.setEdgeInputFormatClass(VertigoTextEdgeInputFormat.class);
> conf.setVertexClass(VertigoVertex.class); // vertex class
> conf.VERTEX_EDGES_CLASS.set(conf, VertigoEdges.class);
>
> conf.MASTER_COMPUTE_CLASS.set(conf,
> VertigoExtendedMasterComputation.class);
>
> conf.SPLIT_MASTER_WORKER.set(conf, false);
> conf.STATIC_GRAPH.set(conf, true);
> //conf.setLocalTestMode(true);
> conf.setLocalTestMode(false);
> conf.setWorkerConfiguration(1,1, 100);
> conf.NUM_COMPUTE_THREADS.set(conf, 4);
> String splitValue = conf.get(LINE_TOKENIZE_VALUE,
> LINE_TOKENIZE_VALUE_DEFAULT);
> System.out.println("MAX_WORKERS: " + conf.getMaxWorkers());
> System.out.println("LINE_TOKENIZE_VALUE: " + splitValue);
> conf.setZooKeeperConfiguration(new
> String("node0522:2181,node0722:2181,node0922:2181"));
>
>
>
> GiraphFileInputFormat.setEdgeInputPath(conf, this.inputFile);
> GiraphJob job = new GiraphJob(conf, "corejob");
>
>
> // prepare dir for final results
> try {
> // create a directory where superstep results will be stored
>
> Path resultDir = new
> org.apache.hadoop.fs.Path(this.fileSystem.getHomeDirectory(), "results");
> if (this.fileSystem.exists(resultDir)) {
> this.fileSystem.delete(resultDir, true);
> }
> boolean res = this.fileSystem.mkdirs(resultDir);
>
> }
> catch(Exception e) {
> e.printStackTrace();
> }
>
>
> long start = System.currentTimeMillis();
> boolean result = job.run(true);
> long stop = System.currentTimeMillis();
>
> We have not tried to use Giraph1.1.0, as we were not able to compile it on
> Hadoop2.0.5alpha.
>
> Carlo
>
>
> Eli Reisman <ap...@gmail.com>
> 16 Mar 2014 21:01
> it looks like the Pivotal cluster is running on YARN or trying to launch
> Giraph on YARN from the stack trace?
>
>
>
> Carlo Sartiani <sa...@gmail.com>
> 13 Mar 2014 12:34
> Hi all.
>
> I am encountering a strange problem when running a Giraph application on a
> cluster with Hadoop 2.0.5alpha.gphd2.1.0 (Pivotal Hadoop distribution).
>
> Our application is based on Giraph 1.0 and is executed in mapreduce
> compatibility mode rather than in yarn mode. The application works
> smoothly on a standard Hadoop 2.0.5alpha distribution in our local cluster.
> However, when moved to an external cluster with Pivotal Hadoop 2.0.5alpha,
> it returns the error described by the following log:
>
> 2014-03-12 17:33:21,489 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for
> application appattempt_1388715708455_1597_000001
> 2014-03-12 17:33:21,722 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2014-03-12 17:33:21,723 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 2014-03-12 17:33:21,989 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2014-03-12 17:33:21,989 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 2014-03-12 17:33:22,005 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: jobSubmitDir=
> file:/data5/phd/yarn/nm-local-dir/usercache/csartiani/appcache/application_1388715708455_1597/container_1388715708455_1597_01_000001/jobSubmitDirjobTokenFile=
> file:/data5/phd/yarn/nm-local-dir/usercache/csartiani/appcache/application_1388715708455_1597/container_1388715708455_1597_01_000001/jobSubmitDir/appTokens
> 2014-03-12 17:33:22,005 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: AM Retries: 1 attempt num:
> 1 is last retry: true
> 2014-03-12 17:33:22,013 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred
> newApiCommitter.
> 2014-03-12 17:33:22,315 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in
> config null
> 2014-03-12 17:33:22,377 WARN [main]
> org.apache.giraph.bsp.BspOutputFormat: getOutputCommitter: Returning
> ImmutableOutputCommiter (does nothing).
> 2014-03-12 17:33:22,377 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is
> org.apache.giraph.bsp.ImmutableOutputCommitter
> 2014-03-12 17:33:22,377 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Not starting
> RecoveryService: recoveryEnabled: true recoverySupportedByCommitter: false
> ApplicationAttemptID: 1
> 2014-03-12 17:33:22,394 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.jobhistory.EventType for class
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
> 2014-03-12 17:33:22,395 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
> 2014-03-12 17:33:22,396 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
> 2014-03-12 17:33:22,396 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
> 2014-03-12 17:33:22,397 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
> 2014-03-12 17:33:22,401 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
> 2014-03-12 17:33:22,402 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for
> class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
> 2014-03-12 17:33:22,403 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for
> class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
> 2014-03-12 17:33:22,403 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is
> inited.
> 2014-03-12 17:33:22,403 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService: Service:MRClientService is
> inited.
> 2014-03-12 17:33:22,403 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:CommitterEventHandler is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:TaskHeartbeatHandler is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapred.TaskAttemptListenerImpl is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator is
> inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
> is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
> is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$StagingDirCleaningService
> is inited.
> 2014-03-12 17:33:22,411 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:JobHistoryEventHandler is inited.
> 2014-03-12 17:33:22,411 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster is inited.
> 2014-03-12 17:33:22,460 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
> 2014-03-12 17:33:22,653 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
> hadoop-metrics2.properties
> 2014-03-12 17:33:22,696 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
> period at 10 second(s).
> 2014-03-12 17:33:22,696 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics
> system started
> 2014-03-12 17:33:22,704 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for
> job_1388715708455_1597 to jobTokenSecretManager
> 2014-03-12 17:33:22,785 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing
> job_1388715708455_1597 because: not enabled; too much RAM;
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job
> job_1388715708455_1597 = 0. Number of splits = 1
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for
> job job_1388715708455_1597 = 0
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
> job_1388715708455_1597Job Transitioned from NEW to INITED
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching
> normal, non-uberized, multi-container job job_1388715708455_1597.
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is
> started.
> 2014-03-12 17:33:22,801 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: JOB_CREATE
> job_1388715708455_1597
> 2014-03-12 17:33:22,833 INFO [Socket Reader #1 for port 43578]
> org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 43578
> 2014-03-12 17:33:22,846 INFO [main]
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding
> protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the
> server
> 2014-03-12 17:33:22,847 INFO [IPC Server Responder]
> org.apache.hadoop.ipc.Server: IPC Server Responder: starting
> 2014-03-12 17:33:22,847 INFO [IPC Server listener on 43578]
> org.apache.hadoop.ipc.Server: IPC Server listener on 43578: starting
> 2014-03-12 17:33:22,847 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated
> MRClientService at node0801/10.1.41.1:43578
> 2014-03-12 17:33:22,867 ERROR [main]
> org.apache.hadoop.yarn.service.CompositeService: Error starting services
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> java.lang.NoSuchMethodError:
> org.apache.hadoop.http.HttpServer.<init>(Ljava/lang/String;Ljava/lang/String;IZLorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/security/authorize/AccessControlList;Lorg/mortbay/jetty/Connector;[Ljava/lang/String;)V
>
> at
> org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:196)
> at
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:149)
>
> at
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1022)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1276)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1272)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1228)
> 2014-03-12 17:33:22,868 INFO [main] org.apache.hadoop.ipc.Server:
> Stopping server on 43578
> 2014-03-12 17:33:22,869 INFO [IPC Server listener on 43578]
> org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 43578
> 2014-03-12 17:33:22,869 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is
> stopped.
> 2014-03-12 17:33:22,869 INFO [IPC Server Responder]
> org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
> 2014-03-12 17:33:22,869 FATAL [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
> org.apache.hadoop.yarn.YarnException: Failed to Start
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1022)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1276)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1272)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1228)
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.http.HttpServer.<init>(Ljava/lang/String;Ljava/lang/String;IZLorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/security/authorize/AccessControlList;Lorg/mortbay/jetty/Connector;[Ljava/lang/String;)V
>
> at
> org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:196)
> at
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:149)
>
> at
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
>
> ... 7 more
> 2014-03-12 17:33:22,870 INFO [Thread-1]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster received a
> signal. Signaling RMCommunicator and JobHistoryEventHandler.
> 2014-03-12 17:33:22,871 WARN [Thread-1]
> org.apache.hadoop.util.ShutdownHookManager: ShutdownHook
> 'MRAppMasterShutdownHook' failed, java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.setSignalled(MRAppMaster.java:811)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1249)
>
> at
> org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
>
>
> The error seems quite strange, as the system is reporting the lack of a
> given constructor for org.apache.hadoop.http.HttpServer.
>
> Other applications that we are running on this cluster and that do not use
> Giraph work perfectly. We have no control on this cluster, so we cannot
> modify any cluster setting.
>
> Do you any idea on how to solve this issue?
>
> Thanks in advance.
>
> Best Regards
>
> Carlo Sartiani
> ------------------------------
>
>
>
> --
> "Humanity is overrated", Gregory House, MD
>
Re: Problems when running Giraph1.0 on Hadoop 2.0.5alpha.gphd2.1.0
Posted by Carlo Sartiani <sa...@gmail.com>.
The cluster uses Pivotal own version of Hadoop 2.0.5alpha. We are
submitting the Giraph job by using the standard GIRAPH api. The cluster
tries to activate a mapreducev2 job, but it seems failing in launching
the web server. The strange thing is that, if we use a "standard"
version of Hadoop 2.0.5alpha, everything works fine.
This is the piece of code we use to configure and launch the job:
GiraphConfiguration conf = new GiraphConfiguration();
conf.setEdgeInputFormatClass(VertigoTextEdgeInputFormat.class);
conf.setVertexClass(VertigoVertex.class); // vertex class
conf.VERTEX_EDGES_CLASS.set(conf, VertigoEdges.class);
conf.MASTER_COMPUTE_CLASS.set(conf,
VertigoExtendedMasterComputation.class);
conf.SPLIT_MASTER_WORKER.set(conf, false);
conf.STATIC_GRAPH.set(conf, true);
//conf.setLocalTestMode(true);
conf.setLocalTestMode(false);
conf.setWorkerConfiguration(1,1, 100);
conf.NUM_COMPUTE_THREADS.set(conf, 4);
String splitValue = conf.get(LINE_TOKENIZE_VALUE,
LINE_TOKENIZE_VALUE_DEFAULT);
System.out.println("MAX_WORKERS: " + conf.getMaxWorkers());
System.out.println("LINE_TOKENIZE_VALUE: " + splitValue);
conf.setZooKeeperConfiguration(new
String("node0522:2181,node0722:2181,node0922:2181"));
GiraphFileInputFormat.setEdgeInputPath(conf, this.inputFile);
GiraphJob job = new GiraphJob(conf, "corejob");
// prepare dir for final results
try {
// create a directory where superstep results will be stored
Path resultDir = new
org.apache.hadoop.fs.Path(this.fileSystem.getHomeDirectory(), "results");
if (this.fileSystem.exists(resultDir)) {
this.fileSystem.delete(resultDir, true);
}
boolean res = this.fileSystem.mkdirs(resultDir);
}
catch(Exception e) {
e.printStackTrace();
}
long start = System.currentTimeMillis();
boolean result = job.run(true);
long stop = System.currentTimeMillis();
We have not tried to use Giraph1.1.0, as we were not able to compile it
on Hadoop2.0.5alpha.
Carlo
> Eli Reisman <ma...@gmail.com>
> 16 Mar 2014 21:01
> it looks like the Pivotal cluster is running on YARN or trying to
> launch Giraph on YARN from the stack trace?
>
>
>
> Carlo Sartiani <ma...@gmail.com>
> 13 Mar 2014 12:34
> Hi all.
>
> I am encountering a strange problem when running a Giraph application
> on a cluster with Hadoop 2.0.5alpha.gphd2.1.0 (Pivotal Hadoop
> distribution).
>
> Our application is based on Giraph 1.0 and is executed in mapreduce
> compatibility mode rather than in yarn mode. The application works
> smoothly on a standard Hadoop 2.0.5alpha distribution in our local
> cluster. However, when moved to an external cluster with Pivotal
> Hadoop 2.0.5alpha, it returns the error described by the following log:
>
> 2014-03-12 17:33:21,489 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster
> for application appattempt_1388715708455_1597_000001
> 2014-03-1217:33:21,722 WARN [main]
> org.apache.hadoop.conf.Configuration: job.xml:an attempt to override
> final parameter: mapreduce.job.end-notification.max.retry.interval;
> Ignoring.
> 2014-03-12 17:33:21,723 WARN [main]
> org.apache.hadoop.conf.Configuration: job.xml:an attempt to override
> final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
> 2014-03-12 17:33:21,989 WARN [main]
> org.apache.hadoop.conf.Configuration: job.xml:an attempt to override
> final parameter: mapreduce.job.end-notification.max.retry.interval;
> Ignoring.
> 2014-03-12 17:33:21,989 WARN [main]
> org.apache.hadoop.conf.Configuration: job.xml:an attempt to override
> final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
> 2014-03-12 17:33:22,005 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster:
> jobSubmitDir=file:/data5/phd/yarn/nm-local-dir/usercache/csartiani/appcache/application_1388715708455_1597/container_1388715708455_1597_01_000001/jobSubmitDir
> jobTokenFile=file:/data5/phd/yarn/nm-local-dir/usercache/csartiani/appcache/application_1388715708455_1597/container_1388715708455_1597_01_000001/jobSubmitDir/appTokens
>
> 2014-03-12 17:33:22,005 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: AM Retries: 1 attempt
> num: 1 is last retry: true
> 2014-03-12 17:33:22,013 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred
> newApiCommitter.
> 2014-03-12 17:33:22,315 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in
> config null
> 2014-03-12 17:33:22,377 WARN [main]
> org.apache.giraph.bsp.BspOutputFormat: getOutputCommitter: Returning
> ImmutableOutputCommiter (does nothing).
> 2014-03-12 17:33:22,377 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is
> org.apache.giraph.bsp.ImmutableOutputCommitter
> 2014-03-12 17:33:22,377 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Not starting
> RecoveryService: recoveryEnabled: true recoverySupportedByCommitter:
> false ApplicationAttemptID: 1
> 2014-03-1217:33:22,394 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.jobhistory.EventType for class
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
> 2014-03-12 17:33:22,395 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
> 2014-03-12 17:33:22,396 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
> 2014-03-12 17:33:22,396 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for
> class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
> 2014-03-12 17:33:22,397 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
> 2014-03-12 17:33:22,401 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for
> class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
> 2014-03-12 17:33:22,402 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for
> class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
> 2014-03-12 17:33:22,403 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for
> class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
> 2014-03-12 17:33:22,403 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is
> inited.
> 2014-03-12 17:33:22,403 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:MRClientService is inited.
> 2014-03-12 17:33:22,403 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:CommitterEventHandler is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:TaskHeartbeatHandler is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapred.TaskAttemptListenerImpl is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator
> is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
> is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
> is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$StagingDirCleaningService
> is inited.
> 2014-03-12 17:33:22,411 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:JobHistoryEventHandler is inited.
> 2014-03-12 17:33:22,411 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster is inited.
> 2014-03-12 17:33:22,460 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for
> class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
> 2014-03-12 17:33:22,653 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
> hadoop-metrics2.properties
> 2014-03-12 17:33:22,696 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
> period at 10 second(s).
> 2014-03-12 17:33:22,696 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics
> system started
> 2014-03-12 17:33:22,704 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token
> for job_1388715708455_1597 to jobTokenSecretManager
> 2014-03-12 17:33:22,785 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing
> job_1388715708455_1597 because: not enabled; too much RAM;
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for
> job job_1388715708455_1597 = 0. Number of splits = 1
> 2014-03-1217:33:22,799 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces
> for job job_1388715708455_1597 = 0
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
> job_1388715708455_1597Job Transitioned from NEW to INITED
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching
> normal, non-uberized, multi-container job job_1388715708455_1597.
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is
> started.
> 2014-03-12 17:33:22,801 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator:
> JOB_CREATE job_1388715708455_1597
> 2014-03-12 17:33:22,833 INFO [Socket Reader #1 for port 43578]
> org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 43578
> 2014-03-12 17:33:22,846 INFO [main]
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl:
> Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB
> to the server
> 2014-03-12 17:33:22,847 INFO [IPC Server Responder]
> org.apache.hadoop.ipc.Server: IPC Server Responder: starting
> 2014-03-12 17:33:22,847 INFO [IPC Server listener on 43578]
> org.apache.hadoop.ipc.Server: IPC Server listener on 43578: starting
> 2014-03-12 17:33:22,847 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService:
> Instantiated MRClientService at node0801/10.1.41.1:43578
> 2014-03-12 17:33:22,867 ERROR [main]
> org.apache.hadoop.yarn.service.CompositeService: Error starting
> services org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> java.lang.NoSuchMethodError:
> org.apache.hadoop.http.HttpServer.<init>(Ljava/lang/String;Ljava/lang/String;IZLorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/security/authorize/AccessControlList;Lorg/mortbay/jetty/Connector;[Ljava/lang/String;)V
>
> at
> org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:196)
> at
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:149)
>
> at
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1022)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1276)
>
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1272)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1228)
>
> 2014-03-12 17:33:22,868 INFO [main] org.apache.hadoop.ipc.Server:
> Stopping server on 43578
> 2014-03-12 17:33:22,869 INFO [IPC Server listener on 43578]
> org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 43578
> 2014-03-12 17:33:22,869 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is
> stopped.
> 2014-03-12 17:33:22,869 INFO [IPC Server Responder]
> org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
> 2014-03-12 17:33:22,869 FATAL [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting
> MRAppMaster
> org.apache.hadoop.yarn.YarnException: Failed to Start
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1022)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1276)
>
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1272)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1228)
>
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.http.HttpServer.<init>(Ljava/lang/String;Ljava/lang/String;IZLorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/security/authorize/AccessControlList;Lorg/mortbay/jetty/Connector;[Ljava/lang/String;)V
>
> at
> org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:196)
> at
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:149)
>
> at
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
>
> ... 7 more
> 2014-03-12 17:33:22,870 INFO [Thread-1]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster received a
> signal. Signaling RMCommunicator and JobHistoryEventHandler.
> 2014-03-12 17:33:22,871 WARN [Thread-1]
> org.apache.hadoop.util.ShutdownHookManager: ShutdownHook
> 'MRAppMasterShutdownHook' failed, java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.setSignalled(MRAppMaster.java:811)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1249)
>
> at
> org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
>
>
> The error seems quite strange, as the system is reporting the lack of
> a given constructor for org.apache.hadoop.http.HttpServer.
>
> Other applications that we are running on this cluster and that do not
> use Giraph work perfectly. We have no control on this cluster, so we
> cannot modify any cluster setting.
>
> Do you any idea on how to solve this issue?
>
> Thanks in advance.
>
> Best Regards
>
> Carlo Sartiani
> ------------------------------------------------------------------------
--
"Humanity is overrated", Gregory House, MD
Re: Problems when running Giraph1.0 on Hadoop 2.0.5alpha.gphd2.1.0
Posted by Eli Reisman <ap...@gmail.com>.
it looks like the Pivotal cluster is running on YARN or trying to launch
Giraph on YARN from the stack trace?
On Thu, Mar 13, 2014 at 4:34 AM, Carlo Sartiani <sa...@gmail.com> wrote:
> Hi all.
>
> I am encountering a strange problem when running a Giraph application on a
> cluster with Hadoop 2.0.5alpha.gphd2.1.0 (Pivotal Hadoop distribution).
>
> Our application is based on Giraph 1.0 and is executed in mapreduce
> compatibility mode rather than in yarn mode. The application works
> smoothly on a standard Hadoop 2.0.5alpha distribution in our local cluster.
> However, when moved to an external cluster with Pivotal Hadoop 2.0.5alpha,
> it returns the error described by the following log:
>
> 2014-03-12 17:33:21,489 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for
> application appattempt_1388715708455_1597_000001
> 2014-03-12 17:33:21,722 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2014-03-12 17:33:21,723 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 2014-03-12 17:33:21,989 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2014-03-12 17:33:21,989 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 2014-03-12 17:33:22,005 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: jobSubmitDir=
> file:/data5/phd/yarn/nm-local-dir/usercache/csartiani/appcache/application_1388715708455_1597/container_1388715708455_1597_01_000001/jobSubmitDirjobTokenFile=
> file:/data5/phd/yarn/nm-local-dir/usercache/csartiani/appcache/application_1388715708455_1597/container_1388715708455_1597_01_000001/jobSubmitDir/appTokens
> 2014-03-12 17:33:22,005 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: AM Retries: 1 attempt num:
> 1 is last retry: true
> 2014-03-12 17:33:22,013 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred
> newApiCommitter.
> 2014-03-12 17:33:22,315 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in
> config null
> 2014-03-12 17:33:22,377 WARN [main]
> org.apache.giraph.bsp.BspOutputFormat: getOutputCommitter: Returning
> ImmutableOutputCommiter (does nothing).
> 2014-03-12 17:33:22,377 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is
> org.apache.giraph.bsp.ImmutableOutputCommitter
> 2014-03-12 17:33:22,377 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Not starting
> RecoveryService: recoveryEnabled: true recoverySupportedByCommitter: false
> ApplicationAttemptID: 1
> 2014-03-12 17:33:22,394 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.jobhistory.EventType for class
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
> 2014-03-12 17:33:22,395 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
> 2014-03-12 17:33:22,396 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
> 2014-03-12 17:33:22,396 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
> 2014-03-12 17:33:22,397 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
> 2014-03-12 17:33:22,401 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
> 2014-03-12 17:33:22,402 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for
> class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
> 2014-03-12 17:33:22,403 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for
> class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
> 2014-03-12 17:33:22,403 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is
> inited.
> 2014-03-12 17:33:22,403 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService: Service:MRClientService is
> inited.
> 2014-03-12 17:33:22,403 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:CommitterEventHandler is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:TaskHeartbeatHandler is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapred.TaskAttemptListenerImpl is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator is
> inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
> is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
> is inited.
> 2014-03-12 17:33:22,405 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$StagingDirCleaningService
> is inited.
> 2014-03-12 17:33:22,411 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:JobHistoryEventHandler is inited.
> 2014-03-12 17:33:22,411 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService:
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster is inited.
> 2014-03-12 17:33:22,460 INFO [main]
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
> org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
> 2014-03-12 17:33:22,653 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
> hadoop-metrics2.properties
> 2014-03-12 17:33:22,696 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
> period at 10 second(s).
> 2014-03-12 17:33:22,696 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics
> system started
> 2014-03-12 17:33:22,704 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for
> job_1388715708455_1597 to jobTokenSecretManager
> 2014-03-12 17:33:22,785 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing
> job_1388715708455_1597 because: not enabled; too much RAM;
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job
> job_1388715708455_1597 = 0. Number of splits = 1
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for
> job job_1388715708455_1597 = 0
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
> job_1388715708455_1597Job Transitioned from NEW to INITED
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching
> normal, non-uberized, multi-container job job_1388715708455_1597.
> 2014-03-12 17:33:22,799 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is
> started.
> 2014-03-12 17:33:22,801 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: JOB_CREATE
> job_1388715708455_1597
> 2014-03-12 17:33:22,833 INFO [Socket Reader #1 for port 43578]
> org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 43578
> 2014-03-12 17:33:22,846 INFO [main]
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding
> protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the
> server
> 2014-03-12 17:33:22,847 INFO [IPC Server Responder]
> org.apache.hadoop.ipc.Server: IPC Server Responder: starting
> 2014-03-12 17:33:22,847 INFO [IPC Server listener on 43578]
> org.apache.hadoop.ipc.Server: IPC Server listener on 43578: starting
> 2014-03-12 17:33:22,847 INFO [main]
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated
> MRClientService at node0801/10.1.41.1:43578
> 2014-03-12 17:33:22,867 ERROR [main]
> org.apache.hadoop.yarn.service.CompositeService: Error starting services
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> java.lang.NoSuchMethodError:
> org.apache.hadoop.http.HttpServer.<init>(Ljava/lang/String;Ljava/lang/String;IZLorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/security/authorize/AccessControlList;Lorg/mortbay/jetty/Connector;[Ljava/lang/String;)V
>
> at
> org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:196)
> at
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:149)
>
> at
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1022)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1276)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1272)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1228)
> 2014-03-12 17:33:22,868 INFO [main] org.apache.hadoop.ipc.Server:
> Stopping server on 43578
> 2014-03-12 17:33:22,869 INFO [IPC Server listener on 43578]
> org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 43578
> 2014-03-12 17:33:22,869 INFO [main]
> org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is
> stopped.
> 2014-03-12 17:33:22,869 INFO [IPC Server Responder]
> org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
> 2014-03-12 17:33:22,869 FATAL [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
> org.apache.hadoop.yarn.YarnException: Failed to Start
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1022)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1276)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1272)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1228)
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.http.HttpServer.<init>(Ljava/lang/String;Ljava/lang/String;IZLorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/security/authorize/AccessControlList;Lorg/mortbay/jetty/Connector;[Ljava/lang/String;)V
>
> at
> org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:196)
> at
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:149)
>
> at
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
>
> ... 7 more
> 2014-03-12 17:33:22,870 INFO [Thread-1]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster received a
> signal. Signaling RMCommunicator and JobHistoryEventHandler.
> 2014-03-12 17:33:22,871 WARN [Thread-1]
> org.apache.hadoop.util.ShutdownHookManager: ShutdownHook
> 'MRAppMasterShutdownHook' failed, java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.setSignalled(MRAppMaster.java:811)
>
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1249)
>
> at
> org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
>
>
> The error seems quite strange, as the system is reporting the lack of a
> given constructor for org.apache.hadoop.http.HttpServer.
>
> Other applications that we are running on this cluster and that do not use
> Giraph work perfectly. We have no control on this cluster, so we cannot
> modify any cluster setting.
>
> Do you any idea on how to solve this issue?
>
> Thanks in advance.
>
> Best Regards
>
> Carlo Sartiani
> --
> "Humanity is overrated", Gregory House, MD
>