You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Saptarshi Guha <sa...@gmail.com> on 2008/11/16 22:27:16 UTC

Empty source in map?

Hello,
	I have a JobControl with an unknown number of jobs. The way I execute  
it is add a job, start it and upon finnishing
decide whether to add another job.
	When I add another a job and run it,I get the following(see below)
	What does the "Unknown Source" mean? Does it have anything do with  
"08/11/16 16:11:16 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics  
with processName=JobTracker, sessionId= - already initialized "

	08/11/16 16:16:41 INFO jvm.JvmMetrics: Initializing JVM Metrics with  
processName=JobTracker, sessionId=
		[output of first job (job_1) skipped....]
	08/11/16 16:16:51 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics  
with processName=JobTracker, sessionId= - already initialized
	08/11/16 16:16:51 WARN mapred.JobClient: Use GenericOptionsParser for  
parsing the arguments. Applications should implement Tool for the same.
	08/11/16 16:16:51 INFO mapred.FileInputFormat: Total input paths to  
process : 1
	08/11/16 16:16:51 INFO mapred.FileInputFormat: Total input paths to  
process : 1
	08/11/16 16:16:51 INFO mapred.FileInputFormat: Total input paths to  
process : 1
	08/11/16 16:16:51 INFO mapred.FileInputFormat: Total input paths to  
process : 1
	08/11/16 16:16:51 INFO mapred.MapTask: numReduceTasks: 1
	08/11/16 16:16:51 INFO mapred.MapTask: io.sort.mb = 200
	08/11/16 16:16:52 INFO mapred.MapTask: data buffer =  
159383552/199229440
	08/11/16 16:16:52 INFO mapred.MapTask: record buffer = 524288/655360
	08/11/16 16:16:52 WARN mapred.LocalJobRunner: job_local_0002
	java.lang.NullPointerException
		at org.saptarshiguha.clusters.ClusterCenter 
$ClosestCenterMR.map(Unknown Source)
		at org.saptarshiguha.clusters.ClusterCenter 
$ClosestCenterMR.map(Unknown Source)
		at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
		at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
		at org.apache.hadoop.mapred.LocalJobRunner 
$Job.run(LocalJobRunner.java:157)
	08/11/16 16:16:57 INFO mapred.LocalJobRunner: file:/tmp/input/ 
sample.data:0+308


	The map function is not run at all.
	

	===BEGIN_CODE====
	JobConf jobConf_1 = ColumnRanges.createSetupJob(inPaths_1, outdir);
	Job job_1 = new Job(jobConf_1, null);
	JobControl theControl = new JobControl("Test");
	theControl.addJob(job_1);
	Thread theController = new Thread(theControl);
	theController.start();
	while (!theControl.allFinished()) {
	    System.out.println("Jobs in waiting state: " +  
theControl.getWaitingJobs().size());
	    System.out.println("Jobs in ready state: " +  
theControl.getReadyJobs().size());
	    System.out.println("Jobs in running state: " +  
theControl.getRunningJobs().size());
	    System.out.println("Jobs in success state: " +  
theControl.getSuccessfulJobs().size());
	    System.out.println("Jobs in failed state: " +  
theControl.getFailedJobs().size());
	    System.out.println("\n");
	    try {
		Thread.sleep(5000);
	    } catch (Exception e) {
		System.out.println("Exception\n");
	    }
	}
	theControl.stop();

	if ( job_1.getState() == Job.SUCCESS) {
	    System.out.println("Choosing initial centers");
	    createSeedCenters();
	
  	    FileUtil.fullyDelete(outdir.getFileSystem(jobConf_1 ),outdir);
	    JobConf jobConf = ClusterCenter.createSetupJob(inPaths_1, outdir);
	    Job job = new Job(jobConf, null);
	    theControl.addJob(job);
	    Thread threadControl = new Thread(theControl);
	    threadControl.start();
	    while (!theControl.allFinished()){
		 try {
		     Thread.sleep(5000);
		 } catch (Exception e) {
		     System.out.println("Exception\n");
		 }
	    }
	}