Fair Scheduling

[vinay Bajaj <vb...@gmail.com>]

Hi

I am trying for FAIR scheduling inside the pool that can be done by setting
LocalProperty in sparkContext. But i am Using JavaSparkContext in my java
Code and passing sparkContext Object for making JavaSparkObject.

System.setProperty("spark.scheduler.mode", "FAIR");
System.setProperty("spark.scheduler.allocation.file",
"/opt/spark/spark-0.8.1-incubating/conf/fairscheduler.xml");
JavaSparkContext sc = new JavaSparkContext("spark://
c8-data-store-1.srv.media.net:7077" , "sparkLogAnalyzer",
"/opt/spark/spark-0.8.1-incubating",
"/autoopt/vinay/sparkTesting/sparkLogAnalyser.jar");

 Seq<String> seq = new ArraySeq<String>(1);
 seq.addString(new
StringBuilder("/autoopt/vinay/sparkTesting/sparkLogAnalyser.jar"));

 SparkContext context = new SparkContext(cluster URL , "sparkLogAnalyzer",
"/opt/spark/spark-0.8.1-incubating",seq, null,null);
context.setLocalProperty("spark.scheduler.pool", "pool1");

JavaSparkContext sc = new JavaSparkContext(context);

String path = "hdfs://path/sparkTestingData/smallData.txt";
JavaRDD<String> lines = sc.textFile(path);

currently i am using spark 0.8.1, SetLocalProperty is only function for
sparkContext and is not in JavaSparkContext.

This code is giving me some "class not found exception" but unable to
figure out. Can anyone validate if i am doing it right way and how can one
do FAIR scheduling inside the pool??

Regards
Vinay Bajaj