[jira] [Created] (IGNITE-13209) JavaIgniteCatalogExample doesn't work with a standalone Spark cluster

["Valentin Kulichenko (Jira)" <ji...@apache.org>]

Valentin Kulichenko created IGNITE-13209:
--------------------------------------------

             Summary: JavaIgniteCatalogExample doesn't work with a standalone Spark cluster
                 Key: IGNITE-13209
                 URL: https://issues.apache.org/jira/browse/IGNITE-13209
             Project: Ignite
          Issue Type: Bug
          Components: spark
    Affects Versions: 2.8.1
            Reporter: Valentin Kulichenko


To reproduce the issue:
 # Start Spark master and slave as described here: [http://spark.apache.org/docs/latest/spark-standalone.html]
 # Change the master URL in the {{JavaIgniteCatalogExample}} from "local" to the one just started.
 # Run the example.

Updated code that creates the {{IgniteSparkSession}}:
{code:java}
String libs = "/Users/vkulichenko/GridGain/releases/apache-ignite-2.8.1-bin/libs";
        
IgniteSparkSession igniteSession = IgniteSparkSession.builder()
    .appName("Spark Ignite catalog example")
    .master("spark://Valentin-Kulichenko-MacBook-Pro-1772.local:7077")
    .config("spark.executor.instances", "2")
    .config("spark.executor.extraClassPath", libs + "/*" + ":" + libs + "/ignite-spark/*:" + libs + "/ignite-spring/*")
    .igniteConfig(CONFIG)
    .getOrCreate();
{code}
Execution fails with this exception:
{noformat}
[2020-07-02 15:50:27,627][ERROR][task-result-getter-3][TaskSetManager] Task 0 in stage 0.0 failed 4 times; aborting job
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 3, 10.0.0.11, executor 0): class org.apache.ignite.IgniteIllegalStateException: Ignite instance with provided name doesn't exist. Did you call Ignition.start(..) to start an Ignite instance? [name=testing]
	at org.apache.ignite.internal.IgnitionEx.grid(IgnitionEx.java:1351)
	at org.apache.ignite.Ignition.ignite(Ignition.java:528)
	at org.apache.ignite.spark.impl.package$.ignite(package.scala:65)
	at org.apache.ignite.spark.impl.IgniteRelationProvider$$anonfun$configProvider$1$2.apply(IgniteRelationProvider.scala:238)
	at org.apache.ignite.spark.impl.IgniteRelationProvider$$anonfun$configProvider$1$2.apply(IgniteRelationProvider.scala:235)
	at org.apache.ignite.spark.Once.apply(IgniteContext.scala:222)
	at org.apache.ignite.spark.IgniteContext.ignite(IgniteContext.scala:144)
	at org.apache.ignite.spark.impl.IgniteSQLDataFrameRDD.compute(IgniteSQLDataFrameRDD.scala:65)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:109)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)