You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@ignite.apache.org by Ranjit <ra...@amazon.com> on 2016/08/14 23:47:55 UTC

Ignite cluster external to Spark cluster

Our Ignite clusters live external to the Spark cluster. I understand that
Ignite must be used in stand-alone mode for this to work. Is that correct?

I want to initialize the Ignite client in a mode in which it does NOT
attempt to start a new Ignite cluster, but uses an existing cluster. What's
the correct way of achieving this?

Thanks!
Ranjit





--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cluster-external-to-Spark-cluster-tp7052.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite cluster external to Spark cluster

Posted by vkulichenko <va...@gmail.com>.

Ranjit,

Client node is not a separate process, it will be embedded in the executor.

In IgniteRDD you save key-value pairs. Each pair will be saved in Ignite
cache separately in the binary form.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cluster-external-to-Spark-cluster-tp7052p7078.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite cluster external to Spark cluster

Posted by Ranjit <ra...@amazon.com>.

Hi Val,

Thanks for your response. Why is there a need to launch a new process on the
executor? Can the Ignite client code not run inside the process space of the
Executor?

I'm also curious to know how the RDD is persisted? Does each Executor take a
slice of the contents of the RDD and write it as a byte[] to the cache? If
there is more documentation on how this works, it would be great to get
access to it!

Thanks,
Ranjit





--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cluster-external-to-Spark-cluster-tp7052p7076.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite cluster external to Spark cluster

Posted by vkulichenko <va...@gmail.com>.

Hi Ranjit,

By default IgniteContext should start a client node on executor and connect
to your cluster (assuming your configuration is correct). Do you observe
different behavior?

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cluster-external-to-Spark-cluster-tp7052p7073.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.