RE: compute ignite data with spark

[Stanislav Lukyanov <st...@gmail.com>]

Hi Shawn,

You can use Ignite standalone and you can also use it together with Spark.
Please take a look at these SO question and an article:
https://stackoverflow.com/questions/36036910/apache-spark-vs-apache-ignite
https://insidebigdata.com/2016/06/20/apache-ignite-and-apache-spark-complementary-in-memory-computing-solutions/

Stan

From: shawn.du
Sent: 24 февраля 2018 г. 9:56
To: user
Subject: compute ignite data with spark

Hi,

Spark is a compute engine.  Ignite also provide compute feature. Also Ignite can integrate with spark.
We are using ignite compute map-reduce feature now.  It is very fast.
I am just curious how spark compares with ignite on computing.
it is possible using spark API computing ignite cache data? 

Thanks
Shawn

Re: compute ignite data with spark

[Denis Magda <dm...@apache.org>]

Hi Shawn,

In addition to RDDs, you'll be able to use Data Frames APIs soon with
Ignite as storage for Spark. They will be released within nearest weeks in
Ignite 2.4.

As for your question on how Ignite compares to Spark. The fist is not just
a computational engine. It's a distributed database (or cache depending on
your use case) with a variety of APIs including the compute grid. Though
you can use Ignite as storage for Spark, Ignite native APIs should be more
performant.

--
Denis



On Mon, Feb 26, 2018 at 2:25 AM, Stanislav Lukyanov <st...@gmail.com>
wrote:

> Hi Shawn,
>
>
>
> You can use Ignite standalone and you can also use it together with Spark.
>
> Please take a look at these SO question and an article:
>
> https://stackoverflow.com/questions/36036910/apache-spark-vs-apache-ignite
>
> https://insidebigdata.com/2016/06/20/apache-ignite-and-
> apache-spark-complementary-in-memory-computing-solutions/
>
>
>
> Stan
>
>
>
> *From: *shawn.du <sh...@neulion.com.cn>
> *Sent: *24 февраля 2018 г. 9:56
> *To: *user <us...@ignite.apache.org>
> *Subject: *compute ignite data with spark
>
>
>
> Hi,
>
>
>
> Spark is a compute engine.  Ignite also provide compute feature. Also
> Ignite can integrate with spark.
>
> We are using ignite compute map-reduce feature now.  It is very fast.
>
> I am just curious how spark compares with ignite on computing.
>
> it is possible using spark API computing ignite cache data?
>
>
>
> Thanks
>
> Shawn
>
>
>
>
>