H2O integration

[Simone Robutti <si...@radicalbit.io>]

Hello everyone,

here at RadicalBit we are evaluating the possibility to start a project
similar to SparkingWater to integrate H2O with Flink. I know the subject
has been discussed several time over the course of the last year but noone
seems to be working on it right now.

I'm writing here to enquire about the interest by others to work on this
project and if anyone else began working on it.

I will engage the H2O community on this subject later today to probe their
sentiment on this possibility and ask for technical advices.

Best regards,

Simone

Re: H2O integration

[Slim Baltagi <sb...@gmail.com>]

Hi

This two blog posts might be a  good start 
http://blog.h2o.ai/2014/09/how-sparkling-water-brings-h2o-to-spark/
https://databricks.com/blog/2014/06/30/sparkling-water-h20-spark.html

I am also collecting here all resources related to H2O in general and Sparkling Water in particular: http://sparkbigdata.com/component/tags/tag/63

I hope this helps.

Slim Baltagi

On Feb 16, 2016, at 10:17 AM, Till Rohrmann <tr...@apache.org> wrote:

> Hi Simone,
> 
> as far as I know, there is nobody currently working on an H2O integration.
> I only looked briefly at the sparkling water implementation when it was
> released. If I remember correctly, then the general idea was to start H2O
> from within the Executor thread and to use a special RDD to communicate
> with the H2O thread.
> 
> I think this could also be done for Flink, where you start the H2O thread
> along side the TaskManagers and have some special operators which do the
> communication with H2O.
> 
> This would definitely be an interesting contribution to Flink :-)
> 
> Cheers,
> Till
> 
> On Tue, Feb 16, 2016 at 4:29 PM, Simone Robutti <
> simone.robutti@radicalbit.io> wrote:
> 
>> Hello everyone,
>> 
>> here at RadicalBit we are evaluating the possibility to start a project
>> similar to SparkingWater to integrate H2O with Flink. I know the subject
>> has been discussed several time over the course of the last year but noone
>> seems to be working on it right now.
>> 
>> I'm writing here to enquire about the interest by others to work on this
>> project and if anyone else began working on it.
>> 
>> I will engage the H2O community on this subject later today to probe their
>> sentiment on this possibility and ask for technical advices.
>> 
>> Best regards,
>> 
>> Simone
>> 

Re: H2O integration

[Till Rohrmann <tr...@apache.org>]

Hi Simone,

as far as I know, there is nobody currently working on an H2O integration.
I only looked briefly at the sparkling water implementation when it was
released. If I remember correctly, then the general idea was to start H2O
from within the Executor thread and to use a special RDD to communicate
with the H2O thread.

I think this could also be done for Flink, where you start the H2O thread
along side the TaskManagers and have some special operators which do the
communication with H2O.

This would definitely be an interesting contribution to Flink :-)

Cheers,
Till

On Tue, Feb 16, 2016 at 4:29 PM, Simone Robutti <
simone.robutti@radicalbit.io> wrote:

> Hello everyone,
>
> here at RadicalBit we are evaluating the possibility to start a project
> similar to SparkingWater to integrate H2O with Flink. I know the subject
> has been discussed several time over the course of the last year but noone
> seems to be working on it right now.
>
> I'm writing here to enquire about the interest by others to work on this
> project and if anyone else began working on it.
>
> I will engage the H2O community on this subject later today to probe their
> sentiment on this possibility and ask for technical advices.
>
> Best regards,
>
> Simone
>