You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by sirinath <si...@gmail.com> on 2015/05/12 07:37:02 UTC
Gelly and ML for Streaming
It would be good if you would consider how Gelly and ML can be used on top of
streaming also from the onset.
This would mean that in Gelly you should be able to read Graph mutation
operation from the stream. Initial Graph (Vertices and Edges) if any
followed by mutation stream.
--
View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Gelly-and-ML-for-Streaming-tp5563.html
Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.
Re: Gelly and ML for Streaming
Posted by sirinath <si...@gmail.com>.
In streaming API you can get efficiency as only some data changes.
I am looking into moving window calculations on real time market data to
generate trading signals
--
View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Gelly-and-ML-for-Streaming-tp5563p5580.html
Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.
Re: Gelly and ML for Streaming
Posted by Paris Carbone <pa...@kth.se>.
Hi again Suminda!
The stream ML api will be along the lines of the batch ML library and will have some interesting features. We consider re-using as much as possible from the batch ML (e.g. the same data structures and general abstractions etc.). Faye and Martha (CCed) are looking into stream learners (eg. vertical hoeffding trees, concept drift detectors) and sampling techniques for their theses and part of their work will be most probably integrated into this library which we will be able to share soon.
Furthermore, we are also looking, at an experimental level for now, into different ways of making online learners interact with offline ones in a single pipeline. If you have any ideas that you would want us to consider/look into at this early stage it would be great!
cheers
Paris
> On 12 May 2015, at 10:17, Vasiliki Kalavri <va...@apache.org> wrote:
>
> Hi Suminda,
>
> indeed this is a very exciting idea and we have been working on both Gelly
> streaming and ML streaming for a while here in Stockholm.
>
> Daniel has been looking into graph streaming for his thesis together with
> Paris and myself.
> We have evaluated existing streaming models and algorithms and we are
> working on designing a nice graph streaming API for Flink. This is a quite
> challenging area with a lot of open research issues and we hope we will be
> able to share some nice results soon!
> We already have some implementations of distributed graph streaming
> algorithms, which we are now looking into optimizing further.
>
> Paris can tell you more on the ML streaming work.
>
> By the way, do you have some specific use-case in mind or problem that you
> need graph/ML streaming for? If you would like to shre it, maybe we can
> already help you :-)
>
> Cheers,
> Vasia.
>
> On 12 May 2015 at 09:19, Márton Balassi <ba...@gmail.com> wrote:
>
>> Dear Suminda,
>>
>> Thanks for the suggestion.
>> Some folks at Stockholm have started implementing these, but are a bit
>> quite about it. Paris, Vasia, Gyula and Daniel Bali can tell you more on
>> the topic just to name a few.
>>
>> Best,
>>
>> Marton
>>
>> On Tue, May 12, 2015 at 7:37 AM, sirinath <si...@gmail.com> wrote:
>>
>>> It would be good if you would consider how Gelly and ML can be used on
>> top
>>> of
>>> streaming also from the onset.
>>>
>>> This would mean that in Gelly you should be able to read Graph mutation
>>> operation from the stream. Initial Graph (Vertices and Edges) if any
>>> followed by mutation stream.
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Gelly-and-ML-for-Streaming-tp5563.html
>>> Sent from the Apache Flink Mailing List archive. mailing list archive at
>>> Nabble.com.
>>>
>>
Re: Gelly and ML for Streaming
Posted by Vasiliki Kalavri <va...@apache.org>.
Hi Suminda,
indeed this is a very exciting idea and we have been working on both Gelly
streaming and ML streaming for a while here in Stockholm.
Daniel has been looking into graph streaming for his thesis together with
Paris and myself.
We have evaluated existing streaming models and algorithms and we are
working on designing a nice graph streaming API for Flink. This is a quite
challenging area with a lot of open research issues and we hope we will be
able to share some nice results soon!
We already have some implementations of distributed graph streaming
algorithms, which we are now looking into optimizing further.
Paris can tell you more on the ML streaming work.
By the way, do you have some specific use-case in mind or problem that you
need graph/ML streaming for? If you would like to shre it, maybe we can
already help you :-)
Cheers,
Vasia.
On 12 May 2015 at 09:19, Márton Balassi <ba...@gmail.com> wrote:
> Dear Suminda,
>
> Thanks for the suggestion.
> Some folks at Stockholm have started implementing these, but are a bit
> quite about it. Paris, Vasia, Gyula and Daniel Bali can tell you more on
> the topic just to name a few.
>
> Best,
>
> Marton
>
> On Tue, May 12, 2015 at 7:37 AM, sirinath <si...@gmail.com> wrote:
>
> > It would be good if you would consider how Gelly and ML can be used on
> top
> > of
> > streaming also from the onset.
> >
> > This would mean that in Gelly you should be able to read Graph mutation
> > operation from the stream. Initial Graph (Vertices and Edges) if any
> > followed by mutation stream.
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Gelly-and-ML-for-Streaming-tp5563.html
> > Sent from the Apache Flink Mailing List archive. mailing list archive at
> > Nabble.com.
> >
>
Re: Gelly and ML for Streaming
Posted by Márton Balassi <ba...@gmail.com>.
Dear Suminda,
Thanks for the suggestion.
Some folks at Stockholm have started implementing these, but are a bit
quite about it. Paris, Vasia, Gyula and Daniel Bali can tell you more on
the topic just to name a few.
Best,
Marton
On Tue, May 12, 2015 at 7:37 AM, sirinath <si...@gmail.com> wrote:
> It would be good if you would consider how Gelly and ML can be used on top
> of
> streaming also from the onset.
>
> This would mean that in Gelly you should be able to read Graph mutation
> operation from the stream. Initial Graph (Vertices and Edges) if any
> followed by mutation stream.
>
>
>
> --
> View this message in context:
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Gelly-and-ML-for-Streaming-tp5563.html
> Sent from the Apache Flink Mailing List archive. mailing list archive at
> Nabble.com.
>