You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Ross Duncan <ro...@gmail.com> on 2016/06/02 20:06:24 UTC
Fwd: Kafka Ingester
Hello,
I came across the landing page for Geode contributions today, and I noticed
one of the suggested ideas (no 6) was a Kafka ingester plugin.
https://cwiki.apache.org/confluence/display/GEODE/How+to+Contribute
I dont know terribly much about geode yet, but was wondering if anyone had
attempted this yet?
If not, is there any prior art for ingesters of other technologies that
might be worth a look?
Also I notice Ignite has a Kafka module, seemingly for a similar purpose,
built around their API. Could this be another useful starting point?
Thanks,
Rosco
Re: Fwd: Kafka Ingester
Posted by theseusyang <th...@gmail.com>.
Join the meeting at this link
<https://pivotalcommunity.adobeconnect.com/clubhouse/>. You can add a
reminder to your calendar with this link
<
https://www.google.com/calendar/event?eid=cTlwZGdzOTcyZm9oZ2Q0NTRuZGJia2xnb2cgcGl2b3RhbC5pb191OGtndnVhaGprYm9oMWduZmh2NXRzMnY5Y0Bn&ctz=America/Los_Angeles
>
.
2016-07-28 16:18 GMT+08:00 Olivier NOUGUIER [via Apache Geode (Incubating)
Developers Forum] <ml...@n6.nabble.com>:
> I'd like to try to join the call. At least as a listener, is it possible
> and how/where ?
>
>
> Le jeu. 28 juil. 2016 à 04:06, theseusyang <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=7473&i=0>> a écrit :
>
> > yeah, I will join this meeting
> >
> > 2016-07-28 9:22 GMT+08:00 GregChase [via Apache Geode (Incubating)
> > Developers Forum] <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=7473&i=1>>:
> >
> > > On Wed, Jul 27, 2016 at 6:39 PM, Greg Chase <[hidden email]
> > > <http:///user/SendEmail.jtp?type=node&node=7471&i=0>> wrote:
> > >
> > > > Hello everyone. We have a community call tomorrow at 9AM Pacific.
> > > Would
> > > > you be interested in discussing this after your educational topic?
> > > >
> > >
> > > After *OUR* educational topic - Geode events & continuous query...
> > >
> > >
> > > ------------------------------
> > > If you reply to this email, your message will be added to the
> discussion
> > > below:
> > >
> > >
> >
> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7471.html
> > > To unsubscribe from Fwd: Kafka Ingester, click here
> > > <
> > >
> > > .
> > > NAML
> > > <
> >
> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> > >
> > >
> >
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7472.html
> > Sent from the Apache Geode (Incubating) Developers Forum mailing list
> > archive at Nabble.com.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7473.html
> To unsubscribe from Fwd: Kafka Ingester, click here
> <http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=6407&code=dGhlc2V1c3lhbmdAZ21haWwuY29tfDY0MDd8LTExNzY2NDM1NzU=>
> .
> NAML
> <http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7474.html
Sent from the Apache Geode (Incubating) Developers Forum mailing list archive at Nabble.com.
Re: Fwd: Kafka Ingester
Posted by Olivier NOUGUIER <ol...@gmail.com>.
I'd like to try to join the call. At least as a listener, is it possible
and how/where ?
Le jeu. 28 juil. 2016 à 04:06, theseusyang <th...@gmail.com> a écrit :
> yeah, I will join this meeting
>
> 2016-07-28 9:22 GMT+08:00 GregChase [via Apache Geode (Incubating)
> Developers Forum] <ml...@n6.nabble.com>:
>
> > On Wed, Jul 27, 2016 at 6:39 PM, Greg Chase <[hidden email]
> > <http:///user/SendEmail.jtp?type=node&node=7471&i=0>> wrote:
> >
> > > Hello everyone. We have a community call tomorrow at 9AM Pacific.
> > Would
> > > you be interested in discussing this after your educational topic?
> > >
> >
> > After *OUR* educational topic - Geode events & continuous query...
> >
> >
> > ------------------------------
> > If you reply to this email, your message will be added to the discussion
> > below:
> >
> >
> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7471.html
> > To unsubscribe from Fwd: Kafka Ingester, click here
> > <
> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=6407&code=dGhlc2V1c3lhbmdAZ21haWwuY29tfDY0MDd8LTExNzY2NDM1NzU=
> >
> > .
> > NAML
> > <
> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
> >
>
>
>
>
> --
> View this message in context:
> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7472.html
> Sent from the Apache Geode (Incubating) Developers Forum mailing list
> archive at Nabble.com.
Re: Fwd: Kafka Ingester
Posted by theseusyang <th...@gmail.com>.
yeah, I will join this meeting
2016-07-28 9:22 GMT+08:00 GregChase [via Apache Geode (Incubating)
Developers Forum] <ml...@n6.nabble.com>:
> On Wed, Jul 27, 2016 at 6:39 PM, Greg Chase <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=7471&i=0>> wrote:
>
> > Hello everyone. We have a community call tomorrow at 9AM Pacific.
> Would
> > you be interested in discussing this after your educational topic?
> >
>
> After *OUR* educational topic - Geode events & continuous query...
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7471.html
> To unsubscribe from Fwd: Kafka Ingester, click here
> <http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=6407&code=dGhlc2V1c3lhbmdAZ21haWwuY29tfDY0MDd8LTExNzY2NDM1NzU=>
> .
> NAML
> <http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7472.html
Sent from the Apache Geode (Incubating) Developers Forum mailing list archive at Nabble.com.
Re: Fwd: Kafka Ingester
Posted by Greg Chase <gr...@gregchase.com>.
On Wed, Jul 27, 2016 at 6:39 PM, Greg Chase <gr...@gregchase.com> wrote:
> Hello everyone. We have a community call tomorrow at 9AM Pacific. Would
> you be interested in discussing this after your educational topic?
>
After *OUR* educational topic - Geode events & continuous query...
Re: Fwd: Kafka Ingester
Posted by Greg Chase <gr...@gregchase.com>.
Hello everyone. We have a community call tomorrow at 9AM Pacific. Would
you be interested in discussing this after your educational topic?
-Greg
On Wed, Jul 27, 2016 at 5:46 PM, theseusyang <th...@gmail.com> wrote:
>
> Olivier, it looks like good!
>
>
>
>
> --
> View this message in context:
> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7469.html
> Sent from the Apache Geode (Incubating) Developers Forum mailing list
> archive at Nabble.com.
>
Re: Fwd: Kafka Ingester
Posted by theseusyang <th...@gmail.com>.
Olivier, it looks like good!
--
View this message in context: http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7469.html
Sent from the Apache Geode (Incubating) Developers Forum mailing list archive at Nabble.com.
Re: Fwd: Kafka Ingester
Posted by Olivier NOUGUIER <ol...@gmail.com>.
Oups forgot the link https://github.com/cheleb/reactive-geode
Le mer. 27 juil. 2016 à 13:02, Olivier NOUGUIER <ol...@gmail.com>
a écrit :
> Hi all,
> I've made a very simple component with Akka Stream to sink in / source
> from geode. Then it is trivial to source from kafka ( or any other source !
> ).
>
>
> Le mer. 27 juil. 2016 à 12:57, theseusyang <th...@gmail.com> a
> écrit :
>
>> Hi Rosco,
>> I have written some codes for kafka integration with Geode. we can
>> together
>> make a discuss about this.
>>
>> thank you
>> theseus yang
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7441.html
>> Sent from the Apache Geode (Incubating) Developers Forum mailing list
>> archive at Nabble.com.
>>
>
Re: Fwd: Kafka Ingester
Posted by Olivier NOUGUIER <ol...@gmail.com>.
Hi all,
I've made a very simple component with Akka Stream to sink in / source
from geode. Then it is trivial to source from kafka ( or any other source !
).
Le mer. 27 juil. 2016 à 12:57, theseusyang <th...@gmail.com> a écrit :
> Hi Rosco,
> I have written some codes for kafka integration with Geode. we can together
> make a discuss about this.
>
> thank you
> theseus yang
>
>
>
> --
> View this message in context:
> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7441.html
> Sent from the Apache Geode (Incubating) Developers Forum mailing list
> archive at Nabble.com.
>
Re: Fwd: Kafka Ingester
Posted by theseusyang <th...@gmail.com>.
Hi Rosco,
I have written some codes for kafka integration with Geode. we can together
make a discuss about this.
thank you
theseus yang
--
View this message in context: http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/Fwd-Kafka-Ingester-tp6407p7441.html
Sent from the Apache Geode (Incubating) Developers Forum mailing list archive at Nabble.com.
Re: Kafka Ingester
Posted by William Markito <wm...@pivotal.io>.
Hey Ross
This sounds like a great idea and I'd recommend it to be yet another separate module, like the Spark connector for example.
Kafka and Geode share many similar concepts but support different data structures so there could be many different use cases for such connector.
Looking forward for a proposal or topics to discuss as you find them.
I'm not familiar with Ignite's implementation so can't comment on that.
Sent from my iPhone
> On Jun 2, 2016, at 1:06 PM, Ross Duncan <ro...@gmail.com> wrote:
>
> Hello,
>
> I came across the landing page for Geode contributions today, and I noticed
> one of the suggested ideas (no 6) was a Kafka ingester plugin.
>
>
> https://cwiki.apache.org/confluence/display/GEODE/How+to+Contribute
>
> I dont know terribly much about geode yet, but was wondering if anyone had
> attempted this yet?
>
> If not, is there any prior art for ingesters of other technologies that
> might be worth a look?
>
> Also I notice Ignite has a Kafka module, seemingly for a similar purpose,
> built around their API. Could this be another useful starting point?
>
> Thanks,
> Rosco