You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@geode.apache.org by Jason Huynh <jh...@pivotal.io> on 2020/03/03 18:35:04 UTC
Blog about how to use geode-kafka-connector
Hello Geode Users!
I wrote a blog/how-to about using the geode-kafka-connector (under
development) to connect Apache Kafka and Apache Geode.
https://medium.com/@huynhja/ingest-store-and-search-json-data-with-apache-kafka-and-apache-geode-fc6d0d2f9d9f
I'll continue to share blogs and such if everyone is ok with it, if not
please let me know and I'll find a different way to share.
Any feedback/suggestions are always welcome.
Thanks,
-Jason
Re: Blog about how to use geode-kafka-connector
Posted by Nabarun Nag <nn...@pivotal.io>.
Hi Paul,
Our aim for this connector was to facilitate Apache Geode to be used to
ecosystems where Apache Kafka is an integral part.
Apache Geode can be used in those environments without making Geode
specific code changes in their sources.
Regards
Naba
On Wed, Mar 4, 2020 at 9:06 AM Jason Huynh <jh...@pivotal.io> wrote:
> Hello Paul,
>
>
> Thanks for reaching out. The blog post and sample application were
> working off the premise that Apache Kafka was already being used in an
> environment and not necessarily because one or the other had limitations
> ingesting. That might be an interesting avenue to explore in the future,
> unfortunately I do not have any of that data at this time.
>
>
> Apache Geode, as you know, should definitely be able to handle ingesting/
> sourcing data directly. There might be some applications, like a chat
> application, where a topic might be put to good use and yet we still want
> to be able to do a more optimized search.
>
>
> As an aside, the geode-kafka-connector also provides the ability to use
> Geode as a source for Kafka.
>
>
> Thanks,
>
> -Jason
>
> On Wed, Mar 4, 2020 at 1:19 AM paul perez <pa...@pymma.com> wrote:
>
>> Hello Jason,
>>
>>
>> First, I would like to thank you for the quote and congratulation for
>> your post.
>>
>> I have a simple question about the injection capability of both products.
>>
>> In your case, which constraints pushed you to add Kafka as a buffer
>> between your event sources and Geode.
>>
>> Were your sources unable to access to Geode directly (for any reason)?
>>
>> Did you find (in your case) some limitations in Geode injection?
>>
>> In that case, could you (if it is possible) provide some metrics when the
>> limitations appeared.
>>
>> Thank you
>>
>> Paul
>>
>> On 03.03.2020 22:56, Anilkumar Gingade wrote:
>>
>> Thanks for sharing this.
>>
>> On Tue, Mar 3, 2020 at 10:53 AM Nabarun Nag <nn...@pivotal.io> wrote:
>>
>>> Please do share !!!
>>>
>>> On Tue, Mar 3, 2020 at 10:50 AM Anthony Baker <ab...@pivotal.io> wrote:
>>>
>>>> Nice! Yes, please continue to share!
>>>> Anthony
>>>>
>>>>
>>>> On Mar 3, 2020, at 10:35 AM, Jason Huynh <jh...@pivotal.io> wrote:
>>>>
>>>> Hello Geode Users!
>>>> I wrote a blog/how-to about using the geode-kafka-connector (under
>>>> development) to connect Apache Kafka and Apache Geode.
>>>>
>>>> https://medium.com/@huynhja/ingest-store-and-search-json-data-with-apache-kafka-and-apache-geode-fc6d0d2f9d9f
>>>> I'll continue to share blogs and such if everyone is ok with it, if not
>>>> please let me know and I'll find a different way to share.
>>>> Any feedback/suggestions are always welcome.
>>>> Thanks,
>>>> -Jason
>>>>
>>>> --
>>
>> Best regards
>>
>> Paul Perez Chief Architect
>> Pymma Consulting
>> --------------------------
>> Tel: +44 79 44 36 04 65
>> Skype ID : polperez
>>
>>
>>
Re: Blog about how to use geode-kafka-connector
Posted by Jason Huynh <jh...@pivotal.io>.
Hello Paul,
Thanks for reaching out. The blog post and sample application were working
off the premise that Apache Kafka was already being used in an environment
and not necessarily because one or the other had limitations ingesting.
That might be an interesting avenue to explore in the future, unfortunately
I do not have any of that data at this time.
Apache Geode, as you know, should definitely be able to handle ingesting/
sourcing data directly. There might be some applications, like a chat
application, where a topic might be put to good use and yet we still want
to be able to do a more optimized search.
As an aside, the geode-kafka-connector also provides the ability to use
Geode as a source for Kafka.
Thanks,
-Jason
On Wed, Mar 4, 2020 at 1:19 AM paul perez <pa...@pymma.com> wrote:
> Hello Jason,
>
>
> First, I would like to thank you for the quote and congratulation for your
> post.
>
> I have a simple question about the injection capability of both products.
>
> In your case, which constraints pushed you to add Kafka as a buffer
> between your event sources and Geode.
>
> Were your sources unable to access to Geode directly (for any reason)?
>
> Did you find (in your case) some limitations in Geode injection?
>
> In that case, could you (if it is possible) provide some metrics when the
> limitations appeared.
>
> Thank you
>
> Paul
>
> On 03.03.2020 22:56, Anilkumar Gingade wrote:
>
> Thanks for sharing this.
>
> On Tue, Mar 3, 2020 at 10:53 AM Nabarun Nag <nn...@pivotal.io> wrote:
>
>> Please do share !!!
>>
>> On Tue, Mar 3, 2020 at 10:50 AM Anthony Baker <ab...@pivotal.io> wrote:
>>
>>> Nice! Yes, please continue to share!
>>> Anthony
>>>
>>>
>>> On Mar 3, 2020, at 10:35 AM, Jason Huynh <jh...@pivotal.io> wrote:
>>>
>>> Hello Geode Users!
>>> I wrote a blog/how-to about using the geode-kafka-connector (under
>>> development) to connect Apache Kafka and Apache Geode.
>>>
>>> https://medium.com/@huynhja/ingest-store-and-search-json-data-with-apache-kafka-and-apache-geode-fc6d0d2f9d9f
>>> I'll continue to share blogs and such if everyone is ok with it, if not
>>> please let me know and I'll find a different way to share.
>>> Any feedback/suggestions are always welcome.
>>> Thanks,
>>> -Jason
>>>
>>> --
>
> Best regards
>
> Paul Perez Chief Architect
> Pymma Consulting
> --------------------------
> Tel: +44 79 44 36 04 65
> Skype ID : polperez
>
>
>
Re: Blog about how to use geode-kafka-connector
Posted by paul perez <pa...@pymma.com>.
Hello Jason,
First, I would like to thank you for the quote and
congratulation for your post.
I have a simple question about the
injection capability of both products.
In your case, which constraints
pushed you to add Kafka as a buffer between your event sources and
Geode.
Were your sources unable to access to Geode directly (for any
reason)?
Did you find (in your case) some limitations in Geode
injection?
In that case, could you (if it is possible) provide some
metrics when the limitations appeared.
Thank you
Paul
On
03.03.2020 22:56, Anilkumar Gingade wrote:
> Thanks for sharing this.
>
> On Tue, Mar 3, 2020 at 10:53 AM Nabarun Nag wrote:
>
>> Please
do share !!!
>>
>> On Tue, Mar 3, 2020 at 10:50 AM Anthony Baker
wrote:
>>
>>> Nice! Yes, please continue to share!
>>> Anthony
>>>
>>>> On Mar 3, 2020, at 10:35 AM, Jason Huynh wrote:
>>>>
>>>> Hello
Geode Users!
>>>> I wrote a blog/how-to about using the
geode-kafka-connector (under development) to connect Apache Kafka and
Apache Geode.
>>>>
https://medium.com/@huynhja/ingest-store-and-search-json-data-with-apache-kafka-and-apache-geode-fc6d0d2f9d9f
[2]
>>>> I'll continue to share blogs and such if everyone is ok with
it, if not please let me know and I'll find a different way to share.
>>>> Any feedback/suggestions are always welcome.
>>>> Thanks,
>>>>
-Jason
--
Best regards
Paul Perez Chief Architect
Pymma
Consulting
--------------------------
Tel: +44 79 44 36 04 65
Skype ID
: polperez
Links:
------
[1] mailto:jhuynh@pivotal.io
[2]
https://medium.com/@huynhja/ingest-store-and-search-json-data-with-apache-kafka-and-apache-geode-fc6d0d2f9d9f
[3]
mailto:abaker@pivotal.io
[4] mailto:nnag@pivotal.io
Re: Blog about how to use geode-kafka-connector
Posted by Anilkumar Gingade <ag...@pivotal.io>.
Thanks for sharing this.
On Tue, Mar 3, 2020 at 10:53 AM Nabarun Nag <nn...@pivotal.io> wrote:
> Please do share !!!
>
> On Tue, Mar 3, 2020 at 10:50 AM Anthony Baker <ab...@pivotal.io> wrote:
>
>> Nice! Yes, please continue to share!
>>
>> Anthony
>>
>>
>> On Mar 3, 2020, at 10:35 AM, Jason Huynh <jh...@pivotal.io> wrote:
>>
>> Hello Geode Users!
>>
>> I wrote a blog/how-to about using the geode-kafka-connector (under
>> development) to connect Apache Kafka and Apache Geode.
>>
>>
>> https://medium.com/@huynhja/ingest-store-and-search-json-data-with-apache-kafka-and-apache-geode-fc6d0d2f9d9f
>>
>> I'll continue to share blogs and such if everyone is ok with it, if not
>> please let me know and I'll find a different way to share.
>>
>> Any feedback/suggestions are always welcome.
>>
>> Thanks,
>> -Jason
>>
>>
>>
Re: Blog about how to use geode-kafka-connector
Posted by Nabarun Nag <nn...@pivotal.io>.
Please do share !!!
On Tue, Mar 3, 2020 at 10:50 AM Anthony Baker <ab...@pivotal.io> wrote:
> Nice! Yes, please continue to share!
>
> Anthony
>
>
> On Mar 3, 2020, at 10:35 AM, Jason Huynh <jh...@pivotal.io> wrote:
>
> Hello Geode Users!
>
> I wrote a blog/how-to about using the geode-kafka-connector (under
> development) to connect Apache Kafka and Apache Geode.
>
>
> https://medium.com/@huynhja/ingest-store-and-search-json-data-with-apache-kafka-and-apache-geode-fc6d0d2f9d9f
>
> I'll continue to share blogs and such if everyone is ok with it, if not
> please let me know and I'll find a different way to share.
>
> Any feedback/suggestions are always welcome.
>
> Thanks,
> -Jason
>
>
>
Re: Blog about how to use geode-kafka-connector
Posted by Anthony Baker <ab...@pivotal.io>.
Nice! Yes, please continue to share!
Anthony
> On Mar 3, 2020, at 10:35 AM, Jason Huynh <jh...@pivotal.io> wrote:
>
> Hello Geode Users!
>
> I wrote a blog/how-to about using the geode-kafka-connector (under development) to connect Apache Kafka and Apache Geode.
>
> https://medium.com/@huynhja/ingest-store-and-search-json-data-with-apache-kafka-and-apache-geode-fc6d0d2f9d9f <https://medium.com/@huynhja/ingest-store-and-search-json-data-with-apache-kafka-and-apache-geode-fc6d0d2f9d9f>
>
> I'll continue to share blogs and such if everyone is ok with it, if not please let me know and I'll find a different way to share.
>
> Any feedback/suggestions are always welcome.
>
> Thanks,
> -Jason