You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Michael Pollind <mp...@gmail.com> on 2020/10/25 22:05:29 UTC
[Micronaut] pubsub support
I've started to work on adding some basic pubsub support for
Micronaut-ignite. I only have a draft PR in progress. There are a couple
modules that already implement pub-sub so those modules would be a good
start for reference:
https://github.com/micronaut-projects/micronaut-gcp
https://github.com/micronaut-projects/micronaut-kafka
https://github.com/micronaut-projects/micronaut-mqtt
https://github.com/micronaut-projects/micronaut-ignite/pull/62
Re: [Micronaut] pubsub support
Posted by Saikat Maitra <sa...@gmail.com>.
Hi Denis,
Ignite Messaging apis are considered for depreciation due to mainly being
unused integration. Please find a related presentation that Val presented
to the community.
https://youtu.be/zPuLJgUfLaM?t=1580
Please feel free to start a separate [DISCUSS] thread for IgniteMessaging
api in the dev channel.
Also please find the related thread for Apache Ignite 3.0 release.
http://apache-ignite-developers.2346864.n4.nabble.com/Apache-Ignite-3-0-td48671.html
Regards,
Saikat
On Sun, Nov 1, 2020 at 11:32 AM Saikat Maitra <sa...@gmail.com>
wrote:
> Hi Michael,
>
> My thoughts are that in the intercept method of
> PubSubClientIntroductionAdvice we can publish messages similar to Mqtt.
>
>
> https://github.com/micronaut-projects/micronaut-mqtt/blob/230a6564344ceea21bcf244681d10b79763b2b19/mqtt-core/src/main/java/io/micronaut/mqtt/intercept/AbstractMqttIntroductionAdvice.java#L75
>
> https://github.com/micronaut-projects/micronaut-mqtt/blob/230a6564344ceea21bcf244681d10b79763b2b19/mqttv3/src/main/java/io/micronaut/mqtt/v3/intercept/MqttIntroductionAdvice.java#L48
>
> The publish method will call stmr.addData.
>
> Regards,
> Saikat
>
>
>
> On Sun, Nov 1, 2020 at 3:14 AM Denis Garus <ga...@gmail.com> wrote:
>
>> Hello, Saikat!
>>
>> I know that IgniteMessagin is used in the production environment and
>> I don't see any bugs in Jira with this feature.
>> I wrote some tests for IgniteMessaging in the security suite, and only
>> one confusing thing was noticed [1].
>>
>> Why did we decide to deprecate IgniyeMessaging API in Ignite 3.0?
>>
>> 1.
>> http://apache-ignite-developers.2346864.n4.nabble.com/Inconsistent-behavior-of-IgniteMessaging-td47481.html
>>
>> вс, 1 нояб. 2020 г. в 04:43, Michael Pollind <mp...@gmail.com>:
>>
>>> I'm looking at this data stream api and I'm having a hard time working
>>> out
>>> how this scheme fits into the pub sub model that micronaut uses. Here is
>>> what I've kind of come up with and wonder if this would work.
>>>
>>> @PubSubClientpublic interface MultipleMessage {
>>>
>>> @Topic("animals")
>>> public void send(@key() String key, Animal o);
>>>
>>> @Topic("my_topic")
>>> public void send2(@key() Long key, Object o);
>>> }
>>>
>>>
>>> @PubSubListener
>>>
>>> public class SimpleSub {
>>>
>>> @Subscription("animals")
>>>
>>> public class Listener(@key() String key, Animal o) {
>>>
>>> }
>>>
>>> }
>>>
>>>
>>> On Tue, Oct 27, 2020 at 7:15 PM Saikat Maitra <sa...@gmail.com>
>>> wrote:
>>>
>>> > Hi Michael, Denis
>>> >
>>> > I did an initial review of the PR and wanted to share my thoughts. I
>>> > observed that currently the implementation is using Ignite messaging
>>> apis
>>> > and I am thinking if we should change the implementation to use Ignite
>>> data
>>> > streaming apis. The reason for change is to ensure that it will be
>>> better
>>> > aligned to Ignite 3.0 release as we are planning to deprecate Ignite
>>> > Messaging apis in Ignite 3.0 [1]
>>> >
>>> > My recommendation to use Ignite data streaming apis is also due to the
>>> > fact that we have multiple integration like Google Pub/Sub[2] and Kafka
>>> > Streamer[3] that uses Data streaming apis for integration and it will
>>> keep
>>> > micronaut implementation consistent with other extensions.
>>> >
>>> > Here is an example[4] of how Kafka Streamer can be used to stream data
>>> > into Ignite nodes.
>>> >
>>> >
>>> > [1]
>>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0
>>> > [2]
>>> >
>>> https://github.com/apache/ignite-extensions/blob/master/modules/pub-sub-ext/src/main/java/org/apache/ignite/stream/pubsub/PubSubStreamer.java
>>> > [3]
>>> >
>>> https://github.com/apache/ignite-extensions/blob/master/modules/kafka-ext/src/main/java/org/apache/ignite/stream/kafka/KafkaStreamer.java
>>> > [4]
>>> >
>>> https://github.com/samaitra/streamersk-extensions/blob/main/src/main/kotlin/com/example/streamerskextensions/kafka/KafkaStreamer.kt
>>> >
>>> > Please let me know your feedback.
>>> >
>>> > Regards,
>>> > Saikat
>>> >
>>> > On Tue, Oct 27, 2020 at 12:48 PM Denis Magda <dm...@apache.org>
>>> wrote:
>>> >
>>> >> Michael, glad to see you back!
>>> >>
>>> >> Could you please explain in a few words what this integration does and
>>> >> what
>>> >> it enables for Ignite and Micronaout? So that we are as a community
>>> are on
>>> >> the same page.
>>> >>
>>> >> -
>>> >> Denis
>>> >>
>>> >>
>>> >> On Sun, Oct 25, 2020 at 3:05 PM Michael Pollind <mp...@gmail.com>
>>> >> wrote:
>>> >>
>>> >> > I've started to work on adding some basic pubsub support for
>>> >> > Micronaut-ignite. I only have a draft PR in progress. There are a
>>> couple
>>> >> > modules that already implement pub-sub so those modules would be a
>>> good
>>> >> > start for reference:
>>> >> > https://github.com/micronaut-projects/micronaut-gcp
>>> >> > https://github.com/micronaut-projects/micronaut-kafka
>>> >> > https://github.com/micronaut-projects/micronaut-mqtt
>>> >> >
>>> >> > https://github.com/micronaut-projects/micronaut-ignite/pull/62
>>> >> >
>>> >> >
>>> >> >
>>> >>
>>> >
>>>
>>
Re: [Micronaut] pubsub support
Posted by Saikat Maitra <sa...@gmail.com>.
Hi Michael,
My thoughts are that in the intercept method of
PubSubClientIntroductionAdvice we can publish messages similar to Mqtt.
https://github.com/micronaut-projects/micronaut-mqtt/blob/230a6564344ceea21bcf244681d10b79763b2b19/mqtt-core/src/main/java/io/micronaut/mqtt/intercept/AbstractMqttIntroductionAdvice.java#L75
https://github.com/micronaut-projects/micronaut-mqtt/blob/230a6564344ceea21bcf244681d10b79763b2b19/mqttv3/src/main/java/io/micronaut/mqtt/v3/intercept/MqttIntroductionAdvice.java#L48
The publish method will call stmr.addData.
Regards,
Saikat
On Sun, Nov 1, 2020 at 3:14 AM Denis Garus <ga...@gmail.com> wrote:
> Hello, Saikat!
>
> I know that IgniteMessagin is used in the production environment and
> I don't see any bugs in Jira with this feature.
> I wrote some tests for IgniteMessaging in the security suite, and only one
> confusing thing was noticed [1].
>
> Why did we decide to deprecate IgniyeMessaging API in Ignite 3.0?
>
> 1.
> http://apache-ignite-developers.2346864.n4.nabble.com/Inconsistent-behavior-of-IgniteMessaging-td47481.html
>
> вс, 1 нояб. 2020 г. в 04:43, Michael Pollind <mp...@gmail.com>:
>
>> I'm looking at this data stream api and I'm having a hard time working out
>> how this scheme fits into the pub sub model that micronaut uses. Here is
>> what I've kind of come up with and wonder if this would work.
>>
>> @PubSubClientpublic interface MultipleMessage {
>>
>> @Topic("animals")
>> public void send(@key() String key, Animal o);
>>
>> @Topic("my_topic")
>> public void send2(@key() Long key, Object o);
>> }
>>
>>
>> @PubSubListener
>>
>> public class SimpleSub {
>>
>> @Subscription("animals")
>>
>> public class Listener(@key() String key, Animal o) {
>>
>> }
>>
>> }
>>
>>
>> On Tue, Oct 27, 2020 at 7:15 PM Saikat Maitra <sa...@gmail.com>
>> wrote:
>>
>> > Hi Michael, Denis
>> >
>> > I did an initial review of the PR and wanted to share my thoughts. I
>> > observed that currently the implementation is using Ignite messaging
>> apis
>> > and I am thinking if we should change the implementation to use Ignite
>> data
>> > streaming apis. The reason for change is to ensure that it will be
>> better
>> > aligned to Ignite 3.0 release as we are planning to deprecate Ignite
>> > Messaging apis in Ignite 3.0 [1]
>> >
>> > My recommendation to use Ignite data streaming apis is also due to the
>> > fact that we have multiple integration like Google Pub/Sub[2] and Kafka
>> > Streamer[3] that uses Data streaming apis for integration and it will
>> keep
>> > micronaut implementation consistent with other extensions.
>> >
>> > Here is an example[4] of how Kafka Streamer can be used to stream data
>> > into Ignite nodes.
>> >
>> >
>> > [1]
>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0
>> > [2]
>> >
>> https://github.com/apache/ignite-extensions/blob/master/modules/pub-sub-ext/src/main/java/org/apache/ignite/stream/pubsub/PubSubStreamer.java
>> > [3]
>> >
>> https://github.com/apache/ignite-extensions/blob/master/modules/kafka-ext/src/main/java/org/apache/ignite/stream/kafka/KafkaStreamer.java
>> > [4]
>> >
>> https://github.com/samaitra/streamersk-extensions/blob/main/src/main/kotlin/com/example/streamerskextensions/kafka/KafkaStreamer.kt
>> >
>> > Please let me know your feedback.
>> >
>> > Regards,
>> > Saikat
>> >
>> > On Tue, Oct 27, 2020 at 12:48 PM Denis Magda <dm...@apache.org> wrote:
>> >
>> >> Michael, glad to see you back!
>> >>
>> >> Could you please explain in a few words what this integration does and
>> >> what
>> >> it enables for Ignite and Micronaout? So that we are as a community
>> are on
>> >> the same page.
>> >>
>> >> -
>> >> Denis
>> >>
>> >>
>> >> On Sun, Oct 25, 2020 at 3:05 PM Michael Pollind <mp...@gmail.com>
>> >> wrote:
>> >>
>> >> > I've started to work on adding some basic pubsub support for
>> >> > Micronaut-ignite. I only have a draft PR in progress. There are a
>> couple
>> >> > modules that already implement pub-sub so those modules would be a
>> good
>> >> > start for reference:
>> >> > https://github.com/micronaut-projects/micronaut-gcp
>> >> > https://github.com/micronaut-projects/micronaut-kafka
>> >> > https://github.com/micronaut-projects/micronaut-mqtt
>> >> >
>> >> > https://github.com/micronaut-projects/micronaut-ignite/pull/62
>> >> >
>> >> >
>> >> >
>> >>
>> >
>>
>
Re: [Micronaut] pubsub support
Posted by Denis Garus <ga...@gmail.com>.
Hello, Saikat!
I know that IgniteMessagin is used in the production environment and
I don't see any bugs in Jira with this feature.
I wrote some tests for IgniteMessaging in the security suite, and only one
confusing thing was noticed [1].
Why did we decide to deprecate IgniyeMessaging API in Ignite 3.0?
1.
http://apache-ignite-developers.2346864.n4.nabble.com/Inconsistent-behavior-of-IgniteMessaging-td47481.html
вс, 1 нояб. 2020 г. в 04:43, Michael Pollind <mp...@gmail.com>:
> I'm looking at this data stream api and I'm having a hard time working out
> how this scheme fits into the pub sub model that micronaut uses. Here is
> what I've kind of come up with and wonder if this would work.
>
> @PubSubClientpublic interface MultipleMessage {
>
> @Topic("animals")
> public void send(@key() String key, Animal o);
>
> @Topic("my_topic")
> public void send2(@key() Long key, Object o);
> }
>
>
> @PubSubListener
>
> public class SimpleSub {
>
> @Subscription("animals")
>
> public class Listener(@key() String key, Animal o) {
>
> }
>
> }
>
>
> On Tue, Oct 27, 2020 at 7:15 PM Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hi Michael, Denis
> >
> > I did an initial review of the PR and wanted to share my thoughts. I
> > observed that currently the implementation is using Ignite messaging apis
> > and I am thinking if we should change the implementation to use Ignite
> data
> > streaming apis. The reason for change is to ensure that it will be better
> > aligned to Ignite 3.0 release as we are planning to deprecate Ignite
> > Messaging apis in Ignite 3.0 [1]
> >
> > My recommendation to use Ignite data streaming apis is also due to the
> > fact that we have multiple integration like Google Pub/Sub[2] and Kafka
> > Streamer[3] that uses Data streaming apis for integration and it will
> keep
> > micronaut implementation consistent with other extensions.
> >
> > Here is an example[4] of how Kafka Streamer can be used to stream data
> > into Ignite nodes.
> >
> >
> > [1] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0
> > [2]
> >
> https://github.com/apache/ignite-extensions/blob/master/modules/pub-sub-ext/src/main/java/org/apache/ignite/stream/pubsub/PubSubStreamer.java
> > [3]
> >
> https://github.com/apache/ignite-extensions/blob/master/modules/kafka-ext/src/main/java/org/apache/ignite/stream/kafka/KafkaStreamer.java
> > [4]
> >
> https://github.com/samaitra/streamersk-extensions/blob/main/src/main/kotlin/com/example/streamerskextensions/kafka/KafkaStreamer.kt
> >
> > Please let me know your feedback.
> >
> > Regards,
> > Saikat
> >
> > On Tue, Oct 27, 2020 at 12:48 PM Denis Magda <dm...@apache.org> wrote:
> >
> >> Michael, glad to see you back!
> >>
> >> Could you please explain in a few words what this integration does and
> >> what
> >> it enables for Ignite and Micronaout? So that we are as a community are
> on
> >> the same page.
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Sun, Oct 25, 2020 at 3:05 PM Michael Pollind <mp...@gmail.com>
> >> wrote:
> >>
> >> > I've started to work on adding some basic pubsub support for
> >> > Micronaut-ignite. I only have a draft PR in progress. There are a
> couple
> >> > modules that already implement pub-sub so those modules would be a
> good
> >> > start for reference:
> >> > https://github.com/micronaut-projects/micronaut-gcp
> >> > https://github.com/micronaut-projects/micronaut-kafka
> >> > https://github.com/micronaut-projects/micronaut-mqtt
> >> >
> >> > https://github.com/micronaut-projects/micronaut-ignite/pull/62
> >> >
> >> >
> >> >
> >>
> >
>
Re: [Micronaut] pubsub support
Posted by Michael Pollind <mp...@gmail.com>.
I'm looking at this data stream api and I'm having a hard time working out
how this scheme fits into the pub sub model that micronaut uses. Here is
what I've kind of come up with and wonder if this would work.
@PubSubClientpublic interface MultipleMessage {
@Topic("animals")
public void send(@key() String key, Animal o);
@Topic("my_topic")
public void send2(@key() Long key, Object o);
}
@PubSubListener
public class SimpleSub {
@Subscription("animals")
public class Listener(@key() String key, Animal o) {
}
}
On Tue, Oct 27, 2020 at 7:15 PM Saikat Maitra <sa...@gmail.com>
wrote:
> Hi Michael, Denis
>
> I did an initial review of the PR and wanted to share my thoughts. I
> observed that currently the implementation is using Ignite messaging apis
> and I am thinking if we should change the implementation to use Ignite data
> streaming apis. The reason for change is to ensure that it will be better
> aligned to Ignite 3.0 release as we are planning to deprecate Ignite
> Messaging apis in Ignite 3.0 [1]
>
> My recommendation to use Ignite data streaming apis is also due to the
> fact that we have multiple integration like Google Pub/Sub[2] and Kafka
> Streamer[3] that uses Data streaming apis for integration and it will keep
> micronaut implementation consistent with other extensions.
>
> Here is an example[4] of how Kafka Streamer can be used to stream data
> into Ignite nodes.
>
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0
> [2]
> https://github.com/apache/ignite-extensions/blob/master/modules/pub-sub-ext/src/main/java/org/apache/ignite/stream/pubsub/PubSubStreamer.java
> [3]
> https://github.com/apache/ignite-extensions/blob/master/modules/kafka-ext/src/main/java/org/apache/ignite/stream/kafka/KafkaStreamer.java
> [4]
> https://github.com/samaitra/streamersk-extensions/blob/main/src/main/kotlin/com/example/streamerskextensions/kafka/KafkaStreamer.kt
>
> Please let me know your feedback.
>
> Regards,
> Saikat
>
> On Tue, Oct 27, 2020 at 12:48 PM Denis Magda <dm...@apache.org> wrote:
>
>> Michael, glad to see you back!
>>
>> Could you please explain in a few words what this integration does and
>> what
>> it enables for Ignite and Micronaout? So that we are as a community are on
>> the same page.
>>
>> -
>> Denis
>>
>>
>> On Sun, Oct 25, 2020 at 3:05 PM Michael Pollind <mp...@gmail.com>
>> wrote:
>>
>> > I've started to work on adding some basic pubsub support for
>> > Micronaut-ignite. I only have a draft PR in progress. There are a couple
>> > modules that already implement pub-sub so those modules would be a good
>> > start for reference:
>> > https://github.com/micronaut-projects/micronaut-gcp
>> > https://github.com/micronaut-projects/micronaut-kafka
>> > https://github.com/micronaut-projects/micronaut-mqtt
>> >
>> > https://github.com/micronaut-projects/micronaut-ignite/pull/62
>> >
>> >
>> >
>>
>
Re: [Micronaut] pubsub support
Posted by Saikat Maitra <sa...@gmail.com>.
Hi Michael, Denis
I did an initial review of the PR and wanted to share my thoughts. I
observed that currently the implementation is using Ignite messaging apis
and I am thinking if we should change the implementation to use Ignite data
streaming apis. The reason for change is to ensure that it will be better
aligned to Ignite 3.0 release as we are planning to deprecate Ignite
Messaging apis in Ignite 3.0 [1]
My recommendation to use Ignite data streaming apis is also due to the fact
that we have multiple integration like Google Pub/Sub[2] and Kafka
Streamer[3] that uses Data streaming apis for integration and it will keep
micronaut implementation consistent with other extensions.
Here is an example[4] of how Kafka Streamer can be used to stream data into
Ignite nodes.
[1] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0
[2]
https://github.com/apache/ignite-extensions/blob/master/modules/pub-sub-ext/src/main/java/org/apache/ignite/stream/pubsub/PubSubStreamer.java
[3]
https://github.com/apache/ignite-extensions/blob/master/modules/kafka-ext/src/main/java/org/apache/ignite/stream/kafka/KafkaStreamer.java
[4]
https://github.com/samaitra/streamersk-extensions/blob/main/src/main/kotlin/com/example/streamerskextensions/kafka/KafkaStreamer.kt
Please let me know your feedback.
Regards,
Saikat
On Tue, Oct 27, 2020 at 12:48 PM Denis Magda <dm...@apache.org> wrote:
> Michael, glad to see you back!
>
> Could you please explain in a few words what this integration does and what
> it enables for Ignite and Micronaout? So that we are as a community are on
> the same page.
>
> -
> Denis
>
>
> On Sun, Oct 25, 2020 at 3:05 PM Michael Pollind <mp...@gmail.com>
> wrote:
>
> > I've started to work on adding some basic pubsub support for
> > Micronaut-ignite. I only have a draft PR in progress. There are a couple
> > modules that already implement pub-sub so those modules would be a good
> > start for reference:
> > https://github.com/micronaut-projects/micronaut-gcp
> > https://github.com/micronaut-projects/micronaut-kafka
> > https://github.com/micronaut-projects/micronaut-mqtt
> >
> > https://github.com/micronaut-projects/micronaut-ignite/pull/62
> >
> >
> >
>
Re: [Micronaut] pubsub support
Posted by Denis Magda <dm...@apache.org>.
Michael, glad to see you back!
Could you please explain in a few words what this integration does and what
it enables for Ignite and Micronaout? So that we are as a community are on
the same page.
-
Denis
On Sun, Oct 25, 2020 at 3:05 PM Michael Pollind <mp...@gmail.com> wrote:
> I've started to work on adding some basic pubsub support for
> Micronaut-ignite. I only have a draft PR in progress. There are a couple
> modules that already implement pub-sub so those modules would be a good
> start for reference:
> https://github.com/micronaut-projects/micronaut-gcp
> https://github.com/micronaut-projects/micronaut-kafka
> https://github.com/micronaut-projects/micronaut-mqtt
>
> https://github.com/micronaut-projects/micronaut-ignite/pull/62
>
>
>