You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by "kcheng.mvp" <kc...@gmail.com> on 2016/01/07 13:11:41 UTC
Re: Point to point messaging with Ignite
My use case is below.
*Node-A* and *Node-B* would send message with topic 'Topic-Happy'
*Node-C* and *Node-D* subscribe this topic 'Topic-Happy'
no matter *Node-A* or *Node-B* send a message with topic 'Topic-Happy',
*make sure only one node* (Node-C or Node-D) could consume this message
How to achieve this? If there is a way, does this way is smart enough for
loading balance?
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Point-to-point-messaging-with-Ignite-tp1191p2420.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Point to point messaging with Ignite
Posted by "kcheng.mvp" <kc...@gmail.com>.
Thank you very much!
I will write a demo application.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Point-to-point-messaging-with-Ignite-tp1191p2429.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Point to point messaging with Ignite
Posted by Raul Kripalani <ra...@apache.org>.
Yep, although this is not a message-oriented pattern, but rather a RPC call
and an implementation of the "Command Message" integration pattern.
I was under the impression that the OP wanted to send message/data across,
not necessarily invoke a discrete action per message type...
@kcheng – when I mention Distributed CountDownLatch, AtomicCounter and
Locks, I mean that you could implement a coordination algorithm based on
such data structures, e.g. the first node to acquire a named lock becomes
the consumer. The rest remain listening for an event that the master
consumer disappeared, and compete to acquire the lock and become the master
consumer. You could implement this algorithm in many manners, also using
latches and counters.
*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk
On Thu, Jan 7, 2016 at 1:04 PM, Denis Magda <dm...@gridgain.com> wrote:
> In my opinion the best way to implement your scenario is to leverage
> Ignite Compute API.
> The reason is that the API guarantees delivery of compute tasks,
> fault-tolerance and uses load balancing to spread tasks among available
> nodes.
>
> The pseudo-code looks like this
>
> // Start the nodes that should receive tasks/messages by marking them with 'receiver' flag.IgniteConfiguration cfg = new IgniteConfiguration();
>
> HashMap<String, Object> attrs = new HashMap<>();
> attrs.put("receiver", true);
>
> cfg.setUserAttributes(attrs);
>
> ignite = Ignition.start(cfg);
>
> // On sender nodes' side create a cluster group that will contain only the nodes with this attr being set.ClusterGroup group = ignite.cluster().forAttribute("receiver", true);
> // Task will be executed only on one node from the group. Which one node will be taken depends on a load balancer and whether a node is alive or not.ignite.compute(group).call(new IgniteCallable<Object>() {
> @Override public Object call() throws Exception {
> // Execution logic. }
> });
>
>
> --
> Denis
>
>
> On 1/7/2016 3:36 PM, kcheng.mvp wrote:
>
> Thank you very much!
>
>
> I am going to use Ignite Queue implement this.
>
>
> but you mentioned that* Ignite Queue + [Distributed Countdown Latch + Atomic
> Counter] *
>
>
> why need *Distributed Countdown Latch * or* Atomic Counte* here ? As once an
> entry item from poll from the queue, then other node can not get the same
> entry item.
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Point-to-point-messaging-with-Ignite-tp1191p2426.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
>
>
Re: Point to point messaging with Ignite
Posted by Denis Magda <dm...@gridgain.com>.
Correct, you should either have classes of all compute tasks, closures,
etc. that are sent over compute API on all the destination nodes or can
just leverage P2P.
My preference is to use P2P in development and switch it off copying
required classes to all required nodes in production in case if the
classes are rarely changed.
However if the classes tend to change frequently I would keep using P2P
in production as well.
--
Denis
On 1/7/2016 4:08 PM, kcheng.mvp wrote:
> If you use compute API then the current node and remote nodes in the same
> clusters(group) should have same copy of jar/class. Although it can be
> achieved by P2P class loading, right?
>
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Point-to-point-messaging-with-Ignite-tp1191p2432.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Point to point messaging with Ignite
Posted by "kcheng.mvp" <kc...@gmail.com>.
If you use compute API then the current node and remote nodes in the same
clusters(group) should have same copy of jar/class. Although it can be
achieved by P2P class loading, right?
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Point-to-point-messaging-with-Ignite-tp1191p2432.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Point to point messaging with Ignite
Posted by Denis Magda <dm...@gridgain.com>.
In my opinion the best way to implement your scenario is to leverage
Ignite Compute API.
The reason is that the API guarantees delivery of compute tasks,
fault-tolerance and uses load balancing to spread tasks among available
nodes.
The pseudo-code looks like this
// Start the nodes that should receive tasks/messages by marking them
with 'receiver' flag. IgniteConfiguration cfg =new IgniteConfiguration();
HashMap<String, Object> attrs =new HashMap<>();
attrs.put("receiver",true);
cfg.setUserAttributes(attrs);
ignite = Ignition.start(cfg);
// On sender nodes' side create a cluster group that will contain only
the nodes with this attr being set. ClusterGroup group = ignite.cluster().forAttribute("receiver",true);
// Task will be executed only on one node from the group. Which one node
will be taken depends on a load balancer and whether a node is alive or
not. ignite.compute(group).call(new IgniteCallable<Object>() {
@Override public Object call()throws Exception {
// Execution logic. }
});
--
Denis
On 1/7/2016 3:36 PM, kcheng.mvp wrote:
> Thank you very much!
>
>
> I am going to use Ignite Queue implement this.
>
>
> but you mentioned that* Ignite Queue + [Distributed Countdown Latch + Atomic
> Counter] *
>
>
> why need *Distributed Countdown Latch * or* Atomic Counte* here ? As once an
> entry item from poll from the queue, then other node can not get the same
> entry item.
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Point-to-point-messaging-with-Ignite-tp1191p2426.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Point to point messaging with Ignite
Posted by "kcheng.mvp" <kc...@gmail.com>.
Thank you very much!
I am going to use Ignite Queue implement this.
but you mentioned that* Ignite Queue + [Distributed Countdown Latch + Atomic
Counter] *
why need *Distributed Countdown Latch * or* Atomic Counte* here ? As once an
entry item from poll from the queue, then other node can not get the same
entry item.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Point-to-point-messaging-with-Ignite-tp1191p2426.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Point to point messaging with Ignite
Posted by Raul Kripalani <ra...@apache.org>.
Ah, you could also use distributed locks to represent ownership, and events
to get informed when a node leaves, without blocking a thread indefinitely
until the lock is released. There are many possibilities.
*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk
On Thu, Jan 7, 2016 at 12:32 PM, Raul Kripalani <ra...@apache.org> wrote:
> Take into account that Ignite topics, by nature, are a broadcast channel,
> which means that all nodes subscribed to topic XYZ will receive a copy of
> the event.
>
> Having said that, you could synchronise multiple nodes to ensure only a
> single one is subscribed to a given topic at any point in time, by using a
> Distributed CountDown Latch, or some other similar coordination mechanism
> to make potential subscribers compete during a failover.
>
> You do need to think about your QoS and delivery guarantees. If a message
> is sent during a brief period where no nodes are subscribed, the message
> would be lost. Another possibility would be to use an Ignite Queue +
> [Distributed Countdown Latch + Atomic Counter] for coordination. Queues
> would accumulate unconsumed messages until a node takes them.
>
> Or as Valentin said, use the compute API.
>
> Regards,
>
> *Raúl Kripalani*
> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> Messaging Engineer
> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> http://blog.raulkr.net | twitter: @raulvk
>
> On Thu, Jan 7, 2016 at 12:11 PM, kcheng.mvp <kc...@gmail.com> wrote:
>
>> My use case is below.
>>
>>
>> *Node-A* and *Node-B* would send message with topic 'Topic-Happy'
>>
>> *Node-C* and *Node-D* subscribe this topic 'Topic-Happy'
>>
>>
>> no matter *Node-A* or *Node-B* send a message with topic 'Topic-Happy',
>> *make sure only one node* (Node-C or Node-D) could consume this message
>>
>>
>> How to achieve this? If there is a way, does this way is smart enough for
>> loading balance?
>>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-ignite-users.70518.x6.nabble.com/Point-to-point-messaging-with-Ignite-tp1191p2420.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>
Re: Point to point messaging with Ignite
Posted by Raul Kripalani <ra...@apache.org>.
Take into account that Ignite topics, by nature, are a broadcast channel,
which means that all nodes subscribed to topic XYZ will receive a copy of
the event.
Having said that, you could synchronise multiple nodes to ensure only a
single one is subscribed to a given topic at any point in time, by using a
Distributed CountDown Latch, or some other similar coordination mechanism
to make potential subscribers compete during a failover.
You do need to think about your QoS and delivery guarantees. If a message
is sent during a brief period where no nodes are subscribed, the message
would be lost. Another possibility would be to use an Ignite Queue +
[Distributed Countdown Latch + Atomic Counter] for coordination. Queues
would accumulate unconsumed messages until a node takes them.
Or as Valentin said, use the compute API.
Regards,
*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk
On Thu, Jan 7, 2016 at 12:11 PM, kcheng.mvp <kc...@gmail.com> wrote:
> My use case is below.
>
>
> *Node-A* and *Node-B* would send message with topic 'Topic-Happy'
>
> *Node-C* and *Node-D* subscribe this topic 'Topic-Happy'
>
>
> no matter *Node-A* or *Node-B* send a message with topic 'Topic-Happy',
> *make sure only one node* (Node-C or Node-D) could consume this message
>
>
> How to achieve this? If there is a way, does this way is smart enough for
> loading balance?
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Point-to-point-messaging-with-Ignite-tp1191p2420.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>