You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by drunkKnight <up...@gmail.com> on 2018/12/20 09:59:30 UTC
Concurrent Consumption of Messages Putting a lock on attribute
I have multiple servers that store 10000 users on each. Earlier because of
less time, I was using a Pub/Sub model for different events. It worked as
follows:
Consider the case, where I had to do an OTP verification for a user then I
would write it in DB and later the event to be executed in the DB by a CRON
job and this way it gets consumed.
Now I want to switch to MQs. I am pretty clear with the implementation idea
but there is a small problem where I got stuck.
So the story is:
Consider a user A, on user's onboarding I create events like OTP,
EmailVerification, and AllocateBonus. I'm planning to create a shared queue
between multiple servers that would have the messages (with an attribute
user-id) in the queue and will also publish it to a topic (where the topic
will be the user-id) and a consumer will consume the topic. Once the topic
gets emptied, it will get deleted.
Now the problem is: consider the user A with messages like EmailVerification
and AllocateBonus. Consumer 1 takes up the EmailVerification event, looks
for the user-id in it and then looks for the events in TOPIC- A. At the same
time, consumer 2 takes up the event AllocateBonus and looks for the user-id
in it and then starts executing it.
So before consumer 1 gets completed, consumer 2 will finish and there will
be no bonus allocated to the user because email is not verified.
PROBLEM: I want to acquire a lock on an attribute of the message so that the
events of the same user-id does not get executed by different consumers
without maintaining the order of execution. I have looked up for this but
could not find anything. Is there anyone here who is aware of such
functionality in ActiveMQ? It would be of great help.
Thanks,
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Concurrent Consumption of Messages Putting a lock on attribute
Posted by drunkKnight <up...@gmail.com>.
Thanks a lot for the help.
Since I'm new to MQs, I wanted to ask one more thing. My project is a gradle
build project and dropwizard is the framework (jetty server).
I could not find any ActiveMQ plugin for gradle. Alternatively, I am
planning to add a maven central repository in the build script.
I tried to look up for links where I could get the details/steps for the
integration of ActiveMQ with Jetty and Gradle.
Any help is highly appreciated.
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Concurrent Consumption of Messages Putting a lock on attribute
Posted by Tim Bain <tb...@alumni.duke.edu>.
Correction: this is the actual documentation for message groups in 5.x:
http://activemq.apache.org/message-groups.html
On Thu, Dec 20, 2018, 12:07 PM Tim Bain <tbain@alumni.duke.edu wrote:
> Here's documentation for ActiveMQ 5.x, if that's what you're using.
> http://activemq.apache.org/exclusive-consumer.html
>
> Tim
>
> On Thu, Dec 20, 2018, 10:04 AM Justin Bertram <jbertram@apache.org wrote:
>
>> It sounds to me like you would want to put all the related messages into a
>> single message group [1] together.
>>
>>
>> Justin
>>
>> [1] https://activemq.apache.org/artemis/docs/latest/message-grouping.html
>>
>> On Thu, Dec 20, 2018 at 8:18 AM drunkKnight <up...@gmail.com>
>> wrote:
>>
>> > I have multiple servers that store 10000 users on each. Earlier because
>> of
>> > less time, I was using a Pub/Sub model for different events. It worked
>> as
>> > follows:
>> > Consider the case, where I had to do an OTP verification for a user
>> then I
>> > would write it in DB and later the event to be executed in the DB by a
>> CRON
>> > job and this way it gets consumed.
>> > Now I want to switch to MQs. I am pretty clear with the implementation
>> idea
>> > but there is a small problem where I got stuck.
>> > So the story is:
>> > Consider a user A, on user's onboarding I create events like OTP,
>> > EmailVerification, and AllocateBonus. I'm planning to create a shared
>> queue
>> > between multiple servers that would have the messages (with an attribute
>> > user-id) in the queue and will also publish it to a topic (where the
>> topic
>> > will be the user-id) and a consumer will consume the topic. Once the
>> topic
>> > gets emptied, it will get deleted.
>> >
>> > Now the problem is: consider the user A with messages like
>> > EmailVerification
>> > and AllocateBonus. Consumer 1 takes up the EmailVerification event,
>> looks
>> > for the user-id in it and then looks for the events in TOPIC- A. At the
>> > same
>> > time, consumer 2 takes up the event AllocateBonus and looks for the
>> user-id
>> > in it and then starts executing it.
>> > So before consumer 1 gets completed, consumer 2 will finish and there
>> will
>> > be no bonus allocated to the user because email is not verified.
>> >
>> > PROBLEM: I want to acquire a lock on an attribute of the message so that
>> > the
>> > events of the same user-id does not get executed by different consumers
>> > without maintaining the order of execution. I have looked up for this
>> but
>> > could not find anything. Is there anyone here who is aware of such
>> > functionality in ActiveMQ? It would be of great help.
>> > Thanks,
>> >
>> >
>> >
>> > --
>> > Sent from:
>> > http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>> >
>>
>
Re: Concurrent Consumption of Messages Putting a lock on attribute
Posted by Tim Bain <tb...@alumni.duke.edu>.
Here's documentation for ActiveMQ 5.x, if that's what you're using.
http://activemq.apache.org/exclusive-consumer.html
Tim
On Thu, Dec 20, 2018, 10:04 AM Justin Bertram <jbertram@apache.org wrote:
> It sounds to me like you would want to put all the related messages into a
> single message group [1] together.
>
>
> Justin
>
> [1] https://activemq.apache.org/artemis/docs/latest/message-grouping.html
>
> On Thu, Dec 20, 2018 at 8:18 AM drunkKnight <up...@gmail.com>
> wrote:
>
> > I have multiple servers that store 10000 users on each. Earlier because
> of
> > less time, I was using a Pub/Sub model for different events. It worked as
> > follows:
> > Consider the case, where I had to do an OTP verification for a user then
> I
> > would write it in DB and later the event to be executed in the DB by a
> CRON
> > job and this way it gets consumed.
> > Now I want to switch to MQs. I am pretty clear with the implementation
> idea
> > but there is a small problem where I got stuck.
> > So the story is:
> > Consider a user A, on user's onboarding I create events like OTP,
> > EmailVerification, and AllocateBonus. I'm planning to create a shared
> queue
> > between multiple servers that would have the messages (with an attribute
> > user-id) in the queue and will also publish it to a topic (where the
> topic
> > will be the user-id) and a consumer will consume the topic. Once the
> topic
> > gets emptied, it will get deleted.
> >
> > Now the problem is: consider the user A with messages like
> > EmailVerification
> > and AllocateBonus. Consumer 1 takes up the EmailVerification event, looks
> > for the user-id in it and then looks for the events in TOPIC- A. At the
> > same
> > time, consumer 2 takes up the event AllocateBonus and looks for the
> user-id
> > in it and then starts executing it.
> > So before consumer 1 gets completed, consumer 2 will finish and there
> will
> > be no bonus allocated to the user because email is not verified.
> >
> > PROBLEM: I want to acquire a lock on an attribute of the message so that
> > the
> > events of the same user-id does not get executed by different consumers
> > without maintaining the order of execution. I have looked up for this but
> > could not find anything. Is there anyone here who is aware of such
> > functionality in ActiveMQ? It would be of great help.
> > Thanks,
> >
> >
> >
> > --
> > Sent from:
> > http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
> >
>
Re: Concurrent Consumption of Messages Putting a lock on attribute
Posted by Justin Bertram <jb...@apache.org>.
It sounds to me like you would want to put all the related messages into a
single message group [1] together.
Justin
[1] https://activemq.apache.org/artemis/docs/latest/message-grouping.html
On Thu, Dec 20, 2018 at 8:18 AM drunkKnight <up...@gmail.com>
wrote:
> I have multiple servers that store 10000 users on each. Earlier because of
> less time, I was using a Pub/Sub model for different events. It worked as
> follows:
> Consider the case, where I had to do an OTP verification for a user then I
> would write it in DB and later the event to be executed in the DB by a CRON
> job and this way it gets consumed.
> Now I want to switch to MQs. I am pretty clear with the implementation idea
> but there is a small problem where I got stuck.
> So the story is:
> Consider a user A, on user's onboarding I create events like OTP,
> EmailVerification, and AllocateBonus. I'm planning to create a shared queue
> between multiple servers that would have the messages (with an attribute
> user-id) in the queue and will also publish it to a topic (where the topic
> will be the user-id) and a consumer will consume the topic. Once the topic
> gets emptied, it will get deleted.
>
> Now the problem is: consider the user A with messages like
> EmailVerification
> and AllocateBonus. Consumer 1 takes up the EmailVerification event, looks
> for the user-id in it and then looks for the events in TOPIC- A. At the
> same
> time, consumer 2 takes up the event AllocateBonus and looks for the user-id
> in it and then starts executing it.
> So before consumer 1 gets completed, consumer 2 will finish and there will
> be no bonus allocated to the user because email is not verified.
>
> PROBLEM: I want to acquire a lock on an attribute of the message so that
> the
> events of the same user-id does not get executed by different consumers
> without maintaining the order of execution. I have looked up for this but
> could not find anything. Is there anyone here who is aware of such
> functionality in ActiveMQ? It would be of great help.
> Thanks,
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>