You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Abhimanyu Nagrath <ab...@gmail.com> on 2017/06/08 13:19:55 UTC
ticketing system Design
Hi ,
Is Apache Kafka along with storm can be used to design a ticketing system.
By ticketing system, I mean that there are millions of tasks stored in
Kafka queues and there are processes/humans to take some actions on the
task. there are come constraints that same task should not be assigned to
two processes/humans and if a task flows to a process/human and no action
is performed it should be reassigned.
I am not sure whether this can be solved using Kafka.Any help is
appreciated
Regards,
Abhimanyu
Re: ticketing system Design
Posted by Neil Moore <ne...@adobe.com.INVALID>.
It can be implemented on top of Kafka with some effort, but Kafka doesn't solve this problem for you directly.
Amazon SQS does provide something like what you want, perhaps looking for other projects similar to SQS will yield some results.
________________________________
From: Abhimanyu Nagrath <ab...@gmail.com>
Sent: 20 June 2017 04:01:15
To: users@kafka.apache.org
Subject: Re: ticketing system Design
Hi ,
Can anyone suggest me where I can get the answer for these type of
questions?
Regards,
Abhimanyu
On Thu, Jun 8, 2017 at 6:49 PM, Abhimanyu Nagrath <
abhimanyunagrath@gmail.com> wrote:
> Hi ,
>
> Is Apache Kafka along with storm can be used to design a ticketing system.
> By ticketing system, I mean that there are millions of tasks stored in
> Kafka queues and there are processes/humans to take some actions on the
> task. there are come constraints that same task should not be assigned to
> two processes/humans and if a task flows to a process/human and no action
> is performed it should be reassigned.
> I am not sure whether this can be solved using Kafka.Any help is
> appreciated
>
>
>
> Regards,
> Abhimanyu
>
Re: ticketing system Design
Posted by Sameer Kumar <sa...@gmail.com>.
Hi Abhimanya,
You can very well do it through Kafka, KafkaStreams and something like
redis.
I would design it to be something like this:-
1. Topic 1 - Pending tasks
2. Topic 2 - Reassigned Tasks.
3. Topic 3- Task To Resource Mapping.
Some other components could be:-
4. Redis Hash(task progress info)
We can also need to store a mapping of resources with completed task count
and in progress task count. This shall help us in ensuring that pipeline
remains active and bandwidth is managed between people who are free vs
busy(managed through counts).
5. TaskAssigner - KStreams - listens to the Pending Tasks and Reassigned
queue. sees who can be assigned task through reading redis hash.
6. Task Processor - java/scala program - processes the task.Probably
trigged after resource processes the task.
7. Task rebalancer - java/scala program - triggered at timely intervals to
do the reassignments.
-Sameer.
On Wed, Jun 21, 2017 at 7:55 AM, Tarun Garg <bi...@live.com> wrote:
> need some more input on this.
>
>
> Kafka is a queue it doesn't take any action.
>
>
> sender(producer) sends data to kafka and consumer pulls data from kafka
> queue. so there is no assignment of data to any consumer.
>
> if a process/human cann't take any action then kafka cann't help in this
> case.
>
> hope it answers.
>
> ________________________________
> From: Abhimanyu Nagrath <ab...@gmail.com>
> Sent: Monday, June 19, 2017 8:01 PM
> To: users@kafka.apache.org
> Subject: Re: ticketing system Design
>
> Hi ,
>
> Can anyone suggest me where I can get the answer for these type of
> questions?
>
>
> Regards,
> Abhimanyu
>
> On Thu, Jun 8, 2017 at 6:49 PM, Abhimanyu Nagrath <
> abhimanyunagrath@gmail.com> wrote:
>
> > Hi ,
> >
> > Is Apache Kafka along with storm can be used to design a ticketing
> system.
> > By ticketing system, I mean that there are millions of tasks stored in
> > Kafka queues and there are processes/humans to take some actions on the
> > task. there are come constraints that same task should not be assigned to
> > two processes/humans and if a task flows to a process/human and no action
> > is performed it should be reassigned.
> > I am not sure whether this can be solved using Kafka.Any help is
> > appreciated
> >
> >
> >
> > Regards,
> > Abhimanyu
> >
>
Re: ticketing system Design
Posted by Michal Borowiecki <mi...@openbet.com>.
If your business flow involves human actions, personally I would look at
a business process engine like the open source camunda.
Even if you don't choose to use it in production, you can use it to
prototype and evolve your design at the inception stage.
There's a simple to run example that integrates with kafka here:
https://github.com/flowing/flowing-retail
And a tutorial that involves a human action in the flow here (but no kafka):
https://docs.camunda.org/get-started/bpmn20/
(NB. My personal interest in camunda is for integrating it as a process
manager/saga element in an event-sourced service at some point)
Cheers,
Michał
On 21/06/17 03:25, Tarun Garg wrote:
> need some more input on this.
>
>
> Kafka is a queue it doesn't take any action.
>
>
> sender(producer) sends data to kafka and consumer pulls data from kafka queue. so there is no assignment of data to any consumer.
>
> if a process/human cann't take any action then kafka cann't help in this case.
>
> hope it answers.
>
> ________________________________
> From: Abhimanyu Nagrath <ab...@gmail.com>
> Sent: Monday, June 19, 2017 8:01 PM
> To: users@kafka.apache.org
> Subject: Re: ticketing system Design
>
> Hi ,
>
> Can anyone suggest me where I can get the answer for these type of
> questions?
>
>
> Regards,
> Abhimanyu
>
> On Thu, Jun 8, 2017 at 6:49 PM, Abhimanyu Nagrath <
> abhimanyunagrath@gmail.com> wrote:
>
>> Hi ,
>>
>> Is Apache Kafka along with storm can be used to design a ticketing system.
>> By ticketing system, I mean that there are millions of tasks stored in
>> Kafka queues and there are processes/humans to take some actions on the
>> task. there are come constraints that same task should not be assigned to
>> two processes/humans and if a task flows to a process/human and no action
>> is performed it should be reassigned.
>> I am not sure whether this can be solved using Kafka.Any help is
>> appreciated
>>
>>
>>
>> Regards,
>> Abhimanyu
>>
--
Signature
<http://www.openbet.com/> Michal Borowiecki
Senior Software Engineer L4
T: +44 208 742 1600
+44 203 249 8448
E: michal.borowiecki@openbet.com
W: www.openbet.com <http://www.openbet.com/>
OpenBet Ltd
Chiswick Park Building 9
566 Chiswick High Rd
London
W4 5XT
UK
<https://www.openbet.com/email_promo>
This message is confidential and intended only for the addressee. If you
have received this message in error, please immediately notify the
postmaster@openbet.com <ma...@openbet.com> and delete it
from your system as well as any copies. The content of e-mails as well
as traffic data may be monitored by OpenBet for employment and security
purposes. To protect the environment please do not print this e-mail
unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building
9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company
registered in England and Wales. Registered no. 3134634. VAT no.
GB927523612
Re: ticketing system Design
Posted by Tarun Garg <bi...@live.com>.
need some more input on this.
Kafka is a queue it doesn't take any action.
sender(producer) sends data to kafka and consumer pulls data from kafka queue. so there is no assignment of data to any consumer.
if a process/human cann't take any action then kafka cann't help in this case.
hope it answers.
________________________________
From: Abhimanyu Nagrath <ab...@gmail.com>
Sent: Monday, June 19, 2017 8:01 PM
To: users@kafka.apache.org
Subject: Re: ticketing system Design
Hi ,
Can anyone suggest me where I can get the answer for these type of
questions?
Regards,
Abhimanyu
On Thu, Jun 8, 2017 at 6:49 PM, Abhimanyu Nagrath <
abhimanyunagrath@gmail.com> wrote:
> Hi ,
>
> Is Apache Kafka along with storm can be used to design a ticketing system.
> By ticketing system, I mean that there are millions of tasks stored in
> Kafka queues and there are processes/humans to take some actions on the
> task. there are come constraints that same task should not be assigned to
> two processes/humans and if a task flows to a process/human and no action
> is performed it should be reassigned.
> I am not sure whether this can be solved using Kafka.Any help is
> appreciated
>
>
>
> Regards,
> Abhimanyu
>
Re: ticketing system Design
Posted by Abhimanyu Nagrath <ab...@gmail.com>.
Hi ,
Can anyone suggest me where I can get the answer for these type of
questions?
Regards,
Abhimanyu
On Thu, Jun 8, 2017 at 6:49 PM, Abhimanyu Nagrath <
abhimanyunagrath@gmail.com> wrote:
> Hi ,
>
> Is Apache Kafka along with storm can be used to design a ticketing system.
> By ticketing system, I mean that there are millions of tasks stored in
> Kafka queues and there are processes/humans to take some actions on the
> task. there are come constraints that same task should not be assigned to
> two processes/humans and if a task flows to a process/human and no action
> is performed it should be reassigned.
> I am not sure whether this can be solved using Kafka.Any help is
> appreciated
>
>
>
> Regards,
> Abhimanyu
>