You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by JO...@DENSO-DIAM.COM on 2018/04/19 12:06:55 UTC
Consuming messages from topics based on keys
I am trying to determine how our consumer will send data to a rest API.
The topics are machine based topics meaning they store all the information
about a specific machine in one topic. I then have keys that are
identifying the type of information stored. Here are some examples:
Topic: E347-8 Key: machine-fault
Topic: E347-8 Key: good-part-count
Currently in the rest end point I am running the key through an if/else
and the deciding how to handle it. Is there a better way to handle this?
Can I have my consumer listen for a topic / key pair? I was thinking I
could layout my consumers to be code specific like this example
Consumer 1:
Topic: E347-8 Key: machine_fault
Topic: 186_7 Key: machine_fault
Consumer 2:
Topic: E347-8 Key: good-part-count
Topic: 186_7 Key: good-part-count
Thanks,
The information contained in and transmitted with this Email may be
privileged, proprietary, confidential and protected from disclosure. No
privilege is hereby intended to be waived. This Email is intended only for
the person to whom it is addressed. If you are not the intended
recipient/addressee, any use of the Email and/or its contents, including,
but not limited to, dissemination, distribution or copying is strictly
prohibited and may be unlawful, and you must not take any action in
reliance on it. If you receive this Email in error, please immediately
notify the sender and delete the original message and any copies of it
from your computer system. We deny any liability for damages resulting
from the use of this Email by the unintended recipient, including the
recipient in error.
Re: Consuming messages from topics based on keys
Posted by Rahul Singh <ra...@gmail.com>.
I think this can be done in two ways.
1. Kstream or Ktable filter in a topology.
2. Store data in a persistent store elsewhere and expose via API (like Cassandra)
--
Rahul Singh
rahul.singh@anant.us
Anant Corporation
On Apr 19, 2018, 7:07 AM -0500, JOE_DELBRIDGE@denso-diam.com, wrote:
> I am trying to determine how our consumer will send data to a rest API.
> The topics are machine based topics meaning they store all the information
> about a specific machine in one topic. I then have keys that are
> identifying the type of information stored. Here are some examples:
>
> Topic: E347-8 Key: machine-fault
> Topic: E347-8 Key: good-part-count
>
> Currently in the rest end point I am running the key through an if/else
> and the deciding how to handle it. Is there a better way to handle this?
> Can I have my consumer listen for a topic / key pair? I was thinking I
> could layout my consumers to be code specific like this example
>
> Consumer 1:
> Topic: E347-8 Key: machine_fault
> Topic: 186_7 Key: machine_fault
>
> Consumer 2:
> Topic: E347-8 Key: good-part-count
> Topic: 186_7 Key: good-part-count
>
>
> Thanks,
>
>
>
>
>
>
>
>
> The information contained in and transmitted with this Email may be
> privileged, proprietary, confidential and protected from disclosure. No
> privilege is hereby intended to be waived. This Email is intended only for
> the person to whom it is addressed. If you are not the intended
> recipient/addressee, any use of the Email and/or its contents, including,
> but not limited to, dissemination, distribution or copying is strictly
> prohibited and may be unlawful, and you must not take any action in
> reliance on it. If you receive this Email in error, please immediately
> notify the sender and delete the original message and any copies of it
> from your computer system. We deny any liability for damages resulting
> from the use of this Email by the unintended recipient, including the
> recipient in error.
>