You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "Doyle, Keith" <Ke...@greenwayhealth.com> on 2016/11/16 17:00:49 UTC
Topics vs Partitions
Trying to wrap my head around some of the concepts...
For semantic partitioning, is there any rationale for choosing to divide a = single topic into multiple partitions as opposed to just creating multiple = topics each with a single partition?
--
[Greenway_Health_PNG_large_NO_tag]
Keith Doyle | Senior Software Engineer
Greenway Health | 4301 W. Boy Scout Blvd., Suite 800, Tampa, FL 33607
(702) 256-9911 office | GreenwayHealth.com<http://www.greenwayhealth.com/>
[cid:image003.png@01D04086.868CBCB0]<https://www.linkedin.com/company/greenway-health>[cid:image004.png@01D04086.868CBCB0]<https://www.facebook.com/GreenwayHealthLLC>[cid:image005.png@01D04086.868CBCB0]<https://twitter.com/greenway>
NOTICE: This e-mail message and all attachments transmitted with it may contain legally privileged and confidential information intended solely for the use of the addressee. If the reader of this message is not the intended recipient, you are hereby notified that any reading, dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by electronic mail and delete this message and all copies and backups thereof. Thank you. Greenway Health.
Re: Topics vs Partitions
Posted by Zakee <kz...@netzero.net>.
Messages in partitions are logically the same. Topic is a logical group of partitions for messages that are of same type and/or from same source e.g. if you have two web-apps producing data, you will like to tell data from one source that from another. These are candidates for a topic each. If you see increase in data over time, you may want to consider increasing partitions to achieve better throughputs. As Dave said, partitions help in distributing / dividing the load from a source into multiple parallel streams, for faster throughputs. This means each partition deals with less (part of) data and maintains its replica state on different nodes.
Same thing on the consumer end, you can start as many parallel processes (or threads) as there are partitions of a topic, to achieve better throughputs.
-Zakee
> On Nov 16, 2016, at 9:03 AM, Tauzell, Dave <Da...@surescripts.com> wrote:
>
> Partitions are used to distribute the messages in a topic between several different broker instances. This provides higher throughput. Partitions can also be replicate which allows for high availability.
>
> -Dave
>
> From: Doyle, Keith [mailto:Keith.Doyle@greenwayhealth.com]
> Sent: Wednesday, November 16, 2016 11:01 AM
> To: users@kafka.apache.org
> Subject: Topics vs Partitions
>
>
> Trying to wrap my head around some of the concepts...
>
>
>
> For semantic partitioning, is there any rationale for choosing to divide a = single topic into multiple partitions as opposed to just creating multiple = topics each with a single partition?
>
>
>
>
> --
>
> [Greenway_Health_PNG_large_NO_tag]
>
> Keith Doyle | Senior Software Engineer
> Greenway Health | 4301 W. Boy Scout Blvd., Suite 800, Tampa, FL 33607
> (702) 256-9911 office | GreenwayHealth.com<http://www.greenwayhealth.com/>
> [cid:image003.png@01D04086.868CBCB0]<https://www.linkedin.com/company/greenway-health>[cid:image004.png@01D04086.868CBCB0]<https://www.facebook.com/GreenwayHealthLLC>[cid:image005.png@01D04086.868CBCB0]<https://twitter.com/greenway>
>
> NOTICE: This e-mail message and all attachments transmitted with it may contain legally privileged and confidential information intended solely for the use of the addressee. If the reader of this message is not the intended recipient, you are hereby notified that any reading, dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by electronic mail and delete this message and all copies and backups thereof. Thank you. Greenway Health.
> This e-mail and any files transmitted with it are confidential, may contain sensitive information, and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by reply e-mail immediately and destroy all copies of the e-mail and any attachments.
____________________________________________________________
3 Signs You May Have a Fatty Liver [Watch]
livecellresearch.com
http://thirdpartyoffers.netzero.net/TGL3231/582cb5ecce48a35ec2d52st03vuc
RE: Topics vs Partitions
Posted by "Tauzell, Dave" <Da...@surescripts.com>.
Partitions are used to distribute the messages in a topic between several different broker instances. This provides higher throughput. Partitions can also be replicate which allows for high availability.
-Dave
From: Doyle, Keith [mailto:Keith.Doyle@greenwayhealth.com]
Sent: Wednesday, November 16, 2016 11:01 AM
To: users@kafka.apache.org
Subject: Topics vs Partitions
Trying to wrap my head around some of the concepts...
For semantic partitioning, is there any rationale for choosing to divide a = single topic into multiple partitions as opposed to just creating multiple = topics each with a single partition?
--
[Greenway_Health_PNG_large_NO_tag]
Keith Doyle | Senior Software Engineer
Greenway Health | 4301 W. Boy Scout Blvd., Suite 800, Tampa, FL 33607
(702) 256-9911 office | GreenwayHealth.com<http://www.greenwayhealth.com/>
[cid:image003.png@01D04086.868CBCB0]<https://www.linkedin.com/company/greenway-health>[cid:image004.png@01D04086.868CBCB0]<https://www.facebook.com/GreenwayHealthLLC>[cid:image005.png@01D04086.868CBCB0]<https://twitter.com/greenway>
NOTICE: This e-mail message and all attachments transmitted with it may contain legally privileged and confidential information intended solely for the use of the addressee. If the reader of this message is not the intended recipient, you are hereby notified that any reading, dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by electronic mail and delete this message and all copies and backups thereof. Thank you. Greenway Health.
This e-mail and any files transmitted with it are confidential, may contain sensitive information, and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by reply e-mail immediately and destroy all copies of the e-mail and any attachments.