Apache Kafka Questions

[Jatin Chhabriya <ja...@circularedge.com>]

Hello Team

Upon careful perusal of documentation and tutorials, our team has a few open questions, and we would appreciate to have these clarified by a hands-on practitioner.

a. Can there by more than one Producer in a Publish-Subscribe model?

b. More details on level of abstraction between Producer/Consumer. Is one aware of others identity and Zookeepers offering on same

c. Can multiple Producers send message to the same Topic

d. How long are messages retained in Publish->Subscribe model?

e. How are messages stored?

Thanks and Regards
C. Jatin Shyam | Devops Engineer | Circular Edge Solutions, Chennai, India

Re: Apache Kafka Questions

["Tauzell, Dave" <Da...@surescripts.com>]

From: Jatin Chhabriya <ja...@circularedge.com>
Date: Wednesday, March 16, 2022 at 9:20 AM
To: users@kafka.apache.org <us...@kafka.apache.org>
Cc: Murali Krishna <mu...@circularedge.com>
Subject: [EXTERNAL] Apache Kafka Questions
Hello Team

Upon careful perusal of documentation and tutorials, our team has a few open questions, and we would appreciate to have these clarified by a hands-on practitioner.

a. Can there by more than one Producer in a Publish-Subscribe model?
Yes


b. More details on level of abstraction between Producer/Consumer. Is one aware of others identity and Zookeepers offering on same
No,  Consumers and Producers only need to know about the topic.   Consumers belong to a “group” which allows many consumers to read from the same topic but only process each message once.


c. Can multiple Producers send message to the same Topic
Yes


d. How long are messages retained in Publish->Subscribe model?
Kafka has time and size based retention policies: https://www.baeldung.com/kafka-message-retention


e. How are messages stored?
On disk,  possibly replicated: https://www.baeldung.com/kafka-message-retention


Thanks and Regards
C. Jatin Shyam | Devops Engineer | Circular Edge Solutions, Chennai, India
This e-mail and any files transmitted with it are confidential, may contain sensitive information, and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by reply e-mail immediately and destroy all copies of the e-mail and any attachments.

Re: Apache Kafka Questions

[Jatin Chhabriya <ja...@circularedge.com>]

Hello Team

Thank you for answering the questions I reached out earlier. We have a new set, and I would really like to have them verified and confirmed by an expert. Request to answer them with specification in a way they can be replicated in local user environment.

f. How are Consumer Groups created? Are new consumers automatically bound to a default group (using consumer group id)?
g. How do Offsets help in achieving Fault Tolerance?
h. Can we identify the IP Address of the Broker Leader from which the message originated via the Meta.Properties file?
i. In the server.properties file if I set the partition count to >1, are partitions implicitly created, if yes, are they all within the same node?

Thanks and Regards,
C. Jatin Shyam
T: @jatinchhab
________________________________
From: Jatin Chhabriya
Sent: Wednesday, March 16, 2022 7:32 PM
To: users@kafka.apache.org <us...@kafka.apache.org>
Cc: Murali Krishna <mu...@circularedge.com>
Subject: Apache Kafka Questions

Hello Team

Upon careful perusal of documentation and tutorials, our team has a few open questions, and we would appreciate to have these clarified by a hands-on practitioner.

a. Can there by more than one Producer in a Publish-Subscribe model?

b. More details on level of abstraction between Producer/Consumer. Is one aware of others identity and Zookeepers offering on same

c. Can multiple Producers send message to the same Topic

d. How long are messages retained in Publish->Subscribe model?

e. How are messages stored?

Thanks and Regards
C. Jatin Shyam | Devops Engineer | Circular Edge Solutions, Chennai, India