You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Gauri Padbidri <gp...@genius-minds.com> on 2016/09/05 07:17:46 UTC
Kafka related queries
Hi All,
I am very new to Kafka. I am still on the verge of deciding the platform to
choose between Kafka and Kinesis for our requirement. Also it may take me
more time to come up with a full proof answer, hence thought of
parallelizing my search. Following are a few queries around Kafka :
. Does Kafka support sending Streaming Records over raw TCP ?
. Does Kafka expect the packet to be BASE 64 encoded ?
. How much egress and ingress capacity does Kafka support per Partition
?
. We would be using Kafka for ONLY FOR Data Ingest, do read replicas
exist for Partitions and if they get replicated to other nodes in the Kafka
cluster?
Having these queries answered will help me a lot to freeze on Kafka. Look
forward to your quick response. Thanks !
Best Regards,
Gauri Padbidri
Re: Kafka related queries
Posted by Marko Bonaći <ma...@sematext.com>.
BTW regarding latency:
https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines
Marko Bonaći
Monitoring | Alerting | Anomaly Detection | Centralized Log Management
Solr & Elasticsearch Support
Sematext <http://sematext.com/> | Contact
<http://sematext.com/about/contact.html>
On Mon, Sep 5, 2016 at 7:13 PM, Marko Bonaći <ma...@sematext.com>
wrote:
> Hi Gauri,
> I'll take a stab at your questions (others will undoubtedly correct me if
> I'm wrong).
>
> Kafka does work over TCP (what else :)
> No format expectations (binary).
> End to end latency depends on various parameters, like network, memory,
> but it is (if you consume straight away; i.e. no consumer lag; i.e. lag can
> fit in the OS page cache so you're not even hitting disk when consuming)
> measured in low 10s of ms.
> No read replicas. You only read from partition's master. I.e. replicas are
> used to achieve redundancy.
>
>
> Marko Bonaći
> Monitoring | Alerting | Anomaly Detection | Centralized Log Management
> Solr & Elasticsearch Support
> Sematext <http://sematext.com/> | Contact
> <http://sematext.com/about/contact.html>
>
> On Mon, Sep 5, 2016 at 9:17 AM, Gauri Padbidri <gpadbidri@genius-minds.com
> > wrote:
>
>> Hi All,
>>
>>
>>
>> I am very new to Kafka. I am still on the verge of deciding the platform
>> to
>> choose between Kafka and Kinesis for our requirement. Also it may take me
>> more time to come up with a full proof answer, hence thought of
>> parallelizing my search. Following are a few queries around Kafka :
>>
>>
>>
>> . Does Kafka support sending Streaming Records over raw TCP ?
>>
>> . Does Kafka expect the packet to be BASE 64 encoded ?
>>
>> . How much egress and ingress capacity does Kafka support per
>> Partition
>> ?
>>
>> . We would be using Kafka for ONLY FOR Data Ingest, do read replicas
>> exist for Partitions and if they get replicated to other nodes in the
>> Kafka
>> cluster?
>>
>>
>>
>> Having these queries answered will help me a lot to freeze on Kafka. Look
>> forward to your quick response. Thanks !
>>
>>
>>
>> Best Regards,
>>
>> Gauri Padbidri
>>
>>
>
Re: Kafka related queries
Posted by Marko Bonaći <ma...@sematext.com>.
Hi Gauri,
I'll take a stab at your questions (others will undoubtedly correct me if
I'm wrong).
Kafka does work over TCP (what else :)
No format expectations (binary).
End to end latency depends on various parameters, like network, memory, but
it is (if you consume straight away; i.e. no consumer lag; i.e. lag can fit
in the OS page cache so you're not even hitting disk when consuming)
measured in low 10s of ms.
No read replicas. You only read from partition's master. I.e. replicas are
used to achieve redundancy.
Marko Bonaći
Monitoring | Alerting | Anomaly Detection | Centralized Log Management
Solr & Elasticsearch Support
Sematext <http://sematext.com/> | Contact
<http://sematext.com/about/contact.html>
On Mon, Sep 5, 2016 at 9:17 AM, Gauri Padbidri <gp...@genius-minds.com>
wrote:
> Hi All,
>
>
>
> I am very new to Kafka. I am still on the verge of deciding the platform to
> choose between Kafka and Kinesis for our requirement. Also it may take me
> more time to come up with a full proof answer, hence thought of
> parallelizing my search. Following are a few queries around Kafka :
>
>
>
> . Does Kafka support sending Streaming Records over raw TCP ?
>
> . Does Kafka expect the packet to be BASE 64 encoded ?
>
> . How much egress and ingress capacity does Kafka support per
> Partition
> ?
>
> . We would be using Kafka for ONLY FOR Data Ingest, do read replicas
> exist for Partitions and if they get replicated to other nodes in the Kafka
> cluster?
>
>
>
> Having these queries answered will help me a lot to freeze on Kafka. Look
> forward to your quick response. Thanks !
>
>
>
> Best Regards,
>
> Gauri Padbidri
>
>