You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Jan <cn...@yahoo.com.INVALID> on 2016/03/03 06:52:33 UTC
Kafka over Satellite links
Hi folks;
does anyone know of Kafka's ability to work over Satellite links. We have a IoT Telemetry application that uses Satellite communication to send data from remote sites to a Central hub.
Any help/ input/ links/ gotchas would be much appreciated.
Regards,Jan
Re: Kafka over Satellite links
Posted by Mathias Herberts <ma...@gmail.com>.
Sat links induce roughly 250ms of latency, you have to tweak TCP Windows so
you can saturate the link as much as you can, but besides that Sat links
are rather common.
On Mar 4, 2016 7:26 AM, "Jan" <cn...@yahoo.com.invalid> wrote:
> Thanks for the input.
> The IoT application could have a http ingestion receiver to funnel data
> into Kafka. However, I am thinking about Kafka's ability to be able to
> injest traffic directly from IoT devices.
> Does anyone have any recommendations while using Kafka Connect between
> producer & consumer sides over Satellite links. Does the Satellite
> link really make any difference v/s a slow Internet link ?
> Input/ suggestions would be much appreciated.
> ThanksJan
>
>
> On Thursday, March 3, 2016 1:16 AM, Christian Csar <ch...@csar.us>
> wrote:
>
>
> I would not do that. I admit I may be a bit biased due to working for
> Buddy Platform (IoT backend stuff including telemetry collection), but
> you want to send the data via some protocol (HTTP? MQTT? COAP?) to the
> central hub and then have those servers put the data into Kafka. Now
> if you want to use Kafka there are the various HTTP front ends that
> will basically put the data into Kafka for you without the client
> needing to deal with the partition management part. But putting data
> into Kafka directly really seems like a bad idea even if it's a large
> number of messages per second per node, even if the security parts
> work out for you.
>
> Christian
>
> On Wed, Mar 2, 2016 at 9:52 PM, Jan <cn...@yahoo.com.invalid> wrote:
> > Hi folks;
> > does anyone know of Kafka's ability to work over Satellite links. We
> have a IoT Telemetry application that uses Satellite communication to send
> data from remote sites to a Central hub.
> > Any help/ input/ links/ gotchas would be much appreciated.
> > Regards,Jan
>
>
>
Re: Kafka over Satellite links
Posted by Jan <cn...@yahoo.com.INVALID>.
Thanks for the input.
The IoT application could have a http ingestion receiver to funnel data into Kafka. However, I am thinking about Kafka's ability to be able to injest traffic directly from IoT devices.
Does anyone have any recommendations while using Kafka Connect between producer & consumer sides over Satellite links. Does the Satellite link really make any difference v/s a slow Internet link ?
Input/ suggestions would be much appreciated.
ThanksJan
On Thursday, March 3, 2016 1:16 AM, Christian Csar <ch...@csar.us> wrote:
I would not do that. I admit I may be a bit biased due to working for
Buddy Platform (IoT backend stuff including telemetry collection), but
you want to send the data via some protocol (HTTP? MQTT? COAP?) to the
central hub and then have those servers put the data into Kafka. Now
if you want to use Kafka there are the various HTTP front ends that
will basically put the data into Kafka for you without the client
needing to deal with the partition management part. But putting data
into Kafka directly really seems like a bad idea even if it's a large
number of messages per second per node, even if the security parts
work out for you.
Christian
On Wed, Mar 2, 2016 at 9:52 PM, Jan <cn...@yahoo.com.invalid> wrote:
> Hi folks;
> does anyone know of Kafka's ability to work over Satellite links. We have a IoT Telemetry application that uses Satellite communication to send data from remote sites to a Central hub.
> Any help/ input/ links/ gotchas would be much appreciated.
> Regards,Jan
Re: Kafka over Satellite links
Posted by Christian Csar <ch...@csar.us>.
I would not do that. I admit I may be a bit biased due to working for
Buddy Platform (IoT backend stuff including telemetry collection), but
you want to send the data via some protocol (HTTP? MQTT? COAP?) to the
central hub and then have those servers put the data into Kafka. Now
if you want to use Kafka there are the various HTTP front ends that
will basically put the data into Kafka for you without the client
needing to deal with the partition management part. But putting data
into Kafka directly really seems like a bad idea even if it's a large
number of messages per second per node, even if the security parts
work out for you.
Christian
On Wed, Mar 2, 2016 at 9:52 PM, Jan <cn...@yahoo.com.invalid> wrote:
> Hi folks;
> does anyone know of Kafka's ability to work over Satellite links. We have a IoT Telemetry application that uses Satellite communication to send data from remote sites to a Central hub.
> Any help/ input/ links/ gotchas would be much appreciated.
> Regards,Jan