You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Qi Xu <sh...@gmail.com> on 2015/07/08 03:27:17 UTC
request forwarding in Kafka?
Hi Kafka folks,
I'm investigating about using Kafka as a data ingress endpoint for our
analytic service.
In our scenario, Kafka cluster is behind NLB which has a public IP, and
the data source service is under another public IP. Basically the data
source service cannot access Kafka cluster nodes directly. And a request
hitting the public IP will be routed to any broker node. If Kafka supports
request forwarding, then our scenario will work. I noticed there's an
article "
http://grokbase.com/t/kafka/users/1441aa74t3/kafka-cluster-behind-a-hardware-load-balancer"
and Jun Rao clearly stated that Kafka does not do request forwarding.
I'm wondering if this conclusion is still true for latest Kafka version? If
that, I'm wondering do you have any plan to add the support of request
forwarding?
If not, what's the best practice do you suggest to solve the problem?
Here're what I think of:
1) Make the data source service have direct access to Kafka cluster
nodes---unfortunately this is impossible for us.
2) Make all Kafka cluster nodes have public IP so that they can be
accessed from our data source service. The concerns are a) we may not have
enough IPs, b) the security.
3) Add a proxy service which in the same network of Kafka, to act as data
ingress service.
What do you suggest?
I'm new to Kafka and looking forward your reply. Thanks a lot.
Tony
Re: request forwarding in Kafka?
Posted by Qi Xu <sh...@gmail.com>.
ping~
On Tue, Jul 7, 2015 at 6:27 PM, Qi Xu <sh...@gmail.com> wrote:
> Hi Kafka folks,
> I'm investigating about using Kafka as a data ingress endpoint for our
> analytic service.
> In our scenario, Kafka cluster is behind NLB which has a public IP, and
> the data source service is under another public IP. Basically the data
> source service cannot access Kafka cluster nodes directly. And a request
> hitting the public IP will be routed to any broker node. If Kafka supports
> request forwarding, then our scenario will work. I noticed there's an
> article "
> http://grokbase.com/t/kafka/users/1441aa74t3/kafka-cluster-behind-a-hardware-load-balancer"
> and Jun Rao clearly stated that Kafka does not do request forwarding.
>
> I'm wondering if this conclusion is still true for latest Kafka version?
> If that, I'm wondering do you have any plan to add the support of request
> forwarding?
>
> If not, what's the best practice do you suggest to solve the problem?
> Here're what I think of:
> 1) Make the data source service have direct access to Kafka cluster
> nodes---unfortunately this is impossible for us.
> 2) Make all Kafka cluster nodes have public IP so that they can be
> accessed from our data source service. The concerns are a) we may not have
> enough IPs, b) the security.
> 3) Add a proxy service which in the same network of Kafka, to act as data
> ingress service.
> What do you suggest?
>
> I'm new to Kafka and looking forward your reply. Thanks a lot.
>
>
> Tony
>
>
>