You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by BigData dev <bi...@gmail.com> on 2017/02/15 19:03:39 UTC

Reg: Kafka HDFS Connector with (HDFS SSL enabled)

Hi,

Does Kafka HDFS Connect work with HDFS (SSL). As I see only properties in
security is
hdfs.authentication.kerberos, connect.hdfs.keytab,hdfs.namenode.principal
as these properties are all related to HDFS Kerberos.

As from the configuration and code I see we pass only Kerberos parameters,
not seen SSL configuration, so want to confirm will the Kafka HDFS
Connector works with HDFS (SSL enabled)?

Could you please provide any information on this.


Thanks

Re: Reg: Kafka HDFS Connector with (HDFS SSL enabled)

Posted by BigData dev <bi...@gmail.com>.
Hi Colin,
I have configured SSL in HDFS and used SWebHDFS.
I am able to make it work with Kafka HDFS Connector.


Thanks,
Bharat


On Fri, Feb 17, 2017 at 1:47 PM, Colin McCabe <cm...@apache.org> wrote:

> Hi,
>
> Just to be clear, HDFS doesn't use HTTP or HTTPS as its primary
> transport mechanism.  Instead, HDFS uses the Hadoop RPC transport
> mechanism.  So in general, it should not be necessary to configure SSL
> to connect a client to HDFS.
>
> HDFS does "support SSL" in the sense that the HDFS web UI can be
> configured to use it.  You can also use HttpFS or WebHDFS to access
> HDFS, which might motivate you to configure SSL.  Are you trying to
> configure one of these?
>
> best,
> Colin
>
>
> On Wed, Feb 15, 2017, at 11:03, BigData dev wrote:
> > Hi,
> >
> > Does Kafka HDFS Connect work with HDFS (SSL). As I see only properties in
> > security is
> > hdfs.authentication.kerberos, connect.hdfs.keytab,hdfs.
> namenode.principal
> > as these properties are all related to HDFS Kerberos.
> >
> > As from the configuration and code I see we pass only Kerberos
> > parameters,
> > not seen SSL configuration, so want to confirm will the Kafka HDFS
> > Connector works with HDFS (SSL enabled)?
> >
> > Could you please provide any information on this.
> >
> >
> > Thanks
>

Re: Reg: Kafka HDFS Connector with (HDFS SSL enabled)

Posted by Colin McCabe <cm...@apache.org>.
Hi,

Just to be clear, HDFS doesn't use HTTP or HTTPS as its primary
transport mechanism.  Instead, HDFS uses the Hadoop RPC transport
mechanism.  So in general, it should not be necessary to configure SSL
to connect a client to HDFS.

HDFS does "support SSL" in the sense that the HDFS web UI can be
configured to use it.  You can also use HttpFS or WebHDFS to access
HDFS, which might motivate you to configure SSL.  Are you trying to
configure one of these?

best,
Colin


On Wed, Feb 15, 2017, at 11:03, BigData dev wrote:
> Hi,
> 
> Does Kafka HDFS Connect work with HDFS (SSL). As I see only properties in
> security is
> hdfs.authentication.kerberos, connect.hdfs.keytab,hdfs.namenode.principal
> as these properties are all related to HDFS Kerberos.
> 
> As from the configuration and code I see we pass only Kerberos
> parameters,
> not seen SSL configuration, so want to confirm will the Kafka HDFS
> Connector works with HDFS (SSL enabled)?
> 
> Could you please provide any information on this.
> 
> 
> Thanks