You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by PasLe Choix <Pa...@gmail.com> on 2019/01/08 01:32:06 UTC
jdbc impala url
I need to create a DBCPConnectionPool to connect to my table on Impala in
Cloudera cluster, wondering who can provide a sample URL for the jdbc
connection?
At the moment my URL is as below and is not working:
jdbc:impala://nydc-pcdhwrk01:21050;
AuthMech=1;KrbRealm=BD.NOVANTAS.PRI;KrbHostFQDN=nydc-pcdhwrk01.novantas.com
;KrbServiceName=pchoix
Thank you very much.
*------------------------------------------------*
*Sincerely yours,*
*PC*
Re: jdbc impala url
Posted by Boris Tyukin <bo...@boristyukin.com>.
also check out my post - there were a few pitfalls
https://boristyukin.com/how-to-connect-apache-nifi-to-apache-impala/
On Wed, Jan 9, 2019 at 1:58 PM Boris Tyukin <bo...@boristyukin.com> wrote:
> this is how our connection looks like on a kerberized CDH cluster
>
> jdbc:impala://_host_:21050/default;AuthMech=1;KrbRealm=blabla.domain.net
> ;KrbHostFQDN=blabla.domain.net;KrbServiceName=impalaservicename
>
> Boris
>
>
> On Wed, Jan 9, 2019 at 10:49 AM Jeff <jt...@gmail.com> wrote:
>
>> Hello,
>>
>> I'm working on some NiFi/Impala integration examples currently, and will
>> be taking a look at kerberized Impala in a few days. I'll try to follow up
>> with you on this list after I have things working. If you beat me to it,
>> please let us know!
>>
>> - Jeff
>>
>> On Mon, Jan 7, 2019 at 8:32 PM PasLe Choix <Pa...@gmail.com> wrote:
>>
>>> I need to create a DBCPConnectionPool to connect to my table on Impala
>>> in Cloudera cluster, wondering who can provide a sample URL for the jdbc
>>> connection?
>>> At the moment my URL is as below and is not working:
>>> jdbc:impala://nydc-pcdhwrk01:21050;
>>> AuthMech=1;KrbRealm=BD.NOVANTAS.PRI;KrbHostFQDN=
>>> nydc-pcdhwrk01.novantas.com;KrbServiceName=pchoix
>>>
>>>
>>> Thank you very much.
>>>
>>> *------------------------------------------------*
>>> *Sincerely yours,*
>>>
>>>
>>> *PC*
>>>
>>
Re: jdbc impala url
Posted by Boris Tyukin <bo...@boristyukin.com>.
Hi Matt,
yes it is very similar. In fact Hive, driver can be used to access Impala
https://www.cloudera.com/documentation/enterprise/5-9-x/topics/impala_jdbc.html#jdbc_driver_choice
.
Since you mentioned Hive, we were struggling with NiFi connecting to Hive
and did not want to compile NiFi with CDH distro but recently discovered
that we can use regular DBCPConnectionPool with Hive, and then use Groovy
script to get connection from a pool and run HiveQL queries. That way it
was also easy to get proper exceptions and error messages from Hive. Worked
much better than using hive cli.
But Impala was really easy and troublesome to use with NiFi.
With Cloudera and HW merger, I really look forward to the future ;)
On Wed, Jan 9, 2019 at 2:07 PM Matt Burgess <ma...@apache.org> wrote:
> Boris,
>
> Is your Impala instance using Kerberos to authenticate JDBC clients
> (the database users, e.g.)? I don't know Impala well but the URL
> reminds me of the Hive JDBC URL, where you specify a service principal
> for HS2 to talk to the Hive metastore for example. If Hive client
> connections are secured with Kerberos, then you have to provide user
> credentials (principal and keytab, e.g.) as well as the service
> principal stuff on the URL. That's why HiveConnectionPool has the
> additional Kerberos properties.
>
> If we need to perform authentication on behalf of Impala clients,
> we'll likely need to add Kerberos properties to DBCPConnectionPool. I
> am already looking at doing that, while testing against a
> Kerberos-secured MariaDB instance. I think something similar applies
> to SQL Server using Active Directory with Kerberos, and if so, I hope
> to put forth a generic solution (or review a PR if someone else wants
> to do it :)
>
> Regards,
> Matt
>
> On Wed, Jan 9, 2019 at 2:00 PM Boris Tyukin <bo...@boristyukin.com> wrote:
> >
> > this is how our connection looks like on a kerberized CDH cluster
> >
> > jdbc:impala://_host_:21050/default;AuthMech=1;KrbRealm=blabla.domain.net
> ;KrbHostFQDN=blabla.domain.net;KrbServiceName=impalaservicename
> >
> > Boris
> >
> >
> > On Wed, Jan 9, 2019 at 10:49 AM Jeff <jt...@gmail.com> wrote:
> >>
> >> Hello,
> >>
> >> I'm working on some NiFi/Impala integration examples currently, and
> will be taking a look at kerberized Impala in a few days. I'll try to
> follow up with you on this list after I have things working. If you beat
> me to it, please let us know!
> >>
> >> - Jeff
> >>
> >> On Mon, Jan 7, 2019 at 8:32 PM PasLe Choix <Pa...@gmail.com>
> wrote:
> >>>
> >>> I need to create a DBCPConnectionPool to connect to my table on
> Impala in Cloudera cluster, wondering who can provide a sample URL for the
> jdbc connection?
> >>> At the moment my URL is as below and is not working:
> >>> jdbc:impala://nydc-pcdhwrk01:21050;
> AuthMech=1;KrbRealm=BD.NOVANTAS.PRI;KrbHostFQDN=
> nydc-pcdhwrk01.novantas.com;KrbServiceName=pchoix
> >>>
> >>>
> >>> Thank you very much.
> >>>
> >>> ------------------------------------------------
> >>> Sincerely yours,
> >>>
> >>>
> >>> PC
>
Re: jdbc impala url
Posted by Matt Burgess <ma...@apache.org>.
Boris,
Is your Impala instance using Kerberos to authenticate JDBC clients
(the database users, e.g.)? I don't know Impala well but the URL
reminds me of the Hive JDBC URL, where you specify a service principal
for HS2 to talk to the Hive metastore for example. If Hive client
connections are secured with Kerberos, then you have to provide user
credentials (principal and keytab, e.g.) as well as the service
principal stuff on the URL. That's why HiveConnectionPool has the
additional Kerberos properties.
If we need to perform authentication on behalf of Impala clients,
we'll likely need to add Kerberos properties to DBCPConnectionPool. I
am already looking at doing that, while testing against a
Kerberos-secured MariaDB instance. I think something similar applies
to SQL Server using Active Directory with Kerberos, and if so, I hope
to put forth a generic solution (or review a PR if someone else wants
to do it :)
Regards,
Matt
On Wed, Jan 9, 2019 at 2:00 PM Boris Tyukin <bo...@boristyukin.com> wrote:
>
> this is how our connection looks like on a kerberized CDH cluster
>
> jdbc:impala://_host_:21050/default;AuthMech=1;KrbRealm=blabla.domain.net;KrbHostFQDN=blabla.domain.net;KrbServiceName=impalaservicename
>
> Boris
>
>
> On Wed, Jan 9, 2019 at 10:49 AM Jeff <jt...@gmail.com> wrote:
>>
>> Hello,
>>
>> I'm working on some NiFi/Impala integration examples currently, and will be taking a look at kerberized Impala in a few days. I'll try to follow up with you on this list after I have things working. If you beat me to it, please let us know!
>>
>> - Jeff
>>
>> On Mon, Jan 7, 2019 at 8:32 PM PasLe Choix <Pa...@gmail.com> wrote:
>>>
>>> I need to create a DBCPConnectionPool to connect to my table on Impala in Cloudera cluster, wondering who can provide a sample URL for the jdbc connection?
>>> At the moment my URL is as below and is not working:
>>> jdbc:impala://nydc-pcdhwrk01:21050; AuthMech=1;KrbRealm=BD.NOVANTAS.PRI;KrbHostFQDN=nydc-pcdhwrk01.novantas.com;KrbServiceName=pchoix
>>>
>>>
>>> Thank you very much.
>>>
>>> ------------------------------------------------
>>> Sincerely yours,
>>>
>>>
>>> PC
Re: jdbc impala url
Posted by Boris Tyukin <bo...@boristyukin.com>.
this is how our connection looks like on a kerberized CDH cluster
jdbc:impala://_host_:21050/default;AuthMech=1;KrbRealm=blabla.domain.net
;KrbHostFQDN=blabla.domain.net;KrbServiceName=impalaservicename
Boris
On Wed, Jan 9, 2019 at 10:49 AM Jeff <jt...@gmail.com> wrote:
> Hello,
>
> I'm working on some NiFi/Impala integration examples currently, and will
> be taking a look at kerberized Impala in a few days. I'll try to follow up
> with you on this list after I have things working. If you beat me to it,
> please let us know!
>
> - Jeff
>
> On Mon, Jan 7, 2019 at 8:32 PM PasLe Choix <Pa...@gmail.com> wrote:
>
>> I need to create a DBCPConnectionPool to connect to my table on Impala
>> in Cloudera cluster, wondering who can provide a sample URL for the jdbc
>> connection?
>> At the moment my URL is as below and is not working:
>> jdbc:impala://nydc-pcdhwrk01:21050;
>> AuthMech=1;KrbRealm=BD.NOVANTAS.PRI;KrbHostFQDN=
>> nydc-pcdhwrk01.novantas.com;KrbServiceName=pchoix
>>
>>
>> Thank you very much.
>>
>> *------------------------------------------------*
>> *Sincerely yours,*
>>
>>
>> *PC*
>>
>
Re: jdbc impala url
Posted by Jeff <jt...@gmail.com>.
Hello,
I'm working on some NiFi/Impala integration examples currently, and will be
taking a look at kerberized Impala in a few days. I'll try to follow up
with you on this list after I have things working. If you beat me to it,
please let us know!
- Jeff
On Mon, Jan 7, 2019 at 8:32 PM PasLe Choix <Pa...@gmail.com> wrote:
> I need to create a DBCPConnectionPool to connect to my table on Impala
> in Cloudera cluster, wondering who can provide a sample URL for the jdbc
> connection?
> At the moment my URL is as below and is not working:
> jdbc:impala://nydc-pcdhwrk01:21050;
> AuthMech=1;KrbRealm=BD.NOVANTAS.PRI;KrbHostFQDN=
> nydc-pcdhwrk01.novantas.com;KrbServiceName=pchoix
>
>
> Thank you very much.
>
> *------------------------------------------------*
> *Sincerely yours,*
>
>
> *PC*
>