You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by Mohammad Islam <mi...@yahoo.com.INVALID> on 2016/11/04 00:29:23 UTC
Supporting Hive JDBC connection with preauth (passing Http Header)
Hi ,I'm wandering if hive JDBC connection (say using beeline ) supports HeaderPreAuth.
In general, preauth gets authenticated user name through http header.However, I'm not sure how to pass HTTP header as part of JDBC URL. If the question is not clear , I can explain it further.
Regards,Mohammad
Re: Supporting Hive JDBC connection with preauth (passing Http Header)
Posted by larry mccay <lm...@apache.org>.
Hi Mohammad -
Great questions.
1. Since JDBC connection with http headers works for beeline, do you think
any Hive/Java thrift API in Java will also work in the same way?
Not really sure what you mean here but JDBC from a Java EE application is
what was the original usecase for the header support. This would still
require Thrift/HTTP, of course.
2. Performance : What is the performance impact for using Knox proxy? For
example, is there any comparison of performance when a large file is moved
through Knox+WebHDFS path vs direct WebHDFS using SSL + Kerberos? If not,
how much is expected?
I don't have any hard numbers for you but I can tell you that this is
probably the most popular usecase for Knox and is in use in many
deployments. Adding a separate hop is certainly never going to speed it up
but the performance hasn't been a show stopper for anyone that I am aware
of. YMMV.
3. Is it possible to submit Spark job through Knox? If not, is there any
discussion?
We don't currently have support but there has been some interest in support
for proxying the Livy REST API for this usecase.
Feel free to file a JIRA for it - we can close it as a dupe if there is
already one filed.
Thank you for the questions - it helps the community know where interests
lie.
--larry
On Fri, Nov 4, 2016 at 12:12 PM, Mohammad Islam <mi...@yahoo.com> wrote:
> Hi Larry,
> Just few follow-up few questions:
>
> 1. Since JDBC connection with http headers works for beeline, do you think
> any Hive/Java thrift API in Java will also work in the same way?
> 2. Performance : What is the performance impact for using Knox proxy? For
> example, is there any comparison of performance when a large file is moved
> through Knox+WebHDFS path vs direct WebHDFS using SSL + Kerberos? If not,
> how much is expected?
>
> 3. Is it possible to submit Spark job through Knox? If not, is there any
> discussion?
>
> So many questions in one email :)
>
> Regards,
> Mohammad
>
>
>
>
>
> On Thursday, November 3, 2016 11:21 PM, Mohammad Islam <mi...@yahoo.com>
> wrote:
>
>
> Thanks Larry again.
> That's link I was looking for.
> I will follow your proposal.
> Regards,
> Mohammad
>
>
>
>
> On Thursday, November 3, 2016 6:47 PM, larry mccay <lm...@apache.org>
> wrote:
>
>
> Hi Mohammad -
>
> This may be of interest: https://cwiki.apache.org/confluence/display/
> Hive/HiveServer2+Clients#HiveServer2Clients-PassingHTTPHeaderKey/
> ValuePairsviaJDBCDriver
>
> You could certainly set SM_USER and SM_GROUPS through this.
>
> Obviously, you would have to ensure that no one can spoof an
> authentication and that you only accept such connections from trusted
> sources.
> I would suggest SSL mutual authentication. See: http://knox.apache.org/
> books/knox-0-9-1/user-guide.html#Mutual+Authentication+with+SSL
>
> Hope that helps.
>
> --larry
>
>
> On Thu, Nov 3, 2016 at 8:29 PM, Mohammad Islam <mi...@yahoo.com> wrote:
>
> Hi ,
> I'm wandering if hive JDBC connection (say using beeline ) supports
> HeaderPreAuth.
>
> In general, preauth gets authenticated user name through http header.
> However, I'm not sure how to pass HTTP header as part of JDBC URL. If the
> question is not clear , I can explain it further.
>
> Regards,
> Mohammad
>
>
>
>
>
>
>
>
>
>
Re: Supporting Hive JDBC connection with preauth (passing Http
Header)
Posted by Mohammad Islam <mi...@yahoo.com>.
Hi Larry,Just few follow-up few questions:
1. Since JDBC connection with http headers works for beeline, do you think any Hive/Java thrift API in Java will also work in the same way?2. Performance : What is the performance impact for using Knox proxy? For example, is there any comparison of performance when a large file is moved through Knox+WebHDFS path vs direct WebHDFS using SSL + Kerberos? If not, how much is expected?
3. Is it possible to submit Spark job through Knox? If not, is there any discussion?
So many questions in one email :)
Regards,Mohammad
On Thursday, November 3, 2016 11:21 PM, Mohammad Islam <mi...@yahoo.com> wrote:
Thanks Larry again.That's link I was looking for.I will follow your proposal.Regards,Mohammad
On Thursday, November 3, 2016 6:47 PM, larry mccay <lm...@apache.org> wrote:
Hi Mohammad -
This may be of interest: https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-PassingHTTPHeaderKey/ValuePairsviaJDBCDriver
You could certainly set SM_USER and SM_GROUPS through this.
Obviously, you would have to ensure that no one can spoof an authentication and that you only accept such connections from trusted sources.I would suggest SSL mutual authentication. See: http://knox.apache.org/books/knox-0-9-1/user-guide.html#Mutual+Authentication+with+SSL
Hope that helps.
--larry
On Thu, Nov 3, 2016 at 8:29 PM, Mohammad Islam <mi...@yahoo.com> wrote:
Hi ,I'm wandering if hive JDBC connection (say using beeline ) supports HeaderPreAuth.
In general, preauth gets authenticated user name through http header.However, I'm not sure how to pass HTTP header as part of JDBC URL. If the question is not clear , I can explain it further.
Regards,Mohammad
Re: Supporting Hive JDBC connection with preauth (passing Http
Header)
Posted by Mohammad Islam <mi...@yahoo.com>.
Thanks Larry again.That's link I was looking for.I will follow your proposal.Regards,Mohammad
On Thursday, November 3, 2016 6:47 PM, larry mccay <lm...@apache.org> wrote:
Hi Mohammad -
This may be of interest: https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-PassingHTTPHeaderKey/ValuePairsviaJDBCDriver
You could certainly set SM_USER and SM_GROUPS through this.
Obviously, you would have to ensure that no one can spoof an authentication and that you only accept such connections from trusted sources.I would suggest SSL mutual authentication. See: http://knox.apache.org/books/knox-0-9-1/user-guide.html#Mutual+Authentication+with+SSL
Hope that helps.
--larry
On Thu, Nov 3, 2016 at 8:29 PM, Mohammad Islam <mi...@yahoo.com> wrote:
Hi ,I'm wandering if hive JDBC connection (say using beeline ) supports HeaderPreAuth.
In general, preauth gets authenticated user name through http header.However, I'm not sure how to pass HTTP header as part of JDBC URL. If the question is not clear , I can explain it further.
Regards,Mohammad
Re: Supporting Hive JDBC connection with preauth (passing Http Header)
Posted by larry mccay <lm...@apache.org>.
Hi Mohammad -
This may be of interest:
https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-PassingHTTPHeaderKey/ValuePairsviaJDBCDriver
You could certainly set SM_USER and SM_GROUPS through this.
Obviously, you would have to ensure that no one can spoof an authentication
and that you only accept such connections from trusted sources.
I would suggest SSL mutual authentication. See:
http://knox.apache.org/books/knox-0-9-1/user-guide.html#Mutual+Authentication+with+SSL
Hope that helps.
--larry
On Thu, Nov 3, 2016 at 8:29 PM, Mohammad Islam <mi...@yahoo.com> wrote:
> Hi ,
> I'm wandering if hive JDBC connection (say using beeline ) supports
> HeaderPreAuth.
>
> In general, preauth gets authenticated user name through http header.
> However, I'm not sure how to pass HTTP header as part of JDBC URL. If the
> question is not clear , I can explain it further.
>
> Regards,
> Mohammad
>
>
>
>