You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Yaar Reuveni <ya...@liveperson.com> on 2017/01/11 13:04:29 UTC
Re: Using CDH dynamic resource pools with Zeppelin
Hey,
Since no answer yet, I'll try a simpler question.
I have Zeppelin defined with a *JDBC* interpreter configured with *Impala*
that works against a CDH5.5 Hadoop cluster.
When I run queries from Zeppelin, these queries run without a user in
Hadoop, also no user seen in the Cloudera manager.
How can I configure it so there is a user defined on the connection and on
the running queries?
Thanks,
Yaar
On Tue, Dec 20, 2016 at 10:25 AM, Yaar Reuveni <ya...@liveperson.com> wrote:
> Hey,
> We're using a cloudera distribution hadoop.
> We want to know how can we configure Zeppelin user authentication and link
> between users and resource pools
> <https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cm_mc_resource_pools.html>
> in our YARN Hadoop cluster
>
> Thanks,
> Yaar
>
> --
>
>
>
--
This message may contain confidential and/or privileged information.
If you are not the addressee or authorized to receive this on behalf of the
addressee you must not use, copy, disclose or take action based on this
message or any information herein.
If you have received this message in error, please advise the sender
immediately by reply email and delete this message. Thank you.
Re: Using CDH dynamic resource pools with Zeppelin
Posted by Jongyoul Lee <jo...@gmail.com>.
PMC raised the release issue on 0.7.0 and community is discussing it.
AFAIK, one of committers will make rc1 within next week.
On Fri, Jan 13, 2017 at 4:33 PM, Yaar Reuveni <ya...@liveperson.com> wrote:
> Is it known when v0.7.0 is expected to be released?
>
> On Wed, Jan 11, 2017 at 4:09 PM, Paul Brenner <pb...@placeiq.com>
> wrote:
>
>> My understanding is that this kind of user specific control isn’t coming
>> until v0.70. Currently when we run zeppelin all tasks are submitted by the
>> user that started the zeppelin process (so we start zeppelin from the yarn
>> account and everything is submitted as yarn). At least for spark there is a
>> user queue parameter that can be set in the interpreter which ensures that
>> users are only getting the resources they are allowed. We just create a
>> different interpreter for each user and set that parameter. It isn’t
>> perfect, and might not even be available for your JDBC, but I thought the
>> detail might help.
>>
>> <http://www.placeiq.com/> <http://www.placeiq.com/>
>> <http://www.placeiq.com/> Paul Brenner <https://twitter.com/placeiq>
>> <https://twitter.com/placeiq> <https://twitter.com/placeiq>
>> <https://www.facebook.com/PlaceIQ> <https://www.facebook.com/PlaceIQ>
>> <https://www.linkedin.com/company/placeiq>
>> <https://www.linkedin.com/company/placeiq>
>> DATA SCIENTIST
>> *(217) 390-3033 *
>>
>> <http://www.placeiq.com/2015/05/26/placeiq-named-winner-of-prestigious-2015-oracle-data-cloud-activate-award/>
>> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
>> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
>> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
>> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
>> <http://placeiq.com/2016/03/08/measuring-addressable-tv-campaigns-is-now-possible/>
>> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
>> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
>> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
>> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
>> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
>> <http://pages.placeiq.com/Location-Data-Accuracy-Whitepaper-Download.html?utm_source=Signature&utm_medium=Email&utm_campaign=AccuracyWP>
>> <http://placeiq.com/2016/08/03/placeiq-bolsters-location-intelligence-platform-with-mastercard-insights/>
>> <http://placeiq.com/2016/10/26/the-making-of-a-location-data-industry-milestone/>[image:
>> PlaceIQ:Location Data Accuracy]
>> <http://placeiq.com/2016/12/07/placeiq-introduces-landmark-a-groundbreaking-offering-that-delivers-access-to-the-highest-quality-location-data-for-insights-that-fuel-limitless-business-decisions/>
>>
>> On Wed, Jan 11, 2017 at 8:04 AM Yaar Reuveni <Yaar Reuveni
>> <Yaar+Reuveni+%3Cyaarr@liveperson.com%3E>> wrote:
>>
>>> Hey,
>>> Since no answer yet, I'll try a simpler question.
>>> I have Zeppelin defined with a *JDBC* interpreter configured with
>>> *Impala* that works against a CDH5.5 Hadoop cluster.
>>> When I run queries from Zeppelin, these queries run without a user in
>>> Hadoop, also no user seen in the Cloudera manager.
>>> How can I configure it so there is a user defined on the connection and
>>> on the running queries?
>>>
>>> Thanks,
>>> Yaar
>>>
>>> On Tue, Dec 20, 2016 at 10:25 AM, Yaar Reuveni <ya...@liveperson.com>
>>> wrote:
>>>
>>>> Hey,
>>>> We're using a cloudera distribution hadoop.
>>>> We want to know how can we configure Zeppelin user authentication and
>>>> link between users and resource pools
>>>> <https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cm_mc_resource_pools.html>
>>>> in our YARN Hadoop cluster
>>>>
>>>> Thanks,
>>>> Yaar
>>>>
>>>> --
>>>>
>>>>
>>>>
>> This message may contain confidential and/or privileged information.
>> If you are not the addressee or authorized to receive this on behalf of
>> the addressee you must not use, copy, disclose or take action based on this
>> message or any information herein.
>> If you have received this message in error, please advise the sender
>> immediately by reply email and delete this message. Thank you.
>>
>>
>
>
> This message may contain confidential and/or privileged information.
> If you are not the addressee or authorized to receive this on behalf of
> the addressee you must not use, copy, disclose or take action based on this
> message or any information herein.
> If you have received this message in error, please advise the sender
> immediately by reply email and delete this message. Thank you.
>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
Re: Using CDH dynamic resource pools with Zeppelin
Posted by Yaar Reuveni <ya...@liveperson.com>.
Is it known when v0.7.0 is expected to be released?
On Wed, Jan 11, 2017 at 4:09 PM, Paul Brenner <pb...@placeiq.com> wrote:
> My understanding is that this kind of user specific control isn’t coming
> until v0.70. Currently when we run zeppelin all tasks are submitted by the
> user that started the zeppelin process (so we start zeppelin from the yarn
> account and everything is submitted as yarn). At least for spark there is a
> user queue parameter that can be set in the interpreter which ensures that
> users are only getting the resources they are allowed. We just create a
> different interpreter for each user and set that parameter. It isn’t
> perfect, and might not even be available for your JDBC, but I thought the
> detail might help.
>
> <http://www.placeiq.com/> <http://www.placeiq.com/>
> <http://www.placeiq.com/> Paul Brenner <https://twitter.com/placeiq>
> <https://twitter.com/placeiq> <https://twitter.com/placeiq>
> <https://www.facebook.com/PlaceIQ> <https://www.facebook.com/PlaceIQ>
> <https://www.linkedin.com/company/placeiq>
> <https://www.linkedin.com/company/placeiq>
> DATA SCIENTIST
> *(217) 390-3033 *
>
> <http://www.placeiq.com/2015/05/26/placeiq-named-winner-of-prestigious-2015-oracle-data-cloud-activate-award/>
> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
> <http://placeiq.com/2016/03/08/measuring-addressable-tv-campaigns-is-now-possible/>
> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
> <http://pages.placeiq.com/Location-Data-Accuracy-Whitepaper-Download.html?utm_source=Signature&utm_medium=Email&utm_campaign=AccuracyWP>
> <http://placeiq.com/2016/08/03/placeiq-bolsters-location-intelligence-platform-with-mastercard-insights/>
> <http://placeiq.com/2016/10/26/the-making-of-a-location-data-industry-milestone/>[image:
> PlaceIQ:Location Data Accuracy]
> <http://placeiq.com/2016/12/07/placeiq-introduces-landmark-a-groundbreaking-offering-that-delivers-access-to-the-highest-quality-location-data-for-insights-that-fuel-limitless-business-decisions/>
>
> On Wed, Jan 11, 2017 at 8:04 AM Yaar Reuveni <Yaar Reuveni
> <Yaar+Reuveni+%3Cyaarr@liveperson.com%3E>> wrote:
>
>> Hey,
>> Since no answer yet, I'll try a simpler question.
>> I have Zeppelin defined with a *JDBC* interpreter configured with
>> *Impala* that works against a CDH5.5 Hadoop cluster.
>> When I run queries from Zeppelin, these queries run without a user in
>> Hadoop, also no user seen in the Cloudera manager.
>> How can I configure it so there is a user defined on the connection and
>> on the running queries?
>>
>> Thanks,
>> Yaar
>>
>> On Tue, Dec 20, 2016 at 10:25 AM, Yaar Reuveni <ya...@liveperson.com>
>> wrote:
>>
>>> Hey,
>>> We're using a cloudera distribution hadoop.
>>> We want to know how can we configure Zeppelin user authentication and
>>> link between users and resource pools
>>> <https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cm_mc_resource_pools.html>
>>> in our YARN Hadoop cluster
>>>
>>> Thanks,
>>> Yaar
>>>
>>> --
>>>
>>>
>>>
> This message may contain confidential and/or privileged information.
> If you are not the addressee or authorized to receive this on behalf of
> the addressee you must not use, copy, disclose or take action based on this
> message or any information herein.
> If you have received this message in error, please advise the sender
> immediately by reply email and delete this message. Thank you.
>
>
--
This message may contain confidential and/or privileged information.
If you are not the addressee or authorized to receive this on behalf of the
addressee you must not use, copy, disclose or take action based on this
message or any information herein.
If you have received this message in error, please advise the sender
immediately by reply email and delete this message. Thank you.
Re: Using CDH dynamic resource pools with Zeppelin
Posted by Paul Brenner <pb...@placeiq.com>.
My understanding is that this kind of user specific control isn’t coming until v0.70. Currently when we run zeppelin all tasks are submitted by the user that started the zeppelin process (so we start zeppelin from the yarn account and everything is submitted as yarn). At least for spark there is a user queue parameter that can be set in the interpreter which ensures that users are only getting the resources they are allowed. We just create a different interpreter for each user and set that parameter. It isn’t perfect, and might not even be available for your JDBC, but I thought the detail might help.
http://www.placeiq.com/ http://www.placeiq.com/ http://www.placeiq.com/
Paul Brenner
https://twitter.com/placeiq https://twitter.com/placeiq https://twitter.com/placeiq
https://www.facebook.com/PlaceIQ https://www.facebook.com/PlaceIQ
https://www.linkedin.com/company/placeiq https://www.linkedin.com/company/placeiq
DATA SCIENTIST
(217) 390-3033
http://www.placeiq.com/2015/05/26/placeiq-named-winner-of-prestigious-2015-oracle-data-cloud-activate-award/ http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/ http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/ http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/ http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/ http://placeiq.com/2016/03/08/measuring-addressable-tv-campaigns-is-now-possible/ http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/ http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/ http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/ http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/ http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/ http://pages.placeiq.com/Location-Data-Accuracy-Whitepaper-Download.html?utm_source=Signature&utm_medium=Email&utm_campaign=AccuracyWP http://placeiq.com/2016/08/03/placeiq-bolsters-location-intelligence-platform-with-mastercard-insights/ http://placeiq.com/2016/10/26/the-making-of-a-location-data-industry-milestone/ http://placeiq.com/2016/12/07/placeiq-introduces-landmark-a-groundbreaking-offering-that-delivers-access-to-the-highest-quality-location-data-for-insights-that-fuel-limitless-business-decisions/
On Wed, Jan 11, 2017 at 8:04 AM Yaar Reuveni
<
mailto:Yaar Reuveni <ya...@liveperson.com>
> wrote:
a, pre, code, a:link, body { word-wrap: break-word !important; }
Hey,
Since no answer yet, I'll try a simpler question.
I have Zeppelin defined with a
JDBC
interpreter configured with
Impala
that works against a CDH5.5 Hadoop cluster.
When I run queries from Zeppelin, these queries run without a user in Hadoop, also no user seen in the Cloudera manager.
How can I configure it so there is a user defined on the connection and on the running queries?
Thanks,
Yaar
On Tue, Dec 20, 2016 at 10:25 AM, Yaar Reuveni
<
mailto:yaarr@liveperson.com
>
wrote:
Hey,
We're using a cloudera distribution hadoop.
We want to know how can we configure Zeppelin user authentication and link between users and
https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cm_mc_resource_pools.html
in our YARN Hadoop cluster
Thanks,
Yaar
--
This message may contain confidential and/or privileged information.
If you are not the addressee or authorized to receive this on behalf of the addressee you must not use, copy, disclose or take action based on this message or any information herein.
If you have received this message in error, please advise the sender immediately by reply email and delete this message. Thank you.