You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sentry.apache.org by Alexander Kolbasov <ak...@cloudera.com> on 2018/04/25 23:58:28 UTC
HMS catalogs and Sentry
Hive 3 introduced the notion of catalog. What do people think about
supporting it in Sentry as well? At some point we will move to Hive-3 and
then it will become a requirement I guess.
- Alex
Re: HMS catalogs and Sentry
Posted by Kalyan Kumar Kalvagadda <kk...@cloudera.com>.
sasha,
Thanks for the explanation. Will look closer into the discussion in Jira.
*Thanks,Kalyan Kumar Kalvagadda* | Software Engineer
t. (469) 279- <0000000000>5732
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
On Mon, Apr 30, 2018 at 6:30 PM, Alexander Kolbasov <ak...@cloudera.com>
wrote:
> I think the way Alan Gates describes it is that the reason is coexistence
> of different databases within HMS for example Kafka may store its metadata
> within 'kafka' catalog, etc. The data within each catalog seems to be the
> same though - it is still databases, tables and partitions. There is some
> discussion and explanation in the JIRA.
>
> On Thu, Apr 26, 2018 at 10:21 AM, Kalyan Kumar Kalvagadda <
> kkalyan@cloudera.com> wrote:
>
> > I feel, we need to understand it better before considering to put in
> sentry
> > wish list.
> >
> >
> > *Thanks,Kalyan Kumar Kalvagadda* | Software Engineer
> > t. (469) 279- <0000000000>5732
> > cloudera.com <https://www.cloudera.com>
> >
> > [image: Cloudera] <https://www.cloudera.com/>
> >
> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera
> > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > ------------------------------
> >
> > On Thu, Apr 26, 2018 at 11:31 AM, Na Li <li...@cloudera.com> wrote:
> >
> > > Sasha,
> > >
> > > The jira https://issues.apache.org/jira/browse/HIVE-18755 has no
> > > description on what's the motivation of having catalog and how it
> works.
> > >
> > > Is catalog similar to namespace, so all databases in a catalog can be
> > > managed by one entity?
> > >
> > > Do you know any description that would help us understand how it work
> > other
> > > than read the hive code?
> > >
> > > Thanks,
> > >
> > > Lina
> > >
> > > On Thu, Apr 26, 2018 at 10:47 AM, Sergio Pena <
> sergio.pena@cloudera.com>
> > > wrote:
> > >
> > > > What is the reason for having catalogs in HMS? I was looking at the
> > JIRA
> > > > but I don't see how catalogs will be used.
> > > >
> > > > Nevertheless, if HMS is going to support catalogs, then we should
> > support
> > > > them when we decide to move to Hive 3.0 at some point.
> > > > Should we put this on our wishlist for Sentry?
> > > >
> > > > On Wed, Apr 25, 2018 at 7:45 PM, Alexander Kolbasov <
> > akolb@cloudera.com>
> > > > wrote:
> > > >
> > > > > See https://issues.apache.org/jira/browse/HIVE-18755. Basically
> > > catalog
> > > > is
> > > > > a container for databases - all databases belong to a catalog. This
> > > means
> > > > > that Sentry may need to support catalog-level privileges.
> > > > >
> > > > > - Alex
> > > > >
> > > > > On Wed, Apr 25, 2018 at 5:06 PM, Kalyan Kumar Kalvagadda <
> > > > > kkalyan@cloudera.com> wrote:
> > > > >
> > > > > > Sasha,
> > > > > >
> > > > > > Could you share information on what catalog is?
> > > > > >
> > > > > > -Kalyan
> > > > > >
> > > > > >
> > > > > > *Thanks,Kalyan Kumar Kalvagadda* | Software Engineer
> > > > > > t. (469) 279- <0000000000>5732
> > > > > > cloudera.com <https://www.cloudera.com>
> > > > > >
> > > > > > [image: Cloudera] <https://www.cloudera.com/>
> > > > > >
> > > > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera>
> > [image:
> > > > > > Cloudera on Facebook] <https://www.facebook.com/cloudera>
> [image:
> > > > > Cloudera
> > > > > > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > > > > ------------------------------
> > > > > >
> > > > > > On Wed, Apr 25, 2018 at 6:58 PM, Alexander Kolbasov <
> > > > akolb@cloudera.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hive 3 introduced the notion of catalog. What do people think
> > about
> > > > > > > supporting it in Sentry as well? At some point we will move to
> > > Hive-3
> > > > > and
> > > > > > > then it will become a requirement I guess.
> > > > > > >
> > > > > > > - Alex
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: HMS catalogs and Sentry
Posted by Alexander Kolbasov <ak...@cloudera.com>.
I think the way Alan Gates describes it is that the reason is coexistence
of different databases within HMS for example Kafka may store its metadata
within 'kafka' catalog, etc. The data within each catalog seems to be the
same though - it is still databases, tables and partitions. There is some
discussion and explanation in the JIRA.
On Thu, Apr 26, 2018 at 10:21 AM, Kalyan Kumar Kalvagadda <
kkalyan@cloudera.com> wrote:
> I feel, we need to understand it better before considering to put in sentry
> wish list.
>
>
> *Thanks,Kalyan Kumar Kalvagadda* | Software Engineer
> t. (469) 279- <0000000000>5732
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------------------------------
>
> On Thu, Apr 26, 2018 at 11:31 AM, Na Li <li...@cloudera.com> wrote:
>
> > Sasha,
> >
> > The jira https://issues.apache.org/jira/browse/HIVE-18755 has no
> > description on what's the motivation of having catalog and how it works.
> >
> > Is catalog similar to namespace, so all databases in a catalog can be
> > managed by one entity?
> >
> > Do you know any description that would help us understand how it work
> other
> > than read the hive code?
> >
> > Thanks,
> >
> > Lina
> >
> > On Thu, Apr 26, 2018 at 10:47 AM, Sergio Pena <se...@cloudera.com>
> > wrote:
> >
> > > What is the reason for having catalogs in HMS? I was looking at the
> JIRA
> > > but I don't see how catalogs will be used.
> > >
> > > Nevertheless, if HMS is going to support catalogs, then we should
> support
> > > them when we decide to move to Hive 3.0 at some point.
> > > Should we put this on our wishlist for Sentry?
> > >
> > > On Wed, Apr 25, 2018 at 7:45 PM, Alexander Kolbasov <
> akolb@cloudera.com>
> > > wrote:
> > >
> > > > See https://issues.apache.org/jira/browse/HIVE-18755. Basically
> > catalog
> > > is
> > > > a container for databases - all databases belong to a catalog. This
> > means
> > > > that Sentry may need to support catalog-level privileges.
> > > >
> > > > - Alex
> > > >
> > > > On Wed, Apr 25, 2018 at 5:06 PM, Kalyan Kumar Kalvagadda <
> > > > kkalyan@cloudera.com> wrote:
> > > >
> > > > > Sasha,
> > > > >
> > > > > Could you share information on what catalog is?
> > > > >
> > > > > -Kalyan
> > > > >
> > > > >
> > > > > *Thanks,Kalyan Kumar Kalvagadda* | Software Engineer
> > > > > t. (469) 279- <0000000000>5732
> > > > > cloudera.com <https://www.cloudera.com>
> > > > >
> > > > > [image: Cloudera] <https://www.cloudera.com/>
> > > > >
> > > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera>
> [image:
> > > > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > > > Cloudera
> > > > > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > > > ------------------------------
> > > > >
> > > > > On Wed, Apr 25, 2018 at 6:58 PM, Alexander Kolbasov <
> > > akolb@cloudera.com>
> > > > > wrote:
> > > > >
> > > > > > Hive 3 introduced the notion of catalog. What do people think
> about
> > > > > > supporting it in Sentry as well? At some point we will move to
> > Hive-3
> > > > and
> > > > > > then it will become a requirement I guess.
> > > > > >
> > > > > > - Alex
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: HMS catalogs and Sentry
Posted by Kalyan Kumar Kalvagadda <kk...@cloudera.com>.
I feel, we need to understand it better before considering to put in sentry
wish list.
*Thanks,Kalyan Kumar Kalvagadda* | Software Engineer
t. (469) 279- <0000000000>5732
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
On Thu, Apr 26, 2018 at 11:31 AM, Na Li <li...@cloudera.com> wrote:
> Sasha,
>
> The jira https://issues.apache.org/jira/browse/HIVE-18755 has no
> description on what's the motivation of having catalog and how it works.
>
> Is catalog similar to namespace, so all databases in a catalog can be
> managed by one entity?
>
> Do you know any description that would help us understand how it work other
> than read the hive code?
>
> Thanks,
>
> Lina
>
> On Thu, Apr 26, 2018 at 10:47 AM, Sergio Pena <se...@cloudera.com>
> wrote:
>
> > What is the reason for having catalogs in HMS? I was looking at the JIRA
> > but I don't see how catalogs will be used.
> >
> > Nevertheless, if HMS is going to support catalogs, then we should support
> > them when we decide to move to Hive 3.0 at some point.
> > Should we put this on our wishlist for Sentry?
> >
> > On Wed, Apr 25, 2018 at 7:45 PM, Alexander Kolbasov <ak...@cloudera.com>
> > wrote:
> >
> > > See https://issues.apache.org/jira/browse/HIVE-18755. Basically
> catalog
> > is
> > > a container for databases - all databases belong to a catalog. This
> means
> > > that Sentry may need to support catalog-level privileges.
> > >
> > > - Alex
> > >
> > > On Wed, Apr 25, 2018 at 5:06 PM, Kalyan Kumar Kalvagadda <
> > > kkalyan@cloudera.com> wrote:
> > >
> > > > Sasha,
> > > >
> > > > Could you share information on what catalog is?
> > > >
> > > > -Kalyan
> > > >
> > > >
> > > > *Thanks,Kalyan Kumar Kalvagadda* | Software Engineer
> > > > t. (469) 279- <0000000000>5732
> > > > cloudera.com <https://www.cloudera.com>
> > > >
> > > > [image: Cloudera] <https://www.cloudera.com/>
> > > >
> > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > > Cloudera
> > > > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > > ------------------------------
> > > >
> > > > On Wed, Apr 25, 2018 at 6:58 PM, Alexander Kolbasov <
> > akolb@cloudera.com>
> > > > wrote:
> > > >
> > > > > Hive 3 introduced the notion of catalog. What do people think about
> > > > > supporting it in Sentry as well? At some point we will move to
> Hive-3
> > > and
> > > > > then it will become a requirement I guess.
> > > > >
> > > > > - Alex
> > > > >
> > > >
> > >
> >
>
Re: HMS catalogs and Sentry
Posted by Na Li <li...@cloudera.com>.
Sasha,
The jira https://issues.apache.org/jira/browse/HIVE-18755 has no
description on what's the motivation of having catalog and how it works.
Is catalog similar to namespace, so all databases in a catalog can be
managed by one entity?
Do you know any description that would help us understand how it work other
than read the hive code?
Thanks,
Lina
On Thu, Apr 26, 2018 at 10:47 AM, Sergio Pena <se...@cloudera.com>
wrote:
> What is the reason for having catalogs in HMS? I was looking at the JIRA
> but I don't see how catalogs will be used.
>
> Nevertheless, if HMS is going to support catalogs, then we should support
> them when we decide to move to Hive 3.0 at some point.
> Should we put this on our wishlist for Sentry?
>
> On Wed, Apr 25, 2018 at 7:45 PM, Alexander Kolbasov <ak...@cloudera.com>
> wrote:
>
> > See https://issues.apache.org/jira/browse/HIVE-18755. Basically catalog
> is
> > a container for databases - all databases belong to a catalog. This means
> > that Sentry may need to support catalog-level privileges.
> >
> > - Alex
> >
> > On Wed, Apr 25, 2018 at 5:06 PM, Kalyan Kumar Kalvagadda <
> > kkalyan@cloudera.com> wrote:
> >
> > > Sasha,
> > >
> > > Could you share information on what catalog is?
> > >
> > > -Kalyan
> > >
> > >
> > > *Thanks,Kalyan Kumar Kalvagadda* | Software Engineer
> > > t. (469) 279- <0000000000>5732
> > > cloudera.com <https://www.cloudera.com>
> > >
> > > [image: Cloudera] <https://www.cloudera.com/>
> > >
> > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > Cloudera
> > > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > ------------------------------
> > >
> > > On Wed, Apr 25, 2018 at 6:58 PM, Alexander Kolbasov <
> akolb@cloudera.com>
> > > wrote:
> > >
> > > > Hive 3 introduced the notion of catalog. What do people think about
> > > > supporting it in Sentry as well? At some point we will move to Hive-3
> > and
> > > > then it will become a requirement I guess.
> > > >
> > > > - Alex
> > > >
> > >
> >
>
Re: HMS catalogs and Sentry
Posted by Sergio Pena <se...@cloudera.com>.
What is the reason for having catalogs in HMS? I was looking at the JIRA
but I don't see how catalogs will be used.
Nevertheless, if HMS is going to support catalogs, then we should support
them when we decide to move to Hive 3.0 at some point.
Should we put this on our wishlist for Sentry?
On Wed, Apr 25, 2018 at 7:45 PM, Alexander Kolbasov <ak...@cloudera.com>
wrote:
> See https://issues.apache.org/jira/browse/HIVE-18755. Basically catalog is
> a container for databases - all databases belong to a catalog. This means
> that Sentry may need to support catalog-level privileges.
>
> - Alex
>
> On Wed, Apr 25, 2018 at 5:06 PM, Kalyan Kumar Kalvagadda <
> kkalyan@cloudera.com> wrote:
>
> > Sasha,
> >
> > Could you share information on what catalog is?
> >
> > -Kalyan
> >
> >
> > *Thanks,Kalyan Kumar Kalvagadda* | Software Engineer
> > t. (469) 279- <0000000000>5732
> > cloudera.com <https://www.cloudera.com>
> >
> > [image: Cloudera] <https://www.cloudera.com/>
> >
> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera
> > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > ------------------------------
> >
> > On Wed, Apr 25, 2018 at 6:58 PM, Alexander Kolbasov <ak...@cloudera.com>
> > wrote:
> >
> > > Hive 3 introduced the notion of catalog. What do people think about
> > > supporting it in Sentry as well? At some point we will move to Hive-3
> and
> > > then it will become a requirement I guess.
> > >
> > > - Alex
> > >
> >
>
Re: HMS catalogs and Sentry
Posted by Alexander Kolbasov <ak...@cloudera.com>.
See https://issues.apache.org/jira/browse/HIVE-18755. Basically catalog is
a container for databases - all databases belong to a catalog. This means
that Sentry may need to support catalog-level privileges.
- Alex
On Wed, Apr 25, 2018 at 5:06 PM, Kalyan Kumar Kalvagadda <
kkalyan@cloudera.com> wrote:
> Sasha,
>
> Could you share information on what catalog is?
>
> -Kalyan
>
>
> *Thanks,Kalyan Kumar Kalvagadda* | Software Engineer
> t. (469) 279- <0000000000>5732
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------------------------------
>
> On Wed, Apr 25, 2018 at 6:58 PM, Alexander Kolbasov <ak...@cloudera.com>
> wrote:
>
> > Hive 3 introduced the notion of catalog. What do people think about
> > supporting it in Sentry as well? At some point we will move to Hive-3 and
> > then it will become a requirement I guess.
> >
> > - Alex
> >
>
Re: HMS catalogs and Sentry
Posted by Kalyan Kumar Kalvagadda <kk...@cloudera.com>.
Sasha,
Could you share information on what catalog is?
-Kalyan
*Thanks,Kalyan Kumar Kalvagadda* | Software Engineer
t. (469) 279- <0000000000>5732
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
On Wed, Apr 25, 2018 at 6:58 PM, Alexander Kolbasov <ak...@cloudera.com>
wrote:
> Hive 3 introduced the notion of catalog. What do people think about
> supporting it in Sentry as well? At some point we will move to Hive-3 and
> then it will become a requirement I guess.
>
> - Alex
>