You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sentry.apache.org by Kalyan Kumar Kalvagadda <kk...@cloudera.com> on 2017/07/18 12:18:10 UTC
Sentry 2.0.0 depends on multiple versions of Hive
Hello all,
Sentry currently supports Hive1.1.0 and Hive 2.0.0 based on the profile
used. I'm currently working on sentry 2.0.0 release upstream to integrate
with Hive 2.1.1.
I'm sending this mail seeking inputs on what is the support we need in
Sentry 2.0.0 release. Would that be just Hive 2.x or both Hive 2.x and Hive
1.1.0?
I'm not sure if we should be supporting Hive 1.1.0 in sentry 2.0.0. As Hive
1.1.0 is pretty old and the notification log implementation in this version
which Sentry-HA is dependent on has a lot of bugs which are fixed in later
releases. Considering this, shouldn't we integrate sentry in release 2.0.0
with Hive 2.1.1 and say that it would no more support older versions of
Hive?
Here are some of the approaches that we could take for Hive integration for
sentry 2.0.0 release.
*Approach-1:* Maintain support for both Hive 1.1.0 and Hive 2.1.1
*Approach-2: *Maintain support for both Hive 1.1.0 and Hive 2.1.1 for now
and remove it by the time sentry 2.0.0 is released.
*Approach-3:* Remove support for Hive 1.1.0 now.
*Downside* for Approach 1 and 2 is that we need to put in lot of effort to
have them work together as Hive 2.1.1 has a lot of API's changes which need
changes to a lot classes in sentry-binding package but still maintain
backward compatibility.
If we want to take approach-1 for any reason, it's worth putting that
effort otherwise we could take approach-3.
My next steps would be based on what I hear form the community.
-Kalyan
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Sergio Pena <se...@cloudera.com>.
Kalyan,
I cherry-picked HIVE-17048 to all 2.x versions: 2.1.2, 2.2.1,2.3.1 and 2.4.0
They are not released yet, but as soon as one of them is released, then we
can bump the Hive version of Sentry to the new version.
- Sergio
On Fri, Aug 11, 2017 at 12:30 PM, Kalyan Kumar Kalvagadda <
kkalyan@cloudera.com> wrote:
> What I've heard from one of the Hive PMC is that they could get the fix to
> Hive 2.4. That is the closest. I will follow-up with him again.
>
> Colm's suggestion is reasonable. Picking Hive 2.0.1 might be safer as it
> might have fixes that were found in Hive 2.0.0
>
>
>
> -Kalyan
>
> On Fri, Aug 11, 2017 at 2:32 AM, Colm O hEigeartaigh <co...@apache.org>
> wrote:
>
> > It's unfortunate, but it seems like option 1 is the only solution really
> > (although we should pick up Hive 2.0.1 at least). Perhaps the Hive
> > community might be receptive to backport the fix to 2.3.x, 2.2.x, 2.1.x
> > given that it is a simple fix and a blocker for Sentry?
> >
> > Colm.
> >
> > On Thu, Aug 10, 2017 at 11:21 PM, Mat Crocker <ma...@cloudera.com>
> > wrote:
> >
> > > My opinion (non-binding) is option-1. Keep it simple and follow HIVE.
> > >
> > > On Thu, Aug 10, 2017 at 5:17 PM, Kalyan Kumar Kalvagadda <
> > > kkalyan@cloudera.com> wrote:
> > >
> > > > Lina,
> > > >
> > > > I don't have that information. I need to dig deeper into apache hive
> > wiki
> > > > to get some understanding of it.
> > > >
> > > > -Kalyan
> > > >
> > > > -Kalyan
> > > >
> > > > On Wed, Aug 9, 2017 at 12:09 PM, Na Li <li...@cloudera.com> wrote:
> > > >
> > > > > What is the timeline of back porting the fix to Hive 2.4? And when
> do
> > > we
> > > > > plan to release sentry 2.0?
> > > > >
> > > > > On Wed, Aug 9, 2017 at 11:57 AM, Kalyan Kumar Kalvagadda <
> > > > > kkalyan@cloudera.com> wrote:
> > > > >
> > > > > > Hello all,
> > > > > >
> > > > > > I would like to share issue that i have discovered while working
> on
> > > > > > SENTRY-1838 <https://issues.apache.org/jira/browse/SENTRY-1838>(
> > > > Support
> > > > > > Hive 2.1.1 for sentry 2.0)
> > > > > >
> > > > > > With the design changes done in Hive between 2.0.0 through 2.1.1,
> > > > > > HiveOperation
> > > > > > is not passed to HiveSemanticAnalyzerHook which is implemented by
> > > > > > sentry. HiveOperation
> > > > > > is very basic information that sentry needs.
> > > > > >
> > > > > > I have reported this issue to hive community.(HIVE-17048
> > > > > > <https://issues.apache.org/jira/browse/HIVE-17048>). This issue
> is
> > > > fixed
> > > > > > in
> > > > > > Hive 3.0.0. I will work Hive community to back-port it to Hive
> 2.4
> > > > > release.
> > > > > >
> > > > > > With that said, we can not integrate with Hive 2.1.1 because of
> > above
> > > > > > issue.
> > > > > >
> > > > > > We have two options here
> > > > > > 1. Settle down to Hive 2.0.0 for sentry 2.0.0 release
> > > > > > 2. Follow-up with Hive community and get this fix ported to Hive
> > 2.4
> > > > > > release and integrate with hive 2.4.
> > > > > >
> > > > > > I give +1 to option-1 and target support for Hive 2.4 in sentry
> > 2.0.1
> > > > > > release.
> > > > > >
> > > > > >
> > > > > > -Kalyan
> > > > > >
> > > > > > On Wed, Jul 19, 2017 at 11:06 AM, Kalyan Kumar Kalvagadda <
> > > > > > kkalyan@cloudera.com> wrote:
> > > > > >
> > > > > > > Lina,
> > > > > > >
> > > > > > > Apache community is currently focused on sentry-ha to be
> > released.
> > > It
> > > > > has
> > > > > > > highest priority over others. If you are contributing to that,
> I
> > > > would
> > > > > > say
> > > > > > > continue doing so.
> > > > > > >
> > > > > > > -Kalyan
> > > > > > >
> > > > > > > On Wed, Jul 19, 2017 at 10:39 AM, Na Li <li...@cloudera.com>
> > > > wrote:
> > > > > > >
> > > > > > >> Kalyan,
> > > > > > >>
> > > > > > >> I can work on those items.
> > > > > > >>
> > > > > > >> Can you create seperate Jiras for each one and add more info.
> > > > > > >>
> > > > > > >> In terms of priority, is finishing sentry-HA a higher priority
> > > than
> > > > > > these
> > > > > > >> (for C6)?
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >>
> > > > > > >> Lina
> > > > > > >>
> > > > > > >> On Wed, Jul 19, 2017 at 10:02 AM, Kalyan Kumar Kalvagadda <
> > > > > > >> kkalyan@cloudera.com> wrote:
> > > > > > >>
> > > > > > >> > Hello all,
> > > > > > >> >
> > > > > > >> > Either way we need to fix couple of things first.
> > > > > > >> >
> > > > > > >> > 1. Fix the test failures when hive-authz2 and datanucleus4
> > > > profiles
> > > > > > are
> > > > > > >> > enabled.
> > > > > > >> > 2. Update e2e tests in sentry-tests-hive-v2 package to have
> > same
> > > > > > >> coverage
> > > > > > >> > as tests in sentry-tests-hive package.
> > > > > > >> >
> > > > > > >> > Does any one wants to take ownership of any of these tasks?
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > -Kalyan
> > > > > > >> >
> > > > > > >> > On Tue, Jul 18, 2017 at 11:02 AM, Na Li <
> lina.li@cloudera.com
> > >
> > > > > wrote:
> > > > > > >> >
> > > > > > >> > > I agree with Colm too. We can support Hive 2.1.1 only in
> > > sentry
> > > > > 2.0
> > > > > > >> > >
> > > > > > >> > > Lina
> > > > > > >> > >
> > > > > > >> > > On Tue, Jul 18, 2017 at 10:03 AM, Sergio Pena <
> > > > > > >> sergio.pena@cloudera.com>
> > > > > > >> > > wrote:
> > > > > > >> > >
> > > > > > >> > > > I agree with Colm too. Hive 1.1 is too old and due to
> the
> > > way
> > > > > how
> > > > > > >> > Sentry
> > > > > > >> > > > works with HMS and notifications, it's not worth the
> > effort
> > > to
> > > > > > keep
> > > > > > >> > > > supporting Hive 1.1. Let's drop it.
> > > > > > >> > > >
> > > > > > >> > > > On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh <
> > > > > > >> > > coheigea@apache.org>
> > > > > > >> > > > wrote:
> > > > > > >> > > >
> > > > > > >> > > > > I think we should drop support for Hive 1.1.0 in
> sentry
> > > > 2.0.0.
> > > > > > >> Hive
> > > > > > >> > > 1.1.0
> > > > > > >> > > > > users can obtain the plugin from the Sentry 1.8.x
> series
> > > of
> > > > > > >> releases.
> > > > > > >> > > > >
> > > > > > >> > > > > Colm.
> > > > > > >> > > > >
> > > > > > >> > > > > On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar
> > Kalvagadda <
> > > > > > >> > > > > kkalyan@cloudera.com> wrote:
> > > > > > >> > > > >
> > > > > > >> > > > > > Hello all,
> > > > > > >> > > > > >
> > > > > > >> > > > > > Sentry currently supports Hive1.1.0 and Hive 2.0.0
> > based
> > > > on
> > > > > > the
> > > > > > >> > > profile
> > > > > > >> > > > > > used. I'm currently working on sentry 2.0.0 release
> > > > upstream
> > > > > > to
> > > > > > >> > > > integrate
> > > > > > >> > > > > > with Hive 2.1.1.
> > > > > > >> > > > > >
> > > > > > >> > > > > > I'm sending this mail seeking inputs on what is the
> > > > support
> > > > > we
> > > > > > >> need
> > > > > > >> > > in
> > > > > > >> > > > > > Sentry 2.0.0 release. Would that be just Hive 2.x or
> > > both
> > > > > Hive
> > > > > > >> 2.x
> > > > > > >> > > and
> > > > > > >> > > > > Hive
> > > > > > >> > > > > > 1.1.0?
> > > > > > >> > > > > >
> > > > > > >> > > > > > I'm not sure if we should be supporting Hive 1.1.0
> in
> > > > sentry
> > > > > > >> 2.0.0.
> > > > > > >> > > As
> > > > > > >> > > > > Hive
> > > > > > >> > > > > > 1.1.0 is pretty old and the notification log
> > > > implementation
> > > > > in
> > > > > > >> this
> > > > > > >> > > > > version
> > > > > > >> > > > > > which Sentry-HA is dependent on has a lot of bugs
> > which
> > > > are
> > > > > > >> fixed
> > > > > > >> > in
> > > > > > >> > > > > later
> > > > > > >> > > > > > releases. Considering this, shouldn't we integrate
> > > sentry
> > > > in
> > > > > > >> > release
> > > > > > >> > > > > 2.0.0
> > > > > > >> > > > > > with Hive 2.1.1 and say that it would no more
> support
> > > > older
> > > > > > >> > versions
> > > > > > >> > > > of
> > > > > > >> > > > > > Hive?
> > > > > > >> > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > > > Here are some of the approaches that we could take
> for
> > > > Hive
> > > > > > >> > > integration
> > > > > > >> > > > > for
> > > > > > >> > > > > > sentry 2.0.0 release.
> > > > > > >> > > > > >
> > > > > > >> > > > > > *Approach-1:* Maintain support for both Hive 1.1.0
> and
> > > > Hive
> > > > > > >> 2.1.1
> > > > > > >> > > > > > *Approach-2: *Maintain support for both Hive 1.1.0
> and
> > > > Hive
> > > > > > >> 2.1.1
> > > > > > >> > for
> > > > > > >> > > > now
> > > > > > >> > > > > > and remove it by the time sentry 2.0.0 is released.
> > > > > > >> > > > > > *Approach-3:* Remove support for Hive 1.1.0 now.
> > > > > > >> > > > > >
> > > > > > >> > > > > > *Downside* for Approach 1 and 2 is that we need to
> put
> > > in
> > > > > lot
> > > > > > of
> > > > > > >> > > effort
> > > > > > >> > > > > to
> > > > > > >> > > > > > have them work together as Hive 2.1.1 has a lot of
> > API's
> > > > > > changes
> > > > > > >> > > which
> > > > > > >> > > > > need
> > > > > > >> > > > > > changes to a lot classes in sentry-binding package
> but
> > > > still
> > > > > > >> > maintain
> > > > > > >> > > > > > backward compatibility.
> > > > > > >> > > > > > If we want to take approach-1 for any reason, it's
> > worth
> > > > > > putting
> > > > > > >> > that
> > > > > > >> > > > > > effort otherwise we could take approach-3.
> > > > > > >> > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > > > My next steps would be based on what I hear form the
> > > > > > community.
> > > > > > >> > > > > >
> > > > > > >> > > > > > -Kalyan
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > > >
> > > > > > >> > > > >
> > > > > > >> > > > > --
> > > > > > >> > > > > Colm O hEigeartaigh
> > > > > > >> > > > >
> > > > > > >> > > > > Talend Community Coder
> > > > > > >> > > > > http://coders.talend.com
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Mat Crocker
> > > Engineering Manager, Cloudera Inc.
> > >
> >
> >
> >
> > --
> > Colm O hEigeartaigh
> >
> > Talend Community Coder
> > http://coders.talend.com
> >
>
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Kalyan Kumar Kalvagadda <kk...@cloudera.com>.
What I've heard from one of the Hive PMC is that they could get the fix to
Hive 2.4. That is the closest. I will follow-up with him again.
Colm's suggestion is reasonable. Picking Hive 2.0.1 might be safer as it
might have fixes that were found in Hive 2.0.0
-Kalyan
On Fri, Aug 11, 2017 at 2:32 AM, Colm O hEigeartaigh <co...@apache.org>
wrote:
> It's unfortunate, but it seems like option 1 is the only solution really
> (although we should pick up Hive 2.0.1 at least). Perhaps the Hive
> community might be receptive to backport the fix to 2.3.x, 2.2.x, 2.1.x
> given that it is a simple fix and a blocker for Sentry?
>
> Colm.
>
> On Thu, Aug 10, 2017 at 11:21 PM, Mat Crocker <ma...@cloudera.com>
> wrote:
>
> > My opinion (non-binding) is option-1. Keep it simple and follow HIVE.
> >
> > On Thu, Aug 10, 2017 at 5:17 PM, Kalyan Kumar Kalvagadda <
> > kkalyan@cloudera.com> wrote:
> >
> > > Lina,
> > >
> > > I don't have that information. I need to dig deeper into apache hive
> wiki
> > > to get some understanding of it.
> > >
> > > -Kalyan
> > >
> > > -Kalyan
> > >
> > > On Wed, Aug 9, 2017 at 12:09 PM, Na Li <li...@cloudera.com> wrote:
> > >
> > > > What is the timeline of back porting the fix to Hive 2.4? And when do
> > we
> > > > plan to release sentry 2.0?
> > > >
> > > > On Wed, Aug 9, 2017 at 11:57 AM, Kalyan Kumar Kalvagadda <
> > > > kkalyan@cloudera.com> wrote:
> > > >
> > > > > Hello all,
> > > > >
> > > > > I would like to share issue that i have discovered while working on
> > > > > SENTRY-1838 <https://issues.apache.org/jira/browse/SENTRY-1838>(
> > > Support
> > > > > Hive 2.1.1 for sentry 2.0)
> > > > >
> > > > > With the design changes done in Hive between 2.0.0 through 2.1.1,
> > > > > HiveOperation
> > > > > is not passed to HiveSemanticAnalyzerHook which is implemented by
> > > > > sentry. HiveOperation
> > > > > is very basic information that sentry needs.
> > > > >
> > > > > I have reported this issue to hive community.(HIVE-17048
> > > > > <https://issues.apache.org/jira/browse/HIVE-17048>). This issue is
> > > fixed
> > > > > in
> > > > > Hive 3.0.0. I will work Hive community to back-port it to Hive 2.4
> > > > release.
> > > > >
> > > > > With that said, we can not integrate with Hive 2.1.1 because of
> above
> > > > > issue.
> > > > >
> > > > > We have two options here
> > > > > 1. Settle down to Hive 2.0.0 for sentry 2.0.0 release
> > > > > 2. Follow-up with Hive community and get this fix ported to Hive
> 2.4
> > > > > release and integrate with hive 2.4.
> > > > >
> > > > > I give +1 to option-1 and target support for Hive 2.4 in sentry
> 2.0.1
> > > > > release.
> > > > >
> > > > >
> > > > > -Kalyan
> > > > >
> > > > > On Wed, Jul 19, 2017 at 11:06 AM, Kalyan Kumar Kalvagadda <
> > > > > kkalyan@cloudera.com> wrote:
> > > > >
> > > > > > Lina,
> > > > > >
> > > > > > Apache community is currently focused on sentry-ha to be
> released.
> > It
> > > > has
> > > > > > highest priority over others. If you are contributing to that, I
> > > would
> > > > > say
> > > > > > continue doing so.
> > > > > >
> > > > > > -Kalyan
> > > > > >
> > > > > > On Wed, Jul 19, 2017 at 10:39 AM, Na Li <li...@cloudera.com>
> > > wrote:
> > > > > >
> > > > > >> Kalyan,
> > > > > >>
> > > > > >> I can work on those items.
> > > > > >>
> > > > > >> Can you create seperate Jiras for each one and add more info.
> > > > > >>
> > > > > >> In terms of priority, is finishing sentry-HA a higher priority
> > than
> > > > > these
> > > > > >> (for C6)?
> > > > > >>
> > > > > >> Thanks,
> > > > > >>
> > > > > >> Lina
> > > > > >>
> > > > > >> On Wed, Jul 19, 2017 at 10:02 AM, Kalyan Kumar Kalvagadda <
> > > > > >> kkalyan@cloudera.com> wrote:
> > > > > >>
> > > > > >> > Hello all,
> > > > > >> >
> > > > > >> > Either way we need to fix couple of things first.
> > > > > >> >
> > > > > >> > 1. Fix the test failures when hive-authz2 and datanucleus4
> > > profiles
> > > > > are
> > > > > >> > enabled.
> > > > > >> > 2. Update e2e tests in sentry-tests-hive-v2 package to have
> same
> > > > > >> coverage
> > > > > >> > as tests in sentry-tests-hive package.
> > > > > >> >
> > > > > >> > Does any one wants to take ownership of any of these tasks?
> > > > > >> >
> > > > > >> >
> > > > > >> > -Kalyan
> > > > > >> >
> > > > > >> > On Tue, Jul 18, 2017 at 11:02 AM, Na Li <lina.li@cloudera.com
> >
> > > > wrote:
> > > > > >> >
> > > > > >> > > I agree with Colm too. We can support Hive 2.1.1 only in
> > sentry
> > > > 2.0
> > > > > >> > >
> > > > > >> > > Lina
> > > > > >> > >
> > > > > >> > > On Tue, Jul 18, 2017 at 10:03 AM, Sergio Pena <
> > > > > >> sergio.pena@cloudera.com>
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > > > I agree with Colm too. Hive 1.1 is too old and due to the
> > way
> > > > how
> > > > > >> > Sentry
> > > > > >> > > > works with HMS and notifications, it's not worth the
> effort
> > to
> > > > > keep
> > > > > >> > > > supporting Hive 1.1. Let's drop it.
> > > > > >> > > >
> > > > > >> > > > On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh <
> > > > > >> > > coheigea@apache.org>
> > > > > >> > > > wrote:
> > > > > >> > > >
> > > > > >> > > > > I think we should drop support for Hive 1.1.0 in sentry
> > > 2.0.0.
> > > > > >> Hive
> > > > > >> > > 1.1.0
> > > > > >> > > > > users can obtain the plugin from the Sentry 1.8.x series
> > of
> > > > > >> releases.
> > > > > >> > > > >
> > > > > >> > > > > Colm.
> > > > > >> > > > >
> > > > > >> > > > > On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar
> Kalvagadda <
> > > > > >> > > > > kkalyan@cloudera.com> wrote:
> > > > > >> > > > >
> > > > > >> > > > > > Hello all,
> > > > > >> > > > > >
> > > > > >> > > > > > Sentry currently supports Hive1.1.0 and Hive 2.0.0
> based
> > > on
> > > > > the
> > > > > >> > > profile
> > > > > >> > > > > > used. I'm currently working on sentry 2.0.0 release
> > > upstream
> > > > > to
> > > > > >> > > > integrate
> > > > > >> > > > > > with Hive 2.1.1.
> > > > > >> > > > > >
> > > > > >> > > > > > I'm sending this mail seeking inputs on what is the
> > > support
> > > > we
> > > > > >> need
> > > > > >> > > in
> > > > > >> > > > > > Sentry 2.0.0 release. Would that be just Hive 2.x or
> > both
> > > > Hive
> > > > > >> 2.x
> > > > > >> > > and
> > > > > >> > > > > Hive
> > > > > >> > > > > > 1.1.0?
> > > > > >> > > > > >
> > > > > >> > > > > > I'm not sure if we should be supporting Hive 1.1.0 in
> > > sentry
> > > > > >> 2.0.0.
> > > > > >> > > As
> > > > > >> > > > > Hive
> > > > > >> > > > > > 1.1.0 is pretty old and the notification log
> > > implementation
> > > > in
> > > > > >> this
> > > > > >> > > > > version
> > > > > >> > > > > > which Sentry-HA is dependent on has a lot of bugs
> which
> > > are
> > > > > >> fixed
> > > > > >> > in
> > > > > >> > > > > later
> > > > > >> > > > > > releases. Considering this, shouldn't we integrate
> > sentry
> > > in
> > > > > >> > release
> > > > > >> > > > > 2.0.0
> > > > > >> > > > > > with Hive 2.1.1 and say that it would no more support
> > > older
> > > > > >> > versions
> > > > > >> > > > of
> > > > > >> > > > > > Hive?
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > Here are some of the approaches that we could take for
> > > Hive
> > > > > >> > > integration
> > > > > >> > > > > for
> > > > > >> > > > > > sentry 2.0.0 release.
> > > > > >> > > > > >
> > > > > >> > > > > > *Approach-1:* Maintain support for both Hive 1.1.0 and
> > > Hive
> > > > > >> 2.1.1
> > > > > >> > > > > > *Approach-2: *Maintain support for both Hive 1.1.0 and
> > > Hive
> > > > > >> 2.1.1
> > > > > >> > for
> > > > > >> > > > now
> > > > > >> > > > > > and remove it by the time sentry 2.0.0 is released.
> > > > > >> > > > > > *Approach-3:* Remove support for Hive 1.1.0 now.
> > > > > >> > > > > >
> > > > > >> > > > > > *Downside* for Approach 1 and 2 is that we need to put
> > in
> > > > lot
> > > > > of
> > > > > >> > > effort
> > > > > >> > > > > to
> > > > > >> > > > > > have them work together as Hive 2.1.1 has a lot of
> API's
> > > > > changes
> > > > > >> > > which
> > > > > >> > > > > need
> > > > > >> > > > > > changes to a lot classes in sentry-binding package but
> > > still
> > > > > >> > maintain
> > > > > >> > > > > > backward compatibility.
> > > > > >> > > > > > If we want to take approach-1 for any reason, it's
> worth
> > > > > putting
> > > > > >> > that
> > > > > >> > > > > > effort otherwise we could take approach-3.
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > My next steps would be based on what I hear form the
> > > > > community.
> > > > > >> > > > > >
> > > > > >> > > > > > -Kalyan
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > > >
> > > > > >> > > > >
> > > > > >> > > > > --
> > > > > >> > > > > Colm O hEigeartaigh
> > > > > >> > > > >
> > > > > >> > > > > Talend Community Coder
> > > > > >> > > > > http://coders.talend.com
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Mat Crocker
> > Engineering Manager, Cloudera Inc.
> >
>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Colm O hEigeartaigh <co...@apache.org>.
It's unfortunate, but it seems like option 1 is the only solution really
(although we should pick up Hive 2.0.1 at least). Perhaps the Hive
community might be receptive to backport the fix to 2.3.x, 2.2.x, 2.1.x
given that it is a simple fix and a blocker for Sentry?
Colm.
On Thu, Aug 10, 2017 at 11:21 PM, Mat Crocker <ma...@cloudera.com>
wrote:
> My opinion (non-binding) is option-1. Keep it simple and follow HIVE.
>
> On Thu, Aug 10, 2017 at 5:17 PM, Kalyan Kumar Kalvagadda <
> kkalyan@cloudera.com> wrote:
>
> > Lina,
> >
> > I don't have that information. I need to dig deeper into apache hive wiki
> > to get some understanding of it.
> >
> > -Kalyan
> >
> > -Kalyan
> >
> > On Wed, Aug 9, 2017 at 12:09 PM, Na Li <li...@cloudera.com> wrote:
> >
> > > What is the timeline of back porting the fix to Hive 2.4? And when do
> we
> > > plan to release sentry 2.0?
> > >
> > > On Wed, Aug 9, 2017 at 11:57 AM, Kalyan Kumar Kalvagadda <
> > > kkalyan@cloudera.com> wrote:
> > >
> > > > Hello all,
> > > >
> > > > I would like to share issue that i have discovered while working on
> > > > SENTRY-1838 <https://issues.apache.org/jira/browse/SENTRY-1838>(
> > Support
> > > > Hive 2.1.1 for sentry 2.0)
> > > >
> > > > With the design changes done in Hive between 2.0.0 through 2.1.1,
> > > > HiveOperation
> > > > is not passed to HiveSemanticAnalyzerHook which is implemented by
> > > > sentry. HiveOperation
> > > > is very basic information that sentry needs.
> > > >
> > > > I have reported this issue to hive community.(HIVE-17048
> > > > <https://issues.apache.org/jira/browse/HIVE-17048>). This issue is
> > fixed
> > > > in
> > > > Hive 3.0.0. I will work Hive community to back-port it to Hive 2.4
> > > release.
> > > >
> > > > With that said, we can not integrate with Hive 2.1.1 because of above
> > > > issue.
> > > >
> > > > We have two options here
> > > > 1. Settle down to Hive 2.0.0 for sentry 2.0.0 release
> > > > 2. Follow-up with Hive community and get this fix ported to Hive 2.4
> > > > release and integrate with hive 2.4.
> > > >
> > > > I give +1 to option-1 and target support for Hive 2.4 in sentry 2.0.1
> > > > release.
> > > >
> > > >
> > > > -Kalyan
> > > >
> > > > On Wed, Jul 19, 2017 at 11:06 AM, Kalyan Kumar Kalvagadda <
> > > > kkalyan@cloudera.com> wrote:
> > > >
> > > > > Lina,
> > > > >
> > > > > Apache community is currently focused on sentry-ha to be released.
> It
> > > has
> > > > > highest priority over others. If you are contributing to that, I
> > would
> > > > say
> > > > > continue doing so.
> > > > >
> > > > > -Kalyan
> > > > >
> > > > > On Wed, Jul 19, 2017 at 10:39 AM, Na Li <li...@cloudera.com>
> > wrote:
> > > > >
> > > > >> Kalyan,
> > > > >>
> > > > >> I can work on those items.
> > > > >>
> > > > >> Can you create seperate Jiras for each one and add more info.
> > > > >>
> > > > >> In terms of priority, is finishing sentry-HA a higher priority
> than
> > > > these
> > > > >> (for C6)?
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> Lina
> > > > >>
> > > > >> On Wed, Jul 19, 2017 at 10:02 AM, Kalyan Kumar Kalvagadda <
> > > > >> kkalyan@cloudera.com> wrote:
> > > > >>
> > > > >> > Hello all,
> > > > >> >
> > > > >> > Either way we need to fix couple of things first.
> > > > >> >
> > > > >> > 1. Fix the test failures when hive-authz2 and datanucleus4
> > profiles
> > > > are
> > > > >> > enabled.
> > > > >> > 2. Update e2e tests in sentry-tests-hive-v2 package to have same
> > > > >> coverage
> > > > >> > as tests in sentry-tests-hive package.
> > > > >> >
> > > > >> > Does any one wants to take ownership of any of these tasks?
> > > > >> >
> > > > >> >
> > > > >> > -Kalyan
> > > > >> >
> > > > >> > On Tue, Jul 18, 2017 at 11:02 AM, Na Li <li...@cloudera.com>
> > > wrote:
> > > > >> >
> > > > >> > > I agree with Colm too. We can support Hive 2.1.1 only in
> sentry
> > > 2.0
> > > > >> > >
> > > > >> > > Lina
> > > > >> > >
> > > > >> > > On Tue, Jul 18, 2017 at 10:03 AM, Sergio Pena <
> > > > >> sergio.pena@cloudera.com>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > I agree with Colm too. Hive 1.1 is too old and due to the
> way
> > > how
> > > > >> > Sentry
> > > > >> > > > works with HMS and notifications, it's not worth the effort
> to
> > > > keep
> > > > >> > > > supporting Hive 1.1. Let's drop it.
> > > > >> > > >
> > > > >> > > > On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh <
> > > > >> > > coheigea@apache.org>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > > I think we should drop support for Hive 1.1.0 in sentry
> > 2.0.0.
> > > > >> Hive
> > > > >> > > 1.1.0
> > > > >> > > > > users can obtain the plugin from the Sentry 1.8.x series
> of
> > > > >> releases.
> > > > >> > > > >
> > > > >> > > > > Colm.
> > > > >> > > > >
> > > > >> > > > > On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar Kalvagadda <
> > > > >> > > > > kkalyan@cloudera.com> wrote:
> > > > >> > > > >
> > > > >> > > > > > Hello all,
> > > > >> > > > > >
> > > > >> > > > > > Sentry currently supports Hive1.1.0 and Hive 2.0.0 based
> > on
> > > > the
> > > > >> > > profile
> > > > >> > > > > > used. I'm currently working on sentry 2.0.0 release
> > upstream
> > > > to
> > > > >> > > > integrate
> > > > >> > > > > > with Hive 2.1.1.
> > > > >> > > > > >
> > > > >> > > > > > I'm sending this mail seeking inputs on what is the
> > support
> > > we
> > > > >> need
> > > > >> > > in
> > > > >> > > > > > Sentry 2.0.0 release. Would that be just Hive 2.x or
> both
> > > Hive
> > > > >> 2.x
> > > > >> > > and
> > > > >> > > > > Hive
> > > > >> > > > > > 1.1.0?
> > > > >> > > > > >
> > > > >> > > > > > I'm not sure if we should be supporting Hive 1.1.0 in
> > sentry
> > > > >> 2.0.0.
> > > > >> > > As
> > > > >> > > > > Hive
> > > > >> > > > > > 1.1.0 is pretty old and the notification log
> > implementation
> > > in
> > > > >> this
> > > > >> > > > > version
> > > > >> > > > > > which Sentry-HA is dependent on has a lot of bugs which
> > are
> > > > >> fixed
> > > > >> > in
> > > > >> > > > > later
> > > > >> > > > > > releases. Considering this, shouldn't we integrate
> sentry
> > in
> > > > >> > release
> > > > >> > > > > 2.0.0
> > > > >> > > > > > with Hive 2.1.1 and say that it would no more support
> > older
> > > > >> > versions
> > > > >> > > > of
> > > > >> > > > > > Hive?
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > Here are some of the approaches that we could take for
> > Hive
> > > > >> > > integration
> > > > >> > > > > for
> > > > >> > > > > > sentry 2.0.0 release.
> > > > >> > > > > >
> > > > >> > > > > > *Approach-1:* Maintain support for both Hive 1.1.0 and
> > Hive
> > > > >> 2.1.1
> > > > >> > > > > > *Approach-2: *Maintain support for both Hive 1.1.0 and
> > Hive
> > > > >> 2.1.1
> > > > >> > for
> > > > >> > > > now
> > > > >> > > > > > and remove it by the time sentry 2.0.0 is released.
> > > > >> > > > > > *Approach-3:* Remove support for Hive 1.1.0 now.
> > > > >> > > > > >
> > > > >> > > > > > *Downside* for Approach 1 and 2 is that we need to put
> in
> > > lot
> > > > of
> > > > >> > > effort
> > > > >> > > > > to
> > > > >> > > > > > have them work together as Hive 2.1.1 has a lot of API's
> > > > changes
> > > > >> > > which
> > > > >> > > > > need
> > > > >> > > > > > changes to a lot classes in sentry-binding package but
> > still
> > > > >> > maintain
> > > > >> > > > > > backward compatibility.
> > > > >> > > > > > If we want to take approach-1 for any reason, it's worth
> > > > putting
> > > > >> > that
> > > > >> > > > > > effort otherwise we could take approach-3.
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > My next steps would be based on what I hear form the
> > > > community.
> > > > >> > > > > >
> > > > >> > > > > > -Kalyan
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > --
> > > > >> > > > > Colm O hEigeartaigh
> > > > >> > > > >
> > > > >> > > > > Talend Community Coder
> > > > >> > > > > http://coders.talend.com
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Mat Crocker
> Engineering Manager, Cloudera Inc.
>
--
Colm O hEigeartaigh
Talend Community Coder
http://coders.talend.com
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Mat Crocker <ma...@cloudera.com>.
My opinion (non-binding) is option-1. Keep it simple and follow HIVE.
On Thu, Aug 10, 2017 at 5:17 PM, Kalyan Kumar Kalvagadda <
kkalyan@cloudera.com> wrote:
> Lina,
>
> I don't have that information. I need to dig deeper into apache hive wiki
> to get some understanding of it.
>
> -Kalyan
>
> -Kalyan
>
> On Wed, Aug 9, 2017 at 12:09 PM, Na Li <li...@cloudera.com> wrote:
>
> > What is the timeline of back porting the fix to Hive 2.4? And when do we
> > plan to release sentry 2.0?
> >
> > On Wed, Aug 9, 2017 at 11:57 AM, Kalyan Kumar Kalvagadda <
> > kkalyan@cloudera.com> wrote:
> >
> > > Hello all,
> > >
> > > I would like to share issue that i have discovered while working on
> > > SENTRY-1838 <https://issues.apache.org/jira/browse/SENTRY-1838>(
> Support
> > > Hive 2.1.1 for sentry 2.0)
> > >
> > > With the design changes done in Hive between 2.0.0 through 2.1.1,
> > > HiveOperation
> > > is not passed to HiveSemanticAnalyzerHook which is implemented by
> > > sentry. HiveOperation
> > > is very basic information that sentry needs.
> > >
> > > I have reported this issue to hive community.(HIVE-17048
> > > <https://issues.apache.org/jira/browse/HIVE-17048>). This issue is
> fixed
> > > in
> > > Hive 3.0.0. I will work Hive community to back-port it to Hive 2.4
> > release.
> > >
> > > With that said, we can not integrate with Hive 2.1.1 because of above
> > > issue.
> > >
> > > We have two options here
> > > 1. Settle down to Hive 2.0.0 for sentry 2.0.0 release
> > > 2. Follow-up with Hive community and get this fix ported to Hive 2.4
> > > release and integrate with hive 2.4.
> > >
> > > I give +1 to option-1 and target support for Hive 2.4 in sentry 2.0.1
> > > release.
> > >
> > >
> > > -Kalyan
> > >
> > > On Wed, Jul 19, 2017 at 11:06 AM, Kalyan Kumar Kalvagadda <
> > > kkalyan@cloudera.com> wrote:
> > >
> > > > Lina,
> > > >
> > > > Apache community is currently focused on sentry-ha to be released. It
> > has
> > > > highest priority over others. If you are contributing to that, I
> would
> > > say
> > > > continue doing so.
> > > >
> > > > -Kalyan
> > > >
> > > > On Wed, Jul 19, 2017 at 10:39 AM, Na Li <li...@cloudera.com>
> wrote:
> > > >
> > > >> Kalyan,
> > > >>
> > > >> I can work on those items.
> > > >>
> > > >> Can you create seperate Jiras for each one and add more info.
> > > >>
> > > >> In terms of priority, is finishing sentry-HA a higher priority than
> > > these
> > > >> (for C6)?
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Lina
> > > >>
> > > >> On Wed, Jul 19, 2017 at 10:02 AM, Kalyan Kumar Kalvagadda <
> > > >> kkalyan@cloudera.com> wrote:
> > > >>
> > > >> > Hello all,
> > > >> >
> > > >> > Either way we need to fix couple of things first.
> > > >> >
> > > >> > 1. Fix the test failures when hive-authz2 and datanucleus4
> profiles
> > > are
> > > >> > enabled.
> > > >> > 2. Update e2e tests in sentry-tests-hive-v2 package to have same
> > > >> coverage
> > > >> > as tests in sentry-tests-hive package.
> > > >> >
> > > >> > Does any one wants to take ownership of any of these tasks?
> > > >> >
> > > >> >
> > > >> > -Kalyan
> > > >> >
> > > >> > On Tue, Jul 18, 2017 at 11:02 AM, Na Li <li...@cloudera.com>
> > wrote:
> > > >> >
> > > >> > > I agree with Colm too. We can support Hive 2.1.1 only in sentry
> > 2.0
> > > >> > >
> > > >> > > Lina
> > > >> > >
> > > >> > > On Tue, Jul 18, 2017 at 10:03 AM, Sergio Pena <
> > > >> sergio.pena@cloudera.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > I agree with Colm too. Hive 1.1 is too old and due to the way
> > how
> > > >> > Sentry
> > > >> > > > works with HMS and notifications, it's not worth the effort to
> > > keep
> > > >> > > > supporting Hive 1.1. Let's drop it.
> > > >> > > >
> > > >> > > > On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh <
> > > >> > > coheigea@apache.org>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > I think we should drop support for Hive 1.1.0 in sentry
> 2.0.0.
> > > >> Hive
> > > >> > > 1.1.0
> > > >> > > > > users can obtain the plugin from the Sentry 1.8.x series of
> > > >> releases.
> > > >> > > > >
> > > >> > > > > Colm.
> > > >> > > > >
> > > >> > > > > On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar Kalvagadda <
> > > >> > > > > kkalyan@cloudera.com> wrote:
> > > >> > > > >
> > > >> > > > > > Hello all,
> > > >> > > > > >
> > > >> > > > > > Sentry currently supports Hive1.1.0 and Hive 2.0.0 based
> on
> > > the
> > > >> > > profile
> > > >> > > > > > used. I'm currently working on sentry 2.0.0 release
> upstream
> > > to
> > > >> > > > integrate
> > > >> > > > > > with Hive 2.1.1.
> > > >> > > > > >
> > > >> > > > > > I'm sending this mail seeking inputs on what is the
> support
> > we
> > > >> need
> > > >> > > in
> > > >> > > > > > Sentry 2.0.0 release. Would that be just Hive 2.x or both
> > Hive
> > > >> 2.x
> > > >> > > and
> > > >> > > > > Hive
> > > >> > > > > > 1.1.0?
> > > >> > > > > >
> > > >> > > > > > I'm not sure if we should be supporting Hive 1.1.0 in
> sentry
> > > >> 2.0.0.
> > > >> > > As
> > > >> > > > > Hive
> > > >> > > > > > 1.1.0 is pretty old and the notification log
> implementation
> > in
> > > >> this
> > > >> > > > > version
> > > >> > > > > > which Sentry-HA is dependent on has a lot of bugs which
> are
> > > >> fixed
> > > >> > in
> > > >> > > > > later
> > > >> > > > > > releases. Considering this, shouldn't we integrate sentry
> in
> > > >> > release
> > > >> > > > > 2.0.0
> > > >> > > > > > with Hive 2.1.1 and say that it would no more support
> older
> > > >> > versions
> > > >> > > > of
> > > >> > > > > > Hive?
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > Here are some of the approaches that we could take for
> Hive
> > > >> > > integration
> > > >> > > > > for
> > > >> > > > > > sentry 2.0.0 release.
> > > >> > > > > >
> > > >> > > > > > *Approach-1:* Maintain support for both Hive 1.1.0 and
> Hive
> > > >> 2.1.1
> > > >> > > > > > *Approach-2: *Maintain support for both Hive 1.1.0 and
> Hive
> > > >> 2.1.1
> > > >> > for
> > > >> > > > now
> > > >> > > > > > and remove it by the time sentry 2.0.0 is released.
> > > >> > > > > > *Approach-3:* Remove support for Hive 1.1.0 now.
> > > >> > > > > >
> > > >> > > > > > *Downside* for Approach 1 and 2 is that we need to put in
> > lot
> > > of
> > > >> > > effort
> > > >> > > > > to
> > > >> > > > > > have them work together as Hive 2.1.1 has a lot of API's
> > > changes
> > > >> > > which
> > > >> > > > > need
> > > >> > > > > > changes to a lot classes in sentry-binding package but
> still
> > > >> > maintain
> > > >> > > > > > backward compatibility.
> > > >> > > > > > If we want to take approach-1 for any reason, it's worth
> > > putting
> > > >> > that
> > > >> > > > > > effort otherwise we could take approach-3.
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > My next steps would be based on what I hear form the
> > > community.
> > > >> > > > > >
> > > >> > > > > > -Kalyan
> > > >> > > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > --
> > > >> > > > > Colm O hEigeartaigh
> > > >> > > > >
> > > >> > > > > Talend Community Coder
> > > >> > > > > http://coders.talend.com
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
--
Mat Crocker
Engineering Manager, Cloudera Inc.
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Kalyan Kumar Kalvagadda <kk...@cloudera.com>.
Lina,
I don't have that information. I need to dig deeper into apache hive wiki
to get some understanding of it.
-Kalyan
-Kalyan
On Wed, Aug 9, 2017 at 12:09 PM, Na Li <li...@cloudera.com> wrote:
> What is the timeline of back porting the fix to Hive 2.4? And when do we
> plan to release sentry 2.0?
>
> On Wed, Aug 9, 2017 at 11:57 AM, Kalyan Kumar Kalvagadda <
> kkalyan@cloudera.com> wrote:
>
> > Hello all,
> >
> > I would like to share issue that i have discovered while working on
> > SENTRY-1838 <https://issues.apache.org/jira/browse/SENTRY-1838>(Support
> > Hive 2.1.1 for sentry 2.0)
> >
> > With the design changes done in Hive between 2.0.0 through 2.1.1,
> > HiveOperation
> > is not passed to HiveSemanticAnalyzerHook which is implemented by
> > sentry. HiveOperation
> > is very basic information that sentry needs.
> >
> > I have reported this issue to hive community.(HIVE-17048
> > <https://issues.apache.org/jira/browse/HIVE-17048>). This issue is fixed
> > in
> > Hive 3.0.0. I will work Hive community to back-port it to Hive 2.4
> release.
> >
> > With that said, we can not integrate with Hive 2.1.1 because of above
> > issue.
> >
> > We have two options here
> > 1. Settle down to Hive 2.0.0 for sentry 2.0.0 release
> > 2. Follow-up with Hive community and get this fix ported to Hive 2.4
> > release and integrate with hive 2.4.
> >
> > I give +1 to option-1 and target support for Hive 2.4 in sentry 2.0.1
> > release.
> >
> >
> > -Kalyan
> >
> > On Wed, Jul 19, 2017 at 11:06 AM, Kalyan Kumar Kalvagadda <
> > kkalyan@cloudera.com> wrote:
> >
> > > Lina,
> > >
> > > Apache community is currently focused on sentry-ha to be released. It
> has
> > > highest priority over others. If you are contributing to that, I would
> > say
> > > continue doing so.
> > >
> > > -Kalyan
> > >
> > > On Wed, Jul 19, 2017 at 10:39 AM, Na Li <li...@cloudera.com> wrote:
> > >
> > >> Kalyan,
> > >>
> > >> I can work on those items.
> > >>
> > >> Can you create seperate Jiras for each one and add more info.
> > >>
> > >> In terms of priority, is finishing sentry-HA a higher priority than
> > these
> > >> (for C6)?
> > >>
> > >> Thanks,
> > >>
> > >> Lina
> > >>
> > >> On Wed, Jul 19, 2017 at 10:02 AM, Kalyan Kumar Kalvagadda <
> > >> kkalyan@cloudera.com> wrote:
> > >>
> > >> > Hello all,
> > >> >
> > >> > Either way we need to fix couple of things first.
> > >> >
> > >> > 1. Fix the test failures when hive-authz2 and datanucleus4 profiles
> > are
> > >> > enabled.
> > >> > 2. Update e2e tests in sentry-tests-hive-v2 package to have same
> > >> coverage
> > >> > as tests in sentry-tests-hive package.
> > >> >
> > >> > Does any one wants to take ownership of any of these tasks?
> > >> >
> > >> >
> > >> > -Kalyan
> > >> >
> > >> > On Tue, Jul 18, 2017 at 11:02 AM, Na Li <li...@cloudera.com>
> wrote:
> > >> >
> > >> > > I agree with Colm too. We can support Hive 2.1.1 only in sentry
> 2.0
> > >> > >
> > >> > > Lina
> > >> > >
> > >> > > On Tue, Jul 18, 2017 at 10:03 AM, Sergio Pena <
> > >> sergio.pena@cloudera.com>
> > >> > > wrote:
> > >> > >
> > >> > > > I agree with Colm too. Hive 1.1 is too old and due to the way
> how
> > >> > Sentry
> > >> > > > works with HMS and notifications, it's not worth the effort to
> > keep
> > >> > > > supporting Hive 1.1. Let's drop it.
> > >> > > >
> > >> > > > On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh <
> > >> > > coheigea@apache.org>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > I think we should drop support for Hive 1.1.0 in sentry 2.0.0.
> > >> Hive
> > >> > > 1.1.0
> > >> > > > > users can obtain the plugin from the Sentry 1.8.x series of
> > >> releases.
> > >> > > > >
> > >> > > > > Colm.
> > >> > > > >
> > >> > > > > On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar Kalvagadda <
> > >> > > > > kkalyan@cloudera.com> wrote:
> > >> > > > >
> > >> > > > > > Hello all,
> > >> > > > > >
> > >> > > > > > Sentry currently supports Hive1.1.0 and Hive 2.0.0 based on
> > the
> > >> > > profile
> > >> > > > > > used. I'm currently working on sentry 2.0.0 release upstream
> > to
> > >> > > > integrate
> > >> > > > > > with Hive 2.1.1.
> > >> > > > > >
> > >> > > > > > I'm sending this mail seeking inputs on what is the support
> we
> > >> need
> > >> > > in
> > >> > > > > > Sentry 2.0.0 release. Would that be just Hive 2.x or both
> Hive
> > >> 2.x
> > >> > > and
> > >> > > > > Hive
> > >> > > > > > 1.1.0?
> > >> > > > > >
> > >> > > > > > I'm not sure if we should be supporting Hive 1.1.0 in sentry
> > >> 2.0.0.
> > >> > > As
> > >> > > > > Hive
> > >> > > > > > 1.1.0 is pretty old and the notification log implementation
> in
> > >> this
> > >> > > > > version
> > >> > > > > > which Sentry-HA is dependent on has a lot of bugs which are
> > >> fixed
> > >> > in
> > >> > > > > later
> > >> > > > > > releases. Considering this, shouldn't we integrate sentry in
> > >> > release
> > >> > > > > 2.0.0
> > >> > > > > > with Hive 2.1.1 and say that it would no more support older
> > >> > versions
> > >> > > > of
> > >> > > > > > Hive?
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > Here are some of the approaches that we could take for Hive
> > >> > > integration
> > >> > > > > for
> > >> > > > > > sentry 2.0.0 release.
> > >> > > > > >
> > >> > > > > > *Approach-1:* Maintain support for both Hive 1.1.0 and Hive
> > >> 2.1.1
> > >> > > > > > *Approach-2: *Maintain support for both Hive 1.1.0 and Hive
> > >> 2.1.1
> > >> > for
> > >> > > > now
> > >> > > > > > and remove it by the time sentry 2.0.0 is released.
> > >> > > > > > *Approach-3:* Remove support for Hive 1.1.0 now.
> > >> > > > > >
> > >> > > > > > *Downside* for Approach 1 and 2 is that we need to put in
> lot
> > of
> > >> > > effort
> > >> > > > > to
> > >> > > > > > have them work together as Hive 2.1.1 has a lot of API's
> > changes
> > >> > > which
> > >> > > > > need
> > >> > > > > > changes to a lot classes in sentry-binding package but still
> > >> > maintain
> > >> > > > > > backward compatibility.
> > >> > > > > > If we want to take approach-1 for any reason, it's worth
> > putting
> > >> > that
> > >> > > > > > effort otherwise we could take approach-3.
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > My next steps would be based on what I hear form the
> > community.
> > >> > > > > >
> > >> > > > > > -Kalyan
> > >> > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > Colm O hEigeartaigh
> > >> > > > >
> > >> > > > > Talend Community Coder
> > >> > > > > http://coders.talend.com
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Na Li <li...@cloudera.com>.
What is the timeline of back porting the fix to Hive 2.4? And when do we
plan to release sentry 2.0?
On Wed, Aug 9, 2017 at 11:57 AM, Kalyan Kumar Kalvagadda <
kkalyan@cloudera.com> wrote:
> Hello all,
>
> I would like to share issue that i have discovered while working on
> SENTRY-1838 <https://issues.apache.org/jira/browse/SENTRY-1838>(Support
> Hive 2.1.1 for sentry 2.0)
>
> With the design changes done in Hive between 2.0.0 through 2.1.1,
> HiveOperation
> is not passed to HiveSemanticAnalyzerHook which is implemented by
> sentry. HiveOperation
> is very basic information that sentry needs.
>
> I have reported this issue to hive community.(HIVE-17048
> <https://issues.apache.org/jira/browse/HIVE-17048>). This issue is fixed
> in
> Hive 3.0.0. I will work Hive community to back-port it to Hive 2.4 release.
>
> With that said, we can not integrate with Hive 2.1.1 because of above
> issue.
>
> We have two options here
> 1. Settle down to Hive 2.0.0 for sentry 2.0.0 release
> 2. Follow-up with Hive community and get this fix ported to Hive 2.4
> release and integrate with hive 2.4.
>
> I give +1 to option-1 and target support for Hive 2.4 in sentry 2.0.1
> release.
>
>
> -Kalyan
>
> On Wed, Jul 19, 2017 at 11:06 AM, Kalyan Kumar Kalvagadda <
> kkalyan@cloudera.com> wrote:
>
> > Lina,
> >
> > Apache community is currently focused on sentry-ha to be released. It has
> > highest priority over others. If you are contributing to that, I would
> say
> > continue doing so.
> >
> > -Kalyan
> >
> > On Wed, Jul 19, 2017 at 10:39 AM, Na Li <li...@cloudera.com> wrote:
> >
> >> Kalyan,
> >>
> >> I can work on those items.
> >>
> >> Can you create seperate Jiras for each one and add more info.
> >>
> >> In terms of priority, is finishing sentry-HA a higher priority than
> these
> >> (for C6)?
> >>
> >> Thanks,
> >>
> >> Lina
> >>
> >> On Wed, Jul 19, 2017 at 10:02 AM, Kalyan Kumar Kalvagadda <
> >> kkalyan@cloudera.com> wrote:
> >>
> >> > Hello all,
> >> >
> >> > Either way we need to fix couple of things first.
> >> >
> >> > 1. Fix the test failures when hive-authz2 and datanucleus4 profiles
> are
> >> > enabled.
> >> > 2. Update e2e tests in sentry-tests-hive-v2 package to have same
> >> coverage
> >> > as tests in sentry-tests-hive package.
> >> >
> >> > Does any one wants to take ownership of any of these tasks?
> >> >
> >> >
> >> > -Kalyan
> >> >
> >> > On Tue, Jul 18, 2017 at 11:02 AM, Na Li <li...@cloudera.com> wrote:
> >> >
> >> > > I agree with Colm too. We can support Hive 2.1.1 only in sentry 2.0
> >> > >
> >> > > Lina
> >> > >
> >> > > On Tue, Jul 18, 2017 at 10:03 AM, Sergio Pena <
> >> sergio.pena@cloudera.com>
> >> > > wrote:
> >> > >
> >> > > > I agree with Colm too. Hive 1.1 is too old and due to the way how
> >> > Sentry
> >> > > > works with HMS and notifications, it's not worth the effort to
> keep
> >> > > > supporting Hive 1.1. Let's drop it.
> >> > > >
> >> > > > On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh <
> >> > > coheigea@apache.org>
> >> > > > wrote:
> >> > > >
> >> > > > > I think we should drop support for Hive 1.1.0 in sentry 2.0.0.
> >> Hive
> >> > > 1.1.0
> >> > > > > users can obtain the plugin from the Sentry 1.8.x series of
> >> releases.
> >> > > > >
> >> > > > > Colm.
> >> > > > >
> >> > > > > On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar Kalvagadda <
> >> > > > > kkalyan@cloudera.com> wrote:
> >> > > > >
> >> > > > > > Hello all,
> >> > > > > >
> >> > > > > > Sentry currently supports Hive1.1.0 and Hive 2.0.0 based on
> the
> >> > > profile
> >> > > > > > used. I'm currently working on sentry 2.0.0 release upstream
> to
> >> > > > integrate
> >> > > > > > with Hive 2.1.1.
> >> > > > > >
> >> > > > > > I'm sending this mail seeking inputs on what is the support we
> >> need
> >> > > in
> >> > > > > > Sentry 2.0.0 release. Would that be just Hive 2.x or both Hive
> >> 2.x
> >> > > and
> >> > > > > Hive
> >> > > > > > 1.1.0?
> >> > > > > >
> >> > > > > > I'm not sure if we should be supporting Hive 1.1.0 in sentry
> >> 2.0.0.
> >> > > As
> >> > > > > Hive
> >> > > > > > 1.1.0 is pretty old and the notification log implementation in
> >> this
> >> > > > > version
> >> > > > > > which Sentry-HA is dependent on has a lot of bugs which are
> >> fixed
> >> > in
> >> > > > > later
> >> > > > > > releases. Considering this, shouldn't we integrate sentry in
> >> > release
> >> > > > > 2.0.0
> >> > > > > > with Hive 2.1.1 and say that it would no more support older
> >> > versions
> >> > > > of
> >> > > > > > Hive?
> >> > > > > >
> >> > > > > >
> >> > > > > > Here are some of the approaches that we could take for Hive
> >> > > integration
> >> > > > > for
> >> > > > > > sentry 2.0.0 release.
> >> > > > > >
> >> > > > > > *Approach-1:* Maintain support for both Hive 1.1.0 and Hive
> >> 2.1.1
> >> > > > > > *Approach-2: *Maintain support for both Hive 1.1.0 and Hive
> >> 2.1.1
> >> > for
> >> > > > now
> >> > > > > > and remove it by the time sentry 2.0.0 is released.
> >> > > > > > *Approach-3:* Remove support for Hive 1.1.0 now.
> >> > > > > >
> >> > > > > > *Downside* for Approach 1 and 2 is that we need to put in lot
> of
> >> > > effort
> >> > > > > to
> >> > > > > > have them work together as Hive 2.1.1 has a lot of API's
> changes
> >> > > which
> >> > > > > need
> >> > > > > > changes to a lot classes in sentry-binding package but still
> >> > maintain
> >> > > > > > backward compatibility.
> >> > > > > > If we want to take approach-1 for any reason, it's worth
> putting
> >> > that
> >> > > > > > effort otherwise we could take approach-3.
> >> > > > > >
> >> > > > > >
> >> > > > > > My next steps would be based on what I hear form the
> community.
> >> > > > > >
> >> > > > > > -Kalyan
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Colm O hEigeartaigh
> >> > > > >
> >> > > > > Talend Community Coder
> >> > > > > http://coders.talend.com
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Kalyan Kumar Kalvagadda <kk...@cloudera.com>.
Hello all,
I would like to share issue that i have discovered while working on
SENTRY-1838 <https://issues.apache.org/jira/browse/SENTRY-1838>(Support
Hive 2.1.1 for sentry 2.0)
With the design changes done in Hive between 2.0.0 through 2.1.1, HiveOperation
is not passed to HiveSemanticAnalyzerHook which is implemented by
sentry. HiveOperation
is very basic information that sentry needs.
I have reported this issue to hive community.(HIVE-17048
<https://issues.apache.org/jira/browse/HIVE-17048>). This issue is fixed in
Hive 3.0.0. I will work Hive community to back-port it to Hive 2.4 release.
With that said, we can not integrate with Hive 2.1.1 because of above issue.
We have two options here
1. Settle down to Hive 2.0.0 for sentry 2.0.0 release
2. Follow-up with Hive community and get this fix ported to Hive 2.4
release and integrate with hive 2.4.
I give +1 to option-1 and target support for Hive 2.4 in sentry 2.0.1
release.
-Kalyan
On Wed, Jul 19, 2017 at 11:06 AM, Kalyan Kumar Kalvagadda <
kkalyan@cloudera.com> wrote:
> Lina,
>
> Apache community is currently focused on sentry-ha to be released. It has
> highest priority over others. If you are contributing to that, I would say
> continue doing so.
>
> -Kalyan
>
> On Wed, Jul 19, 2017 at 10:39 AM, Na Li <li...@cloudera.com> wrote:
>
>> Kalyan,
>>
>> I can work on those items.
>>
>> Can you create seperate Jiras for each one and add more info.
>>
>> In terms of priority, is finishing sentry-HA a higher priority than these
>> (for C6)?
>>
>> Thanks,
>>
>> Lina
>>
>> On Wed, Jul 19, 2017 at 10:02 AM, Kalyan Kumar Kalvagadda <
>> kkalyan@cloudera.com> wrote:
>>
>> > Hello all,
>> >
>> > Either way we need to fix couple of things first.
>> >
>> > 1. Fix the test failures when hive-authz2 and datanucleus4 profiles are
>> > enabled.
>> > 2. Update e2e tests in sentry-tests-hive-v2 package to have same
>> coverage
>> > as tests in sentry-tests-hive package.
>> >
>> > Does any one wants to take ownership of any of these tasks?
>> >
>> >
>> > -Kalyan
>> >
>> > On Tue, Jul 18, 2017 at 11:02 AM, Na Li <li...@cloudera.com> wrote:
>> >
>> > > I agree with Colm too. We can support Hive 2.1.1 only in sentry 2.0
>> > >
>> > > Lina
>> > >
>> > > On Tue, Jul 18, 2017 at 10:03 AM, Sergio Pena <
>> sergio.pena@cloudera.com>
>> > > wrote:
>> > >
>> > > > I agree with Colm too. Hive 1.1 is too old and due to the way how
>> > Sentry
>> > > > works with HMS and notifications, it's not worth the effort to keep
>> > > > supporting Hive 1.1. Let's drop it.
>> > > >
>> > > > On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh <
>> > > coheigea@apache.org>
>> > > > wrote:
>> > > >
>> > > > > I think we should drop support for Hive 1.1.0 in sentry 2.0.0.
>> Hive
>> > > 1.1.0
>> > > > > users can obtain the plugin from the Sentry 1.8.x series of
>> releases.
>> > > > >
>> > > > > Colm.
>> > > > >
>> > > > > On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar Kalvagadda <
>> > > > > kkalyan@cloudera.com> wrote:
>> > > > >
>> > > > > > Hello all,
>> > > > > >
>> > > > > > Sentry currently supports Hive1.1.0 and Hive 2.0.0 based on the
>> > > profile
>> > > > > > used. I'm currently working on sentry 2.0.0 release upstream to
>> > > > integrate
>> > > > > > with Hive 2.1.1.
>> > > > > >
>> > > > > > I'm sending this mail seeking inputs on what is the support we
>> need
>> > > in
>> > > > > > Sentry 2.0.0 release. Would that be just Hive 2.x or both Hive
>> 2.x
>> > > and
>> > > > > Hive
>> > > > > > 1.1.0?
>> > > > > >
>> > > > > > I'm not sure if we should be supporting Hive 1.1.0 in sentry
>> 2.0.0.
>> > > As
>> > > > > Hive
>> > > > > > 1.1.0 is pretty old and the notification log implementation in
>> this
>> > > > > version
>> > > > > > which Sentry-HA is dependent on has a lot of bugs which are
>> fixed
>> > in
>> > > > > later
>> > > > > > releases. Considering this, shouldn't we integrate sentry in
>> > release
>> > > > > 2.0.0
>> > > > > > with Hive 2.1.1 and say that it would no more support older
>> > versions
>> > > > of
>> > > > > > Hive?
>> > > > > >
>> > > > > >
>> > > > > > Here are some of the approaches that we could take for Hive
>> > > integration
>> > > > > for
>> > > > > > sentry 2.0.0 release.
>> > > > > >
>> > > > > > *Approach-1:* Maintain support for both Hive 1.1.0 and Hive
>> 2.1.1
>> > > > > > *Approach-2: *Maintain support for both Hive 1.1.0 and Hive
>> 2.1.1
>> > for
>> > > > now
>> > > > > > and remove it by the time sentry 2.0.0 is released.
>> > > > > > *Approach-3:* Remove support for Hive 1.1.0 now.
>> > > > > >
>> > > > > > *Downside* for Approach 1 and 2 is that we need to put in lot of
>> > > effort
>> > > > > to
>> > > > > > have them work together as Hive 2.1.1 has a lot of API's changes
>> > > which
>> > > > > need
>> > > > > > changes to a lot classes in sentry-binding package but still
>> > maintain
>> > > > > > backward compatibility.
>> > > > > > If we want to take approach-1 for any reason, it's worth putting
>> > that
>> > > > > > effort otherwise we could take approach-3.
>> > > > > >
>> > > > > >
>> > > > > > My next steps would be based on what I hear form the community.
>> > > > > >
>> > > > > > -Kalyan
>> > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Colm O hEigeartaigh
>> > > > >
>> > > > > Talend Community Coder
>> > > > > http://coders.talend.com
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Kalyan Kumar Kalvagadda <kk...@cloudera.com>.
Lina,
Apache community is currently focused on sentry-ha to be released. It has
highest priority over others. If you are contributing to that, I would say
continue doing so.
-Kalyan
On Wed, Jul 19, 2017 at 10:39 AM, Na Li <li...@cloudera.com> wrote:
> Kalyan,
>
> I can work on those items.
>
> Can you create seperate Jiras for each one and add more info.
>
> In terms of priority, is finishing sentry-HA a higher priority than these
> (for C6)?
>
> Thanks,
>
> Lina
>
> On Wed, Jul 19, 2017 at 10:02 AM, Kalyan Kumar Kalvagadda <
> kkalyan@cloudera.com> wrote:
>
> > Hello all,
> >
> > Either way we need to fix couple of things first.
> >
> > 1. Fix the test failures when hive-authz2 and datanucleus4 profiles are
> > enabled.
> > 2. Update e2e tests in sentry-tests-hive-v2 package to have same coverage
> > as tests in sentry-tests-hive package.
> >
> > Does any one wants to take ownership of any of these tasks?
> >
> >
> > -Kalyan
> >
> > On Tue, Jul 18, 2017 at 11:02 AM, Na Li <li...@cloudera.com> wrote:
> >
> > > I agree with Colm too. We can support Hive 2.1.1 only in sentry 2.0
> > >
> > > Lina
> > >
> > > On Tue, Jul 18, 2017 at 10:03 AM, Sergio Pena <
> sergio.pena@cloudera.com>
> > > wrote:
> > >
> > > > I agree with Colm too. Hive 1.1 is too old and due to the way how
> > Sentry
> > > > works with HMS and notifications, it's not worth the effort to keep
> > > > supporting Hive 1.1. Let's drop it.
> > > >
> > > > On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh <
> > > coheigea@apache.org>
> > > > wrote:
> > > >
> > > > > I think we should drop support for Hive 1.1.0 in sentry 2.0.0. Hive
> > > 1.1.0
> > > > > users can obtain the plugin from the Sentry 1.8.x series of
> releases.
> > > > >
> > > > > Colm.
> > > > >
> > > > > On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar Kalvagadda <
> > > > > kkalyan@cloudera.com> wrote:
> > > > >
> > > > > > Hello all,
> > > > > >
> > > > > > Sentry currently supports Hive1.1.0 and Hive 2.0.0 based on the
> > > profile
> > > > > > used. I'm currently working on sentry 2.0.0 release upstream to
> > > > integrate
> > > > > > with Hive 2.1.1.
> > > > > >
> > > > > > I'm sending this mail seeking inputs on what is the support we
> need
> > > in
> > > > > > Sentry 2.0.0 release. Would that be just Hive 2.x or both Hive
> 2.x
> > > and
> > > > > Hive
> > > > > > 1.1.0?
> > > > > >
> > > > > > I'm not sure if we should be supporting Hive 1.1.0 in sentry
> 2.0.0.
> > > As
> > > > > Hive
> > > > > > 1.1.0 is pretty old and the notification log implementation in
> this
> > > > > version
> > > > > > which Sentry-HA is dependent on has a lot of bugs which are fixed
> > in
> > > > > later
> > > > > > releases. Considering this, shouldn't we integrate sentry in
> > release
> > > > > 2.0.0
> > > > > > with Hive 2.1.1 and say that it would no more support older
> > versions
> > > > of
> > > > > > Hive?
> > > > > >
> > > > > >
> > > > > > Here are some of the approaches that we could take for Hive
> > > integration
> > > > > for
> > > > > > sentry 2.0.0 release.
> > > > > >
> > > > > > *Approach-1:* Maintain support for both Hive 1.1.0 and Hive 2.1.1
> > > > > > *Approach-2: *Maintain support for both Hive 1.1.0 and Hive 2.1.1
> > for
> > > > now
> > > > > > and remove it by the time sentry 2.0.0 is released.
> > > > > > *Approach-3:* Remove support for Hive 1.1.0 now.
> > > > > >
> > > > > > *Downside* for Approach 1 and 2 is that we need to put in lot of
> > > effort
> > > > > to
> > > > > > have them work together as Hive 2.1.1 has a lot of API's changes
> > > which
> > > > > need
> > > > > > changes to a lot classes in sentry-binding package but still
> > maintain
> > > > > > backward compatibility.
> > > > > > If we want to take approach-1 for any reason, it's worth putting
> > that
> > > > > > effort otherwise we could take approach-3.
> > > > > >
> > > > > >
> > > > > > My next steps would be based on what I hear form the community.
> > > > > >
> > > > > > -Kalyan
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Colm O hEigeartaigh
> > > > >
> > > > > Talend Community Coder
> > > > > http://coders.talend.com
> > > > >
> > > >
> > >
> >
>
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Na Li <li...@cloudera.com>.
Kalyan,
I can work on those items.
Can you create seperate Jiras for each one and add more info.
In terms of priority, is finishing sentry-HA a higher priority than these
(for C6)?
Thanks,
Lina
On Wed, Jul 19, 2017 at 10:02 AM, Kalyan Kumar Kalvagadda <
kkalyan@cloudera.com> wrote:
> Hello all,
>
> Either way we need to fix couple of things first.
>
> 1. Fix the test failures when hive-authz2 and datanucleus4 profiles are
> enabled.
> 2. Update e2e tests in sentry-tests-hive-v2 package to have same coverage
> as tests in sentry-tests-hive package.
>
> Does any one wants to take ownership of any of these tasks?
>
>
> -Kalyan
>
> On Tue, Jul 18, 2017 at 11:02 AM, Na Li <li...@cloudera.com> wrote:
>
> > I agree with Colm too. We can support Hive 2.1.1 only in sentry 2.0
> >
> > Lina
> >
> > On Tue, Jul 18, 2017 at 10:03 AM, Sergio Pena <se...@cloudera.com>
> > wrote:
> >
> > > I agree with Colm too. Hive 1.1 is too old and due to the way how
> Sentry
> > > works with HMS and notifications, it's not worth the effort to keep
> > > supporting Hive 1.1. Let's drop it.
> > >
> > > On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh <
> > coheigea@apache.org>
> > > wrote:
> > >
> > > > I think we should drop support for Hive 1.1.0 in sentry 2.0.0. Hive
> > 1.1.0
> > > > users can obtain the plugin from the Sentry 1.8.x series of releases.
> > > >
> > > > Colm.
> > > >
> > > > On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar Kalvagadda <
> > > > kkalyan@cloudera.com> wrote:
> > > >
> > > > > Hello all,
> > > > >
> > > > > Sentry currently supports Hive1.1.0 and Hive 2.0.0 based on the
> > profile
> > > > > used. I'm currently working on sentry 2.0.0 release upstream to
> > > integrate
> > > > > with Hive 2.1.1.
> > > > >
> > > > > I'm sending this mail seeking inputs on what is the support we need
> > in
> > > > > Sentry 2.0.0 release. Would that be just Hive 2.x or both Hive 2.x
> > and
> > > > Hive
> > > > > 1.1.0?
> > > > >
> > > > > I'm not sure if we should be supporting Hive 1.1.0 in sentry 2.0.0.
> > As
> > > > Hive
> > > > > 1.1.0 is pretty old and the notification log implementation in this
> > > > version
> > > > > which Sentry-HA is dependent on has a lot of bugs which are fixed
> in
> > > > later
> > > > > releases. Considering this, shouldn't we integrate sentry in
> release
> > > > 2.0.0
> > > > > with Hive 2.1.1 and say that it would no more support older
> versions
> > > of
> > > > > Hive?
> > > > >
> > > > >
> > > > > Here are some of the approaches that we could take for Hive
> > integration
> > > > for
> > > > > sentry 2.0.0 release.
> > > > >
> > > > > *Approach-1:* Maintain support for both Hive 1.1.0 and Hive 2.1.1
> > > > > *Approach-2: *Maintain support for both Hive 1.1.0 and Hive 2.1.1
> for
> > > now
> > > > > and remove it by the time sentry 2.0.0 is released.
> > > > > *Approach-3:* Remove support for Hive 1.1.0 now.
> > > > >
> > > > > *Downside* for Approach 1 and 2 is that we need to put in lot of
> > effort
> > > > to
> > > > > have them work together as Hive 2.1.1 has a lot of API's changes
> > which
> > > > need
> > > > > changes to a lot classes in sentry-binding package but still
> maintain
> > > > > backward compatibility.
> > > > > If we want to take approach-1 for any reason, it's worth putting
> that
> > > > > effort otherwise we could take approach-3.
> > > > >
> > > > >
> > > > > My next steps would be based on what I hear form the community.
> > > > >
> > > > > -Kalyan
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Colm O hEigeartaigh
> > > >
> > > > Talend Community Coder
> > > > http://coders.talend.com
> > > >
> > >
> >
>
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Kalyan Kumar Kalvagadda <kk...@cloudera.com>.
Hello all,
Either way we need to fix couple of things first.
1. Fix the test failures when hive-authz2 and datanucleus4 profiles are
enabled.
2. Update e2e tests in sentry-tests-hive-v2 package to have same coverage
as tests in sentry-tests-hive package.
Does any one wants to take ownership of any of these tasks?
-Kalyan
On Tue, Jul 18, 2017 at 11:02 AM, Na Li <li...@cloudera.com> wrote:
> I agree with Colm too. We can support Hive 2.1.1 only in sentry 2.0
>
> Lina
>
> On Tue, Jul 18, 2017 at 10:03 AM, Sergio Pena <se...@cloudera.com>
> wrote:
>
> > I agree with Colm too. Hive 1.1 is too old and due to the way how Sentry
> > works with HMS and notifications, it's not worth the effort to keep
> > supporting Hive 1.1. Let's drop it.
> >
> > On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh <
> coheigea@apache.org>
> > wrote:
> >
> > > I think we should drop support for Hive 1.1.0 in sentry 2.0.0. Hive
> 1.1.0
> > > users can obtain the plugin from the Sentry 1.8.x series of releases.
> > >
> > > Colm.
> > >
> > > On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar Kalvagadda <
> > > kkalyan@cloudera.com> wrote:
> > >
> > > > Hello all,
> > > >
> > > > Sentry currently supports Hive1.1.0 and Hive 2.0.0 based on the
> profile
> > > > used. I'm currently working on sentry 2.0.0 release upstream to
> > integrate
> > > > with Hive 2.1.1.
> > > >
> > > > I'm sending this mail seeking inputs on what is the support we need
> in
> > > > Sentry 2.0.0 release. Would that be just Hive 2.x or both Hive 2.x
> and
> > > Hive
> > > > 1.1.0?
> > > >
> > > > I'm not sure if we should be supporting Hive 1.1.0 in sentry 2.0.0.
> As
> > > Hive
> > > > 1.1.0 is pretty old and the notification log implementation in this
> > > version
> > > > which Sentry-HA is dependent on has a lot of bugs which are fixed in
> > > later
> > > > releases. Considering this, shouldn't we integrate sentry in release
> > > 2.0.0
> > > > with Hive 2.1.1 and say that it would no more support older versions
> > of
> > > > Hive?
> > > >
> > > >
> > > > Here are some of the approaches that we could take for Hive
> integration
> > > for
> > > > sentry 2.0.0 release.
> > > >
> > > > *Approach-1:* Maintain support for both Hive 1.1.0 and Hive 2.1.1
> > > > *Approach-2: *Maintain support for both Hive 1.1.0 and Hive 2.1.1 for
> > now
> > > > and remove it by the time sentry 2.0.0 is released.
> > > > *Approach-3:* Remove support for Hive 1.1.0 now.
> > > >
> > > > *Downside* for Approach 1 and 2 is that we need to put in lot of
> effort
> > > to
> > > > have them work together as Hive 2.1.1 has a lot of API's changes
> which
> > > need
> > > > changes to a lot classes in sentry-binding package but still maintain
> > > > backward compatibility.
> > > > If we want to take approach-1 for any reason, it's worth putting that
> > > > effort otherwise we could take approach-3.
> > > >
> > > >
> > > > My next steps would be based on what I hear form the community.
> > > >
> > > > -Kalyan
> > > >
> > >
> > >
> > >
> > > --
> > > Colm O hEigeartaigh
> > >
> > > Talend Community Coder
> > > http://coders.talend.com
> > >
> >
>
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Na Li <li...@cloudera.com>.
I agree with Colm too. We can support Hive 2.1.1 only in sentry 2.0
Lina
On Tue, Jul 18, 2017 at 10:03 AM, Sergio Pena <se...@cloudera.com>
wrote:
> I agree with Colm too. Hive 1.1 is too old and due to the way how Sentry
> works with HMS and notifications, it's not worth the effort to keep
> supporting Hive 1.1. Let's drop it.
>
> On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh <co...@apache.org>
> wrote:
>
> > I think we should drop support for Hive 1.1.0 in sentry 2.0.0. Hive 1.1.0
> > users can obtain the plugin from the Sentry 1.8.x series of releases.
> >
> > Colm.
> >
> > On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar Kalvagadda <
> > kkalyan@cloudera.com> wrote:
> >
> > > Hello all,
> > >
> > > Sentry currently supports Hive1.1.0 and Hive 2.0.0 based on the profile
> > > used. I'm currently working on sentry 2.0.0 release upstream to
> integrate
> > > with Hive 2.1.1.
> > >
> > > I'm sending this mail seeking inputs on what is the support we need in
> > > Sentry 2.0.0 release. Would that be just Hive 2.x or both Hive 2.x and
> > Hive
> > > 1.1.0?
> > >
> > > I'm not sure if we should be supporting Hive 1.1.0 in sentry 2.0.0. As
> > Hive
> > > 1.1.0 is pretty old and the notification log implementation in this
> > version
> > > which Sentry-HA is dependent on has a lot of bugs which are fixed in
> > later
> > > releases. Considering this, shouldn't we integrate sentry in release
> > 2.0.0
> > > with Hive 2.1.1 and say that it would no more support older versions
> of
> > > Hive?
> > >
> > >
> > > Here are some of the approaches that we could take for Hive integration
> > for
> > > sentry 2.0.0 release.
> > >
> > > *Approach-1:* Maintain support for both Hive 1.1.0 and Hive 2.1.1
> > > *Approach-2: *Maintain support for both Hive 1.1.0 and Hive 2.1.1 for
> now
> > > and remove it by the time sentry 2.0.0 is released.
> > > *Approach-3:* Remove support for Hive 1.1.0 now.
> > >
> > > *Downside* for Approach 1 and 2 is that we need to put in lot of effort
> > to
> > > have them work together as Hive 2.1.1 has a lot of API's changes which
> > need
> > > changes to a lot classes in sentry-binding package but still maintain
> > > backward compatibility.
> > > If we want to take approach-1 for any reason, it's worth putting that
> > > effort otherwise we could take approach-3.
> > >
> > >
> > > My next steps would be based on what I hear form the community.
> > >
> > > -Kalyan
> > >
> >
> >
> >
> > --
> > Colm O hEigeartaigh
> >
> > Talend Community Coder
> > http://coders.talend.com
> >
>
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Sergio Pena <se...@cloudera.com>.
I agree with Colm too. Hive 1.1 is too old and due to the way how Sentry
works with HMS and notifications, it's not worth the effort to keep
supporting Hive 1.1. Let's drop it.
On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh <co...@apache.org>
wrote:
> I think we should drop support for Hive 1.1.0 in sentry 2.0.0. Hive 1.1.0
> users can obtain the plugin from the Sentry 1.8.x series of releases.
>
> Colm.
>
> On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar Kalvagadda <
> kkalyan@cloudera.com> wrote:
>
> > Hello all,
> >
> > Sentry currently supports Hive1.1.0 and Hive 2.0.0 based on the profile
> > used. I'm currently working on sentry 2.0.0 release upstream to integrate
> > with Hive 2.1.1.
> >
> > I'm sending this mail seeking inputs on what is the support we need in
> > Sentry 2.0.0 release. Would that be just Hive 2.x or both Hive 2.x and
> Hive
> > 1.1.0?
> >
> > I'm not sure if we should be supporting Hive 1.1.0 in sentry 2.0.0. As
> Hive
> > 1.1.0 is pretty old and the notification log implementation in this
> version
> > which Sentry-HA is dependent on has a lot of bugs which are fixed in
> later
> > releases. Considering this, shouldn't we integrate sentry in release
> 2.0.0
> > with Hive 2.1.1 and say that it would no more support older versions of
> > Hive?
> >
> >
> > Here are some of the approaches that we could take for Hive integration
> for
> > sentry 2.0.0 release.
> >
> > *Approach-1:* Maintain support for both Hive 1.1.0 and Hive 2.1.1
> > *Approach-2: *Maintain support for both Hive 1.1.0 and Hive 2.1.1 for now
> > and remove it by the time sentry 2.0.0 is released.
> > *Approach-3:* Remove support for Hive 1.1.0 now.
> >
> > *Downside* for Approach 1 and 2 is that we need to put in lot of effort
> to
> > have them work together as Hive 2.1.1 has a lot of API's changes which
> need
> > changes to a lot classes in sentry-binding package but still maintain
> > backward compatibility.
> > If we want to take approach-1 for any reason, it's worth putting that
> > effort otherwise we could take approach-3.
> >
> >
> > My next steps would be based on what I hear form the community.
> >
> > -Kalyan
> >
>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: Sentry 2.0.0 depends on multiple versions of Hive
Posted by Colm O hEigeartaigh <co...@apache.org>.
I think we should drop support for Hive 1.1.0 in sentry 2.0.0. Hive 1.1.0
users can obtain the plugin from the Sentry 1.8.x series of releases.
Colm.
On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar Kalvagadda <
kkalyan@cloudera.com> wrote:
> Hello all,
>
> Sentry currently supports Hive1.1.0 and Hive 2.0.0 based on the profile
> used. I'm currently working on sentry 2.0.0 release upstream to integrate
> with Hive 2.1.1.
>
> I'm sending this mail seeking inputs on what is the support we need in
> Sentry 2.0.0 release. Would that be just Hive 2.x or both Hive 2.x and Hive
> 1.1.0?
>
> I'm not sure if we should be supporting Hive 1.1.0 in sentry 2.0.0. As Hive
> 1.1.0 is pretty old and the notification log implementation in this version
> which Sentry-HA is dependent on has a lot of bugs which are fixed in later
> releases. Considering this, shouldn't we integrate sentry in release 2.0.0
> with Hive 2.1.1 and say that it would no more support older versions of
> Hive?
>
>
> Here are some of the approaches that we could take for Hive integration for
> sentry 2.0.0 release.
>
> *Approach-1:* Maintain support for both Hive 1.1.0 and Hive 2.1.1
> *Approach-2: *Maintain support for both Hive 1.1.0 and Hive 2.1.1 for now
> and remove it by the time sentry 2.0.0 is released.
> *Approach-3:* Remove support for Hive 1.1.0 now.
>
> *Downside* for Approach 1 and 2 is that we need to put in lot of effort to
> have them work together as Hive 2.1.1 has a lot of API's changes which need
> changes to a lot classes in sentry-binding package but still maintain
> backward compatibility.
> If we want to take approach-1 for any reason, it's worth putting that
> effort otherwise we could take approach-3.
>
>
> My next steps would be based on what I hear form the community.
>
> -Kalyan
>
--
Colm O hEigeartaigh
Talend Community Coder
http://coders.talend.com