You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by Phil Zampino <pz...@apache.org> on 2018/07/13 15:35:52 UTC
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
During some testing of the proposed 1.1.0 code, I've discovered some NPEs
in filters (e.g., AclsAuthorizationFilter, HadoopGroupProviderFilter),
which are concerning.
I've committed a change to address the AclsAuthorizationFilter, but seeing
similar behavior for the HadoopGroupProviderFilter has increased my concern
that there may be a more fundamental problem.
In both cases, it seems that the filters are being invoked prior to (or
during) their respective init() methods have been invoked. Thus, members
which should be initialized in the init() method are not yet initialized.
This can be consistently reproduced, though it is a bit of a pain:
- Install Knox (‘ant install-test-home’, or just unzip knox-1.1.0.zip)
- Start the gateway
- Access the Admin UI
Note that the latest 1.1.0 source has a *fix* for the
AclsAuthorizationFilter NPE, but master does not yet have this change. This
is important because that change effectively hides the issue.
I think we should determine what's happening with this before
producing/testing a release candidate.
On Sat, Feb 24, 2018 at 12:57 PM larry mccay <lm...@apache.org> wrote:
> All -
>
> Sorry for the delay on this topic.
>
> We are going to start of this planning thread with ~85 Unresolved JIRAs in
> either 1.1.0 or 0.15.0 fixVersion.
>
> project = KNOX AND resolution = Unresolved AND fixVersion in (1.1.0,
> 0.15.0) ORDER BY priority DESC, updated DESC
>
> I will spend some time migrating all 0.15.0 to 1.1.0 to begin with and then
> we will need to go through and see what is already taken care of or can
> wait for a 1.2.0 or later.
>
> I also have a couple KIPs in mind to target larger features/themes for this
> release.
>
> Off the top of my head:
>
> * I think we need to address some cloud specific usecases and plan to
> provide a KIP for that. Hybrid cloud/federated knox instances, Azure AD
> integration, ID mapping from Hadoop user to IAM users/roles, etc. Perhaps
> some CASB-like features if they make sense.
>
> * I also think we need one for articulating a reasonable flow for Logout in
> KnoxSSO. There are a lot of little nuances to logout across multiple apps
> and between different IDPs. This will require some discussion.
>
> * Another thing that has been tugging at my interest has been the fact that
> we may be able provide some common libraries to help ecosystem applications
> uptake the trusted proxy pattern and KnoxSSO.
>
> Anyway, these are my initial thoughts, please feel free to raise additional
> ideas/themes for KIPs, etc.
>
> I was thinking that we could try and target an end of March or Mid April
> 1.1.0 release.
>
> Thoughts?
>
> --larry
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by Philip Zampino <pz...@gmail.com>.
The fix for the filter NPEs has been committed to 1.1.0 and master.
I don't believe the other exception is related.
On Thu, Jul 19, 2018 at 4:15 PM larry mccay <lm...@apache.org> wrote:
> @Phil, I see a couple commits land that seem to address the NPE.
> Is that correct?
>
> I have also seen an IllegalStateException during redirect from Admin UI to
> KnoxSSO.
> Has anyone seen this and/or is working on it - is it related to the NPE?
> I don't think it is since I see it more frequently and not always with the
> NPEs.
>
> I'd like to get a new RC cut by end of the week, if possible.
>
> On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <lm...@apache.org> wrote:
>
> > Agreed, Phil.
> > I have cut an RC but we need to address this first. I'll hold off on
> > announcing it.
> >
> > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <pz...@apache.org> wrote:
> >
> >> During some testing of the proposed 1.1.0 code, I've discovered some
> NPEs
> >> in filters (e.g., AclsAuthorizationFilter, HadoopGroupProviderFilter),
> >> which are concerning.
> >>
> >> I've committed a change to address the AclsAuthorizationFilter, but
> seeing
> >> similar behavior for the HadoopGroupProviderFilter has increased my
> >> concern
> >> that there may be a more fundamental problem.
> >> In both cases, it seems that the filters are being invoked prior to (or
> >> during) their respective init() methods have been invoked. Thus, members
> >> which should be initialized in the init() method are not yet
> initialized.
> >>
> >> This can be consistently reproduced, though it is a bit of a pain:
> >>
> >> - Install Knox (‘ant install-test-home’, or just unzip
> knox-1.1.0.zip)
> >> - Start the gateway
> >> - Access the Admin UI
> >>
> >>
> >> Note that the latest 1.1.0 source has a *fix* for the
> >> AclsAuthorizationFilter NPE, but master does not yet have this change.
> >> This
> >> is important because that change effectively hides the issue.
> >>
> >> I think we should determine what's happening with this before
> >> producing/testing a release candidate.
> >>
> >>
> >>
> >>
> >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <lm...@apache.org> wrote:
> >>
> >> > All -
> >> >
> >> > Sorry for the delay on this topic.
> >> >
> >> > We are going to start of this planning thread with ~85 Unresolved
> JIRAs
> >> in
> >> > either 1.1.0 or 0.15.0 fixVersion.
> >> >
> >> > project = KNOX AND resolution = Unresolved AND fixVersion in (1.1.0,
> >> > 0.15.0) ORDER BY priority DESC, updated DESC
> >> >
> >> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin with and
> >> then
> >> > we will need to go through and see what is already taken care of or
> can
> >> > wait for a 1.2.0 or later.
> >> >
> >> > I also have a couple KIPs in mind to target larger features/themes for
> >> this
> >> > release.
> >> >
> >> > Off the top of my head:
> >> >
> >> > * I think we need to address some cloud specific usecases and plan to
> >> > provide a KIP for that. Hybrid cloud/federated knox instances, Azure
> AD
> >> > integration, ID mapping from Hadoop user to IAM users/roles, etc.
> >> Perhaps
> >> > some CASB-like features if they make sense.
> >> >
> >> > * I also think we need one for articulating a reasonable flow for
> >> Logout in
> >> > KnoxSSO. There are a lot of little nuances to logout across multiple
> >> apps
> >> > and between different IDPs. This will require some discussion.
> >> >
> >> > * Another thing that has been tugging at my interest has been the fact
> >> that
> >> > we may be able provide some common libraries to help ecosystem
> >> applications
> >> > uptake the trusted proxy pattern and KnoxSSO.
> >> >
> >> > Anyway, these are my initial thoughts, please feel free to raise
> >> additional
> >> > ideas/themes for KIPs, etc.
> >> >
> >> > I was thinking that we could try and target an end of March or Mid
> April
> >> > 1.1.0 release.
> >> >
> >> > Thoughts?
> >> >
> >> > --larry
> >> >
> >>
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by Sandeep Moré <mo...@gmail.com>.
Great, thanks Larry !
On Fri, Jul 20, 2018 at 10:53 AM larry mccay <lm...@apache.org> wrote:
> I just pushed KNOX-1394 for changing the default whitelist to align with
> the DEMO LDAP config in gateway-site.xml.
> This will realign with the dev and demo environment assumptions that Knox
> has always had OOTB.
>
> I will cut an RC shortly.
>
>
> On Fri, Jul 20, 2018 at 12:40 AM, larry mccay <lm...@apache.org> wrote:
>
> > Yes, along with ERROR level logging when falling back to specific host/ip
> > or localhost variants.
> >
> > On Thu, Jul 19, 2018, 11:48 PM Philip Zampino <pz...@gmail.com>
> wrote:
> >
> >> Sure, but I'm also going to implement the following for the default
> >> behavior (when the DEFAULT value is specified for the whitelist
> property):
> >>
> >>
> >> 1. Attempt to determine the domain from the X-Forwarded-Host header
> >> value
> >> 2. If domain could not be determined, attempt to determine the domain
> >> from the InetAddress.getLocalHost().getCanonicalHostName() value
> >> 3. If domain could not be determined, attempt to determine the domain
> >> from the requested host name
> >> 4. If the domain could be determined from any of these sources, then
> >> the
> >> default whitelist will be based on that domain
> >> 5. If the domain cannot be determined
> >> a. If the requested host name is NOT a variant of localhost, then the
> >> whitelist will be restricted to that specific host name
> >> b. Otherwise, the localhost whitelist will be the default
> >>
> >> Does that sound like a good strategy?
> >>
> >>
> >> On Thu, Jul 19, 2018 at 8:15 PM larry mccay <lm...@apache.org> wrote:
> >>
> >> > Yes, does that sound appropriate to you?
> >> > If the LDAP config in gateway-site.xml gets updated to product the
> >> > whitelist would be in the same place.
> >> >
> >> > On Thu, Jul 19, 2018 at 6:26 PM, Philip Zampino <pz...@gmail.com>
> >> > wrote:
> >> >
> >> > > I am working on a solution for the ip address being treated as a
> >> hostname
> >> > > issue.
> >> > >
> >> > > On Thu, Jul 19, 2018 at 6:24 PM larry mccay <lm...@apache.org>
> >> wrote:
> >> > >
> >> > > > Playing around a bit more, I noticed that there is
> nondeterministic
> >> > > > behavior of the default whitelist feature.
> >> > > > Especially on macs - since the hostname ends up being any number
> of
> >> > > things.
> >> > > > I have noticed the following things when there is no explicit
> >> whitelist
> >> > > > configured:
> >> > > >
> >> > > > * ip address based whitelist being derived which is treated like a
> >> > domain
> >> > > > * localhost is not supported out of the box unless the logic is
> >> unable
> >> > to
> >> > > > determine a domain
> >> > > > * sometimes my host is HW14155.home and sometimes it is
> >> new-host-5.home
> >> > > for
> >> > > > some reason
> >> > > >
> >> > > > Given that all of our samples and docs assume localhost and OOTB
> we
> >> are
> >> > > > setup for DEMO LDAP server, I propose that we at least add
> localhost
> >> > back
> >> > > > for OOTB.
> >> > > > Ip address handling may be worth tackling as well but only if we
> >> can do
> >> > > it
> >> > > > in a day.
> >> > > >
> >> > > > Thoughts?
> >> > > >
> >> > > >
> >> > > > On Thu, Jul 19, 2018 at 6:12 PM, larry mccay <lm...@apache.org>
> >> > wrote:
> >> > > >
> >> > > > > Awesome - just checked it out and I will kick off a new build
> >> > shortly!
> >> > > > >
> >> > > > > On Thu, Jul 19, 2018 at 6:01 PM, Sandeep Moré <
> >> moresandeep@gmail.com
> >> > >
> >> > > > > wrote:
> >> > > > >
> >> > > > >> Hello Larry,
> >> > > > >>
> >> > > > >> I committed the fix to master and v1.1.0, it is under the JIRA
> >> > > KNOX-1391
> >> > > > >> <https://issues.apache.org/jira/browse/KNOX-1391>.
> >> > > > >> we should be good to to cut the RC, provided there are no more
> >> > issues
> >> > > !
> >> > > > >>
> >> > > > >> Thanks !
> >> > > > >> Sandeep
> >> > > > >>
> >> > > > >> On Thu, Jul 19, 2018 at 4:25 PM larry mccay <
> >> larry.mccay@gmail.com>
> >> > > > >> wrote:
> >> > > > >>
> >> > > > >> > Awesome, @sandeep!
> >> > > > >> > I'll keep an eye out.
> >> > > > >> >
> >> > > > >> > Once that lands, you can bump this thread and I'll cut the
> RC.
> >> > > > >> > Obviously, we will need it in both master and v1.1.0
> branches.
> >> > > > >> >
> >> > > > >> > On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <
> >> > > moresandeep@gmail.com>
> >> > > > >> > wrote:
> >> > > > >> >
> >> > > > >> > > Hello Larry,
> >> > > > >> > >
> >> > > > >> > > Yes, I have seen those exceptions, they seem to be
> happening
> >> > > fairly
> >> > > > >> > > consistently and only for KnoxSSO redirects when trying to
> >> > access
> >> > > > >> admin
> >> > > > >> > UI,
> >> > > > >> > > I am taking a look at them as we speak, will open up a JIRA
> >> for
> >> > it
> >> > > > as
> >> > > > >> > well.
> >> > > > >> > > It would be good if we can get it in, I will try to get the
> >> fix
> >> > > out
> >> > > > as
> >> > > > >> > soon
> >> > > > >> > > as I can.
> >> > > > >> > >
> >> > > > >> > > Best,
> >> > > > >> > > Sandeep
> >> > > > >> > >
> >> > > > >> > > On Thu, Jul 19, 2018 at 4:15 PM larry mccay <
> >> lmccay@apache.org>
> >> > > > >> wrote:
> >> > > > >> > >
> >> > > > >> > > > @Phil, I see a couple commits land that seem to address
> the
> >> > NPE.
> >> > > > >> > > > Is that correct?
> >> > > > >> > > >
> >> > > > >> > > > I have also seen an IllegalStateException during redirect
> >> from
> >> > > > >> Admin UI
> >> > > > >> > > to
> >> > > > >> > > > KnoxSSO.
> >> > > > >> > > > Has anyone seen this and/or is working on it - is it
> >> related
> >> > to
> >> > > > the
> >> > > > >> > NPE?
> >> > > > >> > > > I don't think it is since I see it more frequently and
> not
> >> > > always
> >> > > > >> with
> >> > > > >> > > the
> >> > > > >> > > > NPEs.
> >> > > > >> > > >
> >> > > > >> > > > I'd like to get a new RC cut by end of the week, if
> >> possible.
> >> > > > >> > > >
> >> > > > >> > > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <
> >> > lmccay@apache.org
> >> > > >
> >> > > > >> > wrote:
> >> > > > >> > > >
> >> > > > >> > > > > Agreed, Phil.
> >> > > > >> > > > > I have cut an RC but we need to address this first.
> I'll
> >> > hold
> >> > > > >> off on
> >> > > > >> > > > > announcing it.
> >> > > > >> > > > >
> >> > > > >> > > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <
> >> > > > pzampino@apache.org>
> >> > > > >> > > wrote:
> >> > > > >> > > > >
> >> > > > >> > > > >> During some testing of the proposed 1.1.0 code, I've
> >> > > discovered
> >> > > > >> some
> >> > > > >> > > > NPEs
> >> > > > >> > > > >> in filters (e.g., AclsAuthorizationFilter,
> >> > > > >> > HadoopGroupProviderFilter),
> >> > > > >> > > > >> which are concerning.
> >> > > > >> > > > >>
> >> > > > >> > > > >> I've committed a change to address the
> >> > > AclsAuthorizationFilter,
> >> > > > >> but
> >> > > > >> > > > seeing
> >> > > > >> > > > >> similar behavior for the HadoopGroupProviderFilter has
> >> > > > increased
> >> > > > >> my
> >> > > > >> > > > >> concern
> >> > > > >> > > > >> that there may be a more fundamental problem.
> >> > > > >> > > > >> In both cases, it seems that the filters are being
> >> invoked
> >> > > > prior
> >> > > > >> to
> >> > > > >> > > (or
> >> > > > >> > > > >> during) their respective init() methods have been
> >> invoked.
> >> > > > Thus,
> >> > > > >> > > members
> >> > > > >> > > > >> which should be initialized in the init() method are
> not
> >> > yet
> >> > > > >> > > > initialized.
> >> > > > >> > > > >>
> >> > > > >> > > > >> This can be consistently reproduced, though it is a
> bit
> >> of
> >> > a
> >> > > > >> pain:
> >> > > > >> > > > >>
> >> > > > >> > > > >> - Install Knox (‘ant install-test-home’, or just
> >> unzip
> >> > > > >> > > > knox-1.1.0.zip)
> >> > > > >> > > > >> - Start the gateway
> >> > > > >> > > > >> - Access the Admin UI
> >> > > > >> > > > >>
> >> > > > >> > > > >>
> >> > > > >> > > > >> Note that the latest 1.1.0 source has a *fix* for the
> >> > > > >> > > > >> AclsAuthorizationFilter NPE, but master does not yet
> >> have
> >> > > this
> >> > > > >> > change.
> >> > > > >> > > > >> This
> >> > > > >> > > > >> is important because that change effectively hides the
> >> > issue.
> >> > > > >> > > > >>
> >> > > > >> > > > >> I think we should determine what's happening with this
> >> > before
> >> > > > >> > > > >> producing/testing a release candidate.
> >> > > > >> > > > >>
> >> > > > >> > > > >>
> >> > > > >> > > > >>
> >> > > > >> > > > >>
> >> > > > >> > > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <
> >> > > > lmccay@apache.org>
> >> > > > >> > > wrote:
> >> > > > >> > > > >>
> >> > > > >> > > > >> > All -
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > Sorry for the delay on this topic.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > We are going to start of this planning thread with
> ~85
> >> > > > >> Unresolved
> >> > > > >> > > > JIRAs
> >> > > > >> > > > >> in
> >> > > > >> > > > >> > either 1.1.0 or 0.15.0 fixVersion.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > project = KNOX AND resolution = Unresolved AND
> >> fixVersion
> >> > > in
> >> > > > >> > (1.1.0,
> >> > > > >> > > > >> > 0.15.0) ORDER BY priority DESC, updated DESC
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > I will spend some time migrating all 0.15.0 to 1.1.0
> >> to
> >> > > begin
> >> > > > >> with
> >> > > > >> > > and
> >> > > > >> > > > >> then
> >> > > > >> > > > >> > we will need to go through and see what is already
> >> taken
> >> > > care
> >> > > > >> of
> >> > > > >> > or
> >> > > > >> > > > can
> >> > > > >> > > > >> > wait for a 1.2.0 or later.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > I also have a couple KIPs in mind to target larger
> >> > > > >> features/themes
> >> > > > >> > > for
> >> > > > >> > > > >> this
> >> > > > >> > > > >> > release.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > Off the top of my head:
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > * I think we need to address some cloud specific
> >> usecases
> >> > > and
> >> > > > >> plan
> >> > > > >> > > to
> >> > > > >> > > > >> > provide a KIP for that. Hybrid cloud/federated knox
> >> > > > instances,
> >> > > > >> > Azure
> >> > > > >> > > > AD
> >> > > > >> > > > >> > integration, ID mapping from Hadoop user to IAM
> >> > > users/roles,
> >> > > > >> etc.
> >> > > > >> > > > >> Perhaps
> >> > > > >> > > > >> > some CASB-like features if they make sense.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > * I also think we need one for articulating a
> >> reasonable
> >> > > flow
> >> > > > >> for
> >> > > > >> > > > >> Logout in
> >> > > > >> > > > >> > KnoxSSO. There are a lot of little nuances to logout
> >> > across
> >> > > > >> > multiple
> >> > > > >> > > > >> apps
> >> > > > >> > > > >> > and between different IDPs. This will require some
> >> > > > discussion.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > * Another thing that has been tugging at my interest
> >> has
> >> > > been
> >> > > > >> the
> >> > > > >> > > fact
> >> > > > >> > > > >> that
> >> > > > >> > > > >> > we may be able provide some common libraries to help
> >> > > > ecosystem
> >> > > > >> > > > >> applications
> >> > > > >> > > > >> > uptake the trusted proxy pattern and KnoxSSO.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > Anyway, these are my initial thoughts, please feel
> >> free
> >> > to
> >> > > > >> raise
> >> > > > >> > > > >> additional
> >> > > > >> > > > >> > ideas/themes for KIPs, etc.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > I was thinking that we could try and target an end
> of
> >> > March
> >> > > > or
> >> > > > >> Mid
> >> > > > >> > > > April
> >> > > > >> > > > >> > 1.1.0 release.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > Thoughts?
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > --larry
> >> > > > >> > > > >> >
> >> > > > >> > > > >>
> >> > > > >> > > > >
> >> > > > >> > > >
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by larry mccay <lm...@apache.org>.
I just pushed KNOX-1394 for changing the default whitelist to align with
the DEMO LDAP config in gateway-site.xml.
This will realign with the dev and demo environment assumptions that Knox
has always had OOTB.
I will cut an RC shortly.
On Fri, Jul 20, 2018 at 12:40 AM, larry mccay <lm...@apache.org> wrote:
> Yes, along with ERROR level logging when falling back to specific host/ip
> or localhost variants.
>
> On Thu, Jul 19, 2018, 11:48 PM Philip Zampino <pz...@gmail.com> wrote:
>
>> Sure, but I'm also going to implement the following for the default
>> behavior (when the DEFAULT value is specified for the whitelist property):
>>
>>
>> 1. Attempt to determine the domain from the X-Forwarded-Host header
>> value
>> 2. If domain could not be determined, attempt to determine the domain
>> from the InetAddress.getLocalHost().getCanonicalHostName() value
>> 3. If domain could not be determined, attempt to determine the domain
>> from the requested host name
>> 4. If the domain could be determined from any of these sources, then
>> the
>> default whitelist will be based on that domain
>> 5. If the domain cannot be determined
>> a. If the requested host name is NOT a variant of localhost, then the
>> whitelist will be restricted to that specific host name
>> b. Otherwise, the localhost whitelist will be the default
>>
>> Does that sound like a good strategy?
>>
>>
>> On Thu, Jul 19, 2018 at 8:15 PM larry mccay <lm...@apache.org> wrote:
>>
>> > Yes, does that sound appropriate to you?
>> > If the LDAP config in gateway-site.xml gets updated to product the
>> > whitelist would be in the same place.
>> >
>> > On Thu, Jul 19, 2018 at 6:26 PM, Philip Zampino <pz...@gmail.com>
>> > wrote:
>> >
>> > > I am working on a solution for the ip address being treated as a
>> hostname
>> > > issue.
>> > >
>> > > On Thu, Jul 19, 2018 at 6:24 PM larry mccay <lm...@apache.org>
>> wrote:
>> > >
>> > > > Playing around a bit more, I noticed that there is nondeterministic
>> > > > behavior of the default whitelist feature.
>> > > > Especially on macs - since the hostname ends up being any number of
>> > > things.
>> > > > I have noticed the following things when there is no explicit
>> whitelist
>> > > > configured:
>> > > >
>> > > > * ip address based whitelist being derived which is treated like a
>> > domain
>> > > > * localhost is not supported out of the box unless the logic is
>> unable
>> > to
>> > > > determine a domain
>> > > > * sometimes my host is HW14155.home and sometimes it is
>> new-host-5.home
>> > > for
>> > > > some reason
>> > > >
>> > > > Given that all of our samples and docs assume localhost and OOTB we
>> are
>> > > > setup for DEMO LDAP server, I propose that we at least add localhost
>> > back
>> > > > for OOTB.
>> > > > Ip address handling may be worth tackling as well but only if we
>> can do
>> > > it
>> > > > in a day.
>> > > >
>> > > > Thoughts?
>> > > >
>> > > >
>> > > > On Thu, Jul 19, 2018 at 6:12 PM, larry mccay <lm...@apache.org>
>> > wrote:
>> > > >
>> > > > > Awesome - just checked it out and I will kick off a new build
>> > shortly!
>> > > > >
>> > > > > On Thu, Jul 19, 2018 at 6:01 PM, Sandeep Moré <
>> moresandeep@gmail.com
>> > >
>> > > > > wrote:
>> > > > >
>> > > > >> Hello Larry,
>> > > > >>
>> > > > >> I committed the fix to master and v1.1.0, it is under the JIRA
>> > > KNOX-1391
>> > > > >> <https://issues.apache.org/jira/browse/KNOX-1391>.
>> > > > >> we should be good to to cut the RC, provided there are no more
>> > issues
>> > > !
>> > > > >>
>> > > > >> Thanks !
>> > > > >> Sandeep
>> > > > >>
>> > > > >> On Thu, Jul 19, 2018 at 4:25 PM larry mccay <
>> larry.mccay@gmail.com>
>> > > > >> wrote:
>> > > > >>
>> > > > >> > Awesome, @sandeep!
>> > > > >> > I'll keep an eye out.
>> > > > >> >
>> > > > >> > Once that lands, you can bump this thread and I'll cut the RC.
>> > > > >> > Obviously, we will need it in both master and v1.1.0 branches.
>> > > > >> >
>> > > > >> > On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <
>> > > moresandeep@gmail.com>
>> > > > >> > wrote:
>> > > > >> >
>> > > > >> > > Hello Larry,
>> > > > >> > >
>> > > > >> > > Yes, I have seen those exceptions, they seem to be happening
>> > > fairly
>> > > > >> > > consistently and only for KnoxSSO redirects when trying to
>> > access
>> > > > >> admin
>> > > > >> > UI,
>> > > > >> > > I am taking a look at them as we speak, will open up a JIRA
>> for
>> > it
>> > > > as
>> > > > >> > well.
>> > > > >> > > It would be good if we can get it in, I will try to get the
>> fix
>> > > out
>> > > > as
>> > > > >> > soon
>> > > > >> > > as I can.
>> > > > >> > >
>> > > > >> > > Best,
>> > > > >> > > Sandeep
>> > > > >> > >
>> > > > >> > > On Thu, Jul 19, 2018 at 4:15 PM larry mccay <
>> lmccay@apache.org>
>> > > > >> wrote:
>> > > > >> > >
>> > > > >> > > > @Phil, I see a couple commits land that seem to address the
>> > NPE.
>> > > > >> > > > Is that correct?
>> > > > >> > > >
>> > > > >> > > > I have also seen an IllegalStateException during redirect
>> from
>> > > > >> Admin UI
>> > > > >> > > to
>> > > > >> > > > KnoxSSO.
>> > > > >> > > > Has anyone seen this and/or is working on it - is it
>> related
>> > to
>> > > > the
>> > > > >> > NPE?
>> > > > >> > > > I don't think it is since I see it more frequently and not
>> > > always
>> > > > >> with
>> > > > >> > > the
>> > > > >> > > > NPEs.
>> > > > >> > > >
>> > > > >> > > > I'd like to get a new RC cut by end of the week, if
>> possible.
>> > > > >> > > >
>> > > > >> > > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <
>> > lmccay@apache.org
>> > > >
>> > > > >> > wrote:
>> > > > >> > > >
>> > > > >> > > > > Agreed, Phil.
>> > > > >> > > > > I have cut an RC but we need to address this first. I'll
>> > hold
>> > > > >> off on
>> > > > >> > > > > announcing it.
>> > > > >> > > > >
>> > > > >> > > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <
>> > > > pzampino@apache.org>
>> > > > >> > > wrote:
>> > > > >> > > > >
>> > > > >> > > > >> During some testing of the proposed 1.1.0 code, I've
>> > > discovered
>> > > > >> some
>> > > > >> > > > NPEs
>> > > > >> > > > >> in filters (e.g., AclsAuthorizationFilter,
>> > > > >> > HadoopGroupProviderFilter),
>> > > > >> > > > >> which are concerning.
>> > > > >> > > > >>
>> > > > >> > > > >> I've committed a change to address the
>> > > AclsAuthorizationFilter,
>> > > > >> but
>> > > > >> > > > seeing
>> > > > >> > > > >> similar behavior for the HadoopGroupProviderFilter has
>> > > > increased
>> > > > >> my
>> > > > >> > > > >> concern
>> > > > >> > > > >> that there may be a more fundamental problem.
>> > > > >> > > > >> In both cases, it seems that the filters are being
>> invoked
>> > > > prior
>> > > > >> to
>> > > > >> > > (or
>> > > > >> > > > >> during) their respective init() methods have been
>> invoked.
>> > > > Thus,
>> > > > >> > > members
>> > > > >> > > > >> which should be initialized in the init() method are not
>> > yet
>> > > > >> > > > initialized.
>> > > > >> > > > >>
>> > > > >> > > > >> This can be consistently reproduced, though it is a bit
>> of
>> > a
>> > > > >> pain:
>> > > > >> > > > >>
>> > > > >> > > > >> - Install Knox (‘ant install-test-home’, or just
>> unzip
>> > > > >> > > > knox-1.1.0.zip)
>> > > > >> > > > >> - Start the gateway
>> > > > >> > > > >> - Access the Admin UI
>> > > > >> > > > >>
>> > > > >> > > > >>
>> > > > >> > > > >> Note that the latest 1.1.0 source has a *fix* for the
>> > > > >> > > > >> AclsAuthorizationFilter NPE, but master does not yet
>> have
>> > > this
>> > > > >> > change.
>> > > > >> > > > >> This
>> > > > >> > > > >> is important because that change effectively hides the
>> > issue.
>> > > > >> > > > >>
>> > > > >> > > > >> I think we should determine what's happening with this
>> > before
>> > > > >> > > > >> producing/testing a release candidate.
>> > > > >> > > > >>
>> > > > >> > > > >>
>> > > > >> > > > >>
>> > > > >> > > > >>
>> > > > >> > > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <
>> > > > lmccay@apache.org>
>> > > > >> > > wrote:
>> > > > >> > > > >>
>> > > > >> > > > >> > All -
>> > > > >> > > > >> >
>> > > > >> > > > >> > Sorry for the delay on this topic.
>> > > > >> > > > >> >
>> > > > >> > > > >> > We are going to start of this planning thread with ~85
>> > > > >> Unresolved
>> > > > >> > > > JIRAs
>> > > > >> > > > >> in
>> > > > >> > > > >> > either 1.1.0 or 0.15.0 fixVersion.
>> > > > >> > > > >> >
>> > > > >> > > > >> > project = KNOX AND resolution = Unresolved AND
>> fixVersion
>> > > in
>> > > > >> > (1.1.0,
>> > > > >> > > > >> > 0.15.0) ORDER BY priority DESC, updated DESC
>> > > > >> > > > >> >
>> > > > >> > > > >> > I will spend some time migrating all 0.15.0 to 1.1.0
>> to
>> > > begin
>> > > > >> with
>> > > > >> > > and
>> > > > >> > > > >> then
>> > > > >> > > > >> > we will need to go through and see what is already
>> taken
>> > > care
>> > > > >> of
>> > > > >> > or
>> > > > >> > > > can
>> > > > >> > > > >> > wait for a 1.2.0 or later.
>> > > > >> > > > >> >
>> > > > >> > > > >> > I also have a couple KIPs in mind to target larger
>> > > > >> features/themes
>> > > > >> > > for
>> > > > >> > > > >> this
>> > > > >> > > > >> > release.
>> > > > >> > > > >> >
>> > > > >> > > > >> > Off the top of my head:
>> > > > >> > > > >> >
>> > > > >> > > > >> > * I think we need to address some cloud specific
>> usecases
>> > > and
>> > > > >> plan
>> > > > >> > > to
>> > > > >> > > > >> > provide a KIP for that. Hybrid cloud/federated knox
>> > > > instances,
>> > > > >> > Azure
>> > > > >> > > > AD
>> > > > >> > > > >> > integration, ID mapping from Hadoop user to IAM
>> > > users/roles,
>> > > > >> etc.
>> > > > >> > > > >> Perhaps
>> > > > >> > > > >> > some CASB-like features if they make sense.
>> > > > >> > > > >> >
>> > > > >> > > > >> > * I also think we need one for articulating a
>> reasonable
>> > > flow
>> > > > >> for
>> > > > >> > > > >> Logout in
>> > > > >> > > > >> > KnoxSSO. There are a lot of little nuances to logout
>> > across
>> > > > >> > multiple
>> > > > >> > > > >> apps
>> > > > >> > > > >> > and between different IDPs. This will require some
>> > > > discussion.
>> > > > >> > > > >> >
>> > > > >> > > > >> > * Another thing that has been tugging at my interest
>> has
>> > > been
>> > > > >> the
>> > > > >> > > fact
>> > > > >> > > > >> that
>> > > > >> > > > >> > we may be able provide some common libraries to help
>> > > > ecosystem
>> > > > >> > > > >> applications
>> > > > >> > > > >> > uptake the trusted proxy pattern and KnoxSSO.
>> > > > >> > > > >> >
>> > > > >> > > > >> > Anyway, these are my initial thoughts, please feel
>> free
>> > to
>> > > > >> raise
>> > > > >> > > > >> additional
>> > > > >> > > > >> > ideas/themes for KIPs, etc.
>> > > > >> > > > >> >
>> > > > >> > > > >> > I was thinking that we could try and target an end of
>> > March
>> > > > or
>> > > > >> Mid
>> > > > >> > > > April
>> > > > >> > > > >> > 1.1.0 release.
>> > > > >> > > > >> >
>> > > > >> > > > >> > Thoughts?
>> > > > >> > > > >> >
>> > > > >> > > > >> > --larry
>> > > > >> > > > >> >
>> > > > >> > > > >>
>> > > > >> > > > >
>> > > > >> > > >
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by larry mccay <lm...@apache.org>.
Yes, along with ERROR level logging when falling back to specific host/ip
or localhost variants.
On Thu, Jul 19, 2018, 11:48 PM Philip Zampino <pz...@gmail.com> wrote:
> Sure, but I'm also going to implement the following for the default
> behavior (when the DEFAULT value is specified for the whitelist property):
>
>
> 1. Attempt to determine the domain from the X-Forwarded-Host header
> value
> 2. If domain could not be determined, attempt to determine the domain
> from the InetAddress.getLocalHost().getCanonicalHostName() value
> 3. If domain could not be determined, attempt to determine the domain
> from the requested host name
> 4. If the domain could be determined from any of these sources, then the
> default whitelist will be based on that domain
> 5. If the domain cannot be determined
> a. If the requested host name is NOT a variant of localhost, then the
> whitelist will be restricted to that specific host name
> b. Otherwise, the localhost whitelist will be the default
>
> Does that sound like a good strategy?
>
>
> On Thu, Jul 19, 2018 at 8:15 PM larry mccay <lm...@apache.org> wrote:
>
> > Yes, does that sound appropriate to you?
> > If the LDAP config in gateway-site.xml gets updated to product the
> > whitelist would be in the same place.
> >
> > On Thu, Jul 19, 2018 at 6:26 PM, Philip Zampino <pz...@gmail.com>
> > wrote:
> >
> > > I am working on a solution for the ip address being treated as a
> hostname
> > > issue.
> > >
> > > On Thu, Jul 19, 2018 at 6:24 PM larry mccay <lm...@apache.org> wrote:
> > >
> > > > Playing around a bit more, I noticed that there is nondeterministic
> > > > behavior of the default whitelist feature.
> > > > Especially on macs - since the hostname ends up being any number of
> > > things.
> > > > I have noticed the following things when there is no explicit
> whitelist
> > > > configured:
> > > >
> > > > * ip address based whitelist being derived which is treated like a
> > domain
> > > > * localhost is not supported out of the box unless the logic is
> unable
> > to
> > > > determine a domain
> > > > * sometimes my host is HW14155.home and sometimes it is
> new-host-5.home
> > > for
> > > > some reason
> > > >
> > > > Given that all of our samples and docs assume localhost and OOTB we
> are
> > > > setup for DEMO LDAP server, I propose that we at least add localhost
> > back
> > > > for OOTB.
> > > > Ip address handling may be worth tackling as well but only if we can
> do
> > > it
> > > > in a day.
> > > >
> > > > Thoughts?
> > > >
> > > >
> > > > On Thu, Jul 19, 2018 at 6:12 PM, larry mccay <lm...@apache.org>
> > wrote:
> > > >
> > > > > Awesome - just checked it out and I will kick off a new build
> > shortly!
> > > > >
> > > > > On Thu, Jul 19, 2018 at 6:01 PM, Sandeep Moré <
> moresandeep@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > >> Hello Larry,
> > > > >>
> > > > >> I committed the fix to master and v1.1.0, it is under the JIRA
> > > KNOX-1391
> > > > >> <https://issues.apache.org/jira/browse/KNOX-1391>.
> > > > >> we should be good to to cut the RC, provided there are no more
> > issues
> > > !
> > > > >>
> > > > >> Thanks !
> > > > >> Sandeep
> > > > >>
> > > > >> On Thu, Jul 19, 2018 at 4:25 PM larry mccay <
> larry.mccay@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Awesome, @sandeep!
> > > > >> > I'll keep an eye out.
> > > > >> >
> > > > >> > Once that lands, you can bump this thread and I'll cut the RC.
> > > > >> > Obviously, we will need it in both master and v1.1.0 branches.
> > > > >> >
> > > > >> > On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <
> > > moresandeep@gmail.com>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Hello Larry,
> > > > >> > >
> > > > >> > > Yes, I have seen those exceptions, they seem to be happening
> > > fairly
> > > > >> > > consistently and only for KnoxSSO redirects when trying to
> > access
> > > > >> admin
> > > > >> > UI,
> > > > >> > > I am taking a look at them as we speak, will open up a JIRA
> for
> > it
> > > > as
> > > > >> > well.
> > > > >> > > It would be good if we can get it in, I will try to get the
> fix
> > > out
> > > > as
> > > > >> > soon
> > > > >> > > as I can.
> > > > >> > >
> > > > >> > > Best,
> > > > >> > > Sandeep
> > > > >> > >
> > > > >> > > On Thu, Jul 19, 2018 at 4:15 PM larry mccay <
> lmccay@apache.org>
> > > > >> wrote:
> > > > >> > >
> > > > >> > > > @Phil, I see a couple commits land that seem to address the
> > NPE.
> > > > >> > > > Is that correct?
> > > > >> > > >
> > > > >> > > > I have also seen an IllegalStateException during redirect
> from
> > > > >> Admin UI
> > > > >> > > to
> > > > >> > > > KnoxSSO.
> > > > >> > > > Has anyone seen this and/or is working on it - is it related
> > to
> > > > the
> > > > >> > NPE?
> > > > >> > > > I don't think it is since I see it more frequently and not
> > > always
> > > > >> with
> > > > >> > > the
> > > > >> > > > NPEs.
> > > > >> > > >
> > > > >> > > > I'd like to get a new RC cut by end of the week, if
> possible.
> > > > >> > > >
> > > > >> > > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <
> > lmccay@apache.org
> > > >
> > > > >> > wrote:
> > > > >> > > >
> > > > >> > > > > Agreed, Phil.
> > > > >> > > > > I have cut an RC but we need to address this first. I'll
> > hold
> > > > >> off on
> > > > >> > > > > announcing it.
> > > > >> > > > >
> > > > >> > > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <
> > > > pzampino@apache.org>
> > > > >> > > wrote:
> > > > >> > > > >
> > > > >> > > > >> During some testing of the proposed 1.1.0 code, I've
> > > discovered
> > > > >> some
> > > > >> > > > NPEs
> > > > >> > > > >> in filters (e.g., AclsAuthorizationFilter,
> > > > >> > HadoopGroupProviderFilter),
> > > > >> > > > >> which are concerning.
> > > > >> > > > >>
> > > > >> > > > >> I've committed a change to address the
> > > AclsAuthorizationFilter,
> > > > >> but
> > > > >> > > > seeing
> > > > >> > > > >> similar behavior for the HadoopGroupProviderFilter has
> > > > increased
> > > > >> my
> > > > >> > > > >> concern
> > > > >> > > > >> that there may be a more fundamental problem.
> > > > >> > > > >> In both cases, it seems that the filters are being
> invoked
> > > > prior
> > > > >> to
> > > > >> > > (or
> > > > >> > > > >> during) their respective init() methods have been
> invoked.
> > > > Thus,
> > > > >> > > members
> > > > >> > > > >> which should be initialized in the init() method are not
> > yet
> > > > >> > > > initialized.
> > > > >> > > > >>
> > > > >> > > > >> This can be consistently reproduced, though it is a bit
> of
> > a
> > > > >> pain:
> > > > >> > > > >>
> > > > >> > > > >> - Install Knox (‘ant install-test-home’, or just unzip
> > > > >> > > > knox-1.1.0.zip)
> > > > >> > > > >> - Start the gateway
> > > > >> > > > >> - Access the Admin UI
> > > > >> > > > >>
> > > > >> > > > >>
> > > > >> > > > >> Note that the latest 1.1.0 source has a *fix* for the
> > > > >> > > > >> AclsAuthorizationFilter NPE, but master does not yet have
> > > this
> > > > >> > change.
> > > > >> > > > >> This
> > > > >> > > > >> is important because that change effectively hides the
> > issue.
> > > > >> > > > >>
> > > > >> > > > >> I think we should determine what's happening with this
> > before
> > > > >> > > > >> producing/testing a release candidate.
> > > > >> > > > >>
> > > > >> > > > >>
> > > > >> > > > >>
> > > > >> > > > >>
> > > > >> > > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <
> > > > lmccay@apache.org>
> > > > >> > > wrote:
> > > > >> > > > >>
> > > > >> > > > >> > All -
> > > > >> > > > >> >
> > > > >> > > > >> > Sorry for the delay on this topic.
> > > > >> > > > >> >
> > > > >> > > > >> > We are going to start of this planning thread with ~85
> > > > >> Unresolved
> > > > >> > > > JIRAs
> > > > >> > > > >> in
> > > > >> > > > >> > either 1.1.0 or 0.15.0 fixVersion.
> > > > >> > > > >> >
> > > > >> > > > >> > project = KNOX AND resolution = Unresolved AND
> fixVersion
> > > in
> > > > >> > (1.1.0,
> > > > >> > > > >> > 0.15.0) ORDER BY priority DESC, updated DESC
> > > > >> > > > >> >
> > > > >> > > > >> > I will spend some time migrating all 0.15.0 to 1.1.0 to
> > > begin
> > > > >> with
> > > > >> > > and
> > > > >> > > > >> then
> > > > >> > > > >> > we will need to go through and see what is already
> taken
> > > care
> > > > >> of
> > > > >> > or
> > > > >> > > > can
> > > > >> > > > >> > wait for a 1.2.0 or later.
> > > > >> > > > >> >
> > > > >> > > > >> > I also have a couple KIPs in mind to target larger
> > > > >> features/themes
> > > > >> > > for
> > > > >> > > > >> this
> > > > >> > > > >> > release.
> > > > >> > > > >> >
> > > > >> > > > >> > Off the top of my head:
> > > > >> > > > >> >
> > > > >> > > > >> > * I think we need to address some cloud specific
> usecases
> > > and
> > > > >> plan
> > > > >> > > to
> > > > >> > > > >> > provide a KIP for that. Hybrid cloud/federated knox
> > > > instances,
> > > > >> > Azure
> > > > >> > > > AD
> > > > >> > > > >> > integration, ID mapping from Hadoop user to IAM
> > > users/roles,
> > > > >> etc.
> > > > >> > > > >> Perhaps
> > > > >> > > > >> > some CASB-like features if they make sense.
> > > > >> > > > >> >
> > > > >> > > > >> > * I also think we need one for articulating a
> reasonable
> > > flow
> > > > >> for
> > > > >> > > > >> Logout in
> > > > >> > > > >> > KnoxSSO. There are a lot of little nuances to logout
> > across
> > > > >> > multiple
> > > > >> > > > >> apps
> > > > >> > > > >> > and between different IDPs. This will require some
> > > > discussion.
> > > > >> > > > >> >
> > > > >> > > > >> > * Another thing that has been tugging at my interest
> has
> > > been
> > > > >> the
> > > > >> > > fact
> > > > >> > > > >> that
> > > > >> > > > >> > we may be able provide some common libraries to help
> > > > ecosystem
> > > > >> > > > >> applications
> > > > >> > > > >> > uptake the trusted proxy pattern and KnoxSSO.
> > > > >> > > > >> >
> > > > >> > > > >> > Anyway, these are my initial thoughts, please feel free
> > to
> > > > >> raise
> > > > >> > > > >> additional
> > > > >> > > > >> > ideas/themes for KIPs, etc.
> > > > >> > > > >> >
> > > > >> > > > >> > I was thinking that we could try and target an end of
> > March
> > > > or
> > > > >> Mid
> > > > >> > > > April
> > > > >> > > > >> > 1.1.0 release.
> > > > >> > > > >> >
> > > > >> > > > >> > Thoughts?
> > > > >> > > > >> >
> > > > >> > > > >> > --larry
> > > > >> > > > >> >
> > > > >> > > > >>
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by Philip Zampino <pz...@gmail.com>.
Sure, but I'm also going to implement the following for the default
behavior (when the DEFAULT value is specified for the whitelist property):
1. Attempt to determine the domain from the X-Forwarded-Host header value
2. If domain could not be determined, attempt to determine the domain
from the InetAddress.getLocalHost().getCanonicalHostName() value
3. If domain could not be determined, attempt to determine the domain
from the requested host name
4. If the domain could be determined from any of these sources, then the
default whitelist will be based on that domain
5. If the domain cannot be determined
a. If the requested host name is NOT a variant of localhost, then the
whitelist will be restricted to that specific host name
b. Otherwise, the localhost whitelist will be the default
Does that sound like a good strategy?
On Thu, Jul 19, 2018 at 8:15 PM larry mccay <lm...@apache.org> wrote:
> Yes, does that sound appropriate to you?
> If the LDAP config in gateway-site.xml gets updated to product the
> whitelist would be in the same place.
>
> On Thu, Jul 19, 2018 at 6:26 PM, Philip Zampino <pz...@gmail.com>
> wrote:
>
> > I am working on a solution for the ip address being treated as a hostname
> > issue.
> >
> > On Thu, Jul 19, 2018 at 6:24 PM larry mccay <lm...@apache.org> wrote:
> >
> > > Playing around a bit more, I noticed that there is nondeterministic
> > > behavior of the default whitelist feature.
> > > Especially on macs - since the hostname ends up being any number of
> > things.
> > > I have noticed the following things when there is no explicit whitelist
> > > configured:
> > >
> > > * ip address based whitelist being derived which is treated like a
> domain
> > > * localhost is not supported out of the box unless the logic is unable
> to
> > > determine a domain
> > > * sometimes my host is HW14155.home and sometimes it is new-host-5.home
> > for
> > > some reason
> > >
> > > Given that all of our samples and docs assume localhost and OOTB we are
> > > setup for DEMO LDAP server, I propose that we at least add localhost
> back
> > > for OOTB.
> > > Ip address handling may be worth tackling as well but only if we can do
> > it
> > > in a day.
> > >
> > > Thoughts?
> > >
> > >
> > > On Thu, Jul 19, 2018 at 6:12 PM, larry mccay <lm...@apache.org>
> wrote:
> > >
> > > > Awesome - just checked it out and I will kick off a new build
> shortly!
> > > >
> > > > On Thu, Jul 19, 2018 at 6:01 PM, Sandeep Moré <moresandeep@gmail.com
> >
> > > > wrote:
> > > >
> > > >> Hello Larry,
> > > >>
> > > >> I committed the fix to master and v1.1.0, it is under the JIRA
> > KNOX-1391
> > > >> <https://issues.apache.org/jira/browse/KNOX-1391>.
> > > >> we should be good to to cut the RC, provided there are no more
> issues
> > !
> > > >>
> > > >> Thanks !
> > > >> Sandeep
> > > >>
> > > >> On Thu, Jul 19, 2018 at 4:25 PM larry mccay <la...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Awesome, @sandeep!
> > > >> > I'll keep an eye out.
> > > >> >
> > > >> > Once that lands, you can bump this thread and I'll cut the RC.
> > > >> > Obviously, we will need it in both master and v1.1.0 branches.
> > > >> >
> > > >> > On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <
> > moresandeep@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > Hello Larry,
> > > >> > >
> > > >> > > Yes, I have seen those exceptions, they seem to be happening
> > fairly
> > > >> > > consistently and only for KnoxSSO redirects when trying to
> access
> > > >> admin
> > > >> > UI,
> > > >> > > I am taking a look at them as we speak, will open up a JIRA for
> it
> > > as
> > > >> > well.
> > > >> > > It would be good if we can get it in, I will try to get the fix
> > out
> > > as
> > > >> > soon
> > > >> > > as I can.
> > > >> > >
> > > >> > > Best,
> > > >> > > Sandeep
> > > >> > >
> > > >> > > On Thu, Jul 19, 2018 at 4:15 PM larry mccay <lm...@apache.org>
> > > >> wrote:
> > > >> > >
> > > >> > > > @Phil, I see a couple commits land that seem to address the
> NPE.
> > > >> > > > Is that correct?
> > > >> > > >
> > > >> > > > I have also seen an IllegalStateException during redirect from
> > > >> Admin UI
> > > >> > > to
> > > >> > > > KnoxSSO.
> > > >> > > > Has anyone seen this and/or is working on it - is it related
> to
> > > the
> > > >> > NPE?
> > > >> > > > I don't think it is since I see it more frequently and not
> > always
> > > >> with
> > > >> > > the
> > > >> > > > NPEs.
> > > >> > > >
> > > >> > > > I'd like to get a new RC cut by end of the week, if possible.
> > > >> > > >
> > > >> > > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <
> lmccay@apache.org
> > >
> > > >> > wrote:
> > > >> > > >
> > > >> > > > > Agreed, Phil.
> > > >> > > > > I have cut an RC but we need to address this first. I'll
> hold
> > > >> off on
> > > >> > > > > announcing it.
> > > >> > > > >
> > > >> > > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <
> > > pzampino@apache.org>
> > > >> > > wrote:
> > > >> > > > >
> > > >> > > > >> During some testing of the proposed 1.1.0 code, I've
> > discovered
> > > >> some
> > > >> > > > NPEs
> > > >> > > > >> in filters (e.g., AclsAuthorizationFilter,
> > > >> > HadoopGroupProviderFilter),
> > > >> > > > >> which are concerning.
> > > >> > > > >>
> > > >> > > > >> I've committed a change to address the
> > AclsAuthorizationFilter,
> > > >> but
> > > >> > > > seeing
> > > >> > > > >> similar behavior for the HadoopGroupProviderFilter has
> > > increased
> > > >> my
> > > >> > > > >> concern
> > > >> > > > >> that there may be a more fundamental problem.
> > > >> > > > >> In both cases, it seems that the filters are being invoked
> > > prior
> > > >> to
> > > >> > > (or
> > > >> > > > >> during) their respective init() methods have been invoked.
> > > Thus,
> > > >> > > members
> > > >> > > > >> which should be initialized in the init() method are not
> yet
> > > >> > > > initialized.
> > > >> > > > >>
> > > >> > > > >> This can be consistently reproduced, though it is a bit of
> a
> > > >> pain:
> > > >> > > > >>
> > > >> > > > >> - Install Knox (‘ant install-test-home’, or just unzip
> > > >> > > > knox-1.1.0.zip)
> > > >> > > > >> - Start the gateway
> > > >> > > > >> - Access the Admin UI
> > > >> > > > >>
> > > >> > > > >>
> > > >> > > > >> Note that the latest 1.1.0 source has a *fix* for the
> > > >> > > > >> AclsAuthorizationFilter NPE, but master does not yet have
> > this
> > > >> > change.
> > > >> > > > >> This
> > > >> > > > >> is important because that change effectively hides the
> issue.
> > > >> > > > >>
> > > >> > > > >> I think we should determine what's happening with this
> before
> > > >> > > > >> producing/testing a release candidate.
> > > >> > > > >>
> > > >> > > > >>
> > > >> > > > >>
> > > >> > > > >>
> > > >> > > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <
> > > lmccay@apache.org>
> > > >> > > wrote:
> > > >> > > > >>
> > > >> > > > >> > All -
> > > >> > > > >> >
> > > >> > > > >> > Sorry for the delay on this topic.
> > > >> > > > >> >
> > > >> > > > >> > We are going to start of this planning thread with ~85
> > > >> Unresolved
> > > >> > > > JIRAs
> > > >> > > > >> in
> > > >> > > > >> > either 1.1.0 or 0.15.0 fixVersion.
> > > >> > > > >> >
> > > >> > > > >> > project = KNOX AND resolution = Unresolved AND fixVersion
> > in
> > > >> > (1.1.0,
> > > >> > > > >> > 0.15.0) ORDER BY priority DESC, updated DESC
> > > >> > > > >> >
> > > >> > > > >> > I will spend some time migrating all 0.15.0 to 1.1.0 to
> > begin
> > > >> with
> > > >> > > and
> > > >> > > > >> then
> > > >> > > > >> > we will need to go through and see what is already taken
> > care
> > > >> of
> > > >> > or
> > > >> > > > can
> > > >> > > > >> > wait for a 1.2.0 or later.
> > > >> > > > >> >
> > > >> > > > >> > I also have a couple KIPs in mind to target larger
> > > >> features/themes
> > > >> > > for
> > > >> > > > >> this
> > > >> > > > >> > release.
> > > >> > > > >> >
> > > >> > > > >> > Off the top of my head:
> > > >> > > > >> >
> > > >> > > > >> > * I think we need to address some cloud specific usecases
> > and
> > > >> plan
> > > >> > > to
> > > >> > > > >> > provide a KIP for that. Hybrid cloud/federated knox
> > > instances,
> > > >> > Azure
> > > >> > > > AD
> > > >> > > > >> > integration, ID mapping from Hadoop user to IAM
> > users/roles,
> > > >> etc.
> > > >> > > > >> Perhaps
> > > >> > > > >> > some CASB-like features if they make sense.
> > > >> > > > >> >
> > > >> > > > >> > * I also think we need one for articulating a reasonable
> > flow
> > > >> for
> > > >> > > > >> Logout in
> > > >> > > > >> > KnoxSSO. There are a lot of little nuances to logout
> across
> > > >> > multiple
> > > >> > > > >> apps
> > > >> > > > >> > and between different IDPs. This will require some
> > > discussion.
> > > >> > > > >> >
> > > >> > > > >> > * Another thing that has been tugging at my interest has
> > been
> > > >> the
> > > >> > > fact
> > > >> > > > >> that
> > > >> > > > >> > we may be able provide some common libraries to help
> > > ecosystem
> > > >> > > > >> applications
> > > >> > > > >> > uptake the trusted proxy pattern and KnoxSSO.
> > > >> > > > >> >
> > > >> > > > >> > Anyway, these are my initial thoughts, please feel free
> to
> > > >> raise
> > > >> > > > >> additional
> > > >> > > > >> > ideas/themes for KIPs, etc.
> > > >> > > > >> >
> > > >> > > > >> > I was thinking that we could try and target an end of
> March
> > > or
> > > >> Mid
> > > >> > > > April
> > > >> > > > >> > 1.1.0 release.
> > > >> > > > >> >
> > > >> > > > >> > Thoughts?
> > > >> > > > >> >
> > > >> > > > >> > --larry
> > > >> > > > >> >
> > > >> > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by larry mccay <lm...@apache.org>.
Yes, does that sound appropriate to you?
If the LDAP config in gateway-site.xml gets updated to product the
whitelist would be in the same place.
On Thu, Jul 19, 2018 at 6:26 PM, Philip Zampino <pz...@gmail.com> wrote:
> I am working on a solution for the ip address being treated as a hostname
> issue.
>
> On Thu, Jul 19, 2018 at 6:24 PM larry mccay <lm...@apache.org> wrote:
>
> > Playing around a bit more, I noticed that there is nondeterministic
> > behavior of the default whitelist feature.
> > Especially on macs - since the hostname ends up being any number of
> things.
> > I have noticed the following things when there is no explicit whitelist
> > configured:
> >
> > * ip address based whitelist being derived which is treated like a domain
> > * localhost is not supported out of the box unless the logic is unable to
> > determine a domain
> > * sometimes my host is HW14155.home and sometimes it is new-host-5.home
> for
> > some reason
> >
> > Given that all of our samples and docs assume localhost and OOTB we are
> > setup for DEMO LDAP server, I propose that we at least add localhost back
> > for OOTB.
> > Ip address handling may be worth tackling as well but only if we can do
> it
> > in a day.
> >
> > Thoughts?
> >
> >
> > On Thu, Jul 19, 2018 at 6:12 PM, larry mccay <lm...@apache.org> wrote:
> >
> > > Awesome - just checked it out and I will kick off a new build shortly!
> > >
> > > On Thu, Jul 19, 2018 at 6:01 PM, Sandeep Moré <mo...@gmail.com>
> > > wrote:
> > >
> > >> Hello Larry,
> > >>
> > >> I committed the fix to master and v1.1.0, it is under the JIRA
> KNOX-1391
> > >> <https://issues.apache.org/jira/browse/KNOX-1391>.
> > >> we should be good to to cut the RC, provided there are no more issues
> !
> > >>
> > >> Thanks !
> > >> Sandeep
> > >>
> > >> On Thu, Jul 19, 2018 at 4:25 PM larry mccay <la...@gmail.com>
> > >> wrote:
> > >>
> > >> > Awesome, @sandeep!
> > >> > I'll keep an eye out.
> > >> >
> > >> > Once that lands, you can bump this thread and I'll cut the RC.
> > >> > Obviously, we will need it in both master and v1.1.0 branches.
> > >> >
> > >> > On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <
> moresandeep@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hello Larry,
> > >> > >
> > >> > > Yes, I have seen those exceptions, they seem to be happening
> fairly
> > >> > > consistently and only for KnoxSSO redirects when trying to access
> > >> admin
> > >> > UI,
> > >> > > I am taking a look at them as we speak, will open up a JIRA for it
> > as
> > >> > well.
> > >> > > It would be good if we can get it in, I will try to get the fix
> out
> > as
> > >> > soon
> > >> > > as I can.
> > >> > >
> > >> > > Best,
> > >> > > Sandeep
> > >> > >
> > >> > > On Thu, Jul 19, 2018 at 4:15 PM larry mccay <lm...@apache.org>
> > >> wrote:
> > >> > >
> > >> > > > @Phil, I see a couple commits land that seem to address the NPE.
> > >> > > > Is that correct?
> > >> > > >
> > >> > > > I have also seen an IllegalStateException during redirect from
> > >> Admin UI
> > >> > > to
> > >> > > > KnoxSSO.
> > >> > > > Has anyone seen this and/or is working on it - is it related to
> > the
> > >> > NPE?
> > >> > > > I don't think it is since I see it more frequently and not
> always
> > >> with
> > >> > > the
> > >> > > > NPEs.
> > >> > > >
> > >> > > > I'd like to get a new RC cut by end of the week, if possible.
> > >> > > >
> > >> > > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <lmccay@apache.org
> >
> > >> > wrote:
> > >> > > >
> > >> > > > > Agreed, Phil.
> > >> > > > > I have cut an RC but we need to address this first. I'll hold
> > >> off on
> > >> > > > > announcing it.
> > >> > > > >
> > >> > > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <
> > pzampino@apache.org>
> > >> > > wrote:
> > >> > > > >
> > >> > > > >> During some testing of the proposed 1.1.0 code, I've
> discovered
> > >> some
> > >> > > > NPEs
> > >> > > > >> in filters (e.g., AclsAuthorizationFilter,
> > >> > HadoopGroupProviderFilter),
> > >> > > > >> which are concerning.
> > >> > > > >>
> > >> > > > >> I've committed a change to address the
> AclsAuthorizationFilter,
> > >> but
> > >> > > > seeing
> > >> > > > >> similar behavior for the HadoopGroupProviderFilter has
> > increased
> > >> my
> > >> > > > >> concern
> > >> > > > >> that there may be a more fundamental problem.
> > >> > > > >> In both cases, it seems that the filters are being invoked
> > prior
> > >> to
> > >> > > (or
> > >> > > > >> during) their respective init() methods have been invoked.
> > Thus,
> > >> > > members
> > >> > > > >> which should be initialized in the init() method are not yet
> > >> > > > initialized.
> > >> > > > >>
> > >> > > > >> This can be consistently reproduced, though it is a bit of a
> > >> pain:
> > >> > > > >>
> > >> > > > >> - Install Knox (‘ant install-test-home’, or just unzip
> > >> > > > knox-1.1.0.zip)
> > >> > > > >> - Start the gateway
> > >> > > > >> - Access the Admin UI
> > >> > > > >>
> > >> > > > >>
> > >> > > > >> Note that the latest 1.1.0 source has a *fix* for the
> > >> > > > >> AclsAuthorizationFilter NPE, but master does not yet have
> this
> > >> > change.
> > >> > > > >> This
> > >> > > > >> is important because that change effectively hides the issue.
> > >> > > > >>
> > >> > > > >> I think we should determine what's happening with this before
> > >> > > > >> producing/testing a release candidate.
> > >> > > > >>
> > >> > > > >>
> > >> > > > >>
> > >> > > > >>
> > >> > > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <
> > lmccay@apache.org>
> > >> > > wrote:
> > >> > > > >>
> > >> > > > >> > All -
> > >> > > > >> >
> > >> > > > >> > Sorry for the delay on this topic.
> > >> > > > >> >
> > >> > > > >> > We are going to start of this planning thread with ~85
> > >> Unresolved
> > >> > > > JIRAs
> > >> > > > >> in
> > >> > > > >> > either 1.1.0 or 0.15.0 fixVersion.
> > >> > > > >> >
> > >> > > > >> > project = KNOX AND resolution = Unresolved AND fixVersion
> in
> > >> > (1.1.0,
> > >> > > > >> > 0.15.0) ORDER BY priority DESC, updated DESC
> > >> > > > >> >
> > >> > > > >> > I will spend some time migrating all 0.15.0 to 1.1.0 to
> begin
> > >> with
> > >> > > and
> > >> > > > >> then
> > >> > > > >> > we will need to go through and see what is already taken
> care
> > >> of
> > >> > or
> > >> > > > can
> > >> > > > >> > wait for a 1.2.0 or later.
> > >> > > > >> >
> > >> > > > >> > I also have a couple KIPs in mind to target larger
> > >> features/themes
> > >> > > for
> > >> > > > >> this
> > >> > > > >> > release.
> > >> > > > >> >
> > >> > > > >> > Off the top of my head:
> > >> > > > >> >
> > >> > > > >> > * I think we need to address some cloud specific usecases
> and
> > >> plan
> > >> > > to
> > >> > > > >> > provide a KIP for that. Hybrid cloud/federated knox
> > instances,
> > >> > Azure
> > >> > > > AD
> > >> > > > >> > integration, ID mapping from Hadoop user to IAM
> users/roles,
> > >> etc.
> > >> > > > >> Perhaps
> > >> > > > >> > some CASB-like features if they make sense.
> > >> > > > >> >
> > >> > > > >> > * I also think we need one for articulating a reasonable
> flow
> > >> for
> > >> > > > >> Logout in
> > >> > > > >> > KnoxSSO. There are a lot of little nuances to logout across
> > >> > multiple
> > >> > > > >> apps
> > >> > > > >> > and between different IDPs. This will require some
> > discussion.
> > >> > > > >> >
> > >> > > > >> > * Another thing that has been tugging at my interest has
> been
> > >> the
> > >> > > fact
> > >> > > > >> that
> > >> > > > >> > we may be able provide some common libraries to help
> > ecosystem
> > >> > > > >> applications
> > >> > > > >> > uptake the trusted proxy pattern and KnoxSSO.
> > >> > > > >> >
> > >> > > > >> > Anyway, these are my initial thoughts, please feel free to
> > >> raise
> > >> > > > >> additional
> > >> > > > >> > ideas/themes for KIPs, etc.
> > >> > > > >> >
> > >> > > > >> > I was thinking that we could try and target an end of March
> > or
> > >> Mid
> > >> > > > April
> > >> > > > >> > 1.1.0 release.
> > >> > > > >> >
> > >> > > > >> > Thoughts?
> > >> > > > >> >
> > >> > > > >> > --larry
> > >> > > > >> >
> > >> > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by Philip Zampino <pz...@gmail.com>.
I am working on a solution for the ip address being treated as a hostname
issue.
On Thu, Jul 19, 2018 at 6:24 PM larry mccay <lm...@apache.org> wrote:
> Playing around a bit more, I noticed that there is nondeterministic
> behavior of the default whitelist feature.
> Especially on macs - since the hostname ends up being any number of things.
> I have noticed the following things when there is no explicit whitelist
> configured:
>
> * ip address based whitelist being derived which is treated like a domain
> * localhost is not supported out of the box unless the logic is unable to
> determine a domain
> * sometimes my host is HW14155.home and sometimes it is new-host-5.home for
> some reason
>
> Given that all of our samples and docs assume localhost and OOTB we are
> setup for DEMO LDAP server, I propose that we at least add localhost back
> for OOTB.
> Ip address handling may be worth tackling as well but only if we can do it
> in a day.
>
> Thoughts?
>
>
> On Thu, Jul 19, 2018 at 6:12 PM, larry mccay <lm...@apache.org> wrote:
>
> > Awesome - just checked it out and I will kick off a new build shortly!
> >
> > On Thu, Jul 19, 2018 at 6:01 PM, Sandeep Moré <mo...@gmail.com>
> > wrote:
> >
> >> Hello Larry,
> >>
> >> I committed the fix to master and v1.1.0, it is under the JIRA KNOX-1391
> >> <https://issues.apache.org/jira/browse/KNOX-1391>.
> >> we should be good to to cut the RC, provided there are no more issues !
> >>
> >> Thanks !
> >> Sandeep
> >>
> >> On Thu, Jul 19, 2018 at 4:25 PM larry mccay <la...@gmail.com>
> >> wrote:
> >>
> >> > Awesome, @sandeep!
> >> > I'll keep an eye out.
> >> >
> >> > Once that lands, you can bump this thread and I'll cut the RC.
> >> > Obviously, we will need it in both master and v1.1.0 branches.
> >> >
> >> > On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <mo...@gmail.com>
> >> > wrote:
> >> >
> >> > > Hello Larry,
> >> > >
> >> > > Yes, I have seen those exceptions, they seem to be happening fairly
> >> > > consistently and only for KnoxSSO redirects when trying to access
> >> admin
> >> > UI,
> >> > > I am taking a look at them as we speak, will open up a JIRA for it
> as
> >> > well.
> >> > > It would be good if we can get it in, I will try to get the fix out
> as
> >> > soon
> >> > > as I can.
> >> > >
> >> > > Best,
> >> > > Sandeep
> >> > >
> >> > > On Thu, Jul 19, 2018 at 4:15 PM larry mccay <lm...@apache.org>
> >> wrote:
> >> > >
> >> > > > @Phil, I see a couple commits land that seem to address the NPE.
> >> > > > Is that correct?
> >> > > >
> >> > > > I have also seen an IllegalStateException during redirect from
> >> Admin UI
> >> > > to
> >> > > > KnoxSSO.
> >> > > > Has anyone seen this and/or is working on it - is it related to
> the
> >> > NPE?
> >> > > > I don't think it is since I see it more frequently and not always
> >> with
> >> > > the
> >> > > > NPEs.
> >> > > >
> >> > > > I'd like to get a new RC cut by end of the week, if possible.
> >> > > >
> >> > > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <lm...@apache.org>
> >> > wrote:
> >> > > >
> >> > > > > Agreed, Phil.
> >> > > > > I have cut an RC but we need to address this first. I'll hold
> >> off on
> >> > > > > announcing it.
> >> > > > >
> >> > > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <
> pzampino@apache.org>
> >> > > wrote:
> >> > > > >
> >> > > > >> During some testing of the proposed 1.1.0 code, I've discovered
> >> some
> >> > > > NPEs
> >> > > > >> in filters (e.g., AclsAuthorizationFilter,
> >> > HadoopGroupProviderFilter),
> >> > > > >> which are concerning.
> >> > > > >>
> >> > > > >> I've committed a change to address the AclsAuthorizationFilter,
> >> but
> >> > > > seeing
> >> > > > >> similar behavior for the HadoopGroupProviderFilter has
> increased
> >> my
> >> > > > >> concern
> >> > > > >> that there may be a more fundamental problem.
> >> > > > >> In both cases, it seems that the filters are being invoked
> prior
> >> to
> >> > > (or
> >> > > > >> during) their respective init() methods have been invoked.
> Thus,
> >> > > members
> >> > > > >> which should be initialized in the init() method are not yet
> >> > > > initialized.
> >> > > > >>
> >> > > > >> This can be consistently reproduced, though it is a bit of a
> >> pain:
> >> > > > >>
> >> > > > >> - Install Knox (‘ant install-test-home’, or just unzip
> >> > > > knox-1.1.0.zip)
> >> > > > >> - Start the gateway
> >> > > > >> - Access the Admin UI
> >> > > > >>
> >> > > > >>
> >> > > > >> Note that the latest 1.1.0 source has a *fix* for the
> >> > > > >> AclsAuthorizationFilter NPE, but master does not yet have this
> >> > change.
> >> > > > >> This
> >> > > > >> is important because that change effectively hides the issue.
> >> > > > >>
> >> > > > >> I think we should determine what's happening with this before
> >> > > > >> producing/testing a release candidate.
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <
> lmccay@apache.org>
> >> > > wrote:
> >> > > > >>
> >> > > > >> > All -
> >> > > > >> >
> >> > > > >> > Sorry for the delay on this topic.
> >> > > > >> >
> >> > > > >> > We are going to start of this planning thread with ~85
> >> Unresolved
> >> > > > JIRAs
> >> > > > >> in
> >> > > > >> > either 1.1.0 or 0.15.0 fixVersion.
> >> > > > >> >
> >> > > > >> > project = KNOX AND resolution = Unresolved AND fixVersion in
> >> > (1.1.0,
> >> > > > >> > 0.15.0) ORDER BY priority DESC, updated DESC
> >> > > > >> >
> >> > > > >> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin
> >> with
> >> > > and
> >> > > > >> then
> >> > > > >> > we will need to go through and see what is already taken care
> >> of
> >> > or
> >> > > > can
> >> > > > >> > wait for a 1.2.0 or later.
> >> > > > >> >
> >> > > > >> > I also have a couple KIPs in mind to target larger
> >> features/themes
> >> > > for
> >> > > > >> this
> >> > > > >> > release.
> >> > > > >> >
> >> > > > >> > Off the top of my head:
> >> > > > >> >
> >> > > > >> > * I think we need to address some cloud specific usecases and
> >> plan
> >> > > to
> >> > > > >> > provide a KIP for that. Hybrid cloud/federated knox
> instances,
> >> > Azure
> >> > > > AD
> >> > > > >> > integration, ID mapping from Hadoop user to IAM users/roles,
> >> etc.
> >> > > > >> Perhaps
> >> > > > >> > some CASB-like features if they make sense.
> >> > > > >> >
> >> > > > >> > * I also think we need one for articulating a reasonable flow
> >> for
> >> > > > >> Logout in
> >> > > > >> > KnoxSSO. There are a lot of little nuances to logout across
> >> > multiple
> >> > > > >> apps
> >> > > > >> > and between different IDPs. This will require some
> discussion.
> >> > > > >> >
> >> > > > >> > * Another thing that has been tugging at my interest has been
> >> the
> >> > > fact
> >> > > > >> that
> >> > > > >> > we may be able provide some common libraries to help
> ecosystem
> >> > > > >> applications
> >> > > > >> > uptake the trusted proxy pattern and KnoxSSO.
> >> > > > >> >
> >> > > > >> > Anyway, these are my initial thoughts, please feel free to
> >> raise
> >> > > > >> additional
> >> > > > >> > ideas/themes for KIPs, etc.
> >> > > > >> >
> >> > > > >> > I was thinking that we could try and target an end of March
> or
> >> Mid
> >> > > > April
> >> > > > >> > 1.1.0 release.
> >> > > > >> >
> >> > > > >> > Thoughts?
> >> > > > >> >
> >> > > > >> > --larry
> >> > > > >> >
> >> > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by Philip Zampino <pz...@gmail.com>.
To be clear, you're suggesting an explicit whitelist in gateway.xml, which
permits the various forms of "localhost"?
On Thu, Jul 19, 2018 at 6:24 PM larry mccay <lm...@apache.org> wrote:
> Playing around a bit more, I noticed that there is nondeterministic
> behavior of the default whitelist feature.
> Especially on macs - since the hostname ends up being any number of things.
> I have noticed the following things when there is no explicit whitelist
> configured:
>
> * ip address based whitelist being derived which is treated like a domain
> * localhost is not supported out of the box unless the logic is unable to
> determine a domain
> * sometimes my host is HW14155.home and sometimes it is new-host-5.home for
> some reason
>
> Given that all of our samples and docs assume localhost and OOTB we are
> setup for DEMO LDAP server, I propose that we at least add localhost back
> for OOTB.
> Ip address handling may be worth tackling as well but only if we can do it
> in a day.
>
> Thoughts?
>
>
> On Thu, Jul 19, 2018 at 6:12 PM, larry mccay <lm...@apache.org> wrote:
>
> > Awesome - just checked it out and I will kick off a new build shortly!
> >
> > On Thu, Jul 19, 2018 at 6:01 PM, Sandeep Moré <mo...@gmail.com>
> > wrote:
> >
> >> Hello Larry,
> >>
> >> I committed the fix to master and v1.1.0, it is under the JIRA KNOX-1391
> >> <https://issues.apache.org/jira/browse/KNOX-1391>.
> >> we should be good to to cut the RC, provided there are no more issues !
> >>
> >> Thanks !
> >> Sandeep
> >>
> >> On Thu, Jul 19, 2018 at 4:25 PM larry mccay <la...@gmail.com>
> >> wrote:
> >>
> >> > Awesome, @sandeep!
> >> > I'll keep an eye out.
> >> >
> >> > Once that lands, you can bump this thread and I'll cut the RC.
> >> > Obviously, we will need it in both master and v1.1.0 branches.
> >> >
> >> > On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <mo...@gmail.com>
> >> > wrote:
> >> >
> >> > > Hello Larry,
> >> > >
> >> > > Yes, I have seen those exceptions, they seem to be happening fairly
> >> > > consistently and only for KnoxSSO redirects when trying to access
> >> admin
> >> > UI,
> >> > > I am taking a look at them as we speak, will open up a JIRA for it
> as
> >> > well.
> >> > > It would be good if we can get it in, I will try to get the fix out
> as
> >> > soon
> >> > > as I can.
> >> > >
> >> > > Best,
> >> > > Sandeep
> >> > >
> >> > > On Thu, Jul 19, 2018 at 4:15 PM larry mccay <lm...@apache.org>
> >> wrote:
> >> > >
> >> > > > @Phil, I see a couple commits land that seem to address the NPE.
> >> > > > Is that correct?
> >> > > >
> >> > > > I have also seen an IllegalStateException during redirect from
> >> Admin UI
> >> > > to
> >> > > > KnoxSSO.
> >> > > > Has anyone seen this and/or is working on it - is it related to
> the
> >> > NPE?
> >> > > > I don't think it is since I see it more frequently and not always
> >> with
> >> > > the
> >> > > > NPEs.
> >> > > >
> >> > > > I'd like to get a new RC cut by end of the week, if possible.
> >> > > >
> >> > > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <lm...@apache.org>
> >> > wrote:
> >> > > >
> >> > > > > Agreed, Phil.
> >> > > > > I have cut an RC but we need to address this first. I'll hold
> >> off on
> >> > > > > announcing it.
> >> > > > >
> >> > > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <
> pzampino@apache.org>
> >> > > wrote:
> >> > > > >
> >> > > > >> During some testing of the proposed 1.1.0 code, I've discovered
> >> some
> >> > > > NPEs
> >> > > > >> in filters (e.g., AclsAuthorizationFilter,
> >> > HadoopGroupProviderFilter),
> >> > > > >> which are concerning.
> >> > > > >>
> >> > > > >> I've committed a change to address the AclsAuthorizationFilter,
> >> but
> >> > > > seeing
> >> > > > >> similar behavior for the HadoopGroupProviderFilter has
> increased
> >> my
> >> > > > >> concern
> >> > > > >> that there may be a more fundamental problem.
> >> > > > >> In both cases, it seems that the filters are being invoked
> prior
> >> to
> >> > > (or
> >> > > > >> during) their respective init() methods have been invoked.
> Thus,
> >> > > members
> >> > > > >> which should be initialized in the init() method are not yet
> >> > > > initialized.
> >> > > > >>
> >> > > > >> This can be consistently reproduced, though it is a bit of a
> >> pain:
> >> > > > >>
> >> > > > >> - Install Knox (‘ant install-test-home’, or just unzip
> >> > > > knox-1.1.0.zip)
> >> > > > >> - Start the gateway
> >> > > > >> - Access the Admin UI
> >> > > > >>
> >> > > > >>
> >> > > > >> Note that the latest 1.1.0 source has a *fix* for the
> >> > > > >> AclsAuthorizationFilter NPE, but master does not yet have this
> >> > change.
> >> > > > >> This
> >> > > > >> is important because that change effectively hides the issue.
> >> > > > >>
> >> > > > >> I think we should determine what's happening with this before
> >> > > > >> producing/testing a release candidate.
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <
> lmccay@apache.org>
> >> > > wrote:
> >> > > > >>
> >> > > > >> > All -
> >> > > > >> >
> >> > > > >> > Sorry for the delay on this topic.
> >> > > > >> >
> >> > > > >> > We are going to start of this planning thread with ~85
> >> Unresolved
> >> > > > JIRAs
> >> > > > >> in
> >> > > > >> > either 1.1.0 or 0.15.0 fixVersion.
> >> > > > >> >
> >> > > > >> > project = KNOX AND resolution = Unresolved AND fixVersion in
> >> > (1.1.0,
> >> > > > >> > 0.15.0) ORDER BY priority DESC, updated DESC
> >> > > > >> >
> >> > > > >> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin
> >> with
> >> > > and
> >> > > > >> then
> >> > > > >> > we will need to go through and see what is already taken care
> >> of
> >> > or
> >> > > > can
> >> > > > >> > wait for a 1.2.0 or later.
> >> > > > >> >
> >> > > > >> > I also have a couple KIPs in mind to target larger
> >> features/themes
> >> > > for
> >> > > > >> this
> >> > > > >> > release.
> >> > > > >> >
> >> > > > >> > Off the top of my head:
> >> > > > >> >
> >> > > > >> > * I think we need to address some cloud specific usecases and
> >> plan
> >> > > to
> >> > > > >> > provide a KIP for that. Hybrid cloud/federated knox
> instances,
> >> > Azure
> >> > > > AD
> >> > > > >> > integration, ID mapping from Hadoop user to IAM users/roles,
> >> etc.
> >> > > > >> Perhaps
> >> > > > >> > some CASB-like features if they make sense.
> >> > > > >> >
> >> > > > >> > * I also think we need one for articulating a reasonable flow
> >> for
> >> > > > >> Logout in
> >> > > > >> > KnoxSSO. There are a lot of little nuances to logout across
> >> > multiple
> >> > > > >> apps
> >> > > > >> > and between different IDPs. This will require some
> discussion.
> >> > > > >> >
> >> > > > >> > * Another thing that has been tugging at my interest has been
> >> the
> >> > > fact
> >> > > > >> that
> >> > > > >> > we may be able provide some common libraries to help
> ecosystem
> >> > > > >> applications
> >> > > > >> > uptake the trusted proxy pattern and KnoxSSO.
> >> > > > >> >
> >> > > > >> > Anyway, these are my initial thoughts, please feel free to
> >> raise
> >> > > > >> additional
> >> > > > >> > ideas/themes for KIPs, etc.
> >> > > > >> >
> >> > > > >> > I was thinking that we could try and target an end of March
> or
> >> Mid
> >> > > > April
> >> > > > >> > 1.1.0 release.
> >> > > > >> >
> >> > > > >> > Thoughts?
> >> > > > >> >
> >> > > > >> > --larry
> >> > > > >> >
> >> > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by larry mccay <lm...@apache.org>.
Playing around a bit more, I noticed that there is nondeterministic
behavior of the default whitelist feature.
Especially on macs - since the hostname ends up being any number of things.
I have noticed the following things when there is no explicit whitelist
configured:
* ip address based whitelist being derived which is treated like a domain
* localhost is not supported out of the box unless the logic is unable to
determine a domain
* sometimes my host is HW14155.home and sometimes it is new-host-5.home for
some reason
Given that all of our samples and docs assume localhost and OOTB we are
setup for DEMO LDAP server, I propose that we at least add localhost back
for OOTB.
Ip address handling may be worth tackling as well but only if we can do it
in a day.
Thoughts?
On Thu, Jul 19, 2018 at 6:12 PM, larry mccay <lm...@apache.org> wrote:
> Awesome - just checked it out and I will kick off a new build shortly!
>
> On Thu, Jul 19, 2018 at 6:01 PM, Sandeep Moré <mo...@gmail.com>
> wrote:
>
>> Hello Larry,
>>
>> I committed the fix to master and v1.1.0, it is under the JIRA KNOX-1391
>> <https://issues.apache.org/jira/browse/KNOX-1391>.
>> we should be good to to cut the RC, provided there are no more issues !
>>
>> Thanks !
>> Sandeep
>>
>> On Thu, Jul 19, 2018 at 4:25 PM larry mccay <la...@gmail.com>
>> wrote:
>>
>> > Awesome, @sandeep!
>> > I'll keep an eye out.
>> >
>> > Once that lands, you can bump this thread and I'll cut the RC.
>> > Obviously, we will need it in both master and v1.1.0 branches.
>> >
>> > On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <mo...@gmail.com>
>> > wrote:
>> >
>> > > Hello Larry,
>> > >
>> > > Yes, I have seen those exceptions, they seem to be happening fairly
>> > > consistently and only for KnoxSSO redirects when trying to access
>> admin
>> > UI,
>> > > I am taking a look at them as we speak, will open up a JIRA for it as
>> > well.
>> > > It would be good if we can get it in, I will try to get the fix out as
>> > soon
>> > > as I can.
>> > >
>> > > Best,
>> > > Sandeep
>> > >
>> > > On Thu, Jul 19, 2018 at 4:15 PM larry mccay <lm...@apache.org>
>> wrote:
>> > >
>> > > > @Phil, I see a couple commits land that seem to address the NPE.
>> > > > Is that correct?
>> > > >
>> > > > I have also seen an IllegalStateException during redirect from
>> Admin UI
>> > > to
>> > > > KnoxSSO.
>> > > > Has anyone seen this and/or is working on it - is it related to the
>> > NPE?
>> > > > I don't think it is since I see it more frequently and not always
>> with
>> > > the
>> > > > NPEs.
>> > > >
>> > > > I'd like to get a new RC cut by end of the week, if possible.
>> > > >
>> > > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <lm...@apache.org>
>> > wrote:
>> > > >
>> > > > > Agreed, Phil.
>> > > > > I have cut an RC but we need to address this first. I'll hold
>> off on
>> > > > > announcing it.
>> > > > >
>> > > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <pz...@apache.org>
>> > > wrote:
>> > > > >
>> > > > >> During some testing of the proposed 1.1.0 code, I've discovered
>> some
>> > > > NPEs
>> > > > >> in filters (e.g., AclsAuthorizationFilter,
>> > HadoopGroupProviderFilter),
>> > > > >> which are concerning.
>> > > > >>
>> > > > >> I've committed a change to address the AclsAuthorizationFilter,
>> but
>> > > > seeing
>> > > > >> similar behavior for the HadoopGroupProviderFilter has increased
>> my
>> > > > >> concern
>> > > > >> that there may be a more fundamental problem.
>> > > > >> In both cases, it seems that the filters are being invoked prior
>> to
>> > > (or
>> > > > >> during) their respective init() methods have been invoked. Thus,
>> > > members
>> > > > >> which should be initialized in the init() method are not yet
>> > > > initialized.
>> > > > >>
>> > > > >> This can be consistently reproduced, though it is a bit of a
>> pain:
>> > > > >>
>> > > > >> - Install Knox (‘ant install-test-home’, or just unzip
>> > > > knox-1.1.0.zip)
>> > > > >> - Start the gateway
>> > > > >> - Access the Admin UI
>> > > > >>
>> > > > >>
>> > > > >> Note that the latest 1.1.0 source has a *fix* for the
>> > > > >> AclsAuthorizationFilter NPE, but master does not yet have this
>> > change.
>> > > > >> This
>> > > > >> is important because that change effectively hides the issue.
>> > > > >>
>> > > > >> I think we should determine what's happening with this before
>> > > > >> producing/testing a release candidate.
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <lm...@apache.org>
>> > > wrote:
>> > > > >>
>> > > > >> > All -
>> > > > >> >
>> > > > >> > Sorry for the delay on this topic.
>> > > > >> >
>> > > > >> > We are going to start of this planning thread with ~85
>> Unresolved
>> > > > JIRAs
>> > > > >> in
>> > > > >> > either 1.1.0 or 0.15.0 fixVersion.
>> > > > >> >
>> > > > >> > project = KNOX AND resolution = Unresolved AND fixVersion in
>> > (1.1.0,
>> > > > >> > 0.15.0) ORDER BY priority DESC, updated DESC
>> > > > >> >
>> > > > >> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin
>> with
>> > > and
>> > > > >> then
>> > > > >> > we will need to go through and see what is already taken care
>> of
>> > or
>> > > > can
>> > > > >> > wait for a 1.2.0 or later.
>> > > > >> >
>> > > > >> > I also have a couple KIPs in mind to target larger
>> features/themes
>> > > for
>> > > > >> this
>> > > > >> > release.
>> > > > >> >
>> > > > >> > Off the top of my head:
>> > > > >> >
>> > > > >> > * I think we need to address some cloud specific usecases and
>> plan
>> > > to
>> > > > >> > provide a KIP for that. Hybrid cloud/federated knox instances,
>> > Azure
>> > > > AD
>> > > > >> > integration, ID mapping from Hadoop user to IAM users/roles,
>> etc.
>> > > > >> Perhaps
>> > > > >> > some CASB-like features if they make sense.
>> > > > >> >
>> > > > >> > * I also think we need one for articulating a reasonable flow
>> for
>> > > > >> Logout in
>> > > > >> > KnoxSSO. There are a lot of little nuances to logout across
>> > multiple
>> > > > >> apps
>> > > > >> > and between different IDPs. This will require some discussion.
>> > > > >> >
>> > > > >> > * Another thing that has been tugging at my interest has been
>> the
>> > > fact
>> > > > >> that
>> > > > >> > we may be able provide some common libraries to help ecosystem
>> > > > >> applications
>> > > > >> > uptake the trusted proxy pattern and KnoxSSO.
>> > > > >> >
>> > > > >> > Anyway, these are my initial thoughts, please feel free to
>> raise
>> > > > >> additional
>> > > > >> > ideas/themes for KIPs, etc.
>> > > > >> >
>> > > > >> > I was thinking that we could try and target an end of March or
>> Mid
>> > > > April
>> > > > >> > 1.1.0 release.
>> > > > >> >
>> > > > >> > Thoughts?
>> > > > >> >
>> > > > >> > --larry
>> > > > >> >
>> > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by Sandeep Moré <mo...@gmail.com>.
Perfect, thanks !
On Thu, Jul 19, 2018 at 6:12 PM larry mccay <lm...@apache.org> wrote:
> Awesome - just checked it out and I will kick off a new build shortly!
>
> On Thu, Jul 19, 2018 at 6:01 PM, Sandeep Moré <mo...@gmail.com>
> wrote:
>
> > Hello Larry,
> >
> > I committed the fix to master and v1.1.0, it is under the JIRA KNOX-1391
> > <https://issues.apache.org/jira/browse/KNOX-1391>.
> > we should be good to to cut the RC, provided there are no more issues !
> >
> > Thanks !
> > Sandeep
> >
> > On Thu, Jul 19, 2018 at 4:25 PM larry mccay <la...@gmail.com>
> wrote:
> >
> > > Awesome, @sandeep!
> > > I'll keep an eye out.
> > >
> > > Once that lands, you can bump this thread and I'll cut the RC.
> > > Obviously, we will need it in both master and v1.1.0 branches.
> > >
> > > On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <mo...@gmail.com>
> > > wrote:
> > >
> > > > Hello Larry,
> > > >
> > > > Yes, I have seen those exceptions, they seem to be happening fairly
> > > > consistently and only for KnoxSSO redirects when trying to access
> admin
> > > UI,
> > > > I am taking a look at them as we speak, will open up a JIRA for it as
> > > well.
> > > > It would be good if we can get it in, I will try to get the fix out
> as
> > > soon
> > > > as I can.
> > > >
> > > > Best,
> > > > Sandeep
> > > >
> > > > On Thu, Jul 19, 2018 at 4:15 PM larry mccay <lm...@apache.org>
> wrote:
> > > >
> > > > > @Phil, I see a couple commits land that seem to address the NPE.
> > > > > Is that correct?
> > > > >
> > > > > I have also seen an IllegalStateException during redirect from
> Admin
> > UI
> > > > to
> > > > > KnoxSSO.
> > > > > Has anyone seen this and/or is working on it - is it related to the
> > > NPE?
> > > > > I don't think it is since I see it more frequently and not always
> > with
> > > > the
> > > > > NPEs.
> > > > >
> > > > > I'd like to get a new RC cut by end of the week, if possible.
> > > > >
> > > > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <lm...@apache.org>
> > > wrote:
> > > > >
> > > > > > Agreed, Phil.
> > > > > > I have cut an RC but we need to address this first. I'll hold
> off
> > on
> > > > > > announcing it.
> > > > > >
> > > > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <pzampino@apache.org
> >
> > > > wrote:
> > > > > >
> > > > > >> During some testing of the proposed 1.1.0 code, I've discovered
> > some
> > > > > NPEs
> > > > > >> in filters (e.g., AclsAuthorizationFilter,
> > > HadoopGroupProviderFilter),
> > > > > >> which are concerning.
> > > > > >>
> > > > > >> I've committed a change to address the AclsAuthorizationFilter,
> > but
> > > > > seeing
> > > > > >> similar behavior for the HadoopGroupProviderFilter has increased
> > my
> > > > > >> concern
> > > > > >> that there may be a more fundamental problem.
> > > > > >> In both cases, it seems that the filters are being invoked prior
> > to
> > > > (or
> > > > > >> during) their respective init() methods have been invoked. Thus,
> > > > members
> > > > > >> which should be initialized in the init() method are not yet
> > > > > initialized.
> > > > > >>
> > > > > >> This can be consistently reproduced, though it is a bit of a
> pain:
> > > > > >>
> > > > > >> - Install Knox (‘ant install-test-home’, or just unzip
> > > > > knox-1.1.0.zip)
> > > > > >> - Start the gateway
> > > > > >> - Access the Admin UI
> > > > > >>
> > > > > >>
> > > > > >> Note that the latest 1.1.0 source has a *fix* for the
> > > > > >> AclsAuthorizationFilter NPE, but master does not yet have this
> > > change.
> > > > > >> This
> > > > > >> is important because that change effectively hides the issue.
> > > > > >>
> > > > > >> I think we should determine what's happening with this before
> > > > > >> producing/testing a release candidate.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <lmccay@apache.org
> >
> > > > wrote:
> > > > > >>
> > > > > >> > All -
> > > > > >> >
> > > > > >> > Sorry for the delay on this topic.
> > > > > >> >
> > > > > >> > We are going to start of this planning thread with ~85
> > Unresolved
> > > > > JIRAs
> > > > > >> in
> > > > > >> > either 1.1.0 or 0.15.0 fixVersion.
> > > > > >> >
> > > > > >> > project = KNOX AND resolution = Unresolved AND fixVersion in
> > > (1.1.0,
> > > > > >> > 0.15.0) ORDER BY priority DESC, updated DESC
> > > > > >> >
> > > > > >> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin
> > with
> > > > and
> > > > > >> then
> > > > > >> > we will need to go through and see what is already taken care
> of
> > > or
> > > > > can
> > > > > >> > wait for a 1.2.0 or later.
> > > > > >> >
> > > > > >> > I also have a couple KIPs in mind to target larger
> > features/themes
> > > > for
> > > > > >> this
> > > > > >> > release.
> > > > > >> >
> > > > > >> > Off the top of my head:
> > > > > >> >
> > > > > >> > * I think we need to address some cloud specific usecases and
> > plan
> > > > to
> > > > > >> > provide a KIP for that. Hybrid cloud/federated knox instances,
> > > Azure
> > > > > AD
> > > > > >> > integration, ID mapping from Hadoop user to IAM users/roles,
> > etc.
> > > > > >> Perhaps
> > > > > >> > some CASB-like features if they make sense.
> > > > > >> >
> > > > > >> > * I also think we need one for articulating a reasonable flow
> > for
> > > > > >> Logout in
> > > > > >> > KnoxSSO. There are a lot of little nuances to logout across
> > > multiple
> > > > > >> apps
> > > > > >> > and between different IDPs. This will require some discussion.
> > > > > >> >
> > > > > >> > * Another thing that has been tugging at my interest has been
> > the
> > > > fact
> > > > > >> that
> > > > > >> > we may be able provide some common libraries to help ecosystem
> > > > > >> applications
> > > > > >> > uptake the trusted proxy pattern and KnoxSSO.
> > > > > >> >
> > > > > >> > Anyway, these are my initial thoughts, please feel free to
> raise
> > > > > >> additional
> > > > > >> > ideas/themes for KIPs, etc.
> > > > > >> >
> > > > > >> > I was thinking that we could try and target an end of March or
> > Mid
> > > > > April
> > > > > >> > 1.1.0 release.
> > > > > >> >
> > > > > >> > Thoughts?
> > > > > >> >
> > > > > >> > --larry
> > > > > >> >
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by larry mccay <lm...@apache.org>.
Awesome - just checked it out and I will kick off a new build shortly!
On Thu, Jul 19, 2018 at 6:01 PM, Sandeep Moré <mo...@gmail.com> wrote:
> Hello Larry,
>
> I committed the fix to master and v1.1.0, it is under the JIRA KNOX-1391
> <https://issues.apache.org/jira/browse/KNOX-1391>.
> we should be good to to cut the RC, provided there are no more issues !
>
> Thanks !
> Sandeep
>
> On Thu, Jul 19, 2018 at 4:25 PM larry mccay <la...@gmail.com> wrote:
>
> > Awesome, @sandeep!
> > I'll keep an eye out.
> >
> > Once that lands, you can bump this thread and I'll cut the RC.
> > Obviously, we will need it in both master and v1.1.0 branches.
> >
> > On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <mo...@gmail.com>
> > wrote:
> >
> > > Hello Larry,
> > >
> > > Yes, I have seen those exceptions, they seem to be happening fairly
> > > consistently and only for KnoxSSO redirects when trying to access admin
> > UI,
> > > I am taking a look at them as we speak, will open up a JIRA for it as
> > well.
> > > It would be good if we can get it in, I will try to get the fix out as
> > soon
> > > as I can.
> > >
> > > Best,
> > > Sandeep
> > >
> > > On Thu, Jul 19, 2018 at 4:15 PM larry mccay <lm...@apache.org> wrote:
> > >
> > > > @Phil, I see a couple commits land that seem to address the NPE.
> > > > Is that correct?
> > > >
> > > > I have also seen an IllegalStateException during redirect from Admin
> UI
> > > to
> > > > KnoxSSO.
> > > > Has anyone seen this and/or is working on it - is it related to the
> > NPE?
> > > > I don't think it is since I see it more frequently and not always
> with
> > > the
> > > > NPEs.
> > > >
> > > > I'd like to get a new RC cut by end of the week, if possible.
> > > >
> > > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <lm...@apache.org>
> > wrote:
> > > >
> > > > > Agreed, Phil.
> > > > > I have cut an RC but we need to address this first. I'll hold off
> on
> > > > > announcing it.
> > > > >
> > > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <pz...@apache.org>
> > > wrote:
> > > > >
> > > > >> During some testing of the proposed 1.1.0 code, I've discovered
> some
> > > > NPEs
> > > > >> in filters (e.g., AclsAuthorizationFilter,
> > HadoopGroupProviderFilter),
> > > > >> which are concerning.
> > > > >>
> > > > >> I've committed a change to address the AclsAuthorizationFilter,
> but
> > > > seeing
> > > > >> similar behavior for the HadoopGroupProviderFilter has increased
> my
> > > > >> concern
> > > > >> that there may be a more fundamental problem.
> > > > >> In both cases, it seems that the filters are being invoked prior
> to
> > > (or
> > > > >> during) their respective init() methods have been invoked. Thus,
> > > members
> > > > >> which should be initialized in the init() method are not yet
> > > > initialized.
> > > > >>
> > > > >> This can be consistently reproduced, though it is a bit of a pain:
> > > > >>
> > > > >> - Install Knox (‘ant install-test-home’, or just unzip
> > > > knox-1.1.0.zip)
> > > > >> - Start the gateway
> > > > >> - Access the Admin UI
> > > > >>
> > > > >>
> > > > >> Note that the latest 1.1.0 source has a *fix* for the
> > > > >> AclsAuthorizationFilter NPE, but master does not yet have this
> > change.
> > > > >> This
> > > > >> is important because that change effectively hides the issue.
> > > > >>
> > > > >> I think we should determine what's happening with this before
> > > > >> producing/testing a release candidate.
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <lm...@apache.org>
> > > wrote:
> > > > >>
> > > > >> > All -
> > > > >> >
> > > > >> > Sorry for the delay on this topic.
> > > > >> >
> > > > >> > We are going to start of this planning thread with ~85
> Unresolved
> > > > JIRAs
> > > > >> in
> > > > >> > either 1.1.0 or 0.15.0 fixVersion.
> > > > >> >
> > > > >> > project = KNOX AND resolution = Unresolved AND fixVersion in
> > (1.1.0,
> > > > >> > 0.15.0) ORDER BY priority DESC, updated DESC
> > > > >> >
> > > > >> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin
> with
> > > and
> > > > >> then
> > > > >> > we will need to go through and see what is already taken care of
> > or
> > > > can
> > > > >> > wait for a 1.2.0 or later.
> > > > >> >
> > > > >> > I also have a couple KIPs in mind to target larger
> features/themes
> > > for
> > > > >> this
> > > > >> > release.
> > > > >> >
> > > > >> > Off the top of my head:
> > > > >> >
> > > > >> > * I think we need to address some cloud specific usecases and
> plan
> > > to
> > > > >> > provide a KIP for that. Hybrid cloud/federated knox instances,
> > Azure
> > > > AD
> > > > >> > integration, ID mapping from Hadoop user to IAM users/roles,
> etc.
> > > > >> Perhaps
> > > > >> > some CASB-like features if they make sense.
> > > > >> >
> > > > >> > * I also think we need one for articulating a reasonable flow
> for
> > > > >> Logout in
> > > > >> > KnoxSSO. There are a lot of little nuances to logout across
> > multiple
> > > > >> apps
> > > > >> > and between different IDPs. This will require some discussion.
> > > > >> >
> > > > >> > * Another thing that has been tugging at my interest has been
> the
> > > fact
> > > > >> that
> > > > >> > we may be able provide some common libraries to help ecosystem
> > > > >> applications
> > > > >> > uptake the trusted proxy pattern and KnoxSSO.
> > > > >> >
> > > > >> > Anyway, these are my initial thoughts, please feel free to raise
> > > > >> additional
> > > > >> > ideas/themes for KIPs, etc.
> > > > >> >
> > > > >> > I was thinking that we could try and target an end of March or
> Mid
> > > > April
> > > > >> > 1.1.0 release.
> > > > >> >
> > > > >> > Thoughts?
> > > > >> >
> > > > >> > --larry
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by Sandeep Moré <mo...@gmail.com>.
Hello Larry,
I committed the fix to master and v1.1.0, it is under the JIRA KNOX-1391
<https://issues.apache.org/jira/browse/KNOX-1391>.
we should be good to to cut the RC, provided there are no more issues !
Thanks !
Sandeep
On Thu, Jul 19, 2018 at 4:25 PM larry mccay <la...@gmail.com> wrote:
> Awesome, @sandeep!
> I'll keep an eye out.
>
> Once that lands, you can bump this thread and I'll cut the RC.
> Obviously, we will need it in both master and v1.1.0 branches.
>
> On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <mo...@gmail.com>
> wrote:
>
> > Hello Larry,
> >
> > Yes, I have seen those exceptions, they seem to be happening fairly
> > consistently and only for KnoxSSO redirects when trying to access admin
> UI,
> > I am taking a look at them as we speak, will open up a JIRA for it as
> well.
> > It would be good if we can get it in, I will try to get the fix out as
> soon
> > as I can.
> >
> > Best,
> > Sandeep
> >
> > On Thu, Jul 19, 2018 at 4:15 PM larry mccay <lm...@apache.org> wrote:
> >
> > > @Phil, I see a couple commits land that seem to address the NPE.
> > > Is that correct?
> > >
> > > I have also seen an IllegalStateException during redirect from Admin UI
> > to
> > > KnoxSSO.
> > > Has anyone seen this and/or is working on it - is it related to the
> NPE?
> > > I don't think it is since I see it more frequently and not always with
> > the
> > > NPEs.
> > >
> > > I'd like to get a new RC cut by end of the week, if possible.
> > >
> > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <lm...@apache.org>
> wrote:
> > >
> > > > Agreed, Phil.
> > > > I have cut an RC but we need to address this first. I'll hold off on
> > > > announcing it.
> > > >
> > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <pz...@apache.org>
> > wrote:
> > > >
> > > >> During some testing of the proposed 1.1.0 code, I've discovered some
> > > NPEs
> > > >> in filters (e.g., AclsAuthorizationFilter,
> HadoopGroupProviderFilter),
> > > >> which are concerning.
> > > >>
> > > >> I've committed a change to address the AclsAuthorizationFilter, but
> > > seeing
> > > >> similar behavior for the HadoopGroupProviderFilter has increased my
> > > >> concern
> > > >> that there may be a more fundamental problem.
> > > >> In both cases, it seems that the filters are being invoked prior to
> > (or
> > > >> during) their respective init() methods have been invoked. Thus,
> > members
> > > >> which should be initialized in the init() method are not yet
> > > initialized.
> > > >>
> > > >> This can be consistently reproduced, though it is a bit of a pain:
> > > >>
> > > >> - Install Knox (‘ant install-test-home’, or just unzip
> > > knox-1.1.0.zip)
> > > >> - Start the gateway
> > > >> - Access the Admin UI
> > > >>
> > > >>
> > > >> Note that the latest 1.1.0 source has a *fix* for the
> > > >> AclsAuthorizationFilter NPE, but master does not yet have this
> change.
> > > >> This
> > > >> is important because that change effectively hides the issue.
> > > >>
> > > >> I think we should determine what's happening with this before
> > > >> producing/testing a release candidate.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <lm...@apache.org>
> > wrote:
> > > >>
> > > >> > All -
> > > >> >
> > > >> > Sorry for the delay on this topic.
> > > >> >
> > > >> > We are going to start of this planning thread with ~85 Unresolved
> > > JIRAs
> > > >> in
> > > >> > either 1.1.0 or 0.15.0 fixVersion.
> > > >> >
> > > >> > project = KNOX AND resolution = Unresolved AND fixVersion in
> (1.1.0,
> > > >> > 0.15.0) ORDER BY priority DESC, updated DESC
> > > >> >
> > > >> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin with
> > and
> > > >> then
> > > >> > we will need to go through and see what is already taken care of
> or
> > > can
> > > >> > wait for a 1.2.0 or later.
> > > >> >
> > > >> > I also have a couple KIPs in mind to target larger features/themes
> > for
> > > >> this
> > > >> > release.
> > > >> >
> > > >> > Off the top of my head:
> > > >> >
> > > >> > * I think we need to address some cloud specific usecases and plan
> > to
> > > >> > provide a KIP for that. Hybrid cloud/federated knox instances,
> Azure
> > > AD
> > > >> > integration, ID mapping from Hadoop user to IAM users/roles, etc.
> > > >> Perhaps
> > > >> > some CASB-like features if they make sense.
> > > >> >
> > > >> > * I also think we need one for articulating a reasonable flow for
> > > >> Logout in
> > > >> > KnoxSSO. There are a lot of little nuances to logout across
> multiple
> > > >> apps
> > > >> > and between different IDPs. This will require some discussion.
> > > >> >
> > > >> > * Another thing that has been tugging at my interest has been the
> > fact
> > > >> that
> > > >> > we may be able provide some common libraries to help ecosystem
> > > >> applications
> > > >> > uptake the trusted proxy pattern and KnoxSSO.
> > > >> >
> > > >> > Anyway, these are my initial thoughts, please feel free to raise
> > > >> additional
> > > >> > ideas/themes for KIPs, etc.
> > > >> >
> > > >> > I was thinking that we could try and target an end of March or Mid
> > > April
> > > >> > 1.1.0 release.
> > > >> >
> > > >> > Thoughts?
> > > >> >
> > > >> > --larry
> > > >> >
> > > >>
> > > >
> > >
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by Sandeep Moré <mo...@gmail.com>.
Cool, will keep the list posted.
On Thu, Jul 19, 2018 at 4:25 PM larry mccay <la...@gmail.com> wrote:
> Awesome, @sandeep!
> I'll keep an eye out.
>
> Once that lands, you can bump this thread and I'll cut the RC.
> Obviously, we will need it in both master and v1.1.0 branches.
>
> On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <mo...@gmail.com>
> wrote:
>
> > Hello Larry,
> >
> > Yes, I have seen those exceptions, they seem to be happening fairly
> > consistently and only for KnoxSSO redirects when trying to access admin
> UI,
> > I am taking a look at them as we speak, will open up a JIRA for it as
> well.
> > It would be good if we can get it in, I will try to get the fix out as
> soon
> > as I can.
> >
> > Best,
> > Sandeep
> >
> > On Thu, Jul 19, 2018 at 4:15 PM larry mccay <lm...@apache.org> wrote:
> >
> > > @Phil, I see a couple commits land that seem to address the NPE.
> > > Is that correct?
> > >
> > > I have also seen an IllegalStateException during redirect from Admin UI
> > to
> > > KnoxSSO.
> > > Has anyone seen this and/or is working on it - is it related to the
> NPE?
> > > I don't think it is since I see it more frequently and not always with
> > the
> > > NPEs.
> > >
> > > I'd like to get a new RC cut by end of the week, if possible.
> > >
> > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <lm...@apache.org>
> wrote:
> > >
> > > > Agreed, Phil.
> > > > I have cut an RC but we need to address this first. I'll hold off on
> > > > announcing it.
> > > >
> > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <pz...@apache.org>
> > wrote:
> > > >
> > > >> During some testing of the proposed 1.1.0 code, I've discovered some
> > > NPEs
> > > >> in filters (e.g., AclsAuthorizationFilter,
> HadoopGroupProviderFilter),
> > > >> which are concerning.
> > > >>
> > > >> I've committed a change to address the AclsAuthorizationFilter, but
> > > seeing
> > > >> similar behavior for the HadoopGroupProviderFilter has increased my
> > > >> concern
> > > >> that there may be a more fundamental problem.
> > > >> In both cases, it seems that the filters are being invoked prior to
> > (or
> > > >> during) their respective init() methods have been invoked. Thus,
> > members
> > > >> which should be initialized in the init() method are not yet
> > > initialized.
> > > >>
> > > >> This can be consistently reproduced, though it is a bit of a pain:
> > > >>
> > > >> - Install Knox (‘ant install-test-home’, or just unzip
> > > knox-1.1.0.zip)
> > > >> - Start the gateway
> > > >> - Access the Admin UI
> > > >>
> > > >>
> > > >> Note that the latest 1.1.0 source has a *fix* for the
> > > >> AclsAuthorizationFilter NPE, but master does not yet have this
> change.
> > > >> This
> > > >> is important because that change effectively hides the issue.
> > > >>
> > > >> I think we should determine what's happening with this before
> > > >> producing/testing a release candidate.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <lm...@apache.org>
> > wrote:
> > > >>
> > > >> > All -
> > > >> >
> > > >> > Sorry for the delay on this topic.
> > > >> >
> > > >> > We are going to start of this planning thread with ~85 Unresolved
> > > JIRAs
> > > >> in
> > > >> > either 1.1.0 or 0.15.0 fixVersion.
> > > >> >
> > > >> > project = KNOX AND resolution = Unresolved AND fixVersion in
> (1.1.0,
> > > >> > 0.15.0) ORDER BY priority DESC, updated DESC
> > > >> >
> > > >> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin with
> > and
> > > >> then
> > > >> > we will need to go through and see what is already taken care of
> or
> > > can
> > > >> > wait for a 1.2.0 or later.
> > > >> >
> > > >> > I also have a couple KIPs in mind to target larger features/themes
> > for
> > > >> this
> > > >> > release.
> > > >> >
> > > >> > Off the top of my head:
> > > >> >
> > > >> > * I think we need to address some cloud specific usecases and plan
> > to
> > > >> > provide a KIP for that. Hybrid cloud/federated knox instances,
> Azure
> > > AD
> > > >> > integration, ID mapping from Hadoop user to IAM users/roles, etc.
> > > >> Perhaps
> > > >> > some CASB-like features if they make sense.
> > > >> >
> > > >> > * I also think we need one for articulating a reasonable flow for
> > > >> Logout in
> > > >> > KnoxSSO. There are a lot of little nuances to logout across
> multiple
> > > >> apps
> > > >> > and between different IDPs. This will require some discussion.
> > > >> >
> > > >> > * Another thing that has been tugging at my interest has been the
> > fact
> > > >> that
> > > >> > we may be able provide some common libraries to help ecosystem
> > > >> applications
> > > >> > uptake the trusted proxy pattern and KnoxSSO.
> > > >> >
> > > >> > Anyway, these are my initial thoughts, please feel free to raise
> > > >> additional
> > > >> > ideas/themes for KIPs, etc.
> > > >> >
> > > >> > I was thinking that we could try and target an end of March or Mid
> > > April
> > > >> > 1.1.0 release.
> > > >> >
> > > >> > Thoughts?
> > > >> >
> > > >> > --larry
> > > >> >
> > > >>
> > > >
> > >
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by larry mccay <la...@gmail.com>.
Awesome, @sandeep!
I'll keep an eye out.
Once that lands, you can bump this thread and I'll cut the RC.
Obviously, we will need it in both master and v1.1.0 branches.
On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré <mo...@gmail.com> wrote:
> Hello Larry,
>
> Yes, I have seen those exceptions, they seem to be happening fairly
> consistently and only for KnoxSSO redirects when trying to access admin UI,
> I am taking a look at them as we speak, will open up a JIRA for it as well.
> It would be good if we can get it in, I will try to get the fix out as soon
> as I can.
>
> Best,
> Sandeep
>
> On Thu, Jul 19, 2018 at 4:15 PM larry mccay <lm...@apache.org> wrote:
>
> > @Phil, I see a couple commits land that seem to address the NPE.
> > Is that correct?
> >
> > I have also seen an IllegalStateException during redirect from Admin UI
> to
> > KnoxSSO.
> > Has anyone seen this and/or is working on it - is it related to the NPE?
> > I don't think it is since I see it more frequently and not always with
> the
> > NPEs.
> >
> > I'd like to get a new RC cut by end of the week, if possible.
> >
> > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <lm...@apache.org> wrote:
> >
> > > Agreed, Phil.
> > > I have cut an RC but we need to address this first. I'll hold off on
> > > announcing it.
> > >
> > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <pz...@apache.org>
> wrote:
> > >
> > >> During some testing of the proposed 1.1.0 code, I've discovered some
> > NPEs
> > >> in filters (e.g., AclsAuthorizationFilter, HadoopGroupProviderFilter),
> > >> which are concerning.
> > >>
> > >> I've committed a change to address the AclsAuthorizationFilter, but
> > seeing
> > >> similar behavior for the HadoopGroupProviderFilter has increased my
> > >> concern
> > >> that there may be a more fundamental problem.
> > >> In both cases, it seems that the filters are being invoked prior to
> (or
> > >> during) their respective init() methods have been invoked. Thus,
> members
> > >> which should be initialized in the init() method are not yet
> > initialized.
> > >>
> > >> This can be consistently reproduced, though it is a bit of a pain:
> > >>
> > >> - Install Knox (‘ant install-test-home’, or just unzip
> > knox-1.1.0.zip)
> > >> - Start the gateway
> > >> - Access the Admin UI
> > >>
> > >>
> > >> Note that the latest 1.1.0 source has a *fix* for the
> > >> AclsAuthorizationFilter NPE, but master does not yet have this change.
> > >> This
> > >> is important because that change effectively hides the issue.
> > >>
> > >> I think we should determine what's happening with this before
> > >> producing/testing a release candidate.
> > >>
> > >>
> > >>
> > >>
> > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <lm...@apache.org>
> wrote:
> > >>
> > >> > All -
> > >> >
> > >> > Sorry for the delay on this topic.
> > >> >
> > >> > We are going to start of this planning thread with ~85 Unresolved
> > JIRAs
> > >> in
> > >> > either 1.1.0 or 0.15.0 fixVersion.
> > >> >
> > >> > project = KNOX AND resolution = Unresolved AND fixVersion in (1.1.0,
> > >> > 0.15.0) ORDER BY priority DESC, updated DESC
> > >> >
> > >> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin with
> and
> > >> then
> > >> > we will need to go through and see what is already taken care of or
> > can
> > >> > wait for a 1.2.0 or later.
> > >> >
> > >> > I also have a couple KIPs in mind to target larger features/themes
> for
> > >> this
> > >> > release.
> > >> >
> > >> > Off the top of my head:
> > >> >
> > >> > * I think we need to address some cloud specific usecases and plan
> to
> > >> > provide a KIP for that. Hybrid cloud/federated knox instances, Azure
> > AD
> > >> > integration, ID mapping from Hadoop user to IAM users/roles, etc.
> > >> Perhaps
> > >> > some CASB-like features if they make sense.
> > >> >
> > >> > * I also think we need one for articulating a reasonable flow for
> > >> Logout in
> > >> > KnoxSSO. There are a lot of little nuances to logout across multiple
> > >> apps
> > >> > and between different IDPs. This will require some discussion.
> > >> >
> > >> > * Another thing that has been tugging at my interest has been the
> fact
> > >> that
> > >> > we may be able provide some common libraries to help ecosystem
> > >> applications
> > >> > uptake the trusted proxy pattern and KnoxSSO.
> > >> >
> > >> > Anyway, these are my initial thoughts, please feel free to raise
> > >> additional
> > >> > ideas/themes for KIPs, etc.
> > >> >
> > >> > I was thinking that we could try and target an end of March or Mid
> > April
> > >> > 1.1.0 release.
> > >> >
> > >> > Thoughts?
> > >> >
> > >> > --larry
> > >> >
> > >>
> > >
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by Sandeep Moré <mo...@gmail.com>.
Hello Larry,
Yes, I have seen those exceptions, they seem to be happening fairly
consistently and only for KnoxSSO redirects when trying to access admin UI,
I am taking a look at them as we speak, will open up a JIRA for it as well.
It would be good if we can get it in, I will try to get the fix out as soon
as I can.
Best,
Sandeep
On Thu, Jul 19, 2018 at 4:15 PM larry mccay <lm...@apache.org> wrote:
> @Phil, I see a couple commits land that seem to address the NPE.
> Is that correct?
>
> I have also seen an IllegalStateException during redirect from Admin UI to
> KnoxSSO.
> Has anyone seen this and/or is working on it - is it related to the NPE?
> I don't think it is since I see it more frequently and not always with the
> NPEs.
>
> I'd like to get a new RC cut by end of the week, if possible.
>
> On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <lm...@apache.org> wrote:
>
> > Agreed, Phil.
> > I have cut an RC but we need to address this first. I'll hold off on
> > announcing it.
> >
> > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <pz...@apache.org> wrote:
> >
> >> During some testing of the proposed 1.1.0 code, I've discovered some
> NPEs
> >> in filters (e.g., AclsAuthorizationFilter, HadoopGroupProviderFilter),
> >> which are concerning.
> >>
> >> I've committed a change to address the AclsAuthorizationFilter, but
> seeing
> >> similar behavior for the HadoopGroupProviderFilter has increased my
> >> concern
> >> that there may be a more fundamental problem.
> >> In both cases, it seems that the filters are being invoked prior to (or
> >> during) their respective init() methods have been invoked. Thus, members
> >> which should be initialized in the init() method are not yet
> initialized.
> >>
> >> This can be consistently reproduced, though it is a bit of a pain:
> >>
> >> - Install Knox (‘ant install-test-home’, or just unzip
> knox-1.1.0.zip)
> >> - Start the gateway
> >> - Access the Admin UI
> >>
> >>
> >> Note that the latest 1.1.0 source has a *fix* for the
> >> AclsAuthorizationFilter NPE, but master does not yet have this change.
> >> This
> >> is important because that change effectively hides the issue.
> >>
> >> I think we should determine what's happening with this before
> >> producing/testing a release candidate.
> >>
> >>
> >>
> >>
> >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <lm...@apache.org> wrote:
> >>
> >> > All -
> >> >
> >> > Sorry for the delay on this topic.
> >> >
> >> > We are going to start of this planning thread with ~85 Unresolved
> JIRAs
> >> in
> >> > either 1.1.0 or 0.15.0 fixVersion.
> >> >
> >> > project = KNOX AND resolution = Unresolved AND fixVersion in (1.1.0,
> >> > 0.15.0) ORDER BY priority DESC, updated DESC
> >> >
> >> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin with and
> >> then
> >> > we will need to go through and see what is already taken care of or
> can
> >> > wait for a 1.2.0 or later.
> >> >
> >> > I also have a couple KIPs in mind to target larger features/themes for
> >> this
> >> > release.
> >> >
> >> > Off the top of my head:
> >> >
> >> > * I think we need to address some cloud specific usecases and plan to
> >> > provide a KIP for that. Hybrid cloud/federated knox instances, Azure
> AD
> >> > integration, ID mapping from Hadoop user to IAM users/roles, etc.
> >> Perhaps
> >> > some CASB-like features if they make sense.
> >> >
> >> > * I also think we need one for articulating a reasonable flow for
> >> Logout in
> >> > KnoxSSO. There are a lot of little nuances to logout across multiple
> >> apps
> >> > and between different IDPs. This will require some discussion.
> >> >
> >> > * Another thing that has been tugging at my interest has been the fact
> >> that
> >> > we may be able provide some common libraries to help ecosystem
> >> applications
> >> > uptake the trusted proxy pattern and KnoxSSO.
> >> >
> >> > Anyway, these are my initial thoughts, please feel free to raise
> >> additional
> >> > ideas/themes for KIPs, etc.
> >> >
> >> > I was thinking that we could try and target an end of March or Mid
> April
> >> > 1.1.0 release.
> >> >
> >> > Thoughts?
> >> >
> >> > --larry
> >> >
> >>
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by larry mccay <lm...@apache.org>.
@Phil, I see a couple commits land that seem to address the NPE.
Is that correct?
I have also seen an IllegalStateException during redirect from Admin UI to
KnoxSSO.
Has anyone seen this and/or is working on it - is it related to the NPE?
I don't think it is since I see it more frequently and not always with the
NPEs.
I'd like to get a new RC cut by end of the week, if possible.
On Fri, Jul 13, 2018 at 7:57 PM, larry mccay <lm...@apache.org> wrote:
> Agreed, Phil.
> I have cut an RC but we need to address this first. I'll hold off on
> announcing it.
>
> On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <pz...@apache.org> wrote:
>
>> During some testing of the proposed 1.1.0 code, I've discovered some NPEs
>> in filters (e.g., AclsAuthorizationFilter, HadoopGroupProviderFilter),
>> which are concerning.
>>
>> I've committed a change to address the AclsAuthorizationFilter, but seeing
>> similar behavior for the HadoopGroupProviderFilter has increased my
>> concern
>> that there may be a more fundamental problem.
>> In both cases, it seems that the filters are being invoked prior to (or
>> during) their respective init() methods have been invoked. Thus, members
>> which should be initialized in the init() method are not yet initialized.
>>
>> This can be consistently reproduced, though it is a bit of a pain:
>>
>> - Install Knox (‘ant install-test-home’, or just unzip knox-1.1.0.zip)
>> - Start the gateway
>> - Access the Admin UI
>>
>>
>> Note that the latest 1.1.0 source has a *fix* for the
>> AclsAuthorizationFilter NPE, but master does not yet have this change.
>> This
>> is important because that change effectively hides the issue.
>>
>> I think we should determine what's happening with this before
>> producing/testing a release candidate.
>>
>>
>>
>>
>> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <lm...@apache.org> wrote:
>>
>> > All -
>> >
>> > Sorry for the delay on this topic.
>> >
>> > We are going to start of this planning thread with ~85 Unresolved JIRAs
>> in
>> > either 1.1.0 or 0.15.0 fixVersion.
>> >
>> > project = KNOX AND resolution = Unresolved AND fixVersion in (1.1.0,
>> > 0.15.0) ORDER BY priority DESC, updated DESC
>> >
>> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin with and
>> then
>> > we will need to go through and see what is already taken care of or can
>> > wait for a 1.2.0 or later.
>> >
>> > I also have a couple KIPs in mind to target larger features/themes for
>> this
>> > release.
>> >
>> > Off the top of my head:
>> >
>> > * I think we need to address some cloud specific usecases and plan to
>> > provide a KIP for that. Hybrid cloud/federated knox instances, Azure AD
>> > integration, ID mapping from Hadoop user to IAM users/roles, etc.
>> Perhaps
>> > some CASB-like features if they make sense.
>> >
>> > * I also think we need one for articulating a reasonable flow for
>> Logout in
>> > KnoxSSO. There are a lot of little nuances to logout across multiple
>> apps
>> > and between different IDPs. This will require some discussion.
>> >
>> > * Another thing that has been tugging at my interest has been the fact
>> that
>> > we may be able provide some common libraries to help ecosystem
>> applications
>> > uptake the trusted proxy pattern and KnoxSSO.
>> >
>> > Anyway, these are my initial thoughts, please feel free to raise
>> additional
>> > ideas/themes for KIPs, etc.
>> >
>> > I was thinking that we could try and target an end of March or Mid April
>> > 1.1.0 release.
>> >
>> > Thoughts?
>> >
>> > --larry
>> >
>>
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by larry mccay <lm...@apache.org>.
Agreed, Phil.
I have cut an RC but we need to address this first. I'll hold off on
announcing it.
On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <pz...@apache.org> wrote:
> During some testing of the proposed 1.1.0 code, I've discovered some NPEs
> in filters (e.g., AclsAuthorizationFilter, HadoopGroupProviderFilter),
> which are concerning.
>
> I've committed a change to address the AclsAuthorizationFilter, but seeing
> similar behavior for the HadoopGroupProviderFilter has increased my concern
> that there may be a more fundamental problem.
> In both cases, it seems that the filters are being invoked prior to (or
> during) their respective init() methods have been invoked. Thus, members
> which should be initialized in the init() method are not yet initialized.
>
> This can be consistently reproduced, though it is a bit of a pain:
>
> - Install Knox (‘ant install-test-home’, or just unzip knox-1.1.0.zip)
> - Start the gateway
> - Access the Admin UI
>
>
> Note that the latest 1.1.0 source has a *fix* for the
> AclsAuthorizationFilter NPE, but master does not yet have this change. This
> is important because that change effectively hides the issue.
>
> I think we should determine what's happening with this before
> producing/testing a release candidate.
>
>
>
>
> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <lm...@apache.org> wrote:
>
> > All -
> >
> > Sorry for the delay on this topic.
> >
> > We are going to start of this planning thread with ~85 Unresolved JIRAs
> in
> > either 1.1.0 or 0.15.0 fixVersion.
> >
> > project = KNOX AND resolution = Unresolved AND fixVersion in (1.1.0,
> > 0.15.0) ORDER BY priority DESC, updated DESC
> >
> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin with and
> then
> > we will need to go through and see what is already taken care of or can
> > wait for a 1.2.0 or later.
> >
> > I also have a couple KIPs in mind to target larger features/themes for
> this
> > release.
> >
> > Off the top of my head:
> >
> > * I think we need to address some cloud specific usecases and plan to
> > provide a KIP for that. Hybrid cloud/federated knox instances, Azure AD
> > integration, ID mapping from Hadoop user to IAM users/roles, etc. Perhaps
> > some CASB-like features if they make sense.
> >
> > * I also think we need one for articulating a reasonable flow for Logout
> in
> > KnoxSSO. There are a lot of little nuances to logout across multiple apps
> > and between different IDPs. This will require some discussion.
> >
> > * Another thing that has been tugging at my interest has been the fact
> that
> > we may be able provide some common libraries to help ecosystem
> applications
> > uptake the trusted proxy pattern and KnoxSSO.
> >
> > Anyway, these are my initial thoughts, please feel free to raise
> additional
> > ideas/themes for KIPs, etc.
> >
> > I was thinking that we could try and target an end of March or Mid April
> > 1.1.0 release.
> >
> > Thoughts?
> >
> > --larry
> >
>
Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release
Posted by larry mccay <lm...@apache.org>.
Agreed, Phil.
I have cut an RC but we need to address this first. I'll hold off on
announcing it.
On Fri, Jul 13, 2018, 11:36 AM Phil Zampino <pz...@apache.org> wrote:
> During some testing of the proposed 1.1.0 code, I've discovered some NPEs
> in filters (e.g., AclsAuthorizationFilter, HadoopGroupProviderFilter),
> which are concerning.
>
> I've committed a change to address the AclsAuthorizationFilter, but seeing
> similar behavior for the HadoopGroupProviderFilter has increased my concern
> that there may be a more fundamental problem.
> In both cases, it seems that the filters are being invoked prior to (or
> during) their respective init() methods have been invoked. Thus, members
> which should be initialized in the init() method are not yet initialized.
>
> This can be consistently reproduced, though it is a bit of a pain:
>
> - Install Knox (‘ant install-test-home’, or just unzip knox-1.1.0.zip)
> - Start the gateway
> - Access the Admin UI
>
>
> Note that the latest 1.1.0 source has a *fix* for the
> AclsAuthorizationFilter NPE, but master does not yet have this change. This
> is important because that change effectively hides the issue.
>
> I think we should determine what's happening with this before
> producing/testing a release candidate.
>
>
>
>
> On Sat, Feb 24, 2018 at 12:57 PM larry mccay <lm...@apache.org> wrote:
>
> > All -
> >
> > Sorry for the delay on this topic.
> >
> > We are going to start of this planning thread with ~85 Unresolved JIRAs
> in
> > either 1.1.0 or 0.15.0 fixVersion.
> >
> > project = KNOX AND resolution = Unresolved AND fixVersion in (1.1.0,
> > 0.15.0) ORDER BY priority DESC, updated DESC
> >
> > I will spend some time migrating all 0.15.0 to 1.1.0 to begin with and
> then
> > we will need to go through and see what is already taken care of or can
> > wait for a 1.2.0 or later.
> >
> > I also have a couple KIPs in mind to target larger features/themes for
> this
> > release.
> >
> > Off the top of my head:
> >
> > * I think we need to address some cloud specific usecases and plan to
> > provide a KIP for that. Hybrid cloud/federated knox instances, Azure AD
> > integration, ID mapping from Hadoop user to IAM users/roles, etc. Perhaps
> > some CASB-like features if they make sense.
> >
> > * I also think we need one for articulating a reasonable flow for Logout
> in
> > KnoxSSO. There are a lot of little nuances to logout across multiple apps
> > and between different IDPs. This will require some discussion.
> >
> > * Another thing that has been tugging at my interest has been the fact
> that
> > we may be able provide some common libraries to help ecosystem
> applications
> > uptake the trusted proxy pattern and KnoxSSO.
> >
> > Anyway, these are my initial thoughts, please feel free to raise
> additional
> > ideas/themes for KIPs, etc.
> >
> > I was thinking that we could try and target an end of March or Mid April
> > 1.1.0 release.
> >
> > Thoughts?
> >
> > --larry
> >
>