[DISCUSS] Preparing for 0.7.0 Release

[larry mccay <lm...@apache.org>]

Knox dev's -

We need to start locking down the release for 0.7.0.
In preparation of this, Sumit created a branch a week or so ago and we
should start considering the creation of a release candidate.

I believe that I have to update the CHANGES file with an entry for a patch
that I cherry picked into 0.7.0 branch and I will look into that shortly.

Standout features include: KnoxSSO for WebSSO, HA support for numerous
services, diagnostic commands for KnoxCLI, regex based identity
assertion, better control over thread pool, connection queue and
request/response buffers. The ability to proxy Hadoop UIs, CORS support for
cross origin request sharing and more. As well as a number of important bug
fixes.

We do have an important feature coming from the community - specifically
from Jérôme that will be committed in coming days. KNOX-641 adds a
federation provider that integrates pac4j in order to add: OAuth, Facebook,
CAS, SAML, OpenID Connect. I think that this is an exciting integration
that will require a bit of testing before it can be merged into a release
branch.

In my opinion, the set of features and improvements that are currently in
the v0.7.0 branch more than justify a new release and delaying that any
longer would be less than ideal.

Concentrating on defining and testing the usecases that the pac4j provider
will bring to the table post 0.7.0 and coming up with a compelling story
for that feature set can be used to justify a release of its own. I think
that we should target a feature release which we'll call 0.8.0 for now for
a mid January timeframe.

So, discussion points:

1. Should we move forward with the 0.7.0 release once the CHANGES file is
updated?
2. Thoughts on holding the pac4j provider out until an early 2016 release
when the main usecases are better defined and tested?

thanks,

--larry

Re: [DISCUSS] Preparing for 0.7.0 Release

[Dilli Dorai <di...@gmail.com>]

Vow.
A lot of cool and major features have been done.
Definitely time for release mile stone.
Thanks
Dilli

On Tue, Dec 15, 2015 at 4:57 PM, larry mccay <la...@gmail.com> wrote:

> Okay - CHANGES has been updated in both master and v0.7.0.
>
> We should be ready for a release candidate now.
>
> On Tue, Dec 15, 2015 at 4:29 PM, larry mccay <lm...@apache.org> wrote:
>
> > I will take on the task of merging the lists and prepare a patch for that
> > immediately.
> >
> > On Tue, Dec 15, 2015 at 4:27 PM, Kevin Minder <
> > kevin.minder@hortonworks.com> wrote:
> >
> >> I’m in favor of continuing to stabilize the 0.7.0 branch with the
> current
> >> bits we have and then doing an 0.8.0 with the new KNOX-641 stuff as
> quickly
> >> as possible after that.
> >> KNOX-641 ends up providing a wonderful new big feature set and we are
> >> going to need to bandwidth to learn/absorb it.
> >>
> >> BTW here is my take on all of the commits from the branch point for
> >> 0.6.0.  Seems we are getting better with our CHANGES discipline but
> there
> >> is still a great deal of room for improvement.  The CHANGES file has ~30
> >> entries for 0.7.0 and the list below has about ~90 entries.
> >>
> >> [KNOX-639] - Knoxcli.sh create-master should not allow empty strings
> >> KNOX-640 - Make Cookie Domain Configurable
> >> [KNOX-638] - Hive dispatch failing for secure clusters
> >> KNOX-626 Minor fix to namespace parsing
> >> KNOX-637 - Compilation Error in gateway-service-admin and gateway-test
> >> test projects (arshad.mohammad via lmccay)
> >> KNOX-636 - IdentityAsserterHttpServletRequestWrapper must override
> >> getUserPrincipal
> >> KNOX-635 - open up default whitelist for dev - localhost
> >> KNOX-635 - Provide Whitelisting for Redirect Destinations for KnoxSSO
> >> KNOX-634 - CORS Support as Part of WebAppSec Provider
> >> KNOX-632 added back configuration for 'replayBufferSize'
> >> KNOX-633: Upgrade apache commons-collections
> >> KNOX-632: Oozie dispatch failing for secure clusters. Fix tests.
> >> KNOX-632: Oozie dispatch failing for secure clusters
> >> KNOX-625 initial template file for topology using ui proxy services
> >> KNOX-623: Gateway provider rewriter doesn't support boolean attributes
> in
> >> HTML.
> >> KNOX-622 - Misconfigured providers should cause topology deployment to
> >> fail
> >> KNOX-624: Expose configuration for Jetty's request and response buffer
> >> sizes. Fix property names.
> >> KNOX-624: Expose configuration for Jetty's request and response buffer
> >> sizes
> >> KNOX-621 - Simplify KnoxSSO API Resource Path
> >> KNOX-620: Jenkins Knox-master-verify failing since #725 due to JDK
> >> version issues
> >> KNOX-394: Request and response URLs must be parsed as literals not
> >> templates. Part 2.
> >> KNOX-394: Request and response URLs must be parsed as literals not
> >> templates
> >> KNOX-617 - Add the use of CredentialCollectors to Samples
> >> KNOX-616: XmlUrlRewriteStreamFilter unscapes escaped special characters
> >> KNOX-611: Expose configuration for Jetty's thread pool and connection
> >> queue
> >> KNOX-604: Expose configuration of HttpClient's max connections per route
> >> setting
> >> KNOX-614: Incorrect URI template expansion with {**} query params
> >> #fragments
> >> KNOX-615 Domain Cookies cannot Wildcard IP Addresses
> >> KNOX-613 - Provide Credential Collector Abstraction to Client Shell
> >> KNOX-610 - DefaultTokenService issueToken should never return null
> >> KNOX-609 - Add unit tests for the SSOCookieFederationProvider.
> >> KNOX-608: Improve Knox read and write performance by tuning buffer
> sizes.
> >> KNOX-607 - Fix SSOCookieProvider to Handle null Query Strings
> >> KNOX-602 - protect against NPE in audience validation
> >> KNOX-603: Coverity: Potential resource leak in
> >> BaseKeystoreService.createKeystore
> >> KNOX-602 JWT/SSO Cookie Based Federation Provider
> >> KNOX-601: Knox test failures on windows
> >> KNOX-600 setting all service params as filter params for dispatch
> >> KNOX-593 removed replayBufferSize and CappedBufferHttpEntity references
> >> KNOX-447: Incorrect parsing and expansion of valueless query params
> >> KNOX-599: Template with {**} in queries are expanded with =null for
> query
> >> params without a value
> >> KNOX-598: Concurrent JDBC clients via KNOX to Kerberized HiveServer2
> >> causes HTTP 401 error (due to Kerberos
> >> KNOX-570 added zookeeper lookup capability for HS2 HA
> >> KNOX-596: Add diagnostics to topology depoloyment
> >> KNOX-597: Improve diagnostic logging of HTTP traffic. Update CHANGES.
> >> KNOX-597: Improve diagnostic logging of HTTP traffic
> >> KNOX-593 Moved SPNEGO code to httpclient
> >> KNOX-584 Fix for UT instability in
> GatewayBasicFuncTest.testCLIServiceTest
> >> KNOX-590 - add more ShiroProvider configuration support to KnoxCLI
> >> sys-user-auth-test and user-auth-test
> >> KNOX-582 Query Parameter rewrite does not honor empty string value
> >> (jeffreyr via lmccay)
> >> KNOX-581: Hive dispatch not propagating effective principal name
> >> KNOX-580 Initial refactoring out of default HA dispatch
> >> KNOX-579: Regex based identity assertion provider with static dictionary
> >> lookup
> >> KNOX-576: CLI user-auth-test should print a message when a user
> >> successfully authenticates.
> >> KNOX-565: Supporting All the Quick Links on Ambari Dashboard to Go
> >> Through Knox
> >> KNOX-564: NPE for Topology with no Providers Confgured
> >> KNOX-575: Add more logging for LDAP Authentication issues with
> >> ShiroProvider
> >> KNOX-573: KNOX-574 make SecureOnly and MaxAge configurable for SSO
> >> KNOX-549: Test service connections through Knox with Knox CLI
> >> KNOX-566 - Make the Default Ephemeral DH Key Size 2048 for TLS
> >> KNOX-460: UrlRewriteServletFilterTest failed with IBM JAVA
> >> KNOX-423: XmlFilterReaderTest failed with IBM JVM JAVA
> >> KNOX-548: LDAP Bind in Knox CLI. Fixed help usage.
> >> KNOX-562: Fix Null pointer exceptions in KnoxCLI LDAP commands
> >> KNOX-548: KnoxCLI adds a new system-user-auth-test command to test a
> >> topology's system username and password
> >> KNOX-560: Test LDAP Authentication+Authorization from KnoxCLI
> >> KNOX-561: Allow Knox pid directory to be configured via the knox-env.sh
> >> file
> >> KNOX-559 renaming service definition files
> >> KNOX-558: HttpClient connections are not always returned to the pool for
> >> HBase on Windows
> >> KNOX-554: Cannot access topologies through admin API if gateway.path is
> >> modified
> >> KNOX-556 - fix extraneous imports
> >> KNOX-556 - provide better diagnostics for keystore failures
> >> KNOX-555: Prevent dispatch client from attempting retry and redirects
> >> KNOX-553: Added topology validation from KnoxCLI to TopologyService
> >> deployment.
> >> KNOX-547: Topology Validation in Knox CLI. Fix schema load from JAR
> >> NullPointerException
> >> KNOX-547: Topology Validation in Knox CLI
> >> KNOX-550 reverting back to original hive kerberos dispatch behavior
> >> KNOX-546 Consuming intermediate response during kerberos request
> >> dispatching
> >> KNOX-545 - Simplify Keystore Management for Cluster Scaleout
> >> KNOX-544: Knox process does not exit if startup fails due to credential
> >> store issues
> >> KNOX-476 implementation for X-Forwarded-* headers support and population
> >> KNOX-539 add message to identity mapping audit entries
> >> KNOX-538: Log some important system properties at startup
> >> KNOX-534 auditing shiro authentication exceptions
> >> KNOX-533 - add version component to knoxsso url pattern
> >> KNOX-291: Improve audit for topology deployment process
> >> KNOX-532: Update root pom.xml maven-compiler-plugin configuration.
> >> KNOX-531 fix extraneous audit entries and add additional principal
> >> mapping test
> >> KNOX-529 - second attempt to get all usecases - missed wildcard plus
> >> explicit mappings before
> >> KNOX-530 fixed oozie rewrite rules to handle missing port information
> >> KNOX-529 - Fix wildcard based principal group mapping
> >>
> >>
> >>
> >>
> >>
> >>
> >> On 12/15/15, 3:11 PM, "larry mccay" <lm...@apache.org> wrote:
> >>
> >> >Knox dev's -
> >> >
> >> >We need to start locking down the release for 0.7.0.
> >> >In preparation of this, Sumit created a branch a week or so ago and we
> >> >should start considering the creation of a release candidate.
> >> >
> >> >I believe that I have to update the CHANGES file with an entry for a
> >> patch
> >> >that I cherry picked into 0.7.0 branch and I will look into that
> shortly.
> >> >
> >> >Standout features include: KnoxSSO for WebSSO, HA support for numerous
> >> >services, diagnostic commands for KnoxCLI, regex based identity
> >> >assertion, better control over thread pool, connection queue and
> >> >request/response buffers. The ability to proxy Hadoop UIs, CORS support
> >> for
> >> >cross origin request sharing and more. As well as a number of important
> >> bug
> >> >fixes.
> >> >
> >> >We do have an important feature coming from the community -
> specifically
> >> >from Jérôme that will be committed in coming days. KNOX-641 adds a
> >> >federation provider that integrates pac4j in order to add: OAuth,
> >> Facebook,
> >> >CAS, SAML, OpenID Connect. I think that this is an exciting integration
> >> >that will require a bit of testing before it can be merged into a
> release
> >> >branch.
> >> >
> >> >In my opinion, the set of features and improvements that are currently
> in
> >> >the v0.7.0 branch more than justify a new release and delaying that any
> >> >longer would be less than ideal.
> >> >
> >> >Concentrating on defining and testing the usecases that the pac4j
> >> provider
> >> >will bring to the table post 0.7.0 and coming up with a compelling
> story
> >> >for that feature set can be used to justify a release of its own. I
> think
> >> >that we should target a feature release which we'll call 0.8.0 for now
> >> for
> >> >a mid January timeframe.
> >> >
> >> >So, discussion points:
> >> >
> >> >1. Should we move forward with the 0.7.0 release once the CHANGES file
> is
> >> >updated?
> >> >2. Thoughts on holding the pac4j provider out until an early 2016
> release
> >> >when the main usecases are better defined and tested?
> >> >
> >> >thanks,
> >> >
> >> >--larry
> >>
> >
> >
>

Re: [DISCUSS] Preparing for 0.7.0 Release

[larry mccay <la...@gmail.com>]

Okay - CHANGES has been updated in both master and v0.7.0.

We should be ready for a release candidate now.

On Tue, Dec 15, 2015 at 4:29 PM, larry mccay <lm...@apache.org> wrote:

> I will take on the task of merging the lists and prepare a patch for that
> immediately.
>
> On Tue, Dec 15, 2015 at 4:27 PM, Kevin Minder <
> kevin.minder@hortonworks.com> wrote:
>
>> I’m in favor of continuing to stabilize the 0.7.0 branch with the current
>> bits we have and then doing an 0.8.0 with the new KNOX-641 stuff as quickly
>> as possible after that.
>> KNOX-641 ends up providing a wonderful new big feature set and we are
>> going to need to bandwidth to learn/absorb it.
>>
>> BTW here is my take on all of the commits from the branch point for
>> 0.6.0.  Seems we are getting better with our CHANGES discipline but there
>> is still a great deal of room for improvement.  The CHANGES file has ~30
>> entries for 0.7.0 and the list below has about ~90 entries.
>>
>> [KNOX-639] - Knoxcli.sh create-master should not allow empty strings
>> KNOX-640 - Make Cookie Domain Configurable
>> [KNOX-638] - Hive dispatch failing for secure clusters
>> KNOX-626 Minor fix to namespace parsing
>> KNOX-637 - Compilation Error in gateway-service-admin and gateway-test
>> test projects (arshad.mohammad via lmccay)
>> KNOX-636 - IdentityAsserterHttpServletRequestWrapper must override
>> getUserPrincipal
>> KNOX-635 - open up default whitelist for dev - localhost
>> KNOX-635 - Provide Whitelisting for Redirect Destinations for KnoxSSO
>> KNOX-634 - CORS Support as Part of WebAppSec Provider
>> KNOX-632 added back configuration for 'replayBufferSize'
>> KNOX-633: Upgrade apache commons-collections
>> KNOX-632: Oozie dispatch failing for secure clusters. Fix tests.
>> KNOX-632: Oozie dispatch failing for secure clusters
>> KNOX-625 initial template file for topology using ui proxy services
>> KNOX-623: Gateway provider rewriter doesn't support boolean attributes in
>> HTML.
>> KNOX-622 - Misconfigured providers should cause topology deployment to
>> fail
>> KNOX-624: Expose configuration for Jetty's request and response buffer
>> sizes. Fix property names.
>> KNOX-624: Expose configuration for Jetty's request and response buffer
>> sizes
>> KNOX-621 - Simplify KnoxSSO API Resource Path
>> KNOX-620: Jenkins Knox-master-verify failing since #725 due to JDK
>> version issues
>> KNOX-394: Request and response URLs must be parsed as literals not
>> templates. Part 2.
>> KNOX-394: Request and response URLs must be parsed as literals not
>> templates
>> KNOX-617 - Add the use of CredentialCollectors to Samples
>> KNOX-616: XmlUrlRewriteStreamFilter unscapes escaped special characters
>> KNOX-611: Expose configuration for Jetty's thread pool and connection
>> queue
>> KNOX-604: Expose configuration of HttpClient's max connections per route
>> setting
>> KNOX-614: Incorrect URI template expansion with {**} query params
>> #fragments
>> KNOX-615 Domain Cookies cannot Wildcard IP Addresses
>> KNOX-613 - Provide Credential Collector Abstraction to Client Shell
>> KNOX-610 - DefaultTokenService issueToken should never return null
>> KNOX-609 - Add unit tests for the SSOCookieFederationProvider.
>> KNOX-608: Improve Knox read and write performance by tuning buffer sizes.
>> KNOX-607 - Fix SSOCookieProvider to Handle null Query Strings
>> KNOX-602 - protect against NPE in audience validation
>> KNOX-603: Coverity: Potential resource leak in
>> BaseKeystoreService.createKeystore
>> KNOX-602 JWT/SSO Cookie Based Federation Provider
>> KNOX-601: Knox test failures on windows
>> KNOX-600 setting all service params as filter params for dispatch
>> KNOX-593 removed replayBufferSize and CappedBufferHttpEntity references
>> KNOX-447: Incorrect parsing and expansion of valueless query params
>> KNOX-599: Template with {**} in queries are expanded with =null for query
>> params without a value
>> KNOX-598: Concurrent JDBC clients via KNOX to Kerberized HiveServer2
>> causes HTTP 401 error (due to Kerberos
>> KNOX-570 added zookeeper lookup capability for HS2 HA
>> KNOX-596: Add diagnostics to topology depoloyment
>> KNOX-597: Improve diagnostic logging of HTTP traffic. Update CHANGES.
>> KNOX-597: Improve diagnostic logging of HTTP traffic
>> KNOX-593 Moved SPNEGO code to httpclient
>> KNOX-584 Fix for UT instability in GatewayBasicFuncTest.testCLIServiceTest
>> KNOX-590 - add more ShiroProvider configuration support to KnoxCLI
>> sys-user-auth-test and user-auth-test
>> KNOX-582 Query Parameter rewrite does not honor empty string value
>> (jeffreyr via lmccay)
>> KNOX-581: Hive dispatch not propagating effective principal name
>> KNOX-580 Initial refactoring out of default HA dispatch
>> KNOX-579: Regex based identity assertion provider with static dictionary
>> lookup
>> KNOX-576: CLI user-auth-test should print a message when a user
>> successfully authenticates.
>> KNOX-565: Supporting All the Quick Links on Ambari Dashboard to Go
>> Through Knox
>> KNOX-564: NPE for Topology with no Providers Confgured
>> KNOX-575: Add more logging for LDAP Authentication issues with
>> ShiroProvider
>> KNOX-573: KNOX-574 make SecureOnly and MaxAge configurable for SSO
>> KNOX-549: Test service connections through Knox with Knox CLI
>> KNOX-566 - Make the Default Ephemeral DH Key Size 2048 for TLS
>> KNOX-460: UrlRewriteServletFilterTest failed with IBM JAVA
>> KNOX-423: XmlFilterReaderTest failed with IBM JVM JAVA
>> KNOX-548: LDAP Bind in Knox CLI. Fixed help usage.
>> KNOX-562: Fix Null pointer exceptions in KnoxCLI LDAP commands
>> KNOX-548: KnoxCLI adds a new system-user-auth-test command to test a
>> topology's system username and password
>> KNOX-560: Test LDAP Authentication+Authorization from KnoxCLI
>> KNOX-561: Allow Knox pid directory to be configured via the knox-env.sh
>> file
>> KNOX-559 renaming service definition files
>> KNOX-558: HttpClient connections are not always returned to the pool for
>> HBase on Windows
>> KNOX-554: Cannot access topologies through admin API if gateway.path is
>> modified
>> KNOX-556 - fix extraneous imports
>> KNOX-556 - provide better diagnostics for keystore failures
>> KNOX-555: Prevent dispatch client from attempting retry and redirects
>> KNOX-553: Added topology validation from KnoxCLI to TopologyService
>> deployment.
>> KNOX-547: Topology Validation in Knox CLI. Fix schema load from JAR
>> NullPointerException
>> KNOX-547: Topology Validation in Knox CLI
>> KNOX-550 reverting back to original hive kerberos dispatch behavior
>> KNOX-546 Consuming intermediate response during kerberos request
>> dispatching
>> KNOX-545 - Simplify Keystore Management for Cluster Scaleout
>> KNOX-544: Knox process does not exit if startup fails due to credential
>> store issues
>> KNOX-476 implementation for X-Forwarded-* headers support and population
>> KNOX-539 add message to identity mapping audit entries
>> KNOX-538: Log some important system properties at startup
>> KNOX-534 auditing shiro authentication exceptions
>> KNOX-533 - add version component to knoxsso url pattern
>> KNOX-291: Improve audit for topology deployment process
>> KNOX-532: Update root pom.xml maven-compiler-plugin configuration.
>> KNOX-531 fix extraneous audit entries and add additional principal
>> mapping test
>> KNOX-529 - second attempt to get all usecases - missed wildcard plus
>> explicit mappings before
>> KNOX-530 fixed oozie rewrite rules to handle missing port information
>> KNOX-529 - Fix wildcard based principal group mapping
>>
>>
>>
>>
>>
>>
>> On 12/15/15, 3:11 PM, "larry mccay" <lm...@apache.org> wrote:
>>
>> >Knox dev's -
>> >
>> >We need to start locking down the release for 0.7.0.
>> >In preparation of this, Sumit created a branch a week or so ago and we
>> >should start considering the creation of a release candidate.
>> >
>> >I believe that I have to update the CHANGES file with an entry for a
>> patch
>> >that I cherry picked into 0.7.0 branch and I will look into that shortly.
>> >
>> >Standout features include: KnoxSSO for WebSSO, HA support for numerous
>> >services, diagnostic commands for KnoxCLI, regex based identity
>> >assertion, better control over thread pool, connection queue and
>> >request/response buffers. The ability to proxy Hadoop UIs, CORS support
>> for
>> >cross origin request sharing and more. As well as a number of important
>> bug
>> >fixes.
>> >
>> >We do have an important feature coming from the community - specifically
>> >from Jérôme that will be committed in coming days. KNOX-641 adds a
>> >federation provider that integrates pac4j in order to add: OAuth,
>> Facebook,
>> >CAS, SAML, OpenID Connect. I think that this is an exciting integration
>> >that will require a bit of testing before it can be merged into a release
>> >branch.
>> >
>> >In my opinion, the set of features and improvements that are currently in
>> >the v0.7.0 branch more than justify a new release and delaying that any
>> >longer would be less than ideal.
>> >
>> >Concentrating on defining and testing the usecases that the pac4j
>> provider
>> >will bring to the table post 0.7.0 and coming up with a compelling story
>> >for that feature set can be used to justify a release of its own. I think
>> >that we should target a feature release which we'll call 0.8.0 for now
>> for
>> >a mid January timeframe.
>> >
>> >So, discussion points:
>> >
>> >1. Should we move forward with the 0.7.0 release once the CHANGES file is
>> >updated?
>> >2. Thoughts on holding the pac4j provider out until an early 2016 release
>> >when the main usecases are better defined and tested?
>> >
>> >thanks,
>> >
>> >--larry
>>
>
>

Re: [DISCUSS] Preparing for 0.7.0 Release

[larry mccay <lm...@apache.org>]

errr - rc1 that is...

On Thu, Dec 17, 2015 at 2:14 PM, larry mccay <lm...@apache.org> wrote:

> Great!
>
> We can start a VOTE for releasing rc0 then?
>
>
> On Thu, Dec 17, 2015 at 2:03 PM, Kevin Minder <
> kevin.minder@hortonworks.com> wrote:
>
>> Ok since this seems to have quieted down with no objections I’ve created
>> RC1.
>>
>>
>>
>>
>> On 12/16/15, 11:33 AM, "Kevin Minder" <ke...@hortonworks.com>
>> wrote:
>>
>> >Hi Everyone,
>> >I’ve practiced through the release mechanics so I’ll volunteer for be
>> the release manager for 0.7.0 assuming we all agree to move forward.
>> >Kevin.
>> >
>> >
>> >
>> >
>> >On 12/15/15, 4:29 PM, "larry mccay" <lm...@apache.org> wrote:
>> >
>> >>I will take on the task of merging the lists and prepare a patch for
>> that
>> >>immediately.
>> >>
>> >>On Tue, Dec 15, 2015 at 4:27 PM, Kevin Minder <
>> kevin.minder@hortonworks.com>
>> >>wrote:
>> >>
>> >>> I’m in favor of continuing to stabilize the 0.7.0 branch with the
>> current
>> >>> bits we have and then doing an 0.8.0 with the new KNOX-641 stuff as
>> quickly
>> >>> as possible after that.
>> >>> KNOX-641 ends up providing a wonderful new big feature set and we are
>> >>> going to need to bandwidth to learn/absorb it.
>> >>>
>> >>> BTW here is my take on all of the commits from the branch point for
>> >>> 0.6.0.  Seems we are getting better with our CHANGES discipline but
>> there
>> >>> is still a great deal of room for improvement.  The CHANGES file has
>> ~30
>> >>> entries for 0.7.0 and the list below has about ~90 entries.
>> >>>
>> >>> [KNOX-639] - Knoxcli.sh create-master should not allow empty strings
>> >>> KNOX-640 - Make Cookie Domain Configurable
>> >>> [KNOX-638] - Hive dispatch failing for secure clusters
>> >>> KNOX-626 Minor fix to namespace parsing
>> >>> KNOX-637 - Compilation Error in gateway-service-admin and gateway-test
>> >>> test projects (arshad.mohammad via lmccay)
>> >>> KNOX-636 - IdentityAsserterHttpServletRequestWrapper must override
>> >>> getUserPrincipal
>> >>> KNOX-635 - open up default whitelist for dev - localhost
>> >>> KNOX-635 - Provide Whitelisting for Redirect Destinations for KnoxSSO
>> >>> KNOX-634 - CORS Support as Part of WebAppSec Provider
>> >>> KNOX-632 added back configuration for 'replayBufferSize'
>> >>> KNOX-633: Upgrade apache commons-collections
>> >>> KNOX-632: Oozie dispatch failing for secure clusters. Fix tests.
>> >>> KNOX-632: Oozie dispatch failing for secure clusters
>> >>> KNOX-625 initial template file for topology using ui proxy services
>> >>> KNOX-623: Gateway provider rewriter doesn't support boolean
>> attributes in
>> >>> HTML.
>> >>> KNOX-622 - Misconfigured providers should cause topology deployment
>> to fail
>> >>> KNOX-624: Expose configuration for Jetty's request and response buffer
>> >>> sizes. Fix property names.
>> >>> KNOX-624: Expose configuration for Jetty's request and response buffer
>> >>> sizes
>> >>> KNOX-621 - Simplify KnoxSSO API Resource Path
>> >>> KNOX-620: Jenkins Knox-master-verify failing since #725 due to JDK
>> version
>> >>> issues
>> >>> KNOX-394: Request and response URLs must be parsed as literals not
>> >>> templates. Part 2.
>> >>> KNOX-394: Request and response URLs must be parsed as literals not
>> >>> templates
>> >>> KNOX-617 - Add the use of CredentialCollectors to Samples
>> >>> KNOX-616: XmlUrlRewriteStreamFilter unscapes escaped special
>> characters
>> >>> KNOX-611: Expose configuration for Jetty's thread pool and connection
>> queue
>> >>> KNOX-604: Expose configuration of HttpClient's max connections per
>> route
>> >>> setting
>> >>> KNOX-614: Incorrect URI template expansion with {**} query params
>> >>> #fragments
>> >>> KNOX-615 Domain Cookies cannot Wildcard IP Addresses
>> >>> KNOX-613 - Provide Credential Collector Abstraction to Client Shell
>> >>> KNOX-610 - DefaultTokenService issueToken should never return null
>> >>> KNOX-609 - Add unit tests for the SSOCookieFederationProvider.
>> >>> KNOX-608: Improve Knox read and write performance by tuning buffer
>> sizes.
>> >>> KNOX-607 - Fix SSOCookieProvider to Handle null Query Strings
>> >>> KNOX-602 - protect against NPE in audience validation
>> >>> KNOX-603: Coverity: Potential resource leak in
>> >>> BaseKeystoreService.createKeystore
>> >>> KNOX-602 JWT/SSO Cookie Based Federation Provider
>> >>> KNOX-601: Knox test failures on windows
>> >>> KNOX-600 setting all service params as filter params for dispatch
>> >>> KNOX-593 removed replayBufferSize and CappedBufferHttpEntity
>> references
>> >>> KNOX-447: Incorrect parsing and expansion of valueless query params
>> >>> KNOX-599: Template with {**} in queries are expanded with =null for
>> query
>> >>> params without a value
>> >>> KNOX-598: Concurrent JDBC clients via KNOX to Kerberized HiveServer2
>> >>> causes HTTP 401 error (due to Kerberos
>> >>> KNOX-570 added zookeeper lookup capability for HS2 HA
>> >>> KNOX-596: Add diagnostics to topology depoloyment
>> >>> KNOX-597: Improve diagnostic logging of HTTP traffic. Update CHANGES.
>> >>> KNOX-597: Improve diagnostic logging of HTTP traffic
>> >>> KNOX-593 Moved SPNEGO code to httpclient
>> >>> KNOX-584 Fix for UT instability in
>> GatewayBasicFuncTest.testCLIServiceTest
>> >>> KNOX-590 - add more ShiroProvider configuration support to KnoxCLI
>> >>> sys-user-auth-test and user-auth-test
>> >>> KNOX-582 Query Parameter rewrite does not honor empty string value
>> >>> (jeffreyr via lmccay)
>> >>> KNOX-581: Hive dispatch not propagating effective principal name
>> >>> KNOX-580 Initial refactoring out of default HA dispatch
>> >>> KNOX-579: Regex based identity assertion provider with static
>> dictionary
>> >>> lookup
>> >>> KNOX-576: CLI user-auth-test should print a message when a user
>> >>> successfully authenticates.
>> >>> KNOX-565: Supporting All the Quick Links on Ambari Dashboard to Go
>> Through
>> >>> Knox
>> >>> KNOX-564: NPE for Topology with no Providers Confgured
>> >>> KNOX-575: Add more logging for LDAP Authentication issues with
>> >>> ShiroProvider
>> >>> KNOX-573: KNOX-574 make SecureOnly and MaxAge configurable for SSO
>> >>> KNOX-549: Test service connections through Knox with Knox CLI
>> >>> KNOX-566 - Make the Default Ephemeral DH Key Size 2048 for TLS
>> >>> KNOX-460: UrlRewriteServletFilterTest failed with IBM JAVA
>> >>> KNOX-423: XmlFilterReaderTest failed with IBM JVM JAVA
>> >>> KNOX-548: LDAP Bind in Knox CLI. Fixed help usage.
>> >>> KNOX-562: Fix Null pointer exceptions in KnoxCLI LDAP commands
>> >>> KNOX-548: KnoxCLI adds a new system-user-auth-test command to test a
>> >>> topology's system username and password
>> >>> KNOX-560: Test LDAP Authentication+Authorization from KnoxCLI
>> >>> KNOX-561: Allow Knox pid directory to be configured via the
>> knox-env.sh
>> >>> file
>> >>> KNOX-559 renaming service definition files
>> >>> KNOX-558: HttpClient connections are not always returned to the pool
>> for
>> >>> HBase on Windows
>> >>> KNOX-554: Cannot access topologies through admin API if gateway.path
>> is
>> >>> modified
>> >>> KNOX-556 - fix extraneous imports
>> >>> KNOX-556 - provide better diagnostics for keystore failures
>> >>> KNOX-555: Prevent dispatch client from attempting retry and redirects
>> >>> KNOX-553: Added topology validation from KnoxCLI to TopologyService
>> >>> deployment.
>> >>> KNOX-547: Topology Validation in Knox CLI. Fix schema load from JAR
>> >>> NullPointerException
>> >>> KNOX-547: Topology Validation in Knox CLI
>> >>> KNOX-550 reverting back to original hive kerberos dispatch behavior
>> >>> KNOX-546 Consuming intermediate response during kerberos request
>> >>> dispatching
>> >>> KNOX-545 - Simplify Keystore Management for Cluster Scaleout
>> >>> KNOX-544: Knox process does not exit if startup fails due to
>> credential
>> >>> store issues
>> >>> KNOX-476 implementation for X-Forwarded-* headers support and
>> population
>> >>> KNOX-539 add message to identity mapping audit entries
>> >>> KNOX-538: Log some important system properties at startup
>> >>> KNOX-534 auditing shiro authentication exceptions
>> >>> KNOX-533 - add version component to knoxsso url pattern
>> >>> KNOX-291: Improve audit for topology deployment process
>> >>> KNOX-532: Update root pom.xml maven-compiler-plugin configuration.
>> >>> KNOX-531 fix extraneous audit entries and add additional principal
>> mapping
>> >>> test
>> >>> KNOX-529 - second attempt to get all usecases - missed wildcard plus
>> >>> explicit mappings before
>> >>> KNOX-530 fixed oozie rewrite rules to handle missing port information
>> >>> KNOX-529 - Fix wildcard based principal group mapping
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On 12/15/15, 3:11 PM, "larry mccay" <lm...@apache.org> wrote:
>> >>>
>> >>> >Knox dev's -
>> >>> >
>> >>> >We need to start locking down the release for 0.7.0.
>> >>> >In preparation of this, Sumit created a branch a week or so ago and
>> we
>> >>> >should start considering the creation of a release candidate.
>> >>> >
>> >>> >I believe that I have to update the CHANGES file with an entry for a
>> patch
>> >>> >that I cherry picked into 0.7.0 branch and I will look into that
>> shortly.
>> >>> >
>> >>> >Standout features include: KnoxSSO for WebSSO, HA support for
>> numerous
>> >>> >services, diagnostic commands for KnoxCLI, regex based identity
>> >>> >assertion, better control over thread pool, connection queue and
>> >>> >request/response buffers. The ability to proxy Hadoop UIs, CORS
>> support
>> >>> for
>> >>> >cross origin request sharing and more. As well as a number of
>> important
>> >>> bug
>> >>> >fixes.
>> >>> >
>> >>> >We do have an important feature coming from the community -
>> specifically
>> >>> >from Jérôme that will be committed in coming days. KNOX-641 adds a
>> >>> >federation provider that integrates pac4j in order to add: OAuth,
>> >>> Facebook,
>> >>> >CAS, SAML, OpenID Connect. I think that this is an exciting
>> integration
>> >>> >that will require a bit of testing before it can be merged into a
>> release
>> >>> >branch.
>> >>> >
>> >>> >In my opinion, the set of features and improvements that are
>> currently in
>> >>> >the v0.7.0 branch more than justify a new release and delaying that
>> any
>> >>> >longer would be less than ideal.
>> >>> >
>> >>> >Concentrating on defining and testing the usecases that the pac4j
>> provider
>> >>> >will bring to the table post 0.7.0 and coming up with a compelling
>> story
>> >>> >for that feature set can be used to justify a release of its own. I
>> think
>> >>> >that we should target a feature release which we'll call 0.8.0 for
>> now for
>> >>> >a mid January timeframe.
>> >>> >
>> >>> >So, discussion points:
>> >>> >
>> >>> >1. Should we move forward with the 0.7.0 release once the CHANGES
>> file is
>> >>> >updated?
>> >>> >2. Thoughts on holding the pac4j provider out until an early 2016
>> release
>> >>> >when the main usecases are better defined and tested?
>> >>> >
>> >>> >thanks,
>> >>> >
>> >>> >--larry
>> >>>
>>
>
>

Re: [DISCUSS] Preparing for 0.7.0 Release

[larry mccay <lm...@apache.org>]

Great!

We can start a VOTE for releasing rc0 then?


On Thu, Dec 17, 2015 at 2:03 PM, Kevin Minder <ke...@hortonworks.com>
wrote:

> Ok since this seems to have quieted down with no objections I’ve created
> RC1.
>
>
>
>
> On 12/16/15, 11:33 AM, "Kevin Minder" <ke...@hortonworks.com>
> wrote:
>
> >Hi Everyone,
> >I’ve practiced through the release mechanics so I’ll volunteer for be the
> release manager for 0.7.0 assuming we all agree to move forward.
> >Kevin.
> >
> >
> >
> >
> >On 12/15/15, 4:29 PM, "larry mccay" <lm...@apache.org> wrote:
> >
> >>I will take on the task of merging the lists and prepare a patch for that
> >>immediately.
> >>
> >>On Tue, Dec 15, 2015 at 4:27 PM, Kevin Minder <
> kevin.minder@hortonworks.com>
> >>wrote:
> >>
> >>> I’m in favor of continuing to stabilize the 0.7.0 branch with the
> current
> >>> bits we have and then doing an 0.8.0 with the new KNOX-641 stuff as
> quickly
> >>> as possible after that.
> >>> KNOX-641 ends up providing a wonderful new big feature set and we are
> >>> going to need to bandwidth to learn/absorb it.
> >>>
> >>> BTW here is my take on all of the commits from the branch point for
> >>> 0.6.0.  Seems we are getting better with our CHANGES discipline but
> there
> >>> is still a great deal of room for improvement.  The CHANGES file has
> ~30
> >>> entries for 0.7.0 and the list below has about ~90 entries.
> >>>
> >>> [KNOX-639] - Knoxcli.sh create-master should not allow empty strings
> >>> KNOX-640 - Make Cookie Domain Configurable
> >>> [KNOX-638] - Hive dispatch failing for secure clusters
> >>> KNOX-626 Minor fix to namespace parsing
> >>> KNOX-637 - Compilation Error in gateway-service-admin and gateway-test
> >>> test projects (arshad.mohammad via lmccay)
> >>> KNOX-636 - IdentityAsserterHttpServletRequestWrapper must override
> >>> getUserPrincipal
> >>> KNOX-635 - open up default whitelist for dev - localhost
> >>> KNOX-635 - Provide Whitelisting for Redirect Destinations for KnoxSSO
> >>> KNOX-634 - CORS Support as Part of WebAppSec Provider
> >>> KNOX-632 added back configuration for 'replayBufferSize'
> >>> KNOX-633: Upgrade apache commons-collections
> >>> KNOX-632: Oozie dispatch failing for secure clusters. Fix tests.
> >>> KNOX-632: Oozie dispatch failing for secure clusters
> >>> KNOX-625 initial template file for topology using ui proxy services
> >>> KNOX-623: Gateway provider rewriter doesn't support boolean attributes
> in
> >>> HTML.
> >>> KNOX-622 - Misconfigured providers should cause topology deployment to
> fail
> >>> KNOX-624: Expose configuration for Jetty's request and response buffer
> >>> sizes. Fix property names.
> >>> KNOX-624: Expose configuration for Jetty's request and response buffer
> >>> sizes
> >>> KNOX-621 - Simplify KnoxSSO API Resource Path
> >>> KNOX-620: Jenkins Knox-master-verify failing since #725 due to JDK
> version
> >>> issues
> >>> KNOX-394: Request and response URLs must be parsed as literals not
> >>> templates. Part 2.
> >>> KNOX-394: Request and response URLs must be parsed as literals not
> >>> templates
> >>> KNOX-617 - Add the use of CredentialCollectors to Samples
> >>> KNOX-616: XmlUrlRewriteStreamFilter unscapes escaped special characters
> >>> KNOX-611: Expose configuration for Jetty's thread pool and connection
> queue
> >>> KNOX-604: Expose configuration of HttpClient's max connections per
> route
> >>> setting
> >>> KNOX-614: Incorrect URI template expansion with {**} query params
> >>> #fragments
> >>> KNOX-615 Domain Cookies cannot Wildcard IP Addresses
> >>> KNOX-613 - Provide Credential Collector Abstraction to Client Shell
> >>> KNOX-610 - DefaultTokenService issueToken should never return null
> >>> KNOX-609 - Add unit tests for the SSOCookieFederationProvider.
> >>> KNOX-608: Improve Knox read and write performance by tuning buffer
> sizes.
> >>> KNOX-607 - Fix SSOCookieProvider to Handle null Query Strings
> >>> KNOX-602 - protect against NPE in audience validation
> >>> KNOX-603: Coverity: Potential resource leak in
> >>> BaseKeystoreService.createKeystore
> >>> KNOX-602 JWT/SSO Cookie Based Federation Provider
> >>> KNOX-601: Knox test failures on windows
> >>> KNOX-600 setting all service params as filter params for dispatch
> >>> KNOX-593 removed replayBufferSize and CappedBufferHttpEntity references
> >>> KNOX-447: Incorrect parsing and expansion of valueless query params
> >>> KNOX-599: Template with {**} in queries are expanded with =null for
> query
> >>> params without a value
> >>> KNOX-598: Concurrent JDBC clients via KNOX to Kerberized HiveServer2
> >>> causes HTTP 401 error (due to Kerberos
> >>> KNOX-570 added zookeeper lookup capability for HS2 HA
> >>> KNOX-596: Add diagnostics to topology depoloyment
> >>> KNOX-597: Improve diagnostic logging of HTTP traffic. Update CHANGES.
> >>> KNOX-597: Improve diagnostic logging of HTTP traffic
> >>> KNOX-593 Moved SPNEGO code to httpclient
> >>> KNOX-584 Fix for UT instability in
> GatewayBasicFuncTest.testCLIServiceTest
> >>> KNOX-590 - add more ShiroProvider configuration support to KnoxCLI
> >>> sys-user-auth-test and user-auth-test
> >>> KNOX-582 Query Parameter rewrite does not honor empty string value
> >>> (jeffreyr via lmccay)
> >>> KNOX-581: Hive dispatch not propagating effective principal name
> >>> KNOX-580 Initial refactoring out of default HA dispatch
> >>> KNOX-579: Regex based identity assertion provider with static
> dictionary
> >>> lookup
> >>> KNOX-576: CLI user-auth-test should print a message when a user
> >>> successfully authenticates.
> >>> KNOX-565: Supporting All the Quick Links on Ambari Dashboard to Go
> Through
> >>> Knox
> >>> KNOX-564: NPE for Topology with no Providers Confgured
> >>> KNOX-575: Add more logging for LDAP Authentication issues with
> >>> ShiroProvider
> >>> KNOX-573: KNOX-574 make SecureOnly and MaxAge configurable for SSO
> >>> KNOX-549: Test service connections through Knox with Knox CLI
> >>> KNOX-566 - Make the Default Ephemeral DH Key Size 2048 for TLS
> >>> KNOX-460: UrlRewriteServletFilterTest failed with IBM JAVA
> >>> KNOX-423: XmlFilterReaderTest failed with IBM JVM JAVA
> >>> KNOX-548: LDAP Bind in Knox CLI. Fixed help usage.
> >>> KNOX-562: Fix Null pointer exceptions in KnoxCLI LDAP commands
> >>> KNOX-548: KnoxCLI adds a new system-user-auth-test command to test a
> >>> topology's system username and password
> >>> KNOX-560: Test LDAP Authentication+Authorization from KnoxCLI
> >>> KNOX-561: Allow Knox pid directory to be configured via the knox-env.sh
> >>> file
> >>> KNOX-559 renaming service definition files
> >>> KNOX-558: HttpClient connections are not always returned to the pool
> for
> >>> HBase on Windows
> >>> KNOX-554: Cannot access topologies through admin API if gateway.path is
> >>> modified
> >>> KNOX-556 - fix extraneous imports
> >>> KNOX-556 - provide better diagnostics for keystore failures
> >>> KNOX-555: Prevent dispatch client from attempting retry and redirects
> >>> KNOX-553: Added topology validation from KnoxCLI to TopologyService
> >>> deployment.
> >>> KNOX-547: Topology Validation in Knox CLI. Fix schema load from JAR
> >>> NullPointerException
> >>> KNOX-547: Topology Validation in Knox CLI
> >>> KNOX-550 reverting back to original hive kerberos dispatch behavior
> >>> KNOX-546 Consuming intermediate response during kerberos request
> >>> dispatching
> >>> KNOX-545 - Simplify Keystore Management for Cluster Scaleout
> >>> KNOX-544: Knox process does not exit if startup fails due to credential
> >>> store issues
> >>> KNOX-476 implementation for X-Forwarded-* headers support and
> population
> >>> KNOX-539 add message to identity mapping audit entries
> >>> KNOX-538: Log some important system properties at startup
> >>> KNOX-534 auditing shiro authentication exceptions
> >>> KNOX-533 - add version component to knoxsso url pattern
> >>> KNOX-291: Improve audit for topology deployment process
> >>> KNOX-532: Update root pom.xml maven-compiler-plugin configuration.
> >>> KNOX-531 fix extraneous audit entries and add additional principal
> mapping
> >>> test
> >>> KNOX-529 - second attempt to get all usecases - missed wildcard plus
> >>> explicit mappings before
> >>> KNOX-530 fixed oozie rewrite rules to handle missing port information
> >>> KNOX-529 - Fix wildcard based principal group mapping
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On 12/15/15, 3:11 PM, "larry mccay" <lm...@apache.org> wrote:
> >>>
> >>> >Knox dev's -
> >>> >
> >>> >We need to start locking down the release for 0.7.0.
> >>> >In preparation of this, Sumit created a branch a week or so ago and we
> >>> >should start considering the creation of a release candidate.
> >>> >
> >>> >I believe that I have to update the CHANGES file with an entry for a
> patch
> >>> >that I cherry picked into 0.7.0 branch and I will look into that
> shortly.
> >>> >
> >>> >Standout features include: KnoxSSO for WebSSO, HA support for numerous
> >>> >services, diagnostic commands for KnoxCLI, regex based identity
> >>> >assertion, better control over thread pool, connection queue and
> >>> >request/response buffers. The ability to proxy Hadoop UIs, CORS
> support
> >>> for
> >>> >cross origin request sharing and more. As well as a number of
> important
> >>> bug
> >>> >fixes.
> >>> >
> >>> >We do have an important feature coming from the community -
> specifically
> >>> >from Jérôme that will be committed in coming days. KNOX-641 adds a
> >>> >federation provider that integrates pac4j in order to add: OAuth,
> >>> Facebook,
> >>> >CAS, SAML, OpenID Connect. I think that this is an exciting
> integration
> >>> >that will require a bit of testing before it can be merged into a
> release
> >>> >branch.
> >>> >
> >>> >In my opinion, the set of features and improvements that are
> currently in
> >>> >the v0.7.0 branch more than justify a new release and delaying that
> any
> >>> >longer would be less than ideal.
> >>> >
> >>> >Concentrating on defining and testing the usecases that the pac4j
> provider
> >>> >will bring to the table post 0.7.0 and coming up with a compelling
> story
> >>> >for that feature set can be used to justify a release of its own. I
> think
> >>> >that we should target a feature release which we'll call 0.8.0 for
> now for
> >>> >a mid January timeframe.
> >>> >
> >>> >So, discussion points:
> >>> >
> >>> >1. Should we move forward with the 0.7.0 release once the CHANGES
> file is
> >>> >updated?
> >>> >2. Thoughts on holding the pac4j provider out until an early 2016
> release
> >>> >when the main usecases are better defined and tested?
> >>> >
> >>> >thanks,
> >>> >
> >>> >--larry
> >>>
>

Re: [DISCUSS] Preparing for 0.7.0 Release

[Kevin Minder <ke...@hortonworks.com>]

Ok since this seems to have quieted down with no objections I’ve created RC1.




On 12/16/15, 11:33 AM, "Kevin Minder" <ke...@hortonworks.com> wrote:

>Hi Everyone,
>I’ve practiced through the release mechanics so I’ll volunteer for be the release manager for 0.7.0 assuming we all agree to move forward.
>Kevin.
>
>
>
>
>On 12/15/15, 4:29 PM, "larry mccay" <lm...@apache.org> wrote:
>
>>I will take on the task of merging the lists and prepare a patch for that
>>immediately.
>>
>>On Tue, Dec 15, 2015 at 4:27 PM, Kevin Minder <ke...@hortonworks.com>
>>wrote:
>>
>>> I’m in favor of continuing to stabilize the 0.7.0 branch with the current
>>> bits we have and then doing an 0.8.0 with the new KNOX-641 stuff as quickly
>>> as possible after that.
>>> KNOX-641 ends up providing a wonderful new big feature set and we are
>>> going to need to bandwidth to learn/absorb it.
>>>
>>> BTW here is my take on all of the commits from the branch point for
>>> 0.6.0.  Seems we are getting better with our CHANGES discipline but there
>>> is still a great deal of room for improvement.  The CHANGES file has ~30
>>> entries for 0.7.0 and the list below has about ~90 entries.
>>>
>>> [KNOX-639] - Knoxcli.sh create-master should not allow empty strings
>>> KNOX-640 - Make Cookie Domain Configurable
>>> [KNOX-638] - Hive dispatch failing for secure clusters
>>> KNOX-626 Minor fix to namespace parsing
>>> KNOX-637 - Compilation Error in gateway-service-admin and gateway-test
>>> test projects (arshad.mohammad via lmccay)
>>> KNOX-636 - IdentityAsserterHttpServletRequestWrapper must override
>>> getUserPrincipal
>>> KNOX-635 - open up default whitelist for dev - localhost
>>> KNOX-635 - Provide Whitelisting for Redirect Destinations for KnoxSSO
>>> KNOX-634 - CORS Support as Part of WebAppSec Provider
>>> KNOX-632 added back configuration for 'replayBufferSize'
>>> KNOX-633: Upgrade apache commons-collections
>>> KNOX-632: Oozie dispatch failing for secure clusters. Fix tests.
>>> KNOX-632: Oozie dispatch failing for secure clusters
>>> KNOX-625 initial template file for topology using ui proxy services
>>> KNOX-623: Gateway provider rewriter doesn't support boolean attributes in
>>> HTML.
>>> KNOX-622 - Misconfigured providers should cause topology deployment to fail
>>> KNOX-624: Expose configuration for Jetty's request and response buffer
>>> sizes. Fix property names.
>>> KNOX-624: Expose configuration for Jetty's request and response buffer
>>> sizes
>>> KNOX-621 - Simplify KnoxSSO API Resource Path
>>> KNOX-620: Jenkins Knox-master-verify failing since #725 due to JDK version
>>> issues
>>> KNOX-394: Request and response URLs must be parsed as literals not
>>> templates. Part 2.
>>> KNOX-394: Request and response URLs must be parsed as literals not
>>> templates
>>> KNOX-617 - Add the use of CredentialCollectors to Samples
>>> KNOX-616: XmlUrlRewriteStreamFilter unscapes escaped special characters
>>> KNOX-611: Expose configuration for Jetty's thread pool and connection queue
>>> KNOX-604: Expose configuration of HttpClient's max connections per route
>>> setting
>>> KNOX-614: Incorrect URI template expansion with {**} query params
>>> #fragments
>>> KNOX-615 Domain Cookies cannot Wildcard IP Addresses
>>> KNOX-613 - Provide Credential Collector Abstraction to Client Shell
>>> KNOX-610 - DefaultTokenService issueToken should never return null
>>> KNOX-609 - Add unit tests for the SSOCookieFederationProvider.
>>> KNOX-608: Improve Knox read and write performance by tuning buffer sizes.
>>> KNOX-607 - Fix SSOCookieProvider to Handle null Query Strings
>>> KNOX-602 - protect against NPE in audience validation
>>> KNOX-603: Coverity: Potential resource leak in
>>> BaseKeystoreService.createKeystore
>>> KNOX-602 JWT/SSO Cookie Based Federation Provider
>>> KNOX-601: Knox test failures on windows
>>> KNOX-600 setting all service params as filter params for dispatch
>>> KNOX-593 removed replayBufferSize and CappedBufferHttpEntity references
>>> KNOX-447: Incorrect parsing and expansion of valueless query params
>>> KNOX-599: Template with {**} in queries are expanded with =null for query
>>> params without a value
>>> KNOX-598: Concurrent JDBC clients via KNOX to Kerberized HiveServer2
>>> causes HTTP 401 error (due to Kerberos
>>> KNOX-570 added zookeeper lookup capability for HS2 HA
>>> KNOX-596: Add diagnostics to topology depoloyment
>>> KNOX-597: Improve diagnostic logging of HTTP traffic. Update CHANGES.
>>> KNOX-597: Improve diagnostic logging of HTTP traffic
>>> KNOX-593 Moved SPNEGO code to httpclient
>>> KNOX-584 Fix for UT instability in GatewayBasicFuncTest.testCLIServiceTest
>>> KNOX-590 - add more ShiroProvider configuration support to KnoxCLI
>>> sys-user-auth-test and user-auth-test
>>> KNOX-582 Query Parameter rewrite does not honor empty string value
>>> (jeffreyr via lmccay)
>>> KNOX-581: Hive dispatch not propagating effective principal name
>>> KNOX-580 Initial refactoring out of default HA dispatch
>>> KNOX-579: Regex based identity assertion provider with static dictionary
>>> lookup
>>> KNOX-576: CLI user-auth-test should print a message when a user
>>> successfully authenticates.
>>> KNOX-565: Supporting All the Quick Links on Ambari Dashboard to Go Through
>>> Knox
>>> KNOX-564: NPE for Topology with no Providers Confgured
>>> KNOX-575: Add more logging for LDAP Authentication issues with
>>> ShiroProvider
>>> KNOX-573: KNOX-574 make SecureOnly and MaxAge configurable for SSO
>>> KNOX-549: Test service connections through Knox with Knox CLI
>>> KNOX-566 - Make the Default Ephemeral DH Key Size 2048 for TLS
>>> KNOX-460: UrlRewriteServletFilterTest failed with IBM JAVA
>>> KNOX-423: XmlFilterReaderTest failed with IBM JVM JAVA
>>> KNOX-548: LDAP Bind in Knox CLI. Fixed help usage.
>>> KNOX-562: Fix Null pointer exceptions in KnoxCLI LDAP commands
>>> KNOX-548: KnoxCLI adds a new system-user-auth-test command to test a
>>> topology's system username and password
>>> KNOX-560: Test LDAP Authentication+Authorization from KnoxCLI
>>> KNOX-561: Allow Knox pid directory to be configured via the knox-env.sh
>>> file
>>> KNOX-559 renaming service definition files
>>> KNOX-558: HttpClient connections are not always returned to the pool for
>>> HBase on Windows
>>> KNOX-554: Cannot access topologies through admin API if gateway.path is
>>> modified
>>> KNOX-556 - fix extraneous imports
>>> KNOX-556 - provide better diagnostics for keystore failures
>>> KNOX-555: Prevent dispatch client from attempting retry and redirects
>>> KNOX-553: Added topology validation from KnoxCLI to TopologyService
>>> deployment.
>>> KNOX-547: Topology Validation in Knox CLI. Fix schema load from JAR
>>> NullPointerException
>>> KNOX-547: Topology Validation in Knox CLI
>>> KNOX-550 reverting back to original hive kerberos dispatch behavior
>>> KNOX-546 Consuming intermediate response during kerberos request
>>> dispatching
>>> KNOX-545 - Simplify Keystore Management for Cluster Scaleout
>>> KNOX-544: Knox process does not exit if startup fails due to credential
>>> store issues
>>> KNOX-476 implementation for X-Forwarded-* headers support and population
>>> KNOX-539 add message to identity mapping audit entries
>>> KNOX-538: Log some important system properties at startup
>>> KNOX-534 auditing shiro authentication exceptions
>>> KNOX-533 - add version component to knoxsso url pattern
>>> KNOX-291: Improve audit for topology deployment process
>>> KNOX-532: Update root pom.xml maven-compiler-plugin configuration.
>>> KNOX-531 fix extraneous audit entries and add additional principal mapping
>>> test
>>> KNOX-529 - second attempt to get all usecases - missed wildcard plus
>>> explicit mappings before
>>> KNOX-530 fixed oozie rewrite rules to handle missing port information
>>> KNOX-529 - Fix wildcard based principal group mapping
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 12/15/15, 3:11 PM, "larry mccay" <lm...@apache.org> wrote:
>>>
>>> >Knox dev's -
>>> >
>>> >We need to start locking down the release for 0.7.0.
>>> >In preparation of this, Sumit created a branch a week or so ago and we
>>> >should start considering the creation of a release candidate.
>>> >
>>> >I believe that I have to update the CHANGES file with an entry for a patch
>>> >that I cherry picked into 0.7.0 branch and I will look into that shortly.
>>> >
>>> >Standout features include: KnoxSSO for WebSSO, HA support for numerous
>>> >services, diagnostic commands for KnoxCLI, regex based identity
>>> >assertion, better control over thread pool, connection queue and
>>> >request/response buffers. The ability to proxy Hadoop UIs, CORS support
>>> for
>>> >cross origin request sharing and more. As well as a number of important
>>> bug
>>> >fixes.
>>> >
>>> >We do have an important feature coming from the community - specifically
>>> >from Jérôme that will be committed in coming days. KNOX-641 adds a
>>> >federation provider that integrates pac4j in order to add: OAuth,
>>> Facebook,
>>> >CAS, SAML, OpenID Connect. I think that this is an exciting integration
>>> >that will require a bit of testing before it can be merged into a release
>>> >branch.
>>> >
>>> >In my opinion, the set of features and improvements that are currently in
>>> >the v0.7.0 branch more than justify a new release and delaying that any
>>> >longer would be less than ideal.
>>> >
>>> >Concentrating on defining and testing the usecases that the pac4j provider
>>> >will bring to the table post 0.7.0 and coming up with a compelling story
>>> >for that feature set can be used to justify a release of its own. I think
>>> >that we should target a feature release which we'll call 0.8.0 for now for
>>> >a mid January timeframe.
>>> >
>>> >So, discussion points:
>>> >
>>> >1. Should we move forward with the 0.7.0 release once the CHANGES file is
>>> >updated?
>>> >2. Thoughts on holding the pac4j provider out until an early 2016 release
>>> >when the main usecases are better defined and tested?
>>> >
>>> >thanks,
>>> >
>>> >--larry
>>>

Re: [DISCUSS] Preparing for 0.7.0 Release

[larry mccay <lm...@apache.org>]

Thanks, Kevin!


On Wed, Dec 16, 2015 at 11:33 AM, Kevin Minder <kevin.minder@hortonworks.com
> wrote:

> Hi Everyone,
> I’ve practiced through the release mechanics so I’ll volunteer for be the
> release manager for 0.7.0 assuming we all agree to move forward.
> Kevin.
>
>
>
>
> On 12/15/15, 4:29 PM, "larry mccay" <lm...@apache.org> wrote:
>
> >I will take on the task of merging the lists and prepare a patch for that
> >immediately.
> >
> >On Tue, Dec 15, 2015 at 4:27 PM, Kevin Minder <
> kevin.minder@hortonworks.com>
> >wrote:
> >
> >> I’m in favor of continuing to stabilize the 0.7.0 branch with the
> current
> >> bits we have and then doing an 0.8.0 with the new KNOX-641 stuff as
> quickly
> >> as possible after that.
> >> KNOX-641 ends up providing a wonderful new big feature set and we are
> >> going to need to bandwidth to learn/absorb it.
> >>
> >> BTW here is my take on all of the commits from the branch point for
> >> 0.6.0.  Seems we are getting better with our CHANGES discipline but
> there
> >> is still a great deal of room for improvement.  The CHANGES file has ~30
> >> entries for 0.7.0 and the list below has about ~90 entries.
> >>
> >> [KNOX-639] - Knoxcli.sh create-master should not allow empty strings
> >> KNOX-640 - Make Cookie Domain Configurable
> >> [KNOX-638] - Hive dispatch failing for secure clusters
> >> KNOX-626 Minor fix to namespace parsing
> >> KNOX-637 - Compilation Error in gateway-service-admin and gateway-test
> >> test projects (arshad.mohammad via lmccay)
> >> KNOX-636 - IdentityAsserterHttpServletRequestWrapper must override
> >> getUserPrincipal
> >> KNOX-635 - open up default whitelist for dev - localhost
> >> KNOX-635 - Provide Whitelisting for Redirect Destinations for KnoxSSO
> >> KNOX-634 - CORS Support as Part of WebAppSec Provider
> >> KNOX-632 added back configuration for 'replayBufferSize'
> >> KNOX-633: Upgrade apache commons-collections
> >> KNOX-632: Oozie dispatch failing for secure clusters. Fix tests.
> >> KNOX-632: Oozie dispatch failing for secure clusters
> >> KNOX-625 initial template file for topology using ui proxy services
> >> KNOX-623: Gateway provider rewriter doesn't support boolean attributes
> in
> >> HTML.
> >> KNOX-622 - Misconfigured providers should cause topology deployment to
> fail
> >> KNOX-624: Expose configuration for Jetty's request and response buffer
> >> sizes. Fix property names.
> >> KNOX-624: Expose configuration for Jetty's request and response buffer
> >> sizes
> >> KNOX-621 - Simplify KnoxSSO API Resource Path
> >> KNOX-620: Jenkins Knox-master-verify failing since #725 due to JDK
> version
> >> issues
> >> KNOX-394: Request and response URLs must be parsed as literals not
> >> templates. Part 2.
> >> KNOX-394: Request and response URLs must be parsed as literals not
> >> templates
> >> KNOX-617 - Add the use of CredentialCollectors to Samples
> >> KNOX-616: XmlUrlRewriteStreamFilter unscapes escaped special characters
> >> KNOX-611: Expose configuration for Jetty's thread pool and connection
> queue
> >> KNOX-604: Expose configuration of HttpClient's max connections per route
> >> setting
> >> KNOX-614: Incorrect URI template expansion with {**} query params
> >> #fragments
> >> KNOX-615 Domain Cookies cannot Wildcard IP Addresses
> >> KNOX-613 - Provide Credential Collector Abstraction to Client Shell
> >> KNOX-610 - DefaultTokenService issueToken should never return null
> >> KNOX-609 - Add unit tests for the SSOCookieFederationProvider.
> >> KNOX-608: Improve Knox read and write performance by tuning buffer
> sizes.
> >> KNOX-607 - Fix SSOCookieProvider to Handle null Query Strings
> >> KNOX-602 - protect against NPE in audience validation
> >> KNOX-603: Coverity: Potential resource leak in
> >> BaseKeystoreService.createKeystore
> >> KNOX-602 JWT/SSO Cookie Based Federation Provider
> >> KNOX-601: Knox test failures on windows
> >> KNOX-600 setting all service params as filter params for dispatch
> >> KNOX-593 removed replayBufferSize and CappedBufferHttpEntity references
> >> KNOX-447: Incorrect parsing and expansion of valueless query params
> >> KNOX-599: Template with {**} in queries are expanded with =null for
> query
> >> params without a value
> >> KNOX-598: Concurrent JDBC clients via KNOX to Kerberized HiveServer2
> >> causes HTTP 401 error (due to Kerberos
> >> KNOX-570 added zookeeper lookup capability for HS2 HA
> >> KNOX-596: Add diagnostics to topology depoloyment
> >> KNOX-597: Improve diagnostic logging of HTTP traffic. Update CHANGES.
> >> KNOX-597: Improve diagnostic logging of HTTP traffic
> >> KNOX-593 Moved SPNEGO code to httpclient
> >> KNOX-584 Fix for UT instability in
> GatewayBasicFuncTest.testCLIServiceTest
> >> KNOX-590 - add more ShiroProvider configuration support to KnoxCLI
> >> sys-user-auth-test and user-auth-test
> >> KNOX-582 Query Parameter rewrite does not honor empty string value
> >> (jeffreyr via lmccay)
> >> KNOX-581: Hive dispatch not propagating effective principal name
> >> KNOX-580 Initial refactoring out of default HA dispatch
> >> KNOX-579: Regex based identity assertion provider with static dictionary
> >> lookup
> >> KNOX-576: CLI user-auth-test should print a message when a user
> >> successfully authenticates.
> >> KNOX-565: Supporting All the Quick Links on Ambari Dashboard to Go
> Through
> >> Knox
> >> KNOX-564: NPE for Topology with no Providers Confgured
> >> KNOX-575: Add more logging for LDAP Authentication issues with
> >> ShiroProvider
> >> KNOX-573: KNOX-574 make SecureOnly and MaxAge configurable for SSO
> >> KNOX-549: Test service connections through Knox with Knox CLI
> >> KNOX-566 - Make the Default Ephemeral DH Key Size 2048 for TLS
> >> KNOX-460: UrlRewriteServletFilterTest failed with IBM JAVA
> >> KNOX-423: XmlFilterReaderTest failed with IBM JVM JAVA
> >> KNOX-548: LDAP Bind in Knox CLI. Fixed help usage.
> >> KNOX-562: Fix Null pointer exceptions in KnoxCLI LDAP commands
> >> KNOX-548: KnoxCLI adds a new system-user-auth-test command to test a
> >> topology's system username and password
> >> KNOX-560: Test LDAP Authentication+Authorization from KnoxCLI
> >> KNOX-561: Allow Knox pid directory to be configured via the knox-env.sh
> >> file
> >> KNOX-559 renaming service definition files
> >> KNOX-558: HttpClient connections are not always returned to the pool for
> >> HBase on Windows
> >> KNOX-554: Cannot access topologies through admin API if gateway.path is
> >> modified
> >> KNOX-556 - fix extraneous imports
> >> KNOX-556 - provide better diagnostics for keystore failures
> >> KNOX-555: Prevent dispatch client from attempting retry and redirects
> >> KNOX-553: Added topology validation from KnoxCLI to TopologyService
> >> deployment.
> >> KNOX-547: Topology Validation in Knox CLI. Fix schema load from JAR
> >> NullPointerException
> >> KNOX-547: Topology Validation in Knox CLI
> >> KNOX-550 reverting back to original hive kerberos dispatch behavior
> >> KNOX-546 Consuming intermediate response during kerberos request
> >> dispatching
> >> KNOX-545 - Simplify Keystore Management for Cluster Scaleout
> >> KNOX-544: Knox process does not exit if startup fails due to credential
> >> store issues
> >> KNOX-476 implementation for X-Forwarded-* headers support and population
> >> KNOX-539 add message to identity mapping audit entries
> >> KNOX-538: Log some important system properties at startup
> >> KNOX-534 auditing shiro authentication exceptions
> >> KNOX-533 - add version component to knoxsso url pattern
> >> KNOX-291: Improve audit for topology deployment process
> >> KNOX-532: Update root pom.xml maven-compiler-plugin configuration.
> >> KNOX-531 fix extraneous audit entries and add additional principal
> mapping
> >> test
> >> KNOX-529 - second attempt to get all usecases - missed wildcard plus
> >> explicit mappings before
> >> KNOX-530 fixed oozie rewrite rules to handle missing port information
> >> KNOX-529 - Fix wildcard based principal group mapping
> >>
> >>
> >>
> >>
> >>
> >>
> >> On 12/15/15, 3:11 PM, "larry mccay" <lm...@apache.org> wrote:
> >>
> >> >Knox dev's -
> >> >
> >> >We need to start locking down the release for 0.7.0.
> >> >In preparation of this, Sumit created a branch a week or so ago and we
> >> >should start considering the creation of a release candidate.
> >> >
> >> >I believe that I have to update the CHANGES file with an entry for a
> patch
> >> >that I cherry picked into 0.7.0 branch and I will look into that
> shortly.
> >> >
> >> >Standout features include: KnoxSSO for WebSSO, HA support for numerous
> >> >services, diagnostic commands for KnoxCLI, regex based identity
> >> >assertion, better control over thread pool, connection queue and
> >> >request/response buffers. The ability to proxy Hadoop UIs, CORS support
> >> for
> >> >cross origin request sharing and more. As well as a number of important
> >> bug
> >> >fixes.
> >> >
> >> >We do have an important feature coming from the community -
> specifically
> >> >from Jérôme that will be committed in coming days. KNOX-641 adds a
> >> >federation provider that integrates pac4j in order to add: OAuth,
> >> Facebook,
> >> >CAS, SAML, OpenID Connect. I think that this is an exciting integration
> >> >that will require a bit of testing before it can be merged into a
> release
> >> >branch.
> >> >
> >> >In my opinion, the set of features and improvements that are currently
> in
> >> >the v0.7.0 branch more than justify a new release and delaying that any
> >> >longer would be less than ideal.
> >> >
> >> >Concentrating on defining and testing the usecases that the pac4j
> provider
> >> >will bring to the table post 0.7.0 and coming up with a compelling
> story
> >> >for that feature set can be used to justify a release of its own. I
> think
> >> >that we should target a feature release which we'll call 0.8.0 for now
> for
> >> >a mid January timeframe.
> >> >
> >> >So, discussion points:
> >> >
> >> >1. Should we move forward with the 0.7.0 release once the CHANGES file
> is
> >> >updated?
> >> >2. Thoughts on holding the pac4j provider out until an early 2016
> release
> >> >when the main usecases are better defined and tested?
> >> >
> >> >thanks,
> >> >
> >> >--larry
> >>
>

Re: [DISCUSS] Preparing for 0.7.0 Release

[Kevin Minder <ke...@hortonworks.com>]

Hi Everyone,
I’ve practiced through the release mechanics so I’ll volunteer for be the release manager for 0.7.0 assuming we all agree to move forward.
Kevin.




On 12/15/15, 4:29 PM, "larry mccay" <lm...@apache.org> wrote:

>I will take on the task of merging the lists and prepare a patch for that
>immediately.
>
>On Tue, Dec 15, 2015 at 4:27 PM, Kevin Minder <ke...@hortonworks.com>
>wrote:
>
>> I’m in favor of continuing to stabilize the 0.7.0 branch with the current
>> bits we have and then doing an 0.8.0 with the new KNOX-641 stuff as quickly
>> as possible after that.
>> KNOX-641 ends up providing a wonderful new big feature set and we are
>> going to need to bandwidth to learn/absorb it.
>>
>> BTW here is my take on all of the commits from the branch point for
>> 0.6.0.  Seems we are getting better with our CHANGES discipline but there
>> is still a great deal of room for improvement.  The CHANGES file has ~30
>> entries for 0.7.0 and the list below has about ~90 entries.
>>
>> [KNOX-639] - Knoxcli.sh create-master should not allow empty strings
>> KNOX-640 - Make Cookie Domain Configurable
>> [KNOX-638] - Hive dispatch failing for secure clusters
>> KNOX-626 Minor fix to namespace parsing
>> KNOX-637 - Compilation Error in gateway-service-admin and gateway-test
>> test projects (arshad.mohammad via lmccay)
>> KNOX-636 - IdentityAsserterHttpServletRequestWrapper must override
>> getUserPrincipal
>> KNOX-635 - open up default whitelist for dev - localhost
>> KNOX-635 - Provide Whitelisting for Redirect Destinations for KnoxSSO
>> KNOX-634 - CORS Support as Part of WebAppSec Provider
>> KNOX-632 added back configuration for 'replayBufferSize'
>> KNOX-633: Upgrade apache commons-collections
>> KNOX-632: Oozie dispatch failing for secure clusters. Fix tests.
>> KNOX-632: Oozie dispatch failing for secure clusters
>> KNOX-625 initial template file for topology using ui proxy services
>> KNOX-623: Gateway provider rewriter doesn't support boolean attributes in
>> HTML.
>> KNOX-622 - Misconfigured providers should cause topology deployment to fail
>> KNOX-624: Expose configuration for Jetty's request and response buffer
>> sizes. Fix property names.
>> KNOX-624: Expose configuration for Jetty's request and response buffer
>> sizes
>> KNOX-621 - Simplify KnoxSSO API Resource Path
>> KNOX-620: Jenkins Knox-master-verify failing since #725 due to JDK version
>> issues
>> KNOX-394: Request and response URLs must be parsed as literals not
>> templates. Part 2.
>> KNOX-394: Request and response URLs must be parsed as literals not
>> templates
>> KNOX-617 - Add the use of CredentialCollectors to Samples
>> KNOX-616: XmlUrlRewriteStreamFilter unscapes escaped special characters
>> KNOX-611: Expose configuration for Jetty's thread pool and connection queue
>> KNOX-604: Expose configuration of HttpClient's max connections per route
>> setting
>> KNOX-614: Incorrect URI template expansion with {**} query params
>> #fragments
>> KNOX-615 Domain Cookies cannot Wildcard IP Addresses
>> KNOX-613 - Provide Credential Collector Abstraction to Client Shell
>> KNOX-610 - DefaultTokenService issueToken should never return null
>> KNOX-609 - Add unit tests for the SSOCookieFederationProvider.
>> KNOX-608: Improve Knox read and write performance by tuning buffer sizes.
>> KNOX-607 - Fix SSOCookieProvider to Handle null Query Strings
>> KNOX-602 - protect against NPE in audience validation
>> KNOX-603: Coverity: Potential resource leak in
>> BaseKeystoreService.createKeystore
>> KNOX-602 JWT/SSO Cookie Based Federation Provider
>> KNOX-601: Knox test failures on windows
>> KNOX-600 setting all service params as filter params for dispatch
>> KNOX-593 removed replayBufferSize and CappedBufferHttpEntity references
>> KNOX-447: Incorrect parsing and expansion of valueless query params
>> KNOX-599: Template with {**} in queries are expanded with =null for query
>> params without a value
>> KNOX-598: Concurrent JDBC clients via KNOX to Kerberized HiveServer2
>> causes HTTP 401 error (due to Kerberos
>> KNOX-570 added zookeeper lookup capability for HS2 HA
>> KNOX-596: Add diagnostics to topology depoloyment
>> KNOX-597: Improve diagnostic logging of HTTP traffic. Update CHANGES.
>> KNOX-597: Improve diagnostic logging of HTTP traffic
>> KNOX-593 Moved SPNEGO code to httpclient
>> KNOX-584 Fix for UT instability in GatewayBasicFuncTest.testCLIServiceTest
>> KNOX-590 - add more ShiroProvider configuration support to KnoxCLI
>> sys-user-auth-test and user-auth-test
>> KNOX-582 Query Parameter rewrite does not honor empty string value
>> (jeffreyr via lmccay)
>> KNOX-581: Hive dispatch not propagating effective principal name
>> KNOX-580 Initial refactoring out of default HA dispatch
>> KNOX-579: Regex based identity assertion provider with static dictionary
>> lookup
>> KNOX-576: CLI user-auth-test should print a message when a user
>> successfully authenticates.
>> KNOX-565: Supporting All the Quick Links on Ambari Dashboard to Go Through
>> Knox
>> KNOX-564: NPE for Topology with no Providers Confgured
>> KNOX-575: Add more logging for LDAP Authentication issues with
>> ShiroProvider
>> KNOX-573: KNOX-574 make SecureOnly and MaxAge configurable for SSO
>> KNOX-549: Test service connections through Knox with Knox CLI
>> KNOX-566 - Make the Default Ephemeral DH Key Size 2048 for TLS
>> KNOX-460: UrlRewriteServletFilterTest failed with IBM JAVA
>> KNOX-423: XmlFilterReaderTest failed with IBM JVM JAVA
>> KNOX-548: LDAP Bind in Knox CLI. Fixed help usage.
>> KNOX-562: Fix Null pointer exceptions in KnoxCLI LDAP commands
>> KNOX-548: KnoxCLI adds a new system-user-auth-test command to test a
>> topology's system username and password
>> KNOX-560: Test LDAP Authentication+Authorization from KnoxCLI
>> KNOX-561: Allow Knox pid directory to be configured via the knox-env.sh
>> file
>> KNOX-559 renaming service definition files
>> KNOX-558: HttpClient connections are not always returned to the pool for
>> HBase on Windows
>> KNOX-554: Cannot access topologies through admin API if gateway.path is
>> modified
>> KNOX-556 - fix extraneous imports
>> KNOX-556 - provide better diagnostics for keystore failures
>> KNOX-555: Prevent dispatch client from attempting retry and redirects
>> KNOX-553: Added topology validation from KnoxCLI to TopologyService
>> deployment.
>> KNOX-547: Topology Validation in Knox CLI. Fix schema load from JAR
>> NullPointerException
>> KNOX-547: Topology Validation in Knox CLI
>> KNOX-550 reverting back to original hive kerberos dispatch behavior
>> KNOX-546 Consuming intermediate response during kerberos request
>> dispatching
>> KNOX-545 - Simplify Keystore Management for Cluster Scaleout
>> KNOX-544: Knox process does not exit if startup fails due to credential
>> store issues
>> KNOX-476 implementation for X-Forwarded-* headers support and population
>> KNOX-539 add message to identity mapping audit entries
>> KNOX-538: Log some important system properties at startup
>> KNOX-534 auditing shiro authentication exceptions
>> KNOX-533 - add version component to knoxsso url pattern
>> KNOX-291: Improve audit for topology deployment process
>> KNOX-532: Update root pom.xml maven-compiler-plugin configuration.
>> KNOX-531 fix extraneous audit entries and add additional principal mapping
>> test
>> KNOX-529 - second attempt to get all usecases - missed wildcard plus
>> explicit mappings before
>> KNOX-530 fixed oozie rewrite rules to handle missing port information
>> KNOX-529 - Fix wildcard based principal group mapping
>>
>>
>>
>>
>>
>>
>> On 12/15/15, 3:11 PM, "larry mccay" <lm...@apache.org> wrote:
>>
>> >Knox dev's -
>> >
>> >We need to start locking down the release for 0.7.0.
>> >In preparation of this, Sumit created a branch a week or so ago and we
>> >should start considering the creation of a release candidate.
>> >
>> >I believe that I have to update the CHANGES file with an entry for a patch
>> >that I cherry picked into 0.7.0 branch and I will look into that shortly.
>> >
>> >Standout features include: KnoxSSO for WebSSO, HA support for numerous
>> >services, diagnostic commands for KnoxCLI, regex based identity
>> >assertion, better control over thread pool, connection queue and
>> >request/response buffers. The ability to proxy Hadoop UIs, CORS support
>> for
>> >cross origin request sharing and more. As well as a number of important
>> bug
>> >fixes.
>> >
>> >We do have an important feature coming from the community - specifically
>> >from Jérôme that will be committed in coming days. KNOX-641 adds a
>> >federation provider that integrates pac4j in order to add: OAuth,
>> Facebook,
>> >CAS, SAML, OpenID Connect. I think that this is an exciting integration
>> >that will require a bit of testing before it can be merged into a release
>> >branch.
>> >
>> >In my opinion, the set of features and improvements that are currently in
>> >the v0.7.0 branch more than justify a new release and delaying that any
>> >longer would be less than ideal.
>> >
>> >Concentrating on defining and testing the usecases that the pac4j provider
>> >will bring to the table post 0.7.0 and coming up with a compelling story
>> >for that feature set can be used to justify a release of its own. I think
>> >that we should target a feature release which we'll call 0.8.0 for now for
>> >a mid January timeframe.
>> >
>> >So, discussion points:
>> >
>> >1. Should we move forward with the 0.7.0 release once the CHANGES file is
>> >updated?
>> >2. Thoughts on holding the pac4j provider out until an early 2016 release
>> >when the main usecases are better defined and tested?
>> >
>> >thanks,
>> >
>> >--larry
>>

Re: [DISCUSS] Preparing for 0.7.0 Release

[larry mccay <lm...@apache.org>]

I will take on the task of merging the lists and prepare a patch for that
immediately.

On Tue, Dec 15, 2015 at 4:27 PM, Kevin Minder <ke...@hortonworks.com>
wrote:

> I’m in favor of continuing to stabilize the 0.7.0 branch with the current
> bits we have and then doing an 0.8.0 with the new KNOX-641 stuff as quickly
> as possible after that.
> KNOX-641 ends up providing a wonderful new big feature set and we are
> going to need to bandwidth to learn/absorb it.
>
> BTW here is my take on all of the commits from the branch point for
> 0.6.0.  Seems we are getting better with our CHANGES discipline but there
> is still a great deal of room for improvement.  The CHANGES file has ~30
> entries for 0.7.0 and the list below has about ~90 entries.
>
> [KNOX-639] - Knoxcli.sh create-master should not allow empty strings
> KNOX-640 - Make Cookie Domain Configurable
> [KNOX-638] - Hive dispatch failing for secure clusters
> KNOX-626 Minor fix to namespace parsing
> KNOX-637 - Compilation Error in gateway-service-admin and gateway-test
> test projects (arshad.mohammad via lmccay)
> KNOX-636 - IdentityAsserterHttpServletRequestWrapper must override
> getUserPrincipal
> KNOX-635 - open up default whitelist for dev - localhost
> KNOX-635 - Provide Whitelisting for Redirect Destinations for KnoxSSO
> KNOX-634 - CORS Support as Part of WebAppSec Provider
> KNOX-632 added back configuration for 'replayBufferSize'
> KNOX-633: Upgrade apache commons-collections
> KNOX-632: Oozie dispatch failing for secure clusters. Fix tests.
> KNOX-632: Oozie dispatch failing for secure clusters
> KNOX-625 initial template file for topology using ui proxy services
> KNOX-623: Gateway provider rewriter doesn't support boolean attributes in
> HTML.
> KNOX-622 - Misconfigured providers should cause topology deployment to fail
> KNOX-624: Expose configuration for Jetty's request and response buffer
> sizes. Fix property names.
> KNOX-624: Expose configuration for Jetty's request and response buffer
> sizes
> KNOX-621 - Simplify KnoxSSO API Resource Path
> KNOX-620: Jenkins Knox-master-verify failing since #725 due to JDK version
> issues
> KNOX-394: Request and response URLs must be parsed as literals not
> templates. Part 2.
> KNOX-394: Request and response URLs must be parsed as literals not
> templates
> KNOX-617 - Add the use of CredentialCollectors to Samples
> KNOX-616: XmlUrlRewriteStreamFilter unscapes escaped special characters
> KNOX-611: Expose configuration for Jetty's thread pool and connection queue
> KNOX-604: Expose configuration of HttpClient's max connections per route
> setting
> KNOX-614: Incorrect URI template expansion with {**} query params
> #fragments
> KNOX-615 Domain Cookies cannot Wildcard IP Addresses
> KNOX-613 - Provide Credential Collector Abstraction to Client Shell
> KNOX-610 - DefaultTokenService issueToken should never return null
> KNOX-609 - Add unit tests for the SSOCookieFederationProvider.
> KNOX-608: Improve Knox read and write performance by tuning buffer sizes.
> KNOX-607 - Fix SSOCookieProvider to Handle null Query Strings
> KNOX-602 - protect against NPE in audience validation
> KNOX-603: Coverity: Potential resource leak in
> BaseKeystoreService.createKeystore
> KNOX-602 JWT/SSO Cookie Based Federation Provider
> KNOX-601: Knox test failures on windows
> KNOX-600 setting all service params as filter params for dispatch
> KNOX-593 removed replayBufferSize and CappedBufferHttpEntity references
> KNOX-447: Incorrect parsing and expansion of valueless query params
> KNOX-599: Template with {**} in queries are expanded with =null for query
> params without a value
> KNOX-598: Concurrent JDBC clients via KNOX to Kerberized HiveServer2
> causes HTTP 401 error (due to Kerberos
> KNOX-570 added zookeeper lookup capability for HS2 HA
> KNOX-596: Add diagnostics to topology depoloyment
> KNOX-597: Improve diagnostic logging of HTTP traffic. Update CHANGES.
> KNOX-597: Improve diagnostic logging of HTTP traffic
> KNOX-593 Moved SPNEGO code to httpclient
> KNOX-584 Fix for UT instability in GatewayBasicFuncTest.testCLIServiceTest
> KNOX-590 - add more ShiroProvider configuration support to KnoxCLI
> sys-user-auth-test and user-auth-test
> KNOX-582 Query Parameter rewrite does not honor empty string value
> (jeffreyr via lmccay)
> KNOX-581: Hive dispatch not propagating effective principal name
> KNOX-580 Initial refactoring out of default HA dispatch
> KNOX-579: Regex based identity assertion provider with static dictionary
> lookup
> KNOX-576: CLI user-auth-test should print a message when a user
> successfully authenticates.
> KNOX-565: Supporting All the Quick Links on Ambari Dashboard to Go Through
> Knox
> KNOX-564: NPE for Topology with no Providers Confgured
> KNOX-575: Add more logging for LDAP Authentication issues with
> ShiroProvider
> KNOX-573: KNOX-574 make SecureOnly and MaxAge configurable for SSO
> KNOX-549: Test service connections through Knox with Knox CLI
> KNOX-566 - Make the Default Ephemeral DH Key Size 2048 for TLS
> KNOX-460: UrlRewriteServletFilterTest failed with IBM JAVA
> KNOX-423: XmlFilterReaderTest failed with IBM JVM JAVA
> KNOX-548: LDAP Bind in Knox CLI. Fixed help usage.
> KNOX-562: Fix Null pointer exceptions in KnoxCLI LDAP commands
> KNOX-548: KnoxCLI adds a new system-user-auth-test command to test a
> topology's system username and password
> KNOX-560: Test LDAP Authentication+Authorization from KnoxCLI
> KNOX-561: Allow Knox pid directory to be configured via the knox-env.sh
> file
> KNOX-559 renaming service definition files
> KNOX-558: HttpClient connections are not always returned to the pool for
> HBase on Windows
> KNOX-554: Cannot access topologies through admin API if gateway.path is
> modified
> KNOX-556 - fix extraneous imports
> KNOX-556 - provide better diagnostics for keystore failures
> KNOX-555: Prevent dispatch client from attempting retry and redirects
> KNOX-553: Added topology validation from KnoxCLI to TopologyService
> deployment.
> KNOX-547: Topology Validation in Knox CLI. Fix schema load from JAR
> NullPointerException
> KNOX-547: Topology Validation in Knox CLI
> KNOX-550 reverting back to original hive kerberos dispatch behavior
> KNOX-546 Consuming intermediate response during kerberos request
> dispatching
> KNOX-545 - Simplify Keystore Management for Cluster Scaleout
> KNOX-544: Knox process does not exit if startup fails due to credential
> store issues
> KNOX-476 implementation for X-Forwarded-* headers support and population
> KNOX-539 add message to identity mapping audit entries
> KNOX-538: Log some important system properties at startup
> KNOX-534 auditing shiro authentication exceptions
> KNOX-533 - add version component to knoxsso url pattern
> KNOX-291: Improve audit for topology deployment process
> KNOX-532: Update root pom.xml maven-compiler-plugin configuration.
> KNOX-531 fix extraneous audit entries and add additional principal mapping
> test
> KNOX-529 - second attempt to get all usecases - missed wildcard plus
> explicit mappings before
> KNOX-530 fixed oozie rewrite rules to handle missing port information
> KNOX-529 - Fix wildcard based principal group mapping
>
>
>
>
>
>
> On 12/15/15, 3:11 PM, "larry mccay" <lm...@apache.org> wrote:
>
> >Knox dev's -
> >
> >We need to start locking down the release for 0.7.0.
> >In preparation of this, Sumit created a branch a week or so ago and we
> >should start considering the creation of a release candidate.
> >
> >I believe that I have to update the CHANGES file with an entry for a patch
> >that I cherry picked into 0.7.0 branch and I will look into that shortly.
> >
> >Standout features include: KnoxSSO for WebSSO, HA support for numerous
> >services, diagnostic commands for KnoxCLI, regex based identity
> >assertion, better control over thread pool, connection queue and
> >request/response buffers. The ability to proxy Hadoop UIs, CORS support
> for
> >cross origin request sharing and more. As well as a number of important
> bug
> >fixes.
> >
> >We do have an important feature coming from the community - specifically
> >from Jérôme that will be committed in coming days. KNOX-641 adds a
> >federation provider that integrates pac4j in order to add: OAuth,
> Facebook,
> >CAS, SAML, OpenID Connect. I think that this is an exciting integration
> >that will require a bit of testing before it can be merged into a release
> >branch.
> >
> >In my opinion, the set of features and improvements that are currently in
> >the v0.7.0 branch more than justify a new release and delaying that any
> >longer would be less than ideal.
> >
> >Concentrating on defining and testing the usecases that the pac4j provider
> >will bring to the table post 0.7.0 and coming up with a compelling story
> >for that feature set can be used to justify a release of its own. I think
> >that we should target a feature release which we'll call 0.8.0 for now for
> >a mid January timeframe.
> >
> >So, discussion points:
> >
> >1. Should we move forward with the 0.7.0 release once the CHANGES file is
> >updated?
> >2. Thoughts on holding the pac4j provider out until an early 2016 release
> >when the main usecases are better defined and tested?
> >
> >thanks,
> >
> >--larry
>

Re: [DISCUSS] Preparing for 0.7.0 Release

[Kevin Minder <ke...@hortonworks.com>]

I’m in favor of continuing to stabilize the 0.7.0 branch with the current bits we have and then doing an 0.8.0 with the new KNOX-641 stuff as quickly as possible after that.
KNOX-641 ends up providing a wonderful new big feature set and we are going to need to bandwidth to learn/absorb it.

BTW here is my take on all of the commits from the branch point for 0.6.0.  Seems we are getting better with our CHANGES discipline but there is still a great deal of room for improvement.  The CHANGES file has ~30 entries for 0.7.0 and the list below has about ~90 entries.

[KNOX-639] - Knoxcli.sh create-master should not allow empty strings
KNOX-640 - Make Cookie Domain Configurable
[KNOX-638] - Hive dispatch failing for secure clusters
KNOX-626 Minor fix to namespace parsing
KNOX-637 - Compilation Error in gateway-service-admin and gateway-test test projects (arshad.mohammad via lmccay)
KNOX-636 - IdentityAsserterHttpServletRequestWrapper must override getUserPrincipal
KNOX-635 - open up default whitelist for dev - localhost
KNOX-635 - Provide Whitelisting for Redirect Destinations for KnoxSSO
KNOX-634 - CORS Support as Part of WebAppSec Provider
KNOX-632 added back configuration for 'replayBufferSize'
KNOX-633: Upgrade apache commons-collections
KNOX-632: Oozie dispatch failing for secure clusters. Fix tests.
KNOX-632: Oozie dispatch failing for secure clusters
KNOX-625 initial template file for topology using ui proxy services
KNOX-623: Gateway provider rewriter doesn't support boolean attributes in HTML.
KNOX-622 - Misconfigured providers should cause topology deployment to fail
KNOX-624: Expose configuration for Jetty's request and response buffer sizes. Fix property names.
KNOX-624: Expose configuration for Jetty's request and response buffer sizes
KNOX-621 - Simplify KnoxSSO API Resource Path
KNOX-620: Jenkins Knox-master-verify failing since #725 due to JDK version issues
KNOX-394: Request and response URLs must be parsed as literals not templates. Part 2.
KNOX-394: Request and response URLs must be parsed as literals not templates
KNOX-617 - Add the use of CredentialCollectors to Samples
KNOX-616: XmlUrlRewriteStreamFilter unscapes escaped special characters
KNOX-611: Expose configuration for Jetty's thread pool and connection queue
KNOX-604: Expose configuration of HttpClient's max connections per route setting
KNOX-614: Incorrect URI template expansion with {**} query params #fragments
KNOX-615 Domain Cookies cannot Wildcard IP Addresses
KNOX-613 - Provide Credential Collector Abstraction to Client Shell
KNOX-610 - DefaultTokenService issueToken should never return null
KNOX-609 - Add unit tests for the SSOCookieFederationProvider.
KNOX-608: Improve Knox read and write performance by tuning buffer sizes.
KNOX-607 - Fix SSOCookieProvider to Handle null Query Strings
KNOX-602 - protect against NPE in audience validation
KNOX-603: Coverity: Potential resource leak in BaseKeystoreService.createKeystore
KNOX-602 JWT/SSO Cookie Based Federation Provider
KNOX-601: Knox test failures on windows
KNOX-600 setting all service params as filter params for dispatch
KNOX-593 removed replayBufferSize and CappedBufferHttpEntity references
KNOX-447: Incorrect parsing and expansion of valueless query params
KNOX-599: Template with {**} in queries are expanded with =null for query params without a value
KNOX-598: Concurrent JDBC clients via KNOX to Kerberized HiveServer2 causes HTTP 401 error (due to Kerberos 
KNOX-570 added zookeeper lookup capability for HS2 HA
KNOX-596: Add diagnostics to topology depoloyment
KNOX-597: Improve diagnostic logging of HTTP traffic. Update CHANGES.
KNOX-597: Improve diagnostic logging of HTTP traffic
KNOX-593 Moved SPNEGO code to httpclient
KNOX-584 Fix for UT instability in GatewayBasicFuncTest.testCLIServiceTest
KNOX-590 - add more ShiroProvider configuration support to KnoxCLI sys-user-auth-test and user-auth-test
KNOX-582 Query Parameter rewrite does not honor empty string value (jeffreyr via lmccay)
KNOX-581: Hive dispatch not propagating effective principal name
KNOX-580 Initial refactoring out of default HA dispatch
KNOX-579: Regex based identity assertion provider with static dictionary lookup
KNOX-576: CLI user-auth-test should print a message when a user successfully authenticates.
KNOX-565: Supporting All the Quick Links on Ambari Dashboard to Go Through Knox
KNOX-564: NPE for Topology with no Providers Confgured
KNOX-575: Add more logging for LDAP Authentication issues with ShiroProvider
KNOX-573: KNOX-574 make SecureOnly and MaxAge configurable for SSO
KNOX-549: Test service connections through Knox with Knox CLI
KNOX-566 - Make the Default Ephemeral DH Key Size 2048 for TLS
KNOX-460: UrlRewriteServletFilterTest failed with IBM JAVA
KNOX-423: XmlFilterReaderTest failed with IBM JVM JAVA
KNOX-548: LDAP Bind in Knox CLI. Fixed help usage.
KNOX-562: Fix Null pointer exceptions in KnoxCLI LDAP commands
KNOX-548: KnoxCLI adds a new system-user-auth-test command to test a topology's system username and password
KNOX-560: Test LDAP Authentication+Authorization from KnoxCLI
KNOX-561: Allow Knox pid directory to be configured via the knox-env.sh file
KNOX-559 renaming service definition files
KNOX-558: HttpClient connections are not always returned to the pool for HBase on Windows
KNOX-554: Cannot access topologies through admin API if gateway.path is modified
KNOX-556 - fix extraneous imports
KNOX-556 - provide better diagnostics for keystore failures
KNOX-555: Prevent dispatch client from attempting retry and redirects
KNOX-553: Added topology validation from KnoxCLI to TopologyService deployment.
KNOX-547: Topology Validation in Knox CLI. Fix schema load from JAR NullPointerException
KNOX-547: Topology Validation in Knox CLI
KNOX-550 reverting back to original hive kerberos dispatch behavior
KNOX-546 Consuming intermediate response during kerberos request dispatching
KNOX-545 - Simplify Keystore Management for Cluster Scaleout
KNOX-544: Knox process does not exit if startup fails due to credential store issues
KNOX-476 implementation for X-Forwarded-* headers support and population
KNOX-539 add message to identity mapping audit entries
KNOX-538: Log some important system properties at startup
KNOX-534 auditing shiro authentication exceptions
KNOX-533 - add version component to knoxsso url pattern
KNOX-291: Improve audit for topology deployment process
KNOX-532: Update root pom.xml maven-compiler-plugin configuration.
KNOX-531 fix extraneous audit entries and add additional principal mapping test
KNOX-529 - second attempt to get all usecases - missed wildcard plus explicit mappings before
KNOX-530 fixed oozie rewrite rules to handle missing port information
KNOX-529 - Fix wildcard based principal group mapping






On 12/15/15, 3:11 PM, "larry mccay" <lm...@apache.org> wrote:

>Knox dev's -
>
>We need to start locking down the release for 0.7.0.
>In preparation of this, Sumit created a branch a week or so ago and we
>should start considering the creation of a release candidate.
>
>I believe that I have to update the CHANGES file with an entry for a patch
>that I cherry picked into 0.7.0 branch and I will look into that shortly.
>
>Standout features include: KnoxSSO for WebSSO, HA support for numerous
>services, diagnostic commands for KnoxCLI, regex based identity
>assertion, better control over thread pool, connection queue and
>request/response buffers. The ability to proxy Hadoop UIs, CORS support for
>cross origin request sharing and more. As well as a number of important bug
>fixes.
>
>We do have an important feature coming from the community - specifically
>from Jérôme that will be committed in coming days. KNOX-641 adds a
>federation provider that integrates pac4j in order to add: OAuth, Facebook,
>CAS, SAML, OpenID Connect. I think that this is an exciting integration
>that will require a bit of testing before it can be merged into a release
>branch.
>
>In my opinion, the set of features and improvements that are currently in
>the v0.7.0 branch more than justify a new release and delaying that any
>longer would be less than ideal.
>
>Concentrating on defining and testing the usecases that the pac4j provider
>will bring to the table post 0.7.0 and coming up with a compelling story
>for that feature set can be used to justify a release of its own. I think
>that we should target a feature release which we'll call 0.8.0 for now for
>a mid January timeframe.
>
>So, discussion points:
>
>1. Should we move forward with the 0.7.0 release once the CHANGES file is
>updated?
>2. Thoughts on holding the pac4j provider out until an early 2016 release
>when the main usecases are better defined and tested?
>
>thanks,
>
>--larry