You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by geek101 <gi...@git.apache.org> on 2017/03/07 04:32:32 UTC
[GitHub] zookeeper pull request #185: Branch 3.5 FLE and ZAB SSL socket support [DO N...
GitHub user geek101 opened a pull request:
https://github.com/apache/zookeeper/pull/185
Branch 3.5 FLE and ZAB SSL socket support [DO NOT MERGE]
# Zookeeper Dynamic Quorum SSL (work in progress)
Provides SSL for Leader Election and ZAB i.e ports 3888 and 2888.
Goal of this patch is to build on top of SSL changes for [branch-3.4](https://github.com/geek101/zookeeper/blob/branch-3.4/README_SSL.md) and in
the spirit of branch-3.5 provide support for dynamic reconfiguration, i.e do
not violate safety and liveliness even when SSL is enabled.
### TODO
* CA based cert verification currently has no support for changing CA in a
fault-tolerant way. Alternative to changing CA is to revoke the quorum peer
to be removed via CRL(s) hence this needs more discussion/debate perhaps.
* Needs test framework and/or test cases to verify new functionality, this is
probably a significant amount of work gating this patch among other things.
### Self Signed Certs How To
The idea here is to propagate the new member(s) certificate fingerprint(s)
via the secure channel to the quorum via the reconfig() API.
Each quorum peer will have its self-signed cert finger print (typically like
a SHA-256 digest) embedded into its server string.
```
server.1=125.23.63.23:2780:2783:participant:cert:SHA-256-XXXX;2791
```
We also have support for both plain and secure port for ZookeeperServer hence
the config has been extended to reflect that as follows.
```
server.1=125.23.63.23:2780:2783:participant:cert:SHA-256-XXXX;plain:2791;
secure:2891
```
A reconfiguration operation would work by submitting the new server
config (or the new quorum list) to reconfig() API.
This has been tested to work with the current state of the patch.
### Some details
* [X509Util](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/common/X509Util.java)
becomes first class citizen and [QuorumX509Util](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/quorum/util/QuorumX509Util.java) and [ServerX509Util](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/util/ServerX509Util.java)
extend it.
* [ZKConfig](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/common/ZKConfig.java)
becomes an abstract class and [QuorumSslConfig](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerConfig.java) and
[ZookeeperServerConfig](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/ZookeeperServerConfig.java) implement it.
* [QuorumServer](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java#L278) gets the parsing code for the extra cert information and gets
help from new [SSLCertCfg](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/SSLCertCfg.java) class. Dynamic config generation is handled
transparently due to this.
* [ZKDynamicX509TrustManager](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/quorum/util/ZKDynamicX509TrustManager.java) handles the dynamic verification of certs and
gets help from QuorumPeer's new API,
[getQuorumServerFingerPrintByElectionAddress()](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java#L1642) and
[getQuorumServerFingerPrintByCert()](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java#L1661)
* Support for a Quorum peer to also be authenticated as a [ZK client](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/util/ServerX509Util.java#L62) (this
will be removed if it breaks security and or is not needed)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/geek101/zookeeper branch-3.5-ssl-review5
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zookeeper/pull/185.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #185
----
commit 9fea6674fbcaead59e090c28dbd7a59e864c442e
Author: Powell Molleti <po...@yahoo.com>
Date: 2016-08-14T02:27:03Z
SSL support for ZAB and FLE.
Code for dynamic reconfig() and SSL support.
commit b249a1d79d7f46ddba3855922348cc00e2c18ad7
Author: Powell Molleti <po...@yahoo.com>
Date: 2016-08-18T05:04:21Z
Pass ZKConfig() around to make things work.
commit 2f2c8cbaadf8db9c7aa4b51588d5c35547163990
Author: Powell Molleti <po...@yahoo.com>
Date: 2016-08-21T20:00:45Z
Fix QuorumPeer constructor call.
commit 98cc6c48e0ead57be221c0bb3729f9f1bde37382
Author: Powell Molleti <po...@yahoo.com>
Date: 2016-08-21T20:43:32Z
Fix a typo, pass the host ip part of the string.
commit 8e35e86bd0de86b4378bcb8a7e0cb920f263b9a5
Author: Powell Molleti <po...@yahoo.com>
Date: 2016-08-21T23:50:04Z
Make ZookeeperServer SSL auth to force client to be
one of the ZK nodes. For now this is ok but this is
not useful for everyone.
TODO: Create a different trust manager system property
for north-south SSL perhaps along with QuorumPeer dynamic
verification.
commit a040e08cb23ec16456277a6cc3e12566e15b369f
Author: Powell Molleti <po...@yahoo.com>
Date: 2016-09-05T22:03:41Z
Seperate SSL config for client and quorum
Quorum servers will have different properties for
SSL config, example:
-Dquorum.ssl.enabled=true
-Dquorum.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks
-Dquorum.ssl.keyStore.password=testpass
-Dquorum.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks
-Dquorum.ssl.trustStore.password=testpass
Enable each zookeeper node will be able to also allowed to be authenticated
as a client using dynamic reconfig.
Basic algorithm for various SSL connections.
Client:
1. Use the given truststore if available
2. Use the given server's digest
Server:
1. Use the given truststore if available
2. Use dynamic config to check against each server's digest
Quorum:
1. Use the given truststore if available
2. Use dyanmic config to check against each server's digest
commit 351de6e5a97980bdba127ba5dbbf265ce8e979d1
Author: Powell Molleti <po...@yahoo.com>
Date: 2016-09-06T06:12:35Z
Fail verification if Truststore has atleast one cert.
With chained verification a truststore with atleast one cert
trumps verification down stream.
No more support for "cacert" in server string.
commit c8f16214609948834a9cc087294cabc217124f8b
Author: Powell Molleti <po...@yahoo.com>
Date: 2016-09-06T08:20:22Z
Self signed certs are not verified via Truststore.
TODO: Fix this later.
commit f736859e06d045bd04aaa9227021d98d938991c5
Author: Powell Molleti <po...@yahoo.com>
Date: 2016-09-14T07:48:30Z
Server config string has both plain and secure address
Now the client parts of server string could look like this:
plain:127.0.1.1:2181;secure:127.0.1.1:2281
Its upto caller to ensure secureClientAddress and
secureClientPortAddress are same in the secure section above.
commit 0b0ce20081832c1630d6da9679cbde81b0ecf413
Author: Powell Molleti <po...@yahoo.com>
Date: 2016-09-15T05:17:13Z
Fix cert parsing a bit.
commit 0ccee0ac33261bc55c49f06cf51cb9a24ea06c63
Author: Powell Molleti <po...@yahoo.com>
Date: 2017-03-07T04:19:41Z
Fix test code due to Quorum Peer constructor change
and also since JUTE_MAXBUFFER is moved.
commit 30450268282d307e04c684dbcb0abb9c9bc59dca
Author: Powell Molleti <po...@yahoo.com>
Date: 2017-03-07T04:25:29Z
Add README to help explain what this code tries to accomplish.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---