You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by markap14 <gi...@git.apache.org> on 2016/05/05 20:31:00 UTC
[GitHub] nifi pull request: NIFI-1745: Refactor how requests are replicated...
GitHub user markap14 opened a pull request:
https://github.com/apache/nifi/pull/413
NIFI-1745: Refactor how requests are replicated across cluster
This still has some work to be done, but I believe it's worth merging into master now, only because we need to merge these changes with changes from a few other branches, as we have some VERY large changesets going on.
NIFI-1745: Refactoring how HTTP Requests are replicated to nodes
Bug fixes and continuing to work on replication refactoring. Still need to handle cluster locking and revisions
Begin work on RevisionManager
Resolved some issues that resulted from rebase
Fixed URIs to align with new URI's that will be used in 1.0.0
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/markap14/nifi NIFI-1745
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/413.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #413
----
commit 847b39f504226d7cc5cbb012214e3c964e8cbea8
Author: Mark Payne <ma...@hotmail.com>
Date: 2016-03-24T15:49:08Z
NIFI-1678: Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart
Ensure that ACL is set properly when creating heartbeat node. Removed unused ControllerStartupFailureMessage.java
Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator
Refactored CuratorHeartbeatMonitor into an abstract heartbeat monitor that is responsible for processing heartbeats and CuratorHeartbeatMonitor that is responsible for retrieving heartbeat information
Refactored so that heartbeats are sent to Cluster Coordinator directly instead of to ZooKeeper. ZooKeeper is used to know which node is the cluster coordinator but heartbeats to the Cluster Coordinator provide additional information about the nodes.
commit 9f206bd1e9d6a90f26b914b20d2aad113a320aed
Author: Mark Payne <ma...@hotmail.com>
Date: 2016-03-24T15:49:08Z
NIFI-1678: Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
commit 9a11af53297a46a38b9f1ee881d7e040d9f0f787
Author: Mark Payne <ma...@hotmail.com>
Date: 2016-04-01T18:01:37Z
NIFI-1678: Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart
commit 6d74740684c2b8e63d90de79411625fd177fcb40
Author: Mark Payne <ma...@hotmail.com>
Date: 2016-04-06T17:56:55Z
NIFI-1678: Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator
commit fb5c00aea21062dc288587540c2d0509c3e17a61
Author: Mark Payne <ma...@hotmail.com>
Date: 2016-04-08T12:17:11Z
NIFI-1727: Refactored logic for merging HTTP Requests that are federated across cluster
commit e4cccbfce39ebd933be39898bf827a783c1c9273
Author: Mark Payne <ma...@hotmail.com>
Date: 2016-04-12T13:43:13Z
NIFI-1745: Refactoring how HTTP Requests are replicated to nodes
Bug fixes and continuing to work on replication refactoring. Still need to handle cluster locking and revisions
Begin work on RevisionManager
Resolved some issues that resulted from rebase
Fixed URIs to align with new URI's that will be used in 1.0.0
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1745: Refactor how requests are replicated...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on the pull request:
https://github.com/apache/nifi/pull/413#issuecomment-217535825
Turns out I had a deployment problem earlier...
Anyways, running great after the most recent update. I agree we need to introduce these changes in stages given the number of moving parts. I've addressed a couple minor issues (URI paths, .gitignore, etc) which I'll be including in your commit.
One suggestion for the next portion of this work would be to re-introduce the wiring of the HTTP Replicator and Response Mapper in the WebClusterManager. While requiring more arguments, this would ultimately help promote testability rather than hard-coding a particular implementation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1745: Refactor how requests are replicated...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/413
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1745: Refactor how requests are replicated...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on the pull request:
https://github.com/apache/nifi/pull/413#issuecomment-217427533
Started reviewing this PR. I deployed to a cluster that had an existing flow. Node startup failed with this error.
```
java.lang.RuntimeException: java.io.InvalidClassException: org.apache.nifi.web.Revision; local class incompatible: stream classdesc serialVersionUID = 988658790374170022, local class serialVersionUID = 4915920088260082709
at org.apache.nifi.web.util.WebUtils.deserializeHexToObject(WebUtils.java:193) ~[nifi-web-utils-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT] at org.apache.nifi.web.filter.NodeRequestFilter.doFilter(NodeRequestFilter.java:83) ~[classes/:na]
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---