Sharing setup Cluster X node on same machine

[Etienne Jouvin <la...@gmail.com>]

Hello all.

I know, this is not a good idea to setup a cluster on the same computer.
But for dev purpose, it could be useful.
As I spent 1 or 2 days to setup the cluster (not secured) on my computer
and read a lot of thread on Internet, I want to share my experience.


The purpose, having three running nodes, in version 1.12.1. Each one
executing the embedded Zookeeper.

Let's say nodes are "installed" to c:\Apps\NiFi with folders :
* node1
* node2
* node3

For each node, here are configurations made. Each time, replace the X by
the node id (1 / 2 / 3)

File *conf/nifi.properties*
Change IU banner

> nifi.ui.banner.text=Cluster NodeX


Embedded Zookeeper

> nifi.state.management.embedded.zookeeper.start=true


Listening port

> nifi.web.http.port=909X


Cluster configuration

> nifi.cluster.is.node=true
> nifi.cluster.node.address=localhost
> nifi.cluster.node.protocol.port=1144X
> nifi.cluster.flow.election.max.wait.time=30 seconds
> nifi.cluster.load.balance.port=6342
> nifi.zookeeper.connect.string=localhost:2181,localhost:2182,localhost:2183


Notice, the Zookeeper connection string, client ports are 2181 / 2182 /
2183, the pattern is 218X
Alose, I reduced the max.wait.time, default 5 minutes, to make it start
faster. (I spent couple hours understanding why my first did not start, it
was just waiting)

*File conf/state-management.xml*
In the nifi.properties file, there is the
configuration nifi.state.management.provider.cluster to register the state
manager id, linked to declaration in file state-manaement.xml.
Just update the default one, zk-provider, to provide the Zookeeper
connection string.

>     <cluster-provider>
>         <id>zk-provider</id>
>
> <class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class>
>         <property name="Connect
> String">localhost:2181,localhost:2182,localhost:2183</property>
>         <property name="Root Node">/nifi</property>
>         <property name="Session Timeout">10 seconds</property>
>         <property name="Access Control">Open</property>
>     </cluster-provider>


*File conf/zookeeper.properties*
Used to register all started embedded Zookeeper.

> server.1=localhost:2818:3818;2181
> server.2=localhost:2828:3828;2182
> server.3=localhost:2838:3838;2183


The pattern is in fact the following :
server.X=localhost:28X8:38X8;218X

*File state/zookeeper/myid*
Just put the node id as file content. For example :
1

And that's it.

Etienne