Re: NiFi Cluster on AWS using ECS and external Zookeeper

[Aswath Nangamangalam Srinivasan <As...@shi-gmbh.com>]

??I'm trying to set up a NiFi Cluster on AWS using ECS with External Zookeeper.

Have created a ECS Cluster for Zookeeper with 3 EC2 instance and running zookeeper ensemble. This external Zookeeper is working fine as I tested with my SolrCloud and also on my local NiFi. The local NiFi cluster was set up based on https://www.nifi.rocks/apache-nifi-docker-compose-cluster
Now that I had made sure that NiFi cluster works well with my external zookeeper that is running on AWS, I created another ECS cluster with 2 EC2 instances for NiFi. I did make sure all the variables are set properly for a NiFi cluster. All checks from the list of env variables given in https://github.com/apache/nifi/tree/master/nifi-docker/dockerhub

Though NiFi is starting on both the 2 EC2 instances and they using my external zookeeper, the 2 NiFi are running as two separate clusters with one node with each cluster. Ideally I'm wanting them to run as 2 nodes in one cluster.

I did compare the nifi.properties from the local NiFi cluster and the AWS Clusters and they all look good.

Am I missing some obvious step here?

Thank you,
Aswath