[jira] [Created] (AMBARI-24748) Option to configure Zookeeper Observers via Ambari

["George Kuzhanthai Raj S (JIRA)" <ji...@apache.org>]

George Kuzhanthai Raj S created AMBARI-24748:
------------------------------------------------

             Summary: Option to configure Zookeeper Observers via Ambari
                 Key: AMBARI-24748
                 URL: https://issues.apache.org/jira/browse/AMBARI-24748
             Project: Ambari
          Issue Type: New Feature
          Components: ambari-server
         Environment: Operating System: RHEL-7 (64-Bit)  
Stack Version: 2.6.4
Ambari Version: 2.6.1
            Reporter: George Kuzhanthai Raj S


Feature use :

 

Zookeeper Observers are non-voting members of an ensemble which only hear the results of votes, not the agreement protocol that leads up to them.

When zookeeper usage increases (ex: heavy Kafka or HBase usage ), we can leverage Observers to “sanctify” / isolate the Quorum and front client connection using Observers.

 

Issue with current implementation :

The current handling of zookeeper configuration in Ambari essentially prevents the use of the features.

As is, Observers cannot properly configured using Ambari. While we setup using manual configurations to enable Observers, they are however overwritten by Ambari at every service restart.

 

Explanation :

As per, https://zookeeper.apache.org/doc/r3.4.6/zookeeperObservers.html, Observers require 2 changes in the zookeeper config file(zoo.cfg).

 

Every node acting as an Observer, must include the following property:

 

peerType=observer

 

Secondly, all nodes of the ensemble acting as an observer must have “:observer” appended to the server definition line of members.

For example:

 

server.1=host1.example.com:2888:3888

server.2=host2.example.com:2888:3888

server.3=host3.example.com:2888:3888

server.4=host4.example.com:2888:3888:observer

 

This tells every member of the ensemble that server.4 is an Observer.

 

From Ambari’s perspective :

The first configuration « peerType = observer » can be easily handled using a specific Ambari configuration group that includes an extra entry in the “custom zoo.cfg” tab of configurations

 

The second configuration is more problematic :

The server list is automatically generated by ambari using the list of node for which a zookeeper service has been installed.

 

=> It cannot be modified manually.

To get around that we have tried to override the property “server.4” by including it in the “custom zoo.cfg” tab.

 

=> this essentially created a second line of the server4 property with the correct property ( dirty but… )

 

For example :

 

server.4=host4.example.com:2888:3888:observer

…

server.1=host1.example.com:2888:3888

server.2=host2.example.com:2888:3888

server.3=host3.example.com:2888:3888

server.4=host4.example.com:2888:3888

 

Note that custom configuration seem to be systematically placed at the beginning of the file.

 

Unfortunately only the last version of the property is actually retained. Ie. In the above example “server.4=host4.example.com:2888:3888:observer” is ignored, making it impossible to properly set configurations for the ensemble

 

Request :

The target would be to make Observers a specific feature of Ambari Zookeeper configurations.

 

As a workaround, would it be possible to modify the way Ambari generates the list of servers in zoo.cfg ?

Here are a few ideas :

Option 1 : Replace automatically generated property and only retain value defined in the custom property tab

Option 2 : Systematically place custom properties at the bottom of the file

Option 3 : Comment the automatically generated property when a custom property of a different value exists ?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)