Reasonable config for ZK

[vikrant singh <vi...@gmail.com>]

Hello All,
I have a 3 server cluster running on virtual machines with 4 core Intel
processor with 25gb RAM.
MaxLatency on servers are around 200-500 ms. Another similar cluster with
more ephemeral nodes (and watches) has MaxLatency of around 2 secs.

I am wondering what should be a reasonable tick time (and other critical
config) I should keep for the same? right now it is 500ms and I do see some
connection loss occasionally.

If it helps, I use curator for creating these connection and most of them
are ephemeral.
Thanks,
Vikrant

Re: Reasonable config for ZK

[vikrant singh <vi...@gmail.com>]

Thanks for your reply.
I do not see any disturbance between server participating in quorum. But I
will double check and send more information with server logs.

This connection loss is local to client. I am using curator framework to
manage connections. Curator framework has a default session timeout of 15
seconds and connection timeout of 60 seconds. defined here
<https://github.com/apache/curator/search?utf8=%E2%9C%93&q=curator-default-session-timeout>
. On server side my ticktime is 500 ms.
I do have these GC settings (and other settings).
-XX:+UseG1GC -Xmx12G -Xms12G -XX:+UnlockExperimentalVMOptions
-XX:MaxGCPauseMillis=200 -XX:GCPauseIntervalMillis=800
-XX:G1ReservePercent=50 -XX:G1OldCSetRegionThresholdPercent=20
-XX:InitiatingHeapOccupancyPercent=20
Please let me know if you some issues which may have side effects.

Thanks

On Thu, Oct 8, 2015 at 10:24 PM, Rakesh Radhakrishnan <
rakeshr.apache@gmail.com> wrote:

> Hi Vikrant,
>
> >>>>I am wondering what should be a reasonable tick time (and other
> critical
> config) I should keep for the same? right now it is 500ms and I do see some
> connection loss occasionally.
>
> Do you see any disturbances in the ZooKeeper quorum, I meant any
> exceptions between Followers and Leader communications ?
>
> If you are seeing connection loss exceptions only at the zkclient side and
> there is no exceptions at the quorum server-to-server communications, then
> you could check your client's
>
> 1) the session timeout value, please give me the configured session timeout
> 2) any GC pauses or see any chances of delay in ZooKeeper client to server
> heartbeats etc.
>
> Also, would be helpful if you can share the logs.
>
> -Rakesh
>
> On Fri, Oct 9, 2015 at 4:09 AM, vikrant singh <vi...@gmail.com>
> wrote:
>
>> Hello All,
>> I have a 3 server cluster running on virtual machines with 4 core Intel
>> processor with 25gb RAM.
>> MaxLatency on servers are around 200-500 ms. Another similar cluster with
>> more ephemeral nodes (and watches) has MaxLatency of around 2 secs.
>>
>> I am wondering what should be a reasonable tick time (and other critical
>> config) I should keep for the same? right now it is 500ms and I do see
>> some
>> connection loss occasionally.
>>
>> If it helps, I use curator for creating these connection and most of them
>> are ephemeral.
>> Thanks,
>> Vikrant
>>
>
>

Re: Reasonable config for ZK

[Rakesh Radhakrishnan <ra...@gmail.com>]

Hi Vikrant,

>>>>I am wondering what should be a reasonable tick time (and other critical
config) I should keep for the same? right now it is 500ms and I do see some
connection loss occasionally.

Do you see any disturbances in the ZooKeeper quorum, I meant any exceptions
between Followers and Leader communications ?

If you are seeing connection loss exceptions only at the zkclient side and
there is no exceptions at the quorum server-to-server communications, then
you could check your client's

1) the session timeout value, please give me the configured session timeout
2) any GC pauses or see any chances of delay in ZooKeeper client to server
heartbeats etc.

Also, would be helpful if you can share the logs.

-Rakesh

On Fri, Oct 9, 2015 at 4:09 AM, vikrant singh <vi...@gmail.com>
wrote:

> Hello All,
> I have a 3 server cluster running on virtual machines with 4 core Intel
> processor with 25gb RAM.
> MaxLatency on servers are around 200-500 ms. Another similar cluster with
> more ephemeral nodes (and watches) has MaxLatency of around 2 secs.
>
> I am wondering what should be a reasonable tick time (and other critical
> config) I should keep for the same? right now it is 500ms and I do see some
> connection loss occasionally.
>
> If it helps, I use curator for creating these connection and most of them
> are ephemeral.
> Thanks,
> Vikrant
>