You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Bruno Wassermann <ba...@hotmail.com> on 2016/01/27 19:02:13 UTC
snapCount, initLimit, syncLimit
What influences the size of snapshots and how does one go about figuring out adequate settings for snapCount, initLimit and syncLimit? It would be great, if someone could share some advice.
Thanks,
-- Bruno
Re: snapCount, initLimit, syncLimit
Posted by Patrick Hunt <ph...@apache.org>.
I've never seen the snapCount be an issue for users. Doesn't mean it
couldn't be, but the default seems pretty reasonable. I've seen many
disk related issues with the tlogs, but those are usually related to
contention rather than snap count. Keep in mind we preallocate the log
files to improve performance so they tend to be large (but empty) to
start with.
I don't believe there is a message for exceeding the init limit, iirc
you see a communication timeout.
Patrick
On Wed, Jan 27, 2016 at 12:15 PM, Bruno Wassermann <ba...@hotmail.com> wrote:
> Hi Patrick,
>
> Do I understand correctly that a large snapCount can lead to fairly large transaction logs, which in turn may impact the recovery time of a server? And on a related note, if a server exceeds its initLimit, should this result in corresponding log messages at the INFO level in version 3.4.6?
>
> Thanks,
> -- Bruno
>
>> From: phunt@apache.org
>> Date: Wed, 27 Jan 2016 11:28:06 -0800
>> Subject: Re: snapCount, initLimit, syncLimit
>> To: user@zookeeper.apache.org
>>
>> Snapshots are a copy of your data in time, so size is relative to what
>> you put into ZK.
>>
>> After snapCount transactions are written to a log file a new snapshot
>> is created and a new transaction log file is created. The default
>> snapCount is 100,000, you almost never have to change that in my
>> experience.
>>
>> I'd recommend what we have in the sample for init and sync limit
>> (effectively 20 and 10 seconds respectively). Although if you have a
>> very large snapshot file you might need to increase these. You don't
>> want to make sync limit too large, otw it takes longer to identify a
>> partition.
>>
>> Patrick
>>
>>
>> On Wed, Jan 27, 2016 at 10:02 AM, Bruno Wassermann <ba...@hotmail.com> wrote:
>> > What influences the size of snapshots and how does one go about figuring out adequate settings for snapCount, initLimit and syncLimit? It would be great, if someone could share some advice.
>> > Thanks,
>> > -- Bruno
>
RE: snapCount, initLimit, syncLimit
Posted by Bruno Wassermann <ba...@hotmail.com>.
Hi Patrick,
Do I understand correctly that a large snapCount can lead to fairly large transaction logs, which in turn may impact the recovery time of a server? And on a related note, if a server exceeds its initLimit, should this result in corresponding log messages at the INFO level in version 3.4.6?
Thanks,
-- Bruno
> From: phunt@apache.org
> Date: Wed, 27 Jan 2016 11:28:06 -0800
> Subject: Re: snapCount, initLimit, syncLimit
> To: user@zookeeper.apache.org
>
> Snapshots are a copy of your data in time, so size is relative to what
> you put into ZK.
>
> After snapCount transactions are written to a log file a new snapshot
> is created and a new transaction log file is created. The default
> snapCount is 100,000, you almost never have to change that in my
> experience.
>
> I'd recommend what we have in the sample for init and sync limit
> (effectively 20 and 10 seconds respectively). Although if you have a
> very large snapshot file you might need to increase these. You don't
> want to make sync limit too large, otw it takes longer to identify a
> partition.
>
> Patrick
>
>
> On Wed, Jan 27, 2016 at 10:02 AM, Bruno Wassermann <ba...@hotmail.com> wrote:
> > What influences the size of snapshots and how does one go about figuring out adequate settings for snapCount, initLimit and syncLimit? It would be great, if someone could share some advice.
> > Thanks,
> > -- Bruno
Re: snapCount, initLimit, syncLimit
Posted by Patrick Hunt <ph...@apache.org>.
Snapshots are a copy of your data in time, so size is relative to what
you put into ZK.
After snapCount transactions are written to a log file a new snapshot
is created and a new transaction log file is created. The default
snapCount is 100,000, you almost never have to change that in my
experience.
I'd recommend what we have in the sample for init and sync limit
(effectively 20 and 10 seconds respectively). Although if you have a
very large snapshot file you might need to increase these. You don't
want to make sync limit too large, otw it takes longer to identify a
partition.
Patrick
On Wed, Jan 27, 2016 at 10:02 AM, Bruno Wassermann <ba...@hotmail.com> wrote:
> What influences the size of snapshots and how does one go about figuring out adequate settings for snapCount, initLimit and syncLimit? It would be great, if someone could share some advice.
> Thanks,
> -- Bruno