Increasing Default Lock Table Size

[Chen Luo <cl...@uci.edu>]

Hi devs,

Recently I found an issue that our default lock table size (1024) is too
small to avoid hash collisions. Currently, our default log buffer size is
4MB * 8 = 32MB, which means there could be at most 32MB unflushed log
records. Suppose each write produces 100 bytes log records (worse case),
then there could be 320K ongoing operations at most. Thus, in
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/8003, I was trying to
increase the default lock table size to 1MB. However, this fails certain
test cases because of out-of-memory.

I'm thinking that maybe we should introduce a new parameter, e.g.,
txn.lock.table.size with default value 1MB, to make the lock size
configurable. We could still use a smaller lock table size to avoid test
case failures. Please let me know your thoughts.

Best regards,
Chen Luo

Re: Increasing Default Lock Table Size

[Taewoo Kim <wa...@gmail.com>]

Hi Chen,

I think making the lock table size configurable is a good idea. I think it
would be nice if we could document how the size of the lock table affects
AsterixDB's operation and document possible side effects of increasing lock
table size including OOM. Also, it might be good if we could find a new
default value that fits?

Best,
Taewoo


On Thu, Sep 24, 2020 at 9:20 AM Chen Luo <cl...@uci.edu> wrote:

> Hi devs,
>
> Recently I found an issue that our default lock table size (1024) is too
> small to avoid hash collisions. Currently, our default log buffer size is
> 4MB * 8 = 32MB, which means there could be at most 32MB unflushed log
> records. Suppose each write produces 100 bytes log records (worse case),
> then there could be 320K ongoing operations at most. Thus, in
> https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/8003, I was trying to
> increase the default lock table size to 1MB. However, this fails certain
> test cases because of out-of-memory.
>
> I'm thinking that maybe we should introduce a new parameter, e.g.,
> txn.lock.table.size with default value 1MB, to make the lock size
> configurable. We could still use a smaller lock table size to avoid test
> case failures. Please let me know your thoughts.
>
> Best regards,
> Chen Luo
>