You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Zhenya Stanilovsky <ar...@mail.ru> on 2021/12/01 06:15:35 UTC
Re[2]: AW: [2.8.1] Having more backups make SQL queries slower.
hello Maximiliano Gazquez, good question ! But there is no one strength answer on it.
> I assumed that queries are distributed and each node answers the query only with its primary partitions and adding backups wouldn’t affect performance.
Ok, but what about overall system performance degradation ? Check page replacement [1] algo, more efficient was introduced in new versions, thus upgrade to new ver is welcomes. Second — wal it`s all about io usage. Do you have monitoring of your disk io activity ? 4 Backups means 5 nodes with equal data, is it really necessary or you just make a research ? Also additional work for index rebuilding i hope.
[1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-62+Page+replacement+improvements
>My problem is 100% with queries, not writes.
>It’s the same cluster, same hardware, but a LOT slower when using 4 backups instead of 2.
>
>Is there any metric that I could check to find out what’s happening?
>
>Thanks!
>On 30 Nov 2021 15:42 -0300, Henrik < hover@magenta.de >, wrote:
>>With more backups the cluster has the worse writing performance since data will be copied by multiple times. But the reading performance should be increased since each node answers the request from local backup.
>>
>>Thanks
>>
>>
>>Gesendet mit der Telekom Mail App
>>-----Original-Nachricht-----
>>Von: Maximiliano Gazquez < maximiliano.628@gmail.com >
>>Betreff: [2.8.1] Having more backups make SQL queries slower.
>>Datum: 01.12.2021, 00:12 Uhr
>>An: < user@ignite.apache.org >
>>Hello everyone.
>>
>>We are doing some testing in a 10 node cluster which we use as a distributed database with persistence enabled.
>>Each node has 6gb region size + 5gb heap.
>>All caches are partitioned, and I connect to the cluster using the thin client.
>>
>>I’ve found a performance issue:
>>* With 2 backups, the performance is pretty great.
>>* With 4 backups the performance is really bad.
>>So I wanted to ask why would this happen.
>>
>>I assumed that queries are distributed and each node answers the query only with its primary partitions and adding backups wouldn’t affect performance.
>>
>>Thanks everyone!
Re: [2.8.1] Having more backups make SQL queries slower.
Posted by Stephen Darlington <st...@gridgain.com>.
Just to be explicit: you’ll need nearly twice as much memory if you increase the number of backups from 2 to 4. With the same size cluster, chances are a lot of the data is going to be on-disk rather than in-memory.
It’s very unusual to need five copies of the data. Backups=1 is sufficient for most use cases.
> On 1 Dec 2021, at 06:15, Zhenya Stanilovsky <ar...@mail.ru> wrote:
>
> hello Maximiliano Gazquez, good question ! But there is no one strength answer on it.
>
> > I assumed that queries are distributed and each node answers the query only with its primary partitions and adding backups wouldn’t affect performance.
> Ok, but what about overall system performance degradation ? Check page replacement [1] algo, more efficient was introduced in new versions, thus upgrade to new ver is welcomes. Second — wal it`s all about io usage. Do you have monitoring of your disk io activity ? 4 Backups means 5 nodes with equal data, is it really necessary or you just make a research ? Also additional work for index rebuilding i hope.
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-62+Page+replacement+improvements <https://cwiki.apache.org/confluence/display/IGNITE/IEP-62+Page+replacement+improvements>
>
>
> My problem is 100% with queries, not writes.
> It’s the same cluster, same hardware, but a LOT slower when using 4 backups instead of 2.
>
> Is there any metric that I could check to find out what’s happening?
>
> Thanks!
> On 30 Nov 2021 15:42 -0300, Henrik <hover@magenta.de <x-...@magenta.de>>, wrote:
>> With more backups the cluster has the worse writing performance since data will be copied by multiple times. But the reading performance should be increased since each node answers the request from local backup.
>>
>> Thanks
>>
>>
>> Gesendet mit der Telekom Mail App <http://www.t-online.de/service/redir/emailmobilapp_ios_smartphone_footerlink.htm>
>> -----Original-Nachricht-----
>> Von: Maximiliano Gazquez <maximiliano.628@gmail.com <x-...@gmail.com>>
>> Betreff: [2.8.1] Having more backups make SQL queries slower.
>> Datum: 01.12.2021, 00:12 Uhr
>> An: <user@ignite.apache.org <x-...@ignite.apache.org>>
>>
>> Hello everyone.
>>
>> We are doing some testing in a 10 node cluster which we use as a distributed database with persistence enabled.
>> Each node has 6gb region size + 5gb heap.
>> All caches are partitioned, and I connect to the cluster using the thin client.
>>
>> I’ve found a performance issue:
>> With 2 backups, the performance is pretty great.
>> With 4 backups the performance is really bad.
>> So I wanted to ask why would this happen.
>>
>> I assumed that queries are distributed and each node answers the query only with its primary partitions and adding backups wouldn’t affect performance.
>>
>> Thanks everyone!
>
>
>
>