You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Christian Lorenz <Ch...@webtrekk.com> on 2017/12/07 11:25:08 UTC
Huge system.batches table after joining a node (Cassandra 3.11.1)
Hi,
after joining a node into an existing cluster, the table system.batches became quite large (100GB) which is about 1/3 of the nodes size.
Is it safe to truncate the table?
Regards,
Christian
Re: Huge system.batches table after joining a node (Cassandra 3.11.1)
Posted by Alexander Dejanovski <al...@thelastpickle.com>.
Just a heads up that (in case you missed it) MVs were retroactively marked
as experimental and that a large part of the community considers they
should not be used in production.
On Thu, Dec 7, 2017 at 4:53 PM Alexander Dejanovski <al...@thelastpickle.com>
wrote:
> Yes, MVs use batches during bootstraps and decommissions.
>
> You can read more about it here :
> https://issues.apache.org/jira/browse/CASSANDRA-13065
> and here : https://issues.apache.org/jira/browse/CASSANDRA-13614
>
> Things will improve in 4.0 only it seems.
>
> On Thu, Dec 7, 2017 at 4:31 PM Christian Lorenz <
> Christian.Lorenz@webtrekk.com> wrote:
>
>> Hi Alexander,
>>
>>
>>
>> yes we use MV’s. The size of the batch table is around 10GB on the
>> existing nodes. Also seems pretty high.
>>
>> So is this table (also) used to process MV building?
>>
>>
>>
>> Regards,
>>
>> Christian
>>
>> *Von: *Alexander Dejanovski <al...@thelastpickle.com>
>> *Antworten an: *"user@cassandra.apache.org" <us...@cassandra.apache.org>
>> *Datum: *Donnerstag, 7. Dezember 2017 um 16:24
>> *An: *"user@cassandra.apache.org" <us...@cassandra.apache.org>
>> *Betreff: *Re: Huge system.batches table after joining a node (Cassandra
>> 3.11.1)
>>
>>
>>
>> Hi Christian,
>>
>>
>>
>> it is probably not safe to drop it because it contains all logged batches
>> that are supposed to be played on the cluster.
>>
>> The size of the batches table should go down as they get processed
>> (although 100GB is a pretty huge batch log...)
>>
>>
>>
>> Do you use Materialized Views in your data model ?
>>
>> You just bootstrapped a new node and the table grew on all other nodes ?
>>
>>
>>
>> On Thu, Dec 7, 2017 at 12:25 PM Christian Lorenz <
>> Christian.Lorenz@webtrekk.com> wrote:
>>
>> Hi,
>>
>>
>>
>> after joining a node into an existing cluster, the table system.batches
>> became quite large (100GB) which is about 1/3 of the nodes size.
>>
>> Is it safe to truncate the table?
>>
>>
>>
>> Regards,
>>
>> Christian
>>
>>
>>
>> --
>>
>> -----------------
>>
>> Alexander Dejanovski
>>
>> France
>>
>> @alexanderdeja
>>
>>
>>
>> Consultant
>>
>> Apache Cassandra Consulting
>>
>> http://www.thelastpickle.com
>>
> --
> -----------------
> Alexander Dejanovski
> France
> @alexanderdeja
>
> Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com
>
--
-----------------
Alexander Dejanovski
France
@alexanderdeja
Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com
Re: Huge system.batches table after joining a node (Cassandra 3.11.1)
Posted by Alexander Dejanovski <al...@thelastpickle.com>.
Yes, MVs use batches during bootstraps and decommissions.
You can read more about it here :
https://issues.apache.org/jira/browse/CASSANDRA-13065
and here : https://issues.apache.org/jira/browse/CASSANDRA-13614
Things will improve in 4.0 only it seems.
On Thu, Dec 7, 2017 at 4:31 PM Christian Lorenz <
Christian.Lorenz@webtrekk.com> wrote:
> Hi Alexander,
>
>
>
> yes we use MV’s. The size of the batch table is around 10GB on the
> existing nodes. Also seems pretty high.
>
> So is this table (also) used to process MV building?
>
>
>
> Regards,
>
> Christian
>
> *Von: *Alexander Dejanovski <al...@thelastpickle.com>
> *Antworten an: *"user@cassandra.apache.org" <us...@cassandra.apache.org>
> *Datum: *Donnerstag, 7. Dezember 2017 um 16:24
> *An: *"user@cassandra.apache.org" <us...@cassandra.apache.org>
> *Betreff: *Re: Huge system.batches table after joining a node (Cassandra
> 3.11.1)
>
>
>
> Hi Christian,
>
>
>
> it is probably not safe to drop it because it contains all logged batches
> that are supposed to be played on the cluster.
>
> The size of the batches table should go down as they get processed
> (although 100GB is a pretty huge batch log...)
>
>
>
> Do you use Materialized Views in your data model ?
>
> You just bootstrapped a new node and the table grew on all other nodes ?
>
>
>
> On Thu, Dec 7, 2017 at 12:25 PM Christian Lorenz <
> Christian.Lorenz@webtrekk.com> wrote:
>
> Hi,
>
>
>
> after joining a node into an existing cluster, the table system.batches
> became quite large (100GB) which is about 1/3 of the nodes size.
>
> Is it safe to truncate the table?
>
>
>
> Regards,
>
> Christian
>
>
>
> --
>
> -----------------
>
> Alexander Dejanovski
>
> France
>
> @alexanderdeja
>
>
>
> Consultant
>
> Apache Cassandra Consulting
>
> http://www.thelastpickle.com
>
--
-----------------
Alexander Dejanovski
France
@alexanderdeja
Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com
Re: Huge system.batches table after joining a node (Cassandra 3.11.1)
Posted by Christian Lorenz <Ch...@webtrekk.com>.
Hi Alexander,
yes we use MV’s. The size of the batch table is around 10GB on the existing nodes. Also seems pretty high.
So is this table (also) used to process MV building?
Regards,
Christian
Von: Alexander Dejanovski <al...@thelastpickle.com>
Antworten an: "user@cassandra.apache.org" <us...@cassandra.apache.org>
Datum: Donnerstag, 7. Dezember 2017 um 16:24
An: "user@cassandra.apache.org" <us...@cassandra.apache.org>
Betreff: Re: Huge system.batches table after joining a node (Cassandra 3.11.1)
Hi Christian,
it is probably not safe to drop it because it contains all logged batches that are supposed to be played on the cluster.
The size of the batches table should go down as they get processed (although 100GB is a pretty huge batch log...)
Do you use Materialized Views in your data model ?
You just bootstrapped a new node and the table grew on all other nodes ?
On Thu, Dec 7, 2017 at 12:25 PM Christian Lorenz <Ch...@webtrekk.com>> wrote:
Hi,
after joining a node into an existing cluster, the table system.batches became quite large (100GB) which is about 1/3 of the nodes size.
Is it safe to truncate the table?
Regards,
Christian
--
-----------------
Alexander Dejanovski
France
@alexanderdeja
Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com<http://www.thelastpickle.com/>
Re: Huge system.batches table after joining a node (Cassandra 3.11.1)
Posted by Alexander Dejanovski <al...@thelastpickle.com>.
Hi Christian,
it is probably not safe to drop it because it contains all logged batches
that are supposed to be played on the cluster.
The size of the batches table should go down as they get processed
(although 100GB is a pretty huge batch log...)
Do you use Materialized Views in your data model ?
You just bootstrapped a new node and the table grew on all other nodes ?
On Thu, Dec 7, 2017 at 12:25 PM Christian Lorenz <
Christian.Lorenz@webtrekk.com> wrote:
> Hi,
>
>
>
> after joining a node into an existing cluster, the table system.batches
> became quite large (100GB) which is about 1/3 of the nodes size.
>
> Is it safe to truncate the table?
>
>
>
> Regards,
>
> Christian
>
>
>
--
-----------------
Alexander Dejanovski
France
@alexanderdeja
Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com