You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by "Langston, Jim" <Ji...@compuware.com> on 2013/07/18 01:17:25 UTC
sstable size ?
Hi all,
Is there a way to get an SSTable to a smaller size ? By this I mean that I
currently have an SSTable that is nearly 1.2G, so that subsequent SSTables
when they compact are trying to grow to that size. The result is that when
the min_compaction_threshold reaches it value and a compaction is needed,
the compaction is taking a long time as the file grows (it is currently at 52MB and
takes ~22s to compact).
I'm not sure how the SSTable initially grew to its current size of 1.2G, since the
servers have been up for a couple of years. I hadn't noticed until I just upgraded to 1.2.6,
but now I see it affects everything.
Jim
Re: sstable size ?
Posted by Nate McCall <zz...@gmail.com>.
Without digging I'd so no - the SSTable versions will be pretty different.
You could test this pretty easily in isolation though just on a local
instance - I think the issue will be 1.1.6 reading the 1.2.6 SSTable
as the other way should be backwards compatible. Someone jump in if
i'm wrong?
On Thu, Jul 18, 2013 at 3:46 PM, Langston, Jim
<Ji...@compuware.com> wrote:
> Thanks, was heading down that path .. after the build it
> creates a 1.1.6 cassandra snapshot, I'm currently on 1.2.6 - will I
> be able to use the tool ?
>
> Jim
>
> On 7/18/13 3:45 PM, "Nate McCall" <zz...@gmail.com> wrote:
>
>>https://github.com/pcmanus/cassandra/tree/sstable_split/src/java/org/apach
>>e/cassandra/tools
>>
>>You'll have to clone Sylvain's 'sstable_split' branch and build from
>>there.
>>
>>(Commiter folks: this is helpful. @Sylvain - can you commit a patch
>>under this ticket (or wherever):
>>https://issues.apache.org/jira/browse/CASSANDRA-4766 - happy to
>>review).
>>
>>On Thu, Jul 18, 2013 at 1:59 PM, Langston, Jim
>><Ji...@compuware.com> wrote:
>>> I have been looking at the stuff in the zip file, and also the
>>> sstablesplit command script. This script is looking for a java
>>> class StandaloneSplitter located in the package
>>>org.apache.cassandra.tools.
>>>
>>> Where is this package located ? I looked in the lib directory but
>>>nothing
>>> contains
>>> the class. Is this something I need to get as well ?
>>>
>>> Thanks,
>>>
>>> Jim
>>>
>>> From: "Langston, Jim" <Ji...@compuware.com>
>>> Reply-To: <us...@cassandra.apache.org>
>>> Date: Thu, 18 Jul 2013 10:28:39 +0000
>>>
>>> To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
>>> Subject: Re: sstable size ?
>>>
>>> I saw that msg in the thread, I pulled the git files and it looks
>>> like a suite of tools, do I install them on their own ? do I replace the
>>> current ones ? its production data but I can copy the data to where
>>> I want and experiment.
>>>
>>> Jim
>>>
>>> From: aaron morton <aa...@thelastpickle.com>
>>> Reply-To: <us...@cassandra.apache.org>
>>> Date: Thu, 18 Jul 2013 21:41:24 +1200
>>> To: <us...@cassandra.apache.org>
>>> Subject: Re: sstable size ?
>>>
>>> Does this help ?
>>> http://www.mail-archive.com/user@cassandra.apache.org/msg30973.html
>>>
>>> Can you pull the data off the node so you can test it somewhere safe ?
>>>
>>> Cheers
>>>
>>> -----------------
>>> Aaron Morton
>>> Cassandra Consultant
>>> New Zealand
>>>
>>> @aaronmorton
>>> http://www.thelastpickle.com
>>>
>>> On 18/07/2013, at 2:20 PM, "Langston, Jim" <Ji...@compuware.com>
>>> wrote:
>>>
>>> Thanks, this does look like what I'm experiencing. Can someone
>>> post a walkthrough ? The README and the sstablesplit script
>>> don't seem to cover it use in any detail.
>>>
>>> Jim
>>>
>>> From: Colin Blower <cb...@barracuda.com>
>>> Reply-To: <us...@cassandra.apache.org>
>>> Date: Wed, 17 Jul 2013 16:49:59 -0700
>>> To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
>>> Subject: Re: sstable size ?
>>>
>>> Take a look at the very recent thread called 'Alternate "major
>>>compaction"'.
>>> There are some ideas in there about splitting up a large SSTable.
>>>
>>> http://www.mail-archive.com/user@cassandra.apache.org/msg30956.html
>>>
>>>
>>> On 07/17/2013 04:17 PM, Langston, Jim wrote:
>>>
>>> Hi all,
>>>
>>> Is there a way to get an SSTable to a smaller size ? By this I mean
>>>that I
>>> currently have an SSTable that is nearly 1.2G, so that subsequent
>>>SSTables
>>> when they compact are trying to grow to that size. The result is that
>>>when
>>> the min_compaction_threshold reaches it value and a compaction is
>>>needed,
>>> the compaction is taking a long time as the file grows (it is currently
>>>at
>>> 52MB and
>>> takes ~22s to compact).
>>>
>>> I'm not sure how the SSTable initially grew to its current size of 1.2G,
>>> since the
>>> servers have been up for a couple of years. I hadn't noticed until I
>>>just
>>> upgraded to 1.2.6,
>>> but now I see it affects everything.
>>>
>>>
>>> Jim
>>>
>>>
>>>
>>> --
>>> Colin Blower
>>> Software Engineer
>>> Barracuda Networks Inc.
>>> +1 408-342-5576 (o)
>>>
>>>
>>
>
>
Re: sstable size ?
Posted by Nate McCall <zz...@gmail.com>.
Awesome. Thanks Sylvain!
On Jul 19, 2013 3:59 AM, "Sylvain Lebresne" <sy...@datastax.com> wrote:
>
> (Commiter folks: this is helpful. @Sylvain - can you commit a patch
>> under this ticket (or wherever):
>> https://issues.apache.org/jira/browse/CASSANDRA-4766 - happy to
>> review).
>>
>
>
> Right, should have done that sooner anyway. I've attached a rebased
> version to the ticket. Should get in soonish, it's not very complicated.
>
> --
> Sylvain
>
>
>>
>> On Thu, Jul 18, 2013 at 1:59 PM, Langston, Jim
>> <Ji...@compuware.com> wrote:
>> > I have been looking at the stuff in the zip file, and also the
>> > sstablesplit command script. This script is looking for a java
>> > class StandaloneSplitter located in the package
>> org.apache.cassandra.tools.
>> >
>> > Where is this package located ? I looked in the lib directory but
>> nothing
>> > contains
>> > the class. Is this something I need to get as well ?
>> >
>> > Thanks,
>> >
>> > Jim
>> >
>> > From: "Langston, Jim" <Ji...@compuware.com>
>> > Reply-To: <us...@cassandra.apache.org>
>> > Date: Thu, 18 Jul 2013 10:28:39 +0000
>> >
>> > To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
>> > Subject: Re: sstable size ?
>> >
>> > I saw that msg in the thread, I pulled the git files and it looks
>> > like a suite of tools, do I install them on their own ? do I replace the
>> > current ones ? its production data but I can copy the data to where
>> > I want and experiment.
>> >
>> > Jim
>> >
>> > From: aaron morton <aa...@thelastpickle.com>
>> > Reply-To: <us...@cassandra.apache.org>
>> > Date: Thu, 18 Jul 2013 21:41:24 +1200
>> > To: <us...@cassandra.apache.org>
>> > Subject: Re: sstable size ?
>> >
>> > Does this help ?
>> > http://www.mail-archive.com/user@cassandra.apache.org/msg30973.html
>> >
>> > Can you pull the data off the node so you can test it somewhere safe ?
>> >
>> > Cheers
>> >
>> > -----------------
>> > Aaron Morton
>> > Cassandra Consultant
>> > New Zealand
>> >
>> > @aaronmorton
>> > http://www.thelastpickle.com
>> >
>> > On 18/07/2013, at 2:20 PM, "Langston, Jim" <Ji...@compuware.com>
>> > wrote:
>> >
>> > Thanks, this does look like what I'm experiencing. Can someone
>> > post a walkthrough ? The README and the sstablesplit script
>> > don't seem to cover it use in any detail.
>> >
>> > Jim
>> >
>> > From: Colin Blower <cb...@barracuda.com>
>> > Reply-To: <us...@cassandra.apache.org>
>> > Date: Wed, 17 Jul 2013 16:49:59 -0700
>> > To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
>> > Subject: Re: sstable size ?
>> >
>> > Take a look at the very recent thread called 'Alternate "major
>> compaction"'.
>> > There are some ideas in there about splitting up a large SSTable.
>> >
>> > http://www.mail-archive.com/user@cassandra.apache.org/msg30956.html
>> >
>> >
>> > On 07/17/2013 04:17 PM, Langston, Jim wrote:
>> >
>> > Hi all,
>> >
>> > Is there a way to get an SSTable to a smaller size ? By this I mean
>> that I
>> > currently have an SSTable that is nearly 1.2G, so that subsequent
>> SSTables
>> > when they compact are trying to grow to that size. The result is that
>> when
>> > the min_compaction_threshold reaches it value and a compaction is
>> needed,
>> > the compaction is taking a long time as the file grows (it is currently
>> at
>> > 52MB and
>> > takes ~22s to compact).
>> >
>> > I'm not sure how the SSTable initially grew to its current size of 1.2G,
>> > since the
>> > servers have been up for a couple of years. I hadn't noticed until I
>> just
>> > upgraded to 1.2.6,
>> > but now I see it affects everything.
>> >
>> >
>> > Jim
>> >
>> >
>> >
>> > --
>> > Colin Blower
>> > Software Engineer
>> > Barracuda Networks Inc.
>> > +1 408-342-5576 (o)
>> >
>> >
>>
>
>
Re: sstable size ?
Posted by Sylvain Lebresne <sy...@datastax.com>.
> (Commiter folks: this is helpful. @Sylvain - can you commit a patch
> under this ticket (or wherever):
> https://issues.apache.org/jira/browse/CASSANDRA-4766 - happy to
> review).
>
Right, should have done that sooner anyway. I've attached a rebased version
to the ticket. Should get in soonish, it's not very complicated.
--
Sylvain
>
> On Thu, Jul 18, 2013 at 1:59 PM, Langston, Jim
> <Ji...@compuware.com> wrote:
> > I have been looking at the stuff in the zip file, and also the
> > sstablesplit command script. This script is looking for a java
> > class StandaloneSplitter located in the package
> org.apache.cassandra.tools.
> >
> > Where is this package located ? I looked in the lib directory but nothing
> > contains
> > the class. Is this something I need to get as well ?
> >
> > Thanks,
> >
> > Jim
> >
> > From: "Langston, Jim" <Ji...@compuware.com>
> > Reply-To: <us...@cassandra.apache.org>
> > Date: Thu, 18 Jul 2013 10:28:39 +0000
> >
> > To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
> > Subject: Re: sstable size ?
> >
> > I saw that msg in the thread, I pulled the git files and it looks
> > like a suite of tools, do I install them on their own ? do I replace the
> > current ones ? its production data but I can copy the data to where
> > I want and experiment.
> >
> > Jim
> >
> > From: aaron morton <aa...@thelastpickle.com>
> > Reply-To: <us...@cassandra.apache.org>
> > Date: Thu, 18 Jul 2013 21:41:24 +1200
> > To: <us...@cassandra.apache.org>
> > Subject: Re: sstable size ?
> >
> > Does this help ?
> > http://www.mail-archive.com/user@cassandra.apache.org/msg30973.html
> >
> > Can you pull the data off the node so you can test it somewhere safe ?
> >
> > Cheers
> >
> > -----------------
> > Aaron Morton
> > Cassandra Consultant
> > New Zealand
> >
> > @aaronmorton
> > http://www.thelastpickle.com
> >
> > On 18/07/2013, at 2:20 PM, "Langston, Jim" <Ji...@compuware.com>
> > wrote:
> >
> > Thanks, this does look like what I'm experiencing. Can someone
> > post a walkthrough ? The README and the sstablesplit script
> > don't seem to cover it use in any detail.
> >
> > Jim
> >
> > From: Colin Blower <cb...@barracuda.com>
> > Reply-To: <us...@cassandra.apache.org>
> > Date: Wed, 17 Jul 2013 16:49:59 -0700
> > To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
> > Subject: Re: sstable size ?
> >
> > Take a look at the very recent thread called 'Alternate "major
> compaction"'.
> > There are some ideas in there about splitting up a large SSTable.
> >
> > http://www.mail-archive.com/user@cassandra.apache.org/msg30956.html
> >
> >
> > On 07/17/2013 04:17 PM, Langston, Jim wrote:
> >
> > Hi all,
> >
> > Is there a way to get an SSTable to a smaller size ? By this I mean that
> I
> > currently have an SSTable that is nearly 1.2G, so that subsequent
> SSTables
> > when they compact are trying to grow to that size. The result is that
> when
> > the min_compaction_threshold reaches it value and a compaction is needed,
> > the compaction is taking a long time as the file grows (it is currently
> at
> > 52MB and
> > takes ~22s to compact).
> >
> > I'm not sure how the SSTable initially grew to its current size of 1.2G,
> > since the
> > servers have been up for a couple of years. I hadn't noticed until I just
> > upgraded to 1.2.6,
> > but now I see it affects everything.
> >
> >
> > Jim
> >
> >
> >
> > --
> > Colin Blower
> > Software Engineer
> > Barracuda Networks Inc.
> > +1 408-342-5576 (o)
> >
> >
>
Re: sstable size ?
Posted by "Langston, Jim" <Ji...@compuware.com>.
Thanks, was heading down that path .. after the build it
creates a 1.1.6 cassandra snapshot, I'm currently on 1.2.6 - will I
be able to use the tool ?
Jim
On 7/18/13 3:45 PM, "Nate McCall" <zz...@gmail.com> wrote:
>https://github.com/pcmanus/cassandra/tree/sstable_split/src/java/org/apach
>e/cassandra/tools
>
>You'll have to clone Sylvain's 'sstable_split' branch and build from
>there.
>
>(Commiter folks: this is helpful. @Sylvain - can you commit a patch
>under this ticket (or wherever):
>https://issues.apache.org/jira/browse/CASSANDRA-4766 - happy to
>review).
>
>On Thu, Jul 18, 2013 at 1:59 PM, Langston, Jim
><Ji...@compuware.com> wrote:
>> I have been looking at the stuff in the zip file, and also the
>> sstablesplit command script. This script is looking for a java
>> class StandaloneSplitter located in the package
>>org.apache.cassandra.tools.
>>
>> Where is this package located ? I looked in the lib directory but
>>nothing
>> contains
>> the class. Is this something I need to get as well ?
>>
>> Thanks,
>>
>> Jim
>>
>> From: "Langston, Jim" <Ji...@compuware.com>
>> Reply-To: <us...@cassandra.apache.org>
>> Date: Thu, 18 Jul 2013 10:28:39 +0000
>>
>> To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
>> Subject: Re: sstable size ?
>>
>> I saw that msg in the thread, I pulled the git files and it looks
>> like a suite of tools, do I install them on their own ? do I replace the
>> current ones ? its production data but I can copy the data to where
>> I want and experiment.
>>
>> Jim
>>
>> From: aaron morton <aa...@thelastpickle.com>
>> Reply-To: <us...@cassandra.apache.org>
>> Date: Thu, 18 Jul 2013 21:41:24 +1200
>> To: <us...@cassandra.apache.org>
>> Subject: Re: sstable size ?
>>
>> Does this help ?
>> http://www.mail-archive.com/user@cassandra.apache.org/msg30973.html
>>
>> Can you pull the data off the node so you can test it somewhere safe ?
>>
>> Cheers
>>
>> -----------------
>> Aaron Morton
>> Cassandra Consultant
>> New Zealand
>>
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 18/07/2013, at 2:20 PM, "Langston, Jim" <Ji...@compuware.com>
>> wrote:
>>
>> Thanks, this does look like what I'm experiencing. Can someone
>> post a walkthrough ? The README and the sstablesplit script
>> don't seem to cover it use in any detail.
>>
>> Jim
>>
>> From: Colin Blower <cb...@barracuda.com>
>> Reply-To: <us...@cassandra.apache.org>
>> Date: Wed, 17 Jul 2013 16:49:59 -0700
>> To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
>> Subject: Re: sstable size ?
>>
>> Take a look at the very recent thread called 'Alternate "major
>>compaction"'.
>> There are some ideas in there about splitting up a large SSTable.
>>
>> http://www.mail-archive.com/user@cassandra.apache.org/msg30956.html
>>
>>
>> On 07/17/2013 04:17 PM, Langston, Jim wrote:
>>
>> Hi all,
>>
>> Is there a way to get an SSTable to a smaller size ? By this I mean
>>that I
>> currently have an SSTable that is nearly 1.2G, so that subsequent
>>SSTables
>> when they compact are trying to grow to that size. The result is that
>>when
>> the min_compaction_threshold reaches it value and a compaction is
>>needed,
>> the compaction is taking a long time as the file grows (it is currently
>>at
>> 52MB and
>> takes ~22s to compact).
>>
>> I'm not sure how the SSTable initially grew to its current size of 1.2G,
>> since the
>> servers have been up for a couple of years. I hadn't noticed until I
>>just
>> upgraded to 1.2.6,
>> but now I see it affects everything.
>>
>>
>> Jim
>>
>>
>>
>> --
>> Colin Blower
>> Software Engineer
>> Barracuda Networks Inc.
>> +1 408-342-5576 (o)
>>
>>
>
Re: sstable size ?
Posted by Nate McCall <zz...@gmail.com>.
https://github.com/pcmanus/cassandra/tree/sstable_split/src/java/org/apache/cassandra/tools
You'll have to clone Sylvain's 'sstable_split' branch and build from there.
(Commiter folks: this is helpful. @Sylvain - can you commit a patch
under this ticket (or wherever):
https://issues.apache.org/jira/browse/CASSANDRA-4766 - happy to
review).
On Thu, Jul 18, 2013 at 1:59 PM, Langston, Jim
<Ji...@compuware.com> wrote:
> I have been looking at the stuff in the zip file, and also the
> sstablesplit command script. This script is looking for a java
> class StandaloneSplitter located in the package org.apache.cassandra.tools.
>
> Where is this package located ? I looked in the lib directory but nothing
> contains
> the class. Is this something I need to get as well ?
>
> Thanks,
>
> Jim
>
> From: "Langston, Jim" <Ji...@compuware.com>
> Reply-To: <us...@cassandra.apache.org>
> Date: Thu, 18 Jul 2013 10:28:39 +0000
>
> To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
> Subject: Re: sstable size ?
>
> I saw that msg in the thread, I pulled the git files and it looks
> like a suite of tools, do I install them on their own ? do I replace the
> current ones ? its production data but I can copy the data to where
> I want and experiment.
>
> Jim
>
> From: aaron morton <aa...@thelastpickle.com>
> Reply-To: <us...@cassandra.apache.org>
> Date: Thu, 18 Jul 2013 21:41:24 +1200
> To: <us...@cassandra.apache.org>
> Subject: Re: sstable size ?
>
> Does this help ?
> http://www.mail-archive.com/user@cassandra.apache.org/msg30973.html
>
> Can you pull the data off the node so you can test it somewhere safe ?
>
> Cheers
>
> -----------------
> Aaron Morton
> Cassandra Consultant
> New Zealand
>
> @aaronmorton
> http://www.thelastpickle.com
>
> On 18/07/2013, at 2:20 PM, "Langston, Jim" <Ji...@compuware.com>
> wrote:
>
> Thanks, this does look like what I'm experiencing. Can someone
> post a walkthrough ? The README and the sstablesplit script
> don't seem to cover it use in any detail.
>
> Jim
>
> From: Colin Blower <cb...@barracuda.com>
> Reply-To: <us...@cassandra.apache.org>
> Date: Wed, 17 Jul 2013 16:49:59 -0700
> To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
> Subject: Re: sstable size ?
>
> Take a look at the very recent thread called 'Alternate "major compaction"'.
> There are some ideas in there about splitting up a large SSTable.
>
> http://www.mail-archive.com/user@cassandra.apache.org/msg30956.html
>
>
> On 07/17/2013 04:17 PM, Langston, Jim wrote:
>
> Hi all,
>
> Is there a way to get an SSTable to a smaller size ? By this I mean that I
> currently have an SSTable that is nearly 1.2G, so that subsequent SSTables
> when they compact are trying to grow to that size. The result is that when
> the min_compaction_threshold reaches it value and a compaction is needed,
> the compaction is taking a long time as the file grows (it is currently at
> 52MB and
> takes ~22s to compact).
>
> I'm not sure how the SSTable initially grew to its current size of 1.2G,
> since the
> servers have been up for a couple of years. I hadn't noticed until I just
> upgraded to 1.2.6,
> but now I see it affects everything.
>
>
> Jim
>
>
>
> --
> Colin Blower
> Software Engineer
> Barracuda Networks Inc.
> +1 408-342-5576 (o)
>
>
Re: sstable size ?
Posted by "Langston, Jim" <Ji...@compuware.com>.
I have been looking at the stuff in the zip file, and also the
sstablesplit command script. This script is looking for a java
class StandaloneSplitter located in the package org.apache.cassandra.tools.
Where is this package located ? I looked in the lib directory but nothing contains
the class. Is this something I need to get as well ?
Thanks,
Jim
From: "Langston, Jim" <Ji...@compuware.com>>
Reply-To: <us...@cassandra.apache.org>>
Date: Thu, 18 Jul 2013 10:28:39 +0000
To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Subject: Re: sstable size ?
I saw that msg in the thread, I pulled the git files and it looks
like a suite of tools, do I install them on their own ? do I replace the
current ones ? its production data but I can copy the data to where
I want and experiment.
Jim
From: aaron morton <aa...@thelastpickle.com>>
Reply-To: <us...@cassandra.apache.org>>
Date: Thu, 18 Jul 2013 21:41:24 +1200
To: <us...@cassandra.apache.org>>
Subject: Re: sstable size ?
Does this help ? http://www.mail-archive.com/user@cassandra.apache.org/msg30973.html
Can you pull the data off the node so you can test it somewhere safe ?
Cheers
-----------------
Aaron Morton
Cassandra Consultant
New Zealand
@aaronmorton
http://www.thelastpickle.com
On 18/07/2013, at 2:20 PM, "Langston, Jim" <Ji...@compuware.com>> wrote:
Thanks, this does look like what I'm experiencing. Can someone
post a walkthrough ? The README and the sstablesplit script
don't seem to cover it use in any detail.
Jim
From: Colin Blower <cb...@barracuda.com>>
Reply-To: <us...@cassandra.apache.org>>
Date: Wed, 17 Jul 2013 16:49:59 -0700
To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Subject: Re: sstable size ?
Take a look at the very recent thread called 'Alternate "major compaction"'. There are some ideas in there about splitting up a large SSTable.
http://www.mail-archive.com/user@cassandra.apache.org/msg30956.html
On 07/17/2013 04:17 PM, Langston, Jim wrote:
Hi all,
Is there a way to get an SSTable to a smaller size ? By this I mean that I
currently have an SSTable that is nearly 1.2G, so that subsequent SSTables
when they compact are trying to grow to that size. The result is that when
the min_compaction_threshold reaches it value and a compaction is needed,
the compaction is taking a long time as the file grows (it is currently at 52MB and
takes ~22s to compact).
I'm not sure how the SSTable initially grew to its current size of 1.2G, since the
servers have been up for a couple of years. I hadn't noticed until I just upgraded to 1.2.6,
but now I see it affects everything.
Jim
--
Colin Blower
Software Engineer
Barracuda Networks Inc.
+1 408-342-5576 (o)
Re: sstable size ?
Posted by "Langston, Jim" <Ji...@compuware.com>.
I saw that msg in the thread, I pulled the git files and it looks
like a suite of tools, do I install them on their own ? do I replace the
current ones ? its production data but I can copy the data to where
I want and experiment.
Jim
From: aaron morton <aa...@thelastpickle.com>>
Reply-To: <us...@cassandra.apache.org>>
Date: Thu, 18 Jul 2013 21:41:24 +1200
To: <us...@cassandra.apache.org>>
Subject: Re: sstable size ?
Does this help ? http://www.mail-archive.com/user@cassandra.apache.org/msg30973.html
Can you pull the data off the node so you can test it somewhere safe ?
Cheers
-----------------
Aaron Morton
Cassandra Consultant
New Zealand
@aaronmorton
http://www.thelastpickle.com
On 18/07/2013, at 2:20 PM, "Langston, Jim" <Ji...@compuware.com>> wrote:
Thanks, this does look like what I'm experiencing. Can someone
post a walkthrough ? The README and the sstablesplit script
don't seem to cover it use in any detail.
Jim
From: Colin Blower <cb...@barracuda.com>>
Reply-To: <us...@cassandra.apache.org>>
Date: Wed, 17 Jul 2013 16:49:59 -0700
To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Subject: Re: sstable size ?
Take a look at the very recent thread called 'Alternate "major compaction"'. There are some ideas in there about splitting up a large SSTable.
http://www.mail-archive.com/user@cassandra.apache.org/msg30956.html
On 07/17/2013 04:17 PM, Langston, Jim wrote:
Hi all,
Is there a way to get an SSTable to a smaller size ? By this I mean that I
currently have an SSTable that is nearly 1.2G, so that subsequent SSTables
when they compact are trying to grow to that size. The result is that when
the min_compaction_threshold reaches it value and a compaction is needed,
the compaction is taking a long time as the file grows (it is currently at 52MB and
takes ~22s to compact).
I'm not sure how the SSTable initially grew to its current size of 1.2G, since the
servers have been up for a couple of years. I hadn't noticed until I just upgraded to 1.2.6,
but now I see it affects everything.
Jim
--
Colin Blower
Software Engineer
Barracuda Networks Inc.
+1 408-342-5576 (o)
Re: sstable size ?
Posted by aaron morton <aa...@thelastpickle.com>.
Does this help ? http://www.mail-archive.com/user@cassandra.apache.org/msg30973.html
Can you pull the data off the node so you can test it somewhere safe ?
Cheers
-----------------
Aaron Morton
Cassandra Consultant
New Zealand
@aaronmorton
http://www.thelastpickle.com
On 18/07/2013, at 2:20 PM, "Langston, Jim" <Ji...@compuware.com> wrote:
> Thanks, this does look like what I'm experiencing. Can someone
> post a walkthrough ? The README and the sstablesplit script
> don't seem to cover it use in any detail.
>
> Jim
>
> From: Colin Blower <cb...@barracuda.com>
> Reply-To: <us...@cassandra.apache.org>
> Date: Wed, 17 Jul 2013 16:49:59 -0700
> To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
> Subject: Re: sstable size ?
>
> Take a look at the very recent thread called 'Alternate "major compaction"'. There are some ideas in there about splitting up a large SSTable.
>
> http://www.mail-archive.com/user@cassandra.apache.org/msg30956.html
>
>
> On 07/17/2013 04:17 PM, Langston, Jim wrote:
>> Hi all,
>>
>> Is there a way to get an SSTable to a smaller size ? By this I mean that I
>> currently have an SSTable that is nearly 1.2G, so that subsequent SSTables
>> when they compact are trying to grow to that size. The result is that when
>> the min_compaction_threshold reaches it value and a compaction is needed,
>> the compaction is taking a long time as the file grows (it is currently at 52MB and
>> takes ~22s to compact).
>>
>> I'm not sure how the SSTable initially grew to its current size of 1.2G, since the
>> servers have been up for a couple of years. I hadn't noticed until I just upgraded to 1.2.6,
>> but now I see it affects everything.
>>
>>
>> Jim
>
>
> --
> Colin Blower
> Software Engineer
> Barracuda Networks Inc.
> +1 408-342-5576 (o)
Re: sstable size ?
Posted by "Langston, Jim" <Ji...@compuware.com>.
Thanks, this does look like what I'm experiencing. Can someone
post a walkthrough ? The README and the sstablesplit script
don't seem to cover it use in any detail.
Jim
From: Colin Blower <cb...@barracuda.com>>
Reply-To: <us...@cassandra.apache.org>>
Date: Wed, 17 Jul 2013 16:49:59 -0700
To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Subject: Re: sstable size ?
Take a look at the very recent thread called 'Alternate "major compaction"'. There are some ideas in there about splitting up a large SSTable.
http://www.mail-archive.com/user@cassandra.apache.org/msg30956.html
On 07/17/2013 04:17 PM, Langston, Jim wrote:
Hi all,
Is there a way to get an SSTable to a smaller size ? By this I mean that I
currently have an SSTable that is nearly 1.2G, so that subsequent SSTables
when they compact are trying to grow to that size. The result is that when
the min_compaction_threshold reaches it value and a compaction is needed,
the compaction is taking a long time as the file grows (it is currently at 52MB and
takes ~22s to compact).
I'm not sure how the SSTable initially grew to its current size of 1.2G, since the
servers have been up for a couple of years. I hadn't noticed until I just upgraded to 1.2.6,
but now I see it affects everything.
Jim
--
Colin Blower
Software Engineer
Barracuda Networks Inc.
+1 408-342-5576 (o)
Re: sstable size ?
Posted by Colin Blower <cb...@barracuda.com>.
Take a look at the very recent thread called 'Alternate "major
compaction"'. There are some ideas in there about splitting up a large
SSTable.
http://www.mail-archive.com/user@cassandra.apache.org/msg30956.html
On 07/17/2013 04:17 PM, Langston, Jim wrote:
> Hi all,
>
> Is there a way to get an SSTable to a smaller size ? By this I mean
> that I
> currently have an SSTable that is nearly 1.2G, so that subsequent SSTables
> when they compact are trying to grow to that size. The result is that
> when
> the min_compaction_threshold reaches it value and a compaction is needed,
> the compaction is taking a long time as the file grows (it is
> currently at 52MB and
> takes ~22s to compact).
>
> I'm not sure how the SSTable initially grew to its current size of
> 1.2G, since the
> servers have been up for a couple of years. I hadn't noticed until I
> just upgraded to 1.2.6,
> but now I see it affects everything.
>
>
> Jim
--
*Colin Blower*
/Software Engineer/
Barracuda Networks Inc.
+1 408-342-5576 (o)