You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Hannu Kröger <hk...@gmail.com> on 2017/08/28 13:02:43 UTC
Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Hello,
Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has a race
condition that causes Cassandra to create broken sstables (stats file in
sstables to be precise).
Bug described here:
https://issues.apache.org/jira/browse/CASSANDRA-13752
This change might be causing it (but not sure):
https://issues.apache.org/jira/browse/CASSANDRA-13038
Other related issues:
https://issues.apache.org/jira/browse/CASSANDRA-13718
https://issues.apache.org/jira/browse/CASSANDRA-13756
I would not recommend using 3.11.0 nor upgrading to 3.0.12 or higher before
this is fixed.
Cheers,
Hannu
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by Jeff Jirsa <jj...@gmail.com>.
On the user side: 3.11.0 or 3.0.14 is probably what you want to use for new
clusters as long as you can avoid calling the getTombstoneRatio() mbean
with your metrics gathering tools.
On Tue, Sep 12, 2017 at 8:35 AM, Sandeep S <sk...@gmail.com> wrote:
> Hi,
>
> What is the current stable version?
>
> Sandeep.
>
> On 11 September 2017 at 23:06, CPC <ac...@gmail.com> wrote:
>
> > Hi,
> >
> > Is this bug fixed in dse 5.1.3? As I understand calling jmx
> > getTombStoneRatio
> > trigers that bug. We are using opscenter as well and do you have any idea
> > whether opscenter using/calling this method?
> >
> > Thanks
> >
> > On Aug 29, 2017 6:35 AM, "Jeff Jirsa" <jj...@gmail.com> wrote:
> >
> > > I shouldn't actually say I don't think it can happen on 3.0 - I haven't
> > > seen this happen on 3.0 without some other code change to enable it,
> but
> > > like I said, we're still investigating.
> > >
> > > --
> > > Jeff Jirsa
> > >
> > >
> > > > On Aug 28, 2017, at 8:30 PM, Jeff Jirsa <jj...@gmail.com> wrote:
> > > >
> > > > For what it's worth, I don't think this impacts 3.0 without adding
> some
> > > other code change (the reporter of the bug on 3.0 had added custom
> > metrics
> > > that exposed a concurrency issue).
> > > >
> > > > We're looking at it on 3.11. I think 13038 made it far more likely to
> > > occur, but I think it could have happened pre-13038 as well (would take
> > > some serious luck with your deletion time distribution though - the
> > > rounding in 13038 does make it more likely, but the race was already
> > there).
> > > >
> > > > --
> > > > Jeff Jirsa
> > > >
> > > >
> > > >> On Aug 28, 2017, at 8:24 PM, Jay Zhuang
> <jay.zhuang@yahoo.com.INVALID
> > >
> > > wrote:
> > > >>
> > > >> We're using 3.0.12+ for a few months and haven't seen the issue like
> > > >> that. Do we know what could trigger the problem? Or is 3.0.x really
> > > >> impacted?
> > > >>
> > > >> Thanks,
> > > >> Jay
> > > >>
> > > >>> On 8/28/17 6:02 AM, Hannu Kröger wrote:
> > > >>> Hello,
> > > >>>
> > > >>> Current latest Cassandra version (3.11.0, possibly also 3.0.12+)
> has
> > a
> > > race
> > > >>> condition that causes Cassandra to create broken sstables (stats
> file
> > > in
> > > >>> sstables to be precise).
> > > >>>
> > > >>> Bug described here:
> > > >>> https://issues.apache.org/jira/browse/CASSANDRA-13752
> > > >>>
> > > >>> This change might be causing it (but not sure):
> > > >>> https://issues.apache.org/jira/browse/CASSANDRA-13038
> > > >>>
> > > >>> Other related issues:
> > > >>> https://issues.apache.org/jira/browse/CASSANDRA-13718
> > > >>> https://issues.apache.org/jira/browse/CASSANDRA-13756
> > > >>>
> > > >>> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or
> higher
> > > before
> > > >>> this is fixed.
> > > >>>
> > > >>> Cheers,
> > > >>> Hannu
> > > >>>
> > > >>
> > > >> ------------------------------------------------------------
> ---------
> > > >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> > > >> For additional commands, e-mail: user-help@cassandra.apache.org
> > > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> > > For additional commands, e-mail: user-help@cassandra.apache.org
> > >
> > >
> >
>
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by Jeff Jirsa <jj...@gmail.com>.
On the user side: 3.11.0 or 3.0.14 is probably what you want to use for new
clusters as long as you can avoid calling the getTombstoneRatio() mbean
with your metrics gathering tools.
On Tue, Sep 12, 2017 at 8:35 AM, Sandeep S <sk...@gmail.com> wrote:
> Hi,
>
> What is the current stable version?
>
> Sandeep.
>
> On 11 September 2017 at 23:06, CPC <ac...@gmail.com> wrote:
>
> > Hi,
> >
> > Is this bug fixed in dse 5.1.3? As I understand calling jmx
> > getTombStoneRatio
> > trigers that bug. We are using opscenter as well and do you have any idea
> > whether opscenter using/calling this method?
> >
> > Thanks
> >
> > On Aug 29, 2017 6:35 AM, "Jeff Jirsa" <jj...@gmail.com> wrote:
> >
> > > I shouldn't actually say I don't think it can happen on 3.0 - I haven't
> > > seen this happen on 3.0 without some other code change to enable it,
> but
> > > like I said, we're still investigating.
> > >
> > > --
> > > Jeff Jirsa
> > >
> > >
> > > > On Aug 28, 2017, at 8:30 PM, Jeff Jirsa <jj...@gmail.com> wrote:
> > > >
> > > > For what it's worth, I don't think this impacts 3.0 without adding
> some
> > > other code change (the reporter of the bug on 3.0 had added custom
> > metrics
> > > that exposed a concurrency issue).
> > > >
> > > > We're looking at it on 3.11. I think 13038 made it far more likely to
> > > occur, but I think it could have happened pre-13038 as well (would take
> > > some serious luck with your deletion time distribution though - the
> > > rounding in 13038 does make it more likely, but the race was already
> > there).
> > > >
> > > > --
> > > > Jeff Jirsa
> > > >
> > > >
> > > >> On Aug 28, 2017, at 8:24 PM, Jay Zhuang
> <jay.zhuang@yahoo.com.INVALID
> > >
> > > wrote:
> > > >>
> > > >> We're using 3.0.12+ for a few months and haven't seen the issue like
> > > >> that. Do we know what could trigger the problem? Or is 3.0.x really
> > > >> impacted?
> > > >>
> > > >> Thanks,
> > > >> Jay
> > > >>
> > > >>> On 8/28/17 6:02 AM, Hannu Kröger wrote:
> > > >>> Hello,
> > > >>>
> > > >>> Current latest Cassandra version (3.11.0, possibly also 3.0.12+)
> has
> > a
> > > race
> > > >>> condition that causes Cassandra to create broken sstables (stats
> file
> > > in
> > > >>> sstables to be precise).
> > > >>>
> > > >>> Bug described here:
> > > >>> https://issues.apache.org/jira/browse/CASSANDRA-13752
> > > >>>
> > > >>> This change might be causing it (but not sure):
> > > >>> https://issues.apache.org/jira/browse/CASSANDRA-13038
> > > >>>
> > > >>> Other related issues:
> > > >>> https://issues.apache.org/jira/browse/CASSANDRA-13718
> > > >>> https://issues.apache.org/jira/browse/CASSANDRA-13756
> > > >>>
> > > >>> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or
> higher
> > > before
> > > >>> this is fixed.
> > > >>>
> > > >>> Cheers,
> > > >>> Hannu
> > > >>>
> > > >>
> > > >> ------------------------------------------------------------
> ---------
> > > >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> > > >> For additional commands, e-mail: user-help@cassandra.apache.org
> > > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> > > For additional commands, e-mail: user-help@cassandra.apache.org
> > >
> > >
> >
>
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by Sandeep S <sk...@gmail.com>.
Hi,
What is the current stable version?
Sandeep.
On 11 September 2017 at 23:06, CPC <ac...@gmail.com> wrote:
> Hi,
>
> Is this bug fixed in dse 5.1.3? As I understand calling jmx
> getTombStoneRatio
> trigers that bug. We are using opscenter as well and do you have any idea
> whether opscenter using/calling this method?
>
> Thanks
>
> On Aug 29, 2017 6:35 AM, "Jeff Jirsa" <jj...@gmail.com> wrote:
>
> > I shouldn't actually say I don't think it can happen on 3.0 - I haven't
> > seen this happen on 3.0 without some other code change to enable it, but
> > like I said, we're still investigating.
> >
> > --
> > Jeff Jirsa
> >
> >
> > > On Aug 28, 2017, at 8:30 PM, Jeff Jirsa <jj...@gmail.com> wrote:
> > >
> > > For what it's worth, I don't think this impacts 3.0 without adding some
> > other code change (the reporter of the bug on 3.0 had added custom
> metrics
> > that exposed a concurrency issue).
> > >
> > > We're looking at it on 3.11. I think 13038 made it far more likely to
> > occur, but I think it could have happened pre-13038 as well (would take
> > some serious luck with your deletion time distribution though - the
> > rounding in 13038 does make it more likely, but the race was already
> there).
> > >
> > > --
> > > Jeff Jirsa
> > >
> > >
> > >> On Aug 28, 2017, at 8:24 PM, Jay Zhuang <jay.zhuang@yahoo.com.INVALID
> >
> > wrote:
> > >>
> > >> We're using 3.0.12+ for a few months and haven't seen the issue like
> > >> that. Do we know what could trigger the problem? Or is 3.0.x really
> > >> impacted?
> > >>
> > >> Thanks,
> > >> Jay
> > >>
> > >>> On 8/28/17 6:02 AM, Hannu Kröger wrote:
> > >>> Hello,
> > >>>
> > >>> Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has
> a
> > race
> > >>> condition that causes Cassandra to create broken sstables (stats file
> > in
> > >>> sstables to be precise).
> > >>>
> > >>> Bug described here:
> > >>> https://issues.apache.org/jira/browse/CASSANDRA-13752
> > >>>
> > >>> This change might be causing it (but not sure):
> > >>> https://issues.apache.org/jira/browse/CASSANDRA-13038
> > >>>
> > >>> Other related issues:
> > >>> https://issues.apache.org/jira/browse/CASSANDRA-13718
> > >>> https://issues.apache.org/jira/browse/CASSANDRA-13756
> > >>>
> > >>> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or higher
> > before
> > >>> this is fixed.
> > >>>
> > >>> Cheers,
> > >>> Hannu
> > >>>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> > >> For additional commands, e-mail: user-help@cassandra.apache.org
> > >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: user-help@cassandra.apache.org
> >
> >
>
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by Chris Lohfink <cl...@apple.com>.
Last Ive seen of it OpsCenter does not collect this metric. I don't think any monitoring tools do.
Chris
> On Sep 11, 2017, at 4:06 PM, CPC <ac...@gmail.com> wrote:
>
> Hi,
>
> Is this bug fixed in dse 5.1.3? As I understand calling jmx getTombStoneRatio
> trigers that bug. We are using opscenter as well and do you have any idea
> whether opscenter using/calling this method?
>
> Thanks
>
> On Aug 29, 2017 6:35 AM, "Jeff Jirsa" <jj...@gmail.com> wrote:
>
>> I shouldn't actually say I don't think it can happen on 3.0 - I haven't
>> seen this happen on 3.0 without some other code change to enable it, but
>> like I said, we're still investigating.
>>
>> --
>> Jeff Jirsa
>>
>>
>>> On Aug 28, 2017, at 8:30 PM, Jeff Jirsa <jj...@gmail.com> wrote:
>>>
>>> For what it's worth, I don't think this impacts 3.0 without adding some
>> other code change (the reporter of the bug on 3.0 had added custom metrics
>> that exposed a concurrency issue).
>>>
>>> We're looking at it on 3.11. I think 13038 made it far more likely to
>> occur, but I think it could have happened pre-13038 as well (would take
>> some serious luck with your deletion time distribution though - the
>> rounding in 13038 does make it more likely, but the race was already there).
>>>
>>> --
>>> Jeff Jirsa
>>>
>>>
>>>> On Aug 28, 2017, at 8:24 PM, Jay Zhuang <ja...@yahoo.com.INVALID>
>> wrote:
>>>>
>>>> We're using 3.0.12+ for a few months and haven't seen the issue like
>>>> that. Do we know what could trigger the problem? Or is 3.0.x really
>>>> impacted?
>>>>
>>>> Thanks,
>>>> Jay
>>>>
>>>>> On 8/28/17 6:02 AM, Hannu Kröger wrote:
>>>>> Hello,
>>>>>
>>>>> Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has a
>> race
>>>>> condition that causes Cassandra to create broken sstables (stats file
>> in
>>>>> sstables to be precise).
>>>>>
>>>>> Bug described here:
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13752
>>>>>
>>>>> This change might be causing it (but not sure):
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13038
>>>>>
>>>>> Other related issues:
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13718
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13756
>>>>>
>>>>> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or higher
>> before
>>>>> this is fixed.
>>>>>
>>>>> Cheers,
>>>>> Hannu
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>>>> For additional commands, e-mail: user-help@cassandra.apache.org
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: user-help@cassandra.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by Chris Lohfink <cl...@apple.com>.
Last Ive seen of it OpsCenter does not collect this metric. I don't think any monitoring tools do.
Chris
> On Sep 11, 2017, at 4:06 PM, CPC <ac...@gmail.com> wrote:
>
> Hi,
>
> Is this bug fixed in dse 5.1.3? As I understand calling jmx getTombStoneRatio
> trigers that bug. We are using opscenter as well and do you have any idea
> whether opscenter using/calling this method?
>
> Thanks
>
> On Aug 29, 2017 6:35 AM, "Jeff Jirsa" <jj...@gmail.com> wrote:
>
>> I shouldn't actually say I don't think it can happen on 3.0 - I haven't
>> seen this happen on 3.0 without some other code change to enable it, but
>> like I said, we're still investigating.
>>
>> --
>> Jeff Jirsa
>>
>>
>>> On Aug 28, 2017, at 8:30 PM, Jeff Jirsa <jj...@gmail.com> wrote:
>>>
>>> For what it's worth, I don't think this impacts 3.0 without adding some
>> other code change (the reporter of the bug on 3.0 had added custom metrics
>> that exposed a concurrency issue).
>>>
>>> We're looking at it on 3.11. I think 13038 made it far more likely to
>> occur, but I think it could have happened pre-13038 as well (would take
>> some serious luck with your deletion time distribution though - the
>> rounding in 13038 does make it more likely, but the race was already there).
>>>
>>> --
>>> Jeff Jirsa
>>>
>>>
>>>> On Aug 28, 2017, at 8:24 PM, Jay Zhuang <ja...@yahoo.com.INVALID>
>> wrote:
>>>>
>>>> We're using 3.0.12+ for a few months and haven't seen the issue like
>>>> that. Do we know what could trigger the problem? Or is 3.0.x really
>>>> impacted?
>>>>
>>>> Thanks,
>>>> Jay
>>>>
>>>>> On 8/28/17 6:02 AM, Hannu Kröger wrote:
>>>>> Hello,
>>>>>
>>>>> Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has a
>> race
>>>>> condition that causes Cassandra to create broken sstables (stats file
>> in
>>>>> sstables to be precise).
>>>>>
>>>>> Bug described here:
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13752
>>>>>
>>>>> This change might be causing it (but not sure):
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13038
>>>>>
>>>>> Other related issues:
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13718
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13756
>>>>>
>>>>> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or higher
>> before
>>>>> this is fixed.
>>>>>
>>>>> Cheers,
>>>>> Hannu
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>>>> For additional commands, e-mail: user-help@cassandra.apache.org
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: user-help@cassandra.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by CPC <ac...@gmail.com>.
Hi,
Is this bug fixed in dse 5.1.3? As I understand calling jmx getTombStoneRatio
trigers that bug. We are using opscenter as well and do you have any idea
whether opscenter using/calling this method?
Thanks
On Aug 29, 2017 6:35 AM, "Jeff Jirsa" <jj...@gmail.com> wrote:
> I shouldn't actually say I don't think it can happen on 3.0 - I haven't
> seen this happen on 3.0 without some other code change to enable it, but
> like I said, we're still investigating.
>
> --
> Jeff Jirsa
>
>
> > On Aug 28, 2017, at 8:30 PM, Jeff Jirsa <jj...@gmail.com> wrote:
> >
> > For what it's worth, I don't think this impacts 3.0 without adding some
> other code change (the reporter of the bug on 3.0 had added custom metrics
> that exposed a concurrency issue).
> >
> > We're looking at it on 3.11. I think 13038 made it far more likely to
> occur, but I think it could have happened pre-13038 as well (would take
> some serious luck with your deletion time distribution though - the
> rounding in 13038 does make it more likely, but the race was already there).
> >
> > --
> > Jeff Jirsa
> >
> >
> >> On Aug 28, 2017, at 8:24 PM, Jay Zhuang <ja...@yahoo.com.INVALID>
> wrote:
> >>
> >> We're using 3.0.12+ for a few months and haven't seen the issue like
> >> that. Do we know what could trigger the problem? Or is 3.0.x really
> >> impacted?
> >>
> >> Thanks,
> >> Jay
> >>
> >>> On 8/28/17 6:02 AM, Hannu Kröger wrote:
> >>> Hello,
> >>>
> >>> Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has a
> race
> >>> condition that causes Cassandra to create broken sstables (stats file
> in
> >>> sstables to be precise).
> >>>
> >>> Bug described here:
> >>> https://issues.apache.org/jira/browse/CASSANDRA-13752
> >>>
> >>> This change might be causing it (but not sure):
> >>> https://issues.apache.org/jira/browse/CASSANDRA-13038
> >>>
> >>> Other related issues:
> >>> https://issues.apache.org/jira/browse/CASSANDRA-13718
> >>> https://issues.apache.org/jira/browse/CASSANDRA-13756
> >>>
> >>> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or higher
> before
> >>> this is fixed.
> >>>
> >>> Cheers,
> >>> Hannu
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> >> For additional commands, e-mail: user-help@cassandra.apache.org
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
>
>
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by CPC <ac...@gmail.com>.
Hi,
Is this bug fixed in dse 5.1.3? As I understand calling jmx getTombStoneRatio
trigers that bug. We are using opscenter as well and do you have any idea
whether opscenter using/calling this method?
Thanks
On Aug 29, 2017 6:35 AM, "Jeff Jirsa" <jj...@gmail.com> wrote:
> I shouldn't actually say I don't think it can happen on 3.0 - I haven't
> seen this happen on 3.0 without some other code change to enable it, but
> like I said, we're still investigating.
>
> --
> Jeff Jirsa
>
>
> > On Aug 28, 2017, at 8:30 PM, Jeff Jirsa <jj...@gmail.com> wrote:
> >
> > For what it's worth, I don't think this impacts 3.0 without adding some
> other code change (the reporter of the bug on 3.0 had added custom metrics
> that exposed a concurrency issue).
> >
> > We're looking at it on 3.11. I think 13038 made it far more likely to
> occur, but I think it could have happened pre-13038 as well (would take
> some serious luck with your deletion time distribution though - the
> rounding in 13038 does make it more likely, but the race was already there).
> >
> > --
> > Jeff Jirsa
> >
> >
> >> On Aug 28, 2017, at 8:24 PM, Jay Zhuang <ja...@yahoo.com.INVALID>
> wrote:
> >>
> >> We're using 3.0.12+ for a few months and haven't seen the issue like
> >> that. Do we know what could trigger the problem? Or is 3.0.x really
> >> impacted?
> >>
> >> Thanks,
> >> Jay
> >>
> >>> On 8/28/17 6:02 AM, Hannu Kröger wrote:
> >>> Hello,
> >>>
> >>> Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has a
> race
> >>> condition that causes Cassandra to create broken sstables (stats file
> in
> >>> sstables to be precise).
> >>>
> >>> Bug described here:
> >>> https://issues.apache.org/jira/browse/CASSANDRA-13752
> >>>
> >>> This change might be causing it (but not sure):
> >>> https://issues.apache.org/jira/browse/CASSANDRA-13038
> >>>
> >>> Other related issues:
> >>> https://issues.apache.org/jira/browse/CASSANDRA-13718
> >>> https://issues.apache.org/jira/browse/CASSANDRA-13756
> >>>
> >>> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or higher
> before
> >>> this is fixed.
> >>>
> >>> Cheers,
> >>> Hannu
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> >> For additional commands, e-mail: user-help@cassandra.apache.org
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
>
>
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by Jeff Jirsa <jj...@gmail.com>.
I shouldn't actually say I don't think it can happen on 3.0 - I haven't seen this happen on 3.0 without some other code change to enable it, but like I said, we're still investigating.
--
Jeff Jirsa
> On Aug 28, 2017, at 8:30 PM, Jeff Jirsa <jj...@gmail.com> wrote:
>
> For what it's worth, I don't think this impacts 3.0 without adding some other code change (the reporter of the bug on 3.0 had added custom metrics that exposed a concurrency issue).
>
> We're looking at it on 3.11. I think 13038 made it far more likely to occur, but I think it could have happened pre-13038 as well (would take some serious luck with your deletion time distribution though - the rounding in 13038 does make it more likely, but the race was already there).
>
> --
> Jeff Jirsa
>
>
>> On Aug 28, 2017, at 8:24 PM, Jay Zhuang <ja...@yahoo.com.INVALID> wrote:
>>
>> We're using 3.0.12+ for a few months and haven't seen the issue like
>> that. Do we know what could trigger the problem? Or is 3.0.x really
>> impacted?
>>
>> Thanks,
>> Jay
>>
>>> On 8/28/17 6:02 AM, Hannu Kröger wrote:
>>> Hello,
>>>
>>> Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has a race
>>> condition that causes Cassandra to create broken sstables (stats file in
>>> sstables to be precise).
>>>
>>> Bug described here:
>>> https://issues.apache.org/jira/browse/CASSANDRA-13752
>>>
>>> This change might be causing it (but not sure):
>>> https://issues.apache.org/jira/browse/CASSANDRA-13038
>>>
>>> Other related issues:
>>> https://issues.apache.org/jira/browse/CASSANDRA-13718
>>> https://issues.apache.org/jira/browse/CASSANDRA-13756
>>>
>>> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or higher before
>>> this is fixed.
>>>
>>> Cheers,
>>> Hannu
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: user-help@cassandra.apache.org
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by Jeff Jirsa <jj...@gmail.com>.
I shouldn't actually say I don't think it can happen on 3.0 - I haven't seen this happen on 3.0 without some other code change to enable it, but like I said, we're still investigating.
--
Jeff Jirsa
> On Aug 28, 2017, at 8:30 PM, Jeff Jirsa <jj...@gmail.com> wrote:
>
> For what it's worth, I don't think this impacts 3.0 without adding some other code change (the reporter of the bug on 3.0 had added custom metrics that exposed a concurrency issue).
>
> We're looking at it on 3.11. I think 13038 made it far more likely to occur, but I think it could have happened pre-13038 as well (would take some serious luck with your deletion time distribution though - the rounding in 13038 does make it more likely, but the race was already there).
>
> --
> Jeff Jirsa
>
>
>> On Aug 28, 2017, at 8:24 PM, Jay Zhuang <ja...@yahoo.com.INVALID> wrote:
>>
>> We're using 3.0.12+ for a few months and haven't seen the issue like
>> that. Do we know what could trigger the problem? Or is 3.0.x really
>> impacted?
>>
>> Thanks,
>> Jay
>>
>>> On 8/28/17 6:02 AM, Hannu Kröger wrote:
>>> Hello,
>>>
>>> Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has a race
>>> condition that causes Cassandra to create broken sstables (stats file in
>>> sstables to be precise).
>>>
>>> Bug described here:
>>> https://issues.apache.org/jira/browse/CASSANDRA-13752
>>>
>>> This change might be causing it (but not sure):
>>> https://issues.apache.org/jira/browse/CASSANDRA-13038
>>>
>>> Other related issues:
>>> https://issues.apache.org/jira/browse/CASSANDRA-13718
>>> https://issues.apache.org/jira/browse/CASSANDRA-13756
>>>
>>> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or higher before
>>> this is fixed.
>>>
>>> Cheers,
>>> Hannu
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: user-help@cassandra.apache.org
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by Jeff Jirsa <jj...@gmail.com>.
For what it's worth, I don't think this impacts 3.0 without adding some other code change (the reporter of the bug on 3.0 had added custom metrics that exposed a concurrency issue).
We're looking at it on 3.11. I think 13038 made it far more likely to occur, but I think it could have happened pre-13038 as well (would take some serious luck with your deletion time distribution though - the rounding in 13038 does make it more likely, but the race was already there).
--
Jeff Jirsa
> On Aug 28, 2017, at 8:24 PM, Jay Zhuang <ja...@yahoo.com.INVALID> wrote:
>
> We're using 3.0.12+ for a few months and haven't seen the issue like
> that. Do we know what could trigger the problem? Or is 3.0.x really
> impacted?
>
> Thanks,
> Jay
>
>> On 8/28/17 6:02 AM, Hannu Kröger wrote:
>> Hello,
>>
>> Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has a race
>> condition that causes Cassandra to create broken sstables (stats file in
>> sstables to be precise).
>>
>> Bug described here:
>> https://issues.apache.org/jira/browse/CASSANDRA-13752
>>
>> This change might be causing it (but not sure):
>> https://issues.apache.org/jira/browse/CASSANDRA-13038
>>
>> Other related issues:
>> https://issues.apache.org/jira/browse/CASSANDRA-13718
>> https://issues.apache.org/jira/browse/CASSANDRA-13756
>>
>> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or higher before
>> this is fixed.
>>
>> Cheers,
>> Hannu
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by Jeff Jirsa <jj...@gmail.com>.
For what it's worth, I don't think this impacts 3.0 without adding some other code change (the reporter of the bug on 3.0 had added custom metrics that exposed a concurrency issue).
We're looking at it on 3.11. I think 13038 made it far more likely to occur, but I think it could have happened pre-13038 as well (would take some serious luck with your deletion time distribution though - the rounding in 13038 does make it more likely, but the race was already there).
--
Jeff Jirsa
> On Aug 28, 2017, at 8:24 PM, Jay Zhuang <ja...@yahoo.com.INVALID> wrote:
>
> We're using 3.0.12+ for a few months and haven't seen the issue like
> that. Do we know what could trigger the problem? Or is 3.0.x really
> impacted?
>
> Thanks,
> Jay
>
>> On 8/28/17 6:02 AM, Hannu Kröger wrote:
>> Hello,
>>
>> Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has a race
>> condition that causes Cassandra to create broken sstables (stats file in
>> sstables to be precise).
>>
>> Bug described here:
>> https://issues.apache.org/jira/browse/CASSANDRA-13752
>>
>> This change might be causing it (but not sure):
>> https://issues.apache.org/jira/browse/CASSANDRA-13038
>>
>> Other related issues:
>> https://issues.apache.org/jira/browse/CASSANDRA-13718
>> https://issues.apache.org/jira/browse/CASSANDRA-13756
>>
>> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or higher before
>> this is fixed.
>>
>> Cheers,
>> Hannu
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by Jay Zhuang <ja...@yahoo.com.INVALID>.
We're using 3.0.12+ for a few months and haven't seen the issue like
that. Do we know what could trigger the problem? Or is 3.0.x really
impacted?
Thanks,
Jay
On 8/28/17 6:02 AM, Hannu Kröger wrote:
> Hello,
>
> Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has a race
> condition that causes Cassandra to create broken sstables (stats file in
> sstables to be precise).
>
> Bug described here:
> https://issues.apache.org/jira/browse/CASSANDRA-13752
>
> This change might be causing it (but not sure):
> https://issues.apache.org/jira/browse/CASSANDRA-13038
>
> Other related issues:
> https://issues.apache.org/jira/browse/CASSANDRA-13718
> https://issues.apache.org/jira/browse/CASSANDRA-13756
>
> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or higher before
> this is fixed.
>
> Cheers,
> Hannu
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org
Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+
Posted by Jay Zhuang <ja...@yahoo.com.INVALID>.
We're using 3.0.12+ for a few months and haven't seen the issue like
that. Do we know what could trigger the problem? Or is 3.0.x really
impacted?
Thanks,
Jay
On 8/28/17 6:02 AM, Hannu Kröger wrote:
> Hello,
>
> Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has a race
> condition that causes Cassandra to create broken sstables (stats file in
> sstables to be precise).
>
> Bug described here:
> https://issues.apache.org/jira/browse/CASSANDRA-13752
>
> This change might be causing it (but not sure):
> https://issues.apache.org/jira/browse/CASSANDRA-13038
>
> Other related issues:
> https://issues.apache.org/jira/browse/CASSANDRA-13718
> https://issues.apache.org/jira/browse/CASSANDRA-13756
>
> I would not recommend using 3.11.0 nor upgrading to 3.0.12 or higher before
> this is fixed.
>
> Cheers,
> Hannu
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org