You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Boris Yen <yu...@gmail.com> on 2013/03/08 11:16:33 UTC
Cassandra 4021
Hi,
We have run into this a few times on 1.0.10. It seems this happens when the
file-rename after compaction is not completed. We have created a patch for
this. However, in order to make sure we do not break anything, I was
wondering if anyone could help us review the changes to see if this would
cause any issue or not.
Thanks and Regards,
Boris
Re: Cassandra 4021
Posted by Jonathan Ellis <jb...@gmail.com>.
I think you've nailed it. I've committed this fix to 1.1.11 and 1.2.3.
On Fri, Mar 8, 2013 at 5:59 PM, Boris Yen <yu...@gmail.com> wrote:
> Use the following sstable on our file system as an example.
>
> rw-rr- 1 byan admin 333 Mar 5 15:26 indexBytes-hd-115-Index.db
> rw-rr- 1 byan admin 66305 Mar 5 15:26 indexBytes-tmp-hd-115-Data.db
> rw-rr- 1 byan admin 67 Mar 5 15:26 indexBytes-tmp-hd-115-Digest.sha1
> rw-rr- 1 byan admin 1936 Mar 5 15:26 indexBytes-tmp-hd-115-Filter.db
>
> The reason why we tweak the hashcode and equals methods is that the
> Descriptor for indexBytes-hd-115-Index.db and indexBytes-tmp-hd-115-Data.db
> are the same. If the Descriptor is used as key in a map and
> indexBytes-hd-115-Index.db
> get listed first, the C* will consider that the there are four components
> for this Descriptor [Digest.sha1, Index.db, Filter.db, Data.db]. In this
> case, it is not correct. When removing orphans for sstable, the Descriptor
> will be used to create the filename to delete files, that is why C* cannot
> find the file to delete.
>
> Regards,
> Boris
>
> On Sat, Mar 9, 2013 at 12:54 AM, Jonathan Ellis <jb...@gmail.com> wrote:
>
>> Thanks, Boris. Can you explain what the cause is, and why tweaking hashCode
>> fixes it?
>> On Mar 8, 2013 4:17 AM, "Boris Yen" <yu...@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > We have run into this a few times on 1.0.10. It seems this happens when
>> > the file-rename after compaction is not completed. We have created a
>> patch
>> > for this. However, in order to make sure we do not break anything, I was
>> > wondering if anyone could help us review the changes to see if this would
>> > cause any issue or not.
>> >
>> > Thanks and Regards,
>> > Boris
>> >
>>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder, http://www.datastax.com
@spyced
Re: Cassandra 4021
Posted by Boris Yen <yu...@gmail.com>.
Hi Jonathan,
If our finding is not correct or the patch is going the wrong direction,
please kindly let us know.
Thank you in advance.
Regards,
Boris
On Sat, Mar 9, 2013 at 7:59 AM, Boris Yen <yu...@gmail.com> wrote:
> Use the following sstable on our file system as an example.
>
> rw-rr- 1 byan admin 333 Mar 5 15:26 indexBytes-hd-115-Index.db
> rw-rr- 1 byan admin 66305 Mar 5 15:26 indexBytes-tmp-hd-115-Data.db
> rw-rr- 1 byan admin 67 Mar 5 15:26 indexBytes-tmp-hd-115-Digest.sha1
> rw-rr- 1 byan admin 1936 Mar 5 15:26 indexBytes-tmp-hd-115-Filter.db
>
> The reason why we tweak the hashcode and equals methods is that the
> Descriptor for indexBytes-hd-115-Index.db and indexBytes-tmp-hd-115-Data.db
> are the same. If the Descriptor is used as key in a map and indexBytes-hd-115-Index.db
> get listed first, the C* will consider that the there are four components
> for this Descriptor [Digest.sha1, Index.db, Filter.db, Data.db]. In this
> case, it is not correct. When removing orphans for sstable, the Descriptor
> will be used to create the filename to delete files, that is why C* cannot
> find the file to delete.
>
> Regards,
> Boris
>
> On Sat, Mar 9, 2013 at 12:54 AM, Jonathan Ellis <jb...@gmail.com> wrote:
>
>> Thanks, Boris. Can you explain what the cause is, and why tweaking
>> hashCode
>> fixes it?
>> On Mar 8, 2013 4:17 AM, "Boris Yen" <yu...@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > We have run into this a few times on 1.0.10. It seems this happens when
>> > the file-rename after compaction is not completed. We have created a
>> patch
>> > for this. However, in order to make sure we do not break anything, I was
>> > wondering if anyone could help us review the changes to see if this
>> would
>> > cause any issue or not.
>> >
>> > Thanks and Regards,
>> > Boris
>> >
>>
>
>
Re: Cassandra 4021
Posted by Boris Yen <yu...@gmail.com>.
Use the following sstable on our file system as an example.
rw-rr- 1 byan admin 333 Mar 5 15:26 indexBytes-hd-115-Index.db
rw-rr- 1 byan admin 66305 Mar 5 15:26 indexBytes-tmp-hd-115-Data.db
rw-rr- 1 byan admin 67 Mar 5 15:26 indexBytes-tmp-hd-115-Digest.sha1
rw-rr- 1 byan admin 1936 Mar 5 15:26 indexBytes-tmp-hd-115-Filter.db
The reason why we tweak the hashcode and equals methods is that the
Descriptor for indexBytes-hd-115-Index.db and indexBytes-tmp-hd-115-Data.db
are the same. If the Descriptor is used as key in a map and
indexBytes-hd-115-Index.db
get listed first, the C* will consider that the there are four components
for this Descriptor [Digest.sha1, Index.db, Filter.db, Data.db]. In this
case, it is not correct. When removing orphans for sstable, the Descriptor
will be used to create the filename to delete files, that is why C* cannot
find the file to delete.
Regards,
Boris
On Sat, Mar 9, 2013 at 12:54 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> Thanks, Boris. Can you explain what the cause is, and why tweaking hashCode
> fixes it?
> On Mar 8, 2013 4:17 AM, "Boris Yen" <yu...@gmail.com> wrote:
>
> > Hi,
> >
> > We have run into this a few times on 1.0.10. It seems this happens when
> > the file-rename after compaction is not completed. We have created a
> patch
> > for this. However, in order to make sure we do not break anything, I was
> > wondering if anyone could help us review the changes to see if this would
> > cause any issue or not.
> >
> > Thanks and Regards,
> > Boris
> >
>
Re: Cassandra 4021
Posted by Jonathan Ellis <jb...@gmail.com>.
Thanks, Boris. Can you explain what the cause is, and why tweaking hashCode
fixes it?
On Mar 8, 2013 4:17 AM, "Boris Yen" <yu...@gmail.com> wrote:
> Hi,
>
> We have run into this a few times on 1.0.10. It seems this happens when
> the file-rename after compaction is not completed. We have created a patch
> for this. However, in order to make sure we do not break anything, I was
> wondering if anyone could help us review the changes to see if this would
> cause any issue or not.
>
> Thanks and Regards,
> Boris
>