You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2009/09/02 22:02:32 UTC
[jira] Created: (CASSANDRA-414) remove sstableLock
remove sstableLock
------------------
Key: CASSANDRA-414
URL: https://issues.apache.org/jira/browse/CASSANDRA-414
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Fix For: 0.5
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751507#action_12751507 ]
Jonathan Ellis commented on CASSANDRA-414:
------------------------------------------
rebased.
02
remove sstableLock. re-order a few ops so that we can never "lose" data temporarily -- always remove old sstable _after_ adding the new ones. so at worst a few read ops will merge data from an sstable that is obsolete -- this is ok and better than Stop The World locking
01
combine addToList and storeLocation; rename to addSSTable
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751824#action_12751824 ]
Jonathan Ellis commented on CASSANDRA-414:
------------------------------------------
I checked the source for NBHM and couldn't find any evidence that my literal reading of the iterator contract ("Iterators and Enumerations return elements reflecting the state of the hash table at some point at or since the creation of the iterator/enumeration") was correct. So I asked the author for clarification here: https://sourceforge.net/forum/message.php?msg_id=7611241.
He replied, and sure enough, Jun's suspicions were correct and even if the compaction thread is careful to add the new SSTR before removing the old ones from sstables_, iterator threads may see the absence of the latter but not the presence of the former.
So I think that this approach is not going to work. But I think we can still cut the lock penalty dramatically from what it is now. I should have some code for that approach Monday.
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12752473#action_12752473 ]
Hudson commented on CASSANDRA-414:
----------------------------------
Integrated in Cassandra #191 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/191/])
Revert "remove sstableLock. re-order a few ops so that we can never "lose" data temporarily -- always remove old sstable references _after_ adding the new ones. so at worst a few read ops will merge data from an sstable that is obsolete -- this is ok and better than Stop The World locking"
and " combine addToList and storeLocation; rename to addSSTable"
These were works in progress (and broken); accidentally committed w/ the 418 fix.
combine addToList and storeLocation; rename to addSSTable
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753972#action_12753972 ]
Jonathan Ellis commented on CASSANDRA-414:
------------------------------------------
It's easy enough to check the logs with e.g. stress.py: the cleanup does happen. If it were the SSTR being enqueued then
r = (FileDeletingReference) finalizerQueue.remove();
would generate ClassCastException and nothing would get cleaned up (until server restart).
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-convert-ssTables_-to-a-Set-since-the-filename-is-enca.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0003-Replace-sstableLock-with-SSTableTracker-which-perform.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-414:
-------------------------------------
Attachment: 0003-Replace-sstableLock-with-SSTableTracker-which-perform.txt
0002-convert-ssTables_-to-a-Set-since-the-filename-is-enca.txt
0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-convert-ssTables_-to-a-Set-since-the-filename-is-enca.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0003-Replace-sstableLock-with-SSTableTracker-which-perform.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-414:
-------------------------------------
Attachment: 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751755#action_12751755 ]
Jonathan Ellis commented on CASSANDRA-414:
------------------------------------------
The iterator will either have a reference to A, or not. If it does, then A will not be deleted until the iterator is done since the delete is no longer done by compaction but by the referencequeue. If it does not, it will have a reference to the new sstable instead.
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751781#action_12751781 ]
Jun Rao commented on CASSANDRA-414:
-----------------------------------
The question is when a reference is added to the referencequeue. If it's at SSTableReader open time, it may be too late. Between file A is iterated and SSTableReader is opened on A, a compaction can remove A, assuming no one else is reading A at that time.
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12752812#action_12752812 ]
Jonathan Ellis commented on CASSANDRA-414:
------------------------------------------
03
Replace sstableLock with SSTableTracker, which performs updates to the sstable list atomically
without readers ever having to block. (Readers will always either see the old list, or the new.)
We avoid a race on the delete of the old SSTable files on-disk by using a ReferenceQueue:
when the last reference is gone, a PhantomReference is added to the queue and can do cleanup.
In case Cassandra is killed between compaction and this cleanup, a -Compacted empty file
is written to disk; Cassandra removes any files thus tagged on startup.
02
convert ssTables_ to a Set, since the filename is encapsulated in the SSTR object now
01
CASSANDRA-414 combine addToList and storeLocation; rename to addSSTable
ready for review.
there is a good summary of how PhantomReference and ReferenceQueue work here: http://www.kdgregory.com/index.php?page=java.refobj
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-convert-ssTables_-to-a-Set-since-the-filename-is-enca.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0003-Replace-sstableLock-with-SSTableTracker-which-perform.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751700#action_12751700 ]
Jun Rao commented on CASSANDRA-414:
-----------------------------------
Keeping references to SSTableReader may not be enough. The iterator could iterate file A. Then just before an SSTableReader is opened on A, A is removed from sstables and deleted on disk. Now, you can get the same "file not found" exception as above.
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-414.
--------------------------------------
Resolution: Fixed
committed
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-convert-ssTables_-to-a-Set-since-the-filename-is-enca.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0003-Replace-sstableLock-with-SSTableTracker-which-perform.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753966#action_12753966 ]
Jun Rao commented on CASSANDRA-414:
-----------------------------------
Maybe I don''t understand how PhantomReference works, but the code in SSTableReader dealing with finalizerQueue doesn't look right to me. What gets enqueued in finalizerQueue is SSTR. It doesn't seem like that you can cast it directly to FileDeletingReference. It seems to me that you have to maintain a map btw SSTR and FileDeletingReference. Every time you dequeue an item from finalizerQueue, you can lookup the map to find the corresponding FileDeletingReference.
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-convert-ssTables_-to-a-Set-since-the-filename-is-enca.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0003-Replace-sstableLock-with-SSTableTracker-which-perform.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753970#action_12753970 ]
Jonathan Ellis commented on CASSANDRA-414:
------------------------------------------
Yeah, it's a little subtle. The article I linked is a good explanation, the javadoc alone isn't sufficient or at least wasn't for me.
A Reference of any type has a get() method that returns the actual referent. Here that would be the SSTR. But ReferenceQueue holds the Reference wrapper, not the actual SSTRs. (When you pass a RQ to the Reference constructor, the Reference will be enqueued on that RQ when its referent is GC'd. The referent itself already GC'd or in the process of being GC'd so it can't be put on the RQ or you would get back to the Bad Old Days of finalizer resurrection bugs.)
Now, if the referent is no longer live, get() will return null. Since the point of the RQ design is to do cleanup after the object is dead, we subclass PhantomReference and store a reference to the path, so we don't actually need the SSTR to do the delete. (In fact for PR in particular get() _always_ returns null but that is not really essential to understanding what is going on here.)
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-convert-ssTables_-to-a-Set-since-the-filename-is-enca.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0003-Replace-sstableLock-with-SSTableTracker-which-perform.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751785#action_12751785 ]
Jonathan Ellis commented on CASSANDRA-414:
------------------------------------------
Remember, SSTR objects are not transitory -- we only have one such object for each file on disk ever. Even the compaction code uses SSTR.get to use a reference to the open object, and does not open new ones (except for the newly compacted files of course, which is in turn the only time open is called on those).
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751624#action_12751624 ]
Jonathan Ellis commented on CASSANDRA-414:
------------------------------------------
when compaction completes it modifies CFS.sstables_ as follows:
- sstables_.put(new compacted sstable)
- remove the source sstables & delete their file on disk
now, NonBlockingHashMap guarantees that "Iterators and Enumerations return elements reflecting the state of the hash table at some point at or since the creation of the iterator/enumeration." BUT this has no effect on other parts of the system, particularly the delete!
so the fundamental problem exhibeted by the unit tests is this:
- thread A starts iterating over sstables_
- compaction thread finishes and does its thing. now some of the files A needs to see a consistent view of the data are gone: the set of sstables being iterated over was in fact consistent but because we're violating encapsulation by doing the delete and remove separately, we can get incorrect results.
I think the simplest solution is to use pseudo-finalizers to do the actual file delete once no references to the owning SSTableReader exist anymore. These can be done using http://java.sun.com/javase/6/docs/api/java/lang/ref/ReferenceQueue.html and http://java.sun.com/javase/6/docs/api/java/lang/ref/PhantomReference.html.
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754116#action_12754116 ]
Hudson commented on CASSANDRA-414:
----------------------------------
Integrated in Cassandra #194 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/194/])
Replace sstableLock with SSTableTracker, which performs updates to the sstable list atomically
without readers ever having to block. (Readers will always either see the old list, or the new.)
We avoid a race on the delete of the old SSTable files on-disk by using a ReferenceQueue:
when the last reference is gone, a PhantomReference is added to the queue and can do cleanup.
In case Cassandra is killed between compaction and this cleanup, a -Compacted empty file
is written to disk; Cassandra removes any files thus tagged on startup.
patch by jbellis; reviewed by Chris Goffinet for
convert ssTables_ to a Set, since the filename is encapsulated in the SSTR object now
patch by jbellis; reviewed by Chris Goffinet for
combine addToList and storeLocation; rename to addSSTable
patch by jbellis; reviewed by Chris Goffinet for
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-convert-ssTables_-to-a-Set-since-the-filename-is-enca.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0003-Replace-sstableLock-with-SSTableTracker-which-perform.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751784#action_12751784 ]
Jonathan Ellis commented on CASSANDRA-414:
------------------------------------------
No, it has nothing to do with whether someone is actually reading (has a BufferedRAF open) on A, only whether any reference to the SSTR object itself exists. Which is no earlier than when the last iterator that might open a BRAF is done.
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751532#action_12751532 ]
Jun Rao commented on CASSANDRA-414:
-----------------------------------
We need to add a comment to CFS.snapshot, indicating that it's a fuzzy one, instead of point-at-time one. Also, the following unit tests failed on me.
[junit] Testcase: testNameSort10(org.apache.cassandra.db.NameSortTest): Caused an ERROR
[junit] /home/junrao/local/cassandra_test/build/test/cassandra/data/Keyspace1/Super1-50-Data.db (No such file or directory)
[junit] java.io.FileNotFoundException: /home/junrao/local/cassandra_test/build/test/cassandra/data/Keyspace1/Super1-50-Data.db (No such file or directory)
[junit] at java.io.RandomAccessFile.open(Native Method)
[junit] at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
[junit] at java.io.RandomAccessFile.<init>(RandomAccessFile.java:98)
[junit] at org.apache.cassandra.io.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:142)
[junit] at org.apache.cassandra.db.filter.SSTableSliceIterator$ColumnGroupReader.<init>(SSTableSliceIterator.java:110)
[junit] at org.apache.cassandra.db.filter.SSTableSliceIterator.<init>(SSTableSliceIterator.java:56)
[junit] at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:64)
[junit] at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1347)
[junit] at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1283)
[junit] at org.apache.cassandra.db.Table.get(Table.java:564)
[junit] at org.apache.cassandra.db.NameSortTest.validateNameSort(NameSortTest.java:113)
[junit] at org.apache.cassandra.db.NameSortTest.testNameSort(NameSortTest.java:94)
[junit] at org.apache.cassandra.db.NameSortTest.testNameSort10(NameSortTest.java:48)
[junit]
[junit]
[junit] Testcase: testNameSort100(org.apache.cassandra.db.NameSortTest): Caused an ERROR
[junit] /home/junrao/local/cassandra_test/build/test/cassandra/data/Keyspace1/Super1-582-Data.db (No such file or directory)
[junit] java.io.FileNotFoundException: /home/junrao/local/cassandra_test/build/test/cassandra/data/Keyspace1/Super1-582-Data.db (No such file or directory
[junit] at java.io.RandomAccessFile.open(Native Method)
[junit] at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
[junit] at java.io.RandomAccessFile.<init>(RandomAccessFile.java:98)
[junit] at org.apache.cassandra.io.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:142)
[junit] at org.apache.cassandra.db.filter.SSTableSliceIterator$ColumnGroupReader.<init>(SSTableSliceIterator.java:110)
[junit] at org.apache.cassandra.db.filter.SSTableSliceIterator.<init>(SSTableSliceIterator.java:56)
[junit] at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:64)
[junit] at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1347)
[junit] at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1283)
[junit] at org.apache.cassandra.db.Table.get(Table.java:564)
[junit] at org.apache.cassandra.db.NameSortTest.validateNameSort(NameSortTest.java:113)
[junit] at org.apache.cassandra.db.NameSortTest.testNameSort(NameSortTest.java:94)
[junit] at org.apache.cassandra.db.NameSortTest.testNameSort100(NameSortTest.java:55)
[junit]
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-414) remove sstableLock
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-414:
-------------------------------------
Attachment: 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-414) remove sstableLock
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753759#action_12753759 ]
Chris Goffinet commented on CASSANDRA-414:
------------------------------------------
+1
> remove sstableLock
> ------------------
>
> Key: CASSANDRA-414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-414
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0001-CASSANDRA-414-combine-addToList-and-storeLocation-ren.txt, 0002-convert-ssTables_-to-a-Set-since-the-filename-is-enca.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0002-remove-sstableLock.-re-order-a-few-ops-so-that-we-can.txt, 0003-Replace-sstableLock-with-SSTableTracker-which-perform.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.