You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by jb...@apache.org on 2011/06/24 16:22:42 UTC
svn commit: r1139319 - in /cassandra/trunk: ./ contrib/ debian/
interface/thrift/gen-java/org/apache/cassandra/thrift/
src/java/org/apache/cassandra/config/ src/java/org/apache/cassandra/db/
src/java/org/apache/cassandra/io/sstable/ src/java/org/apache...
Author: jbellis
Date: Fri Jun 24 14:22:41 2011
New Revision: 1139319
URL: http://svn.apache.org/viewvc?rev=1139319&view=rev
Log:
merge from 0.8
Modified:
cassandra/trunk/ (props changed)
cassandra/trunk/CHANGES.txt
cassandra/trunk/contrib/ (props changed)
cassandra/trunk/debian/control
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed)
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed)
cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
cassandra/trunk/src/java/org/apache/cassandra/db/Table.java
cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
cassandra/trunk/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java
cassandra/trunk/src/java/org/apache/cassandra/streaming/StreamOut.java
Propchange: cassandra/trunk/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Jun 24 14:22:41 2011
@@ -1,7 +1,7 @@
/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291
-/cassandra/branches/cassandra-0.7:1026516-1133874,1135638,1138148,1138996
+/cassandra/branches/cassandra-0.7:1026516-1138710,1138996
/cassandra/branches/cassandra-0.7.0:1053690-1055654
-/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1137982,1137984,1138149,1138998
+/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1138998
/cassandra/branches/cassandra-0.8.0:1125021-1130369
/cassandra/branches/cassandra-0.8.1:1101014-1125018
/cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689
Modified: cassandra/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1139319&r1=1139318&r2=1139319&view=diff
==============================================================================
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Fri Jun 24 14:22:41 2011
@@ -12,8 +12,8 @@
0.8.2
* improve thread-safety in StreamOutSession (CASSANDRA-2792)
- * Expose number of threads blocked on submitting memtable to flush
- (CASSANDRA-2817)
+ * allow deleting a row and updating indexed columns in it in the
+ same mutation (CASSANDRA-2773)
0.8.1
Propchange: cassandra/trunk/contrib/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Jun 24 14:22:41 2011
@@ -1,7 +1,7 @@
/cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009
-/cassandra/branches/cassandra-0.7/contrib:1026516-1133874,1135638,1138148,1138996
+/cassandra/branches/cassandra-0.7/contrib:1026516-1138710,1138996
/cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654
-/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1137982,1137984,1138149,1138998
+/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1138998
/cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369
/cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018
/cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689
Modified: cassandra/trunk/debian/control
URL: http://svn.apache.org/viewvc/cassandra/trunk/debian/control?rev=1139319&r1=1139318&r2=1139319&view=diff
==============================================================================
--- cassandra/trunk/debian/control (original)
+++ cassandra/trunk/debian/control Fri Jun 24 14:22:41 2011
@@ -2,7 +2,7 @@ Source: cassandra
Section: misc
Priority: extra
Maintainer: Eric Evans <ee...@apache.org>
-Build-Depends: debhelper (>= 5), openjdk-6-jdk (>= 6b11) | java6-sdk, ant (>= 1.7)
+Build-Depends: debhelper (>= 5), openjdk-6-jdk (>= 6b11) | java6-sdk, ant (>= 1.7), ant-optional (>= 1.7)
Homepage: http://cassandra.apache.org
Vcs-Svn: https://svn.apache.org/repos/asf/cassandra/trunk
Vcs-Browser: http://svn.apache.org/viewvc/cassandra/trunk
@@ -10,8 +10,7 @@ Standards-Version: 3.8.3
Package: cassandra
Architecture: all
-Depends: openjdk-6-jre-headless (>= 6b11) | java6-runtime, jsvc (>= 1.0), libcommons-daemon-java (>= 1.0), adduser
-Recommends: libjna-java
+Depends: openjdk-6-jre-headless (>= 6b11) | java6-runtime, jsvc (>= 1.0), libcommons-daemon-java (>= 1.0), adduser, libjna-java
Description: distributed storage system for structured data
Cassandra is a distributed (peer-to-peer) system for the management
and storage of structured data.
Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Jun 24 14:22:41 2011
@@ -1,7 +1,7 @@
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1133874,1135638,1138148,1138996
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1138710,1138996
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654
-/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1137982,1137984,1138149,1138998
+/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1138998
/cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369
/cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689
Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Jun 24 14:22:41 2011
@@ -1,7 +1,7 @@
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1131291
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1133874,1135638,1138148,1138996
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1138710,1138996
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654
-/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090934-1125013,1125019-1137982,1137984,1138149,1138998
+/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090934-1125013,1125019-1138998
/cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1125021-1130369
/cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1101014-1125018
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689
Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Jun 24 14:22:41 2011
@@ -1,7 +1,7 @@
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1052356,1052358-1053452,1053454,1053456-1131291
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516-1133874,1135638,1138148,1138996
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516-1138710,1138996
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1053690-1055654
-/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1090934-1125013,1125019-1137982,1137984,1138149,1138998
+/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1090934-1125013,1125019-1138998
/cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1125021-1130369
/cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1101014-1125018
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1051699-1053689
Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Jun 24 14:22:41 2011
@@ -1,7 +1,7 @@
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:922689-1052356,1052358-1053452,1053454,1053456-1131291
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1026516-1133874,1135638,1138148,1138996
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1026516-1138710,1138996
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1053690-1055654
-/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1090934-1125013,1125019-1137982,1137984,1138149,1138998
+/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1090934-1125013,1125019-1138998
/cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1125021-1130369
/cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1101014-1125018
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1051699-1053689
Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Jun 24 14:22:41 2011
@@ -1,7 +1,7 @@
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:922689-1052356,1052358-1053452,1053454,1053456-1131291
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1026516-1133874,1135638,1138148,1138996
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1026516-1138710,1138996
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1053690-1055654
-/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1090934-1125013,1125019-1137982,1137984,1138149,1138998
+/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1090934-1125013,1125019-1138998
/cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1125021-1130369
/cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1101014-1125018
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1051699-1053689
Modified: cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java?rev=1139319&r1=1139318&r2=1139319&view=diff
==============================================================================
--- cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java Fri Jun 24 14:22:41 2011
@@ -63,8 +63,6 @@ public class DatabaseDescriptor
private static SeedProvider seedProvider;
/* Current index into the above list of directories */
private static int currentIndex = 0;
- private static int consistencyThreads = 4; // not configurable
-
static Map<String, KSMetaData> tables = new HashMap<String, KSMetaData>();
@@ -691,11 +689,6 @@ public class DatabaseDescriptor
return conf.phi_convict_threshold;
}
- public static int getConsistencyThreads()
- {
- return consistencyThreads;
- }
-
public static int getConcurrentReaders()
{
return conf.concurrent_reads;
Modified: cassandra/trunk/src/java/org/apache/cassandra/db/Table.java
URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/Table.java?rev=1139319&r1=1139318&r2=1139319&view=diff
==============================================================================
--- cassandra/trunk/src/java/org/apache/cassandra/db/Table.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/Table.java Fri Jun 24 14:22:41 2011
@@ -471,6 +471,8 @@ public class Table
private static void ignoreObsoleteMutations(ColumnFamily cf, SortedSet<ByteBuffer> mutatedIndexedColumns, ColumnFamily oldIndexedColumns)
{
+ // DO NOT modify the cf object here, it can race w/ the CL write (see https://issues.apache.org/jira/browse/CASSANDRA-2604)
+
if (oldIndexedColumns == null)
return;
@@ -479,7 +481,16 @@ public class Table
ByteBuffer name = iter.next();
IColumn newColumn = cf.getColumn(name); // null == row delete or it wouldn't be marked Mutated
if (newColumn != null && cf.isMarkedForDelete())
- throw new UnsupportedOperationException("Index manager cannot support deleting and inserting into a row in the same mutation");
+ {
+ // row is marked for delete, but column was also updated. if column is timestamped less than
+ // the row tombstone, treat it as if it didn't exist. Otherwise we don't care about row
+ // tombstone for the purpose of the index update and we can proceed as usual.
+ if (newColumn.timestamp() <= cf.getMarkedForDeleteAt())
+ {
+ // don't remove from the cf object; that can race w/ CommitLog write. Leaving it is harmless.
+ newColumn = null;
+ }
+ }
IColumn oldColumn = oldIndexedColumns.getColumn(name);
// deletions are irrelevant to the index unless we're changing state from live -> deleted, i.e.,
Modified: cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java?rev=1139319&r1=1139318&r2=1139319&view=diff
==============================================================================
--- cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java Fri Jun 24 14:22:41 2011
@@ -257,8 +257,8 @@ public class SSTableWriter extends SSTab
if (!desc.isLatestVersion)
// TODO: streaming between different versions will fail: need support for
// recovering other versions to provide a stable streaming api
- throw new RuntimeException(String.format("Cannot recover SSTable with version %s (current version %s).",
- desc.version, Descriptor.CURRENT_VERSION));
+ throw new RuntimeException(String.format("Cannot recover SSTable %s due to version mismatch. (current version is %s).", desc.toString()
+ , Descriptor.CURRENT_VERSION));
return new Builder(desc, type);
}
Modified: cassandra/trunk/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java
URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java?rev=1139319&r1=1139318&r2=1139319&view=diff
==============================================================================
--- cassandra/trunk/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java Fri Jun 24 14:22:41 2011
@@ -92,6 +92,7 @@ public abstract class AbstractCassandraD
*/
protected void setup() throws IOException
{
+ logger.info("JVM vendor/version: {}/{}", System.getProperty("java.vm.name"), System.getProperty("java.version") );
logger.info("Heap size: {}/{}", Runtime.getRuntime().totalMemory(), Runtime.getRuntime().maxMemory());
CLibrary.tryMlockall();
Modified: cassandra/trunk/src/java/org/apache/cassandra/streaming/StreamOut.java
URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/streaming/StreamOut.java?rev=1139319&r1=1139318&r2=1139319&view=diff
==============================================================================
--- cassandra/trunk/src/java/org/apache/cassandra/streaming/StreamOut.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/streaming/StreamOut.java Fri Jun 24 14:22:41 2011
@@ -42,18 +42,31 @@ import org.apache.cassandra.utils.Pair;
/**
* This class handles streaming data from one node to another.
*
- * The source node is in charge of the streaming session. It begins the stream by sending
+ * The source node [the Out side] is always in charge of the streaming session. Streams may
+ * be initiated either directly by the source via the methods in this class,
+ * or on demand from the target (via StreamRequest).
+ *
+ * Files to stream are grouped into sessions, which can have callbacks associated
+ * with them so that (for instance) we can mark a new node a full member of the
+ * cluster after all the data it needs has been streamed.
+ *
+ * The source begins a session by sending
* a Message with the stream bit flag in the Header turned on. Part of that Message
* will include a StreamHeader that includes the files that will be streamed as part
* of that session, as well as the first file-to-be-streamed. (Combining session list
* and first file like this is inconvenient, but not as inconvenient as the old
* three-part send-file-list, wait-for-ack, start-first-file dance.)
*
- * After each file, the target will send a StreamReply indicating success
+ * This is done over a separate TCP connection to avoid blocking ordinary intra-node
+ * traffic during the stream. So there is no Handler for the main stream of data --
+ * when a connection sets the Stream bit, IncomingTcpConnection knows what to expect
+ * without any further Messages.
+ *
+ * After each file, the target node [the In side] will send a StreamReply indicating success
* (FILE_FINISHED) or failure (FILE_RETRY).
*
- * When all files have been successfully transferred and integrated the source will send
- * SESSION_FINISHED and the session is complete.
+ * When all files have been successfully transferred and integrated the target will
+ * send an additional SESSION_FINISHED reply and the session is complete.
*
* For Stream requests (for bootstrap), one subtlety is that we always have to
* create at least one stream reply, even if the list of files is empty, otherwise the