You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by GitBox <gi...@apache.org> on 2021/05/17 10:54:05 UTC
[GitHub] [cassandra] bereng opened a new pull request #1012: Flaky ViewComplexTest
bereng opened a new pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng commented on a change in pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng commented on a change in pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#discussion_r638463358
##########
File path: test/unit/org/apache/cassandra/cql3/ViewFilteringPK.java
##########
@@ -0,0 +1,650 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+import org.junit.After;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import org.apache.cassandra.db.SystemKeyspace;
+import org.apache.cassandra.transport.ProtocolVersion;
+
+@RunWith(Parameterized.class)
+public class ViewFilteringPK extends CQLTester
+{
Review comment:
Ops correct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng closed pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng closed pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng edited a comment on pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng edited a comment on pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#issuecomment-847555479
@ekaterinadimitrova2 I renamed the `ViewComplexTest*` files to `ViewComplex*Test` to conform to the rest of the test files name pattern #justfyi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] ekaterinadimitrova2 commented on a change in pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
ekaterinadimitrova2 commented on a change in pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#discussion_r639323018
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexDeletionsTest.java
##########
@@ -356,20 +363,20 @@ private void testCommutativeRowDeletion(boolean flush) throws Throwable
}
@Test
- public void complexTimestampDeletionTestWithFlush() throws Throwable
+ public void testComplexTimestampDeletionTestWithFlush() throws Throwable
{
- complexTimestampWithbaseNonPKColumnsInViewPKDeletionTest(true);
- complexTimestampWithbasePKColumnsInViewPKDeletionTest(true);
+ testComplexTimestampWithbaseNonPKColumnsInViewPKDeletionTest(true);
+ testComplexTimestampWithbasePKColumnsInViewPKDeletionTest(true);
}
Review comment:
Do we need test twice in the name?
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexTestDeletions.java
##########
@@ -0,0 +1,494 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.apache.cassandra.cql3.ViewComplexTest.createView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateViewWithFlush;
+import static org.junit.Assert.assertEquals;
+
+@RunWith(Parameterized.class)
+public class ViewComplexTestDeletions extends CQLTester
+{
+ @Parameterized.Parameter
+ public ProtocolVersion version;
+
+ @Parameterized.Parameters()
+ public static Collection<Object[]> versions()
+ {
+ return ViewComplexTest.versions();
+ }
+
+ private final List<String> views = new ArrayList<>();
+
+ @BeforeClass
+ public static void startup()
+ {
+ ViewComplexTest.startup();
+ }
+
+ @Before
+ public void begin()
+ {
+ ViewComplexTest.beginSetup(views);
+ }
Review comment:
In general yes, in the tests I haven't seen it done. But I leave it to you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng commented on pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng commented on pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#issuecomment-846941006
CI [j11](https://app.circleci.com/pipelines/github/bereng/cassandra/312/workflows/a0abd59b-6323-4ad9-b444-fe8ed2c1d2fc)
Ci [j8](https://app.circleci.com/pipelines/github/bereng/cassandra/312/workflows/cc152b69-de64-4dd5-9175-1016bfe9a63e)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] ekaterinadimitrova2 commented on a change in pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
ekaterinadimitrova2 commented on a change in pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#discussion_r635560642
##########
File path: test/long/org/apache/cassandra/cql3/ViewComplexTest.java
##########
@@ -57,6 +57,9 @@
@RunWith(Parameterized.class)
public class ViewComplexTest extends CQLTester
{
+ @ClassRule
+ public static Timeout classTimeout = new Timeout(20, TimeUnit.MINUTES);
Review comment:
Isn't 20 too much? I see the long tests have by default 8 minutes timeout, is this one going that far?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng commented on pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng commented on pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#issuecomment-849316676
CI [j11](https://app.circleci.com/pipelines/github/bereng/cassandra/319/workflows/1dd551f5-e09e-42ab-bdf9-5d821e360d0d)
CI [j8](https://app.circleci.com/pipelines/github/bereng/cassandra/319/workflows/60191f62-f2df-493e-aaef-9995c52e89b4)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng commented on a change in pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng commented on a change in pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#discussion_r638469777
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexTest.java
##########
@@ -0,0 +1,605 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.google.common.base.Objects;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import com.datastax.driver.core.exceptions.OperationTimedOutException;
+import org.apache.cassandra.concurrent.SEPExecutor;
+import org.apache.cassandra.concurrent.Stage;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+
+/* This class been split into multiple one bc of timeout issues CASSANDRA-16670
+ * Any changes here check if they apply to the other classes
+ */
+@RunWith(Parameterized.class)
+public class ViewComplexTest extends CQLTester
+{
+ @Parameterized.Parameter
+ public ProtocolVersion version;
+
+ @Parameterized.Parameters()
+ public static Collection<Object[]> versions()
+ {
+ return ProtocolVersion.SUPPORTED.stream()
+ .map(v -> new Object[]{v})
+ .collect(Collectors.toList());
+ }
+
+ private final List<String> views = new ArrayList<>();
+
+ @BeforeClass
+ public static void startup()
+ {
+ requireNetwork();
+ }
+
+ @Before
+ public void begin()
+ {
+ beginSetup(views);
+ }
+
+ public static void beginSetup(List<String> views)
+ {
+ views.clear();
+ }
+
Review comment:
Closing as we can follow the conversation in the one left open.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng commented on a change in pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng commented on a change in pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#discussion_r638465377
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexTestDeletions.java
##########
@@ -0,0 +1,494 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.apache.cassandra.cql3.ViewComplexTest.createView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateViewWithFlush;
+import static org.junit.Assert.assertEquals;
+
+@RunWith(Parameterized.class)
+public class ViewComplexTestDeletions extends CQLTester
+{
+ @Parameterized.Parameter
+ public ProtocolVersion version;
+
+ @Parameterized.Parameters()
+ public static Collection<Object[]> versions()
+ {
+ return ViewComplexTest.versions();
+ }
+
+ private final List<String> views = new ArrayList<>();
+
+ @BeforeClass
+ public static void startup()
+ {
+ ViewComplexTest.startup();
+ }
+
+ @Before
+ public void begin()
+ {
+ ViewComplexTest.beginSetup(views);
+ }
+
+ @After
+ public void end() throws Throwable
+ {
+ ViewComplexTest.endTearDown(views, version, this);
+ }
+
+ // for now, unselected column cannot be fully supported, SEE CASSANDRA-11500
+ @Ignore
+ @Test
+ public void testPartialDeleteUnselectedColumn() throws Throwable
+ {
+ boolean flush = true;
+ execute("USE " + keyspace());
+ executeNet(version, "USE " + keyspace());
+ createTable("CREATE TABLE %s (k int, c int, a int, b int, PRIMARY KEY (k, c))");
+ createView("mv",
+ "CREATE MATERIALIZED VIEW %s AS SELECT k,c FROM %%s WHERE k IS NOT NULL AND c IS NOT NULL PRIMARY KEY (k,c)",
+ version,
+ this,
+ views);
+ Keyspace ks = Keyspace.open(keyspace());
+ ks.getColumnFamilyStore("mv").disableAutoCompaction();
+
+ updateView("UPDATE %s USING TIMESTAMP 10 SET b=1 WHERE k=1 AND c=1", version, this);
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, 1));
+ assertRows(execute("SELECT * from mv"), row(1, 1));
+ updateView("DELETE b FROM %s USING TIMESTAMP 11 WHERE k=1 AND c=1", version, this);
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+ assertEmpty(execute("SELECT * from %s"));
+ assertEmpty(execute("SELECT * from mv"));
+ updateView("UPDATE %s USING TIMESTAMP 1 SET a=1 WHERE k=1 AND c=1", version, this);
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+ assertRows(execute("SELECT * from %s"), row(1, 1, 1, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1));
+
+ execute("truncate %s;");
+
+ // removal generated by unselected column should not shadow PK update with smaller timestamp
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 18 SET a=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, 1, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1));
+
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 20 SET a=null WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"));
+ assertRows(execute("SELECT * from mv"));
+
+ updateViewWithFlush("INSERT INTO %s(k,c) VALUES(1,1) USING TIMESTAMP 15", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1));
+ }
+
+ @Test
+ public void testPartialDeleteSelectedColumnWithFlush() throws Throwable
+ {
+ testPartialDeleteSelectedColumn(true);
+ }
+
+ @Test
+ public void testPartialDeleteSelectedColumnWithoutFlush() throws Throwable
+ {
+ testPartialDeleteSelectedColumn(false);
+ }
+
+ private void testPartialDeleteSelectedColumn(boolean flush) throws Throwable
+ {
+ execute("USE " + keyspace());
+ executeNet(version, "USE " + keyspace());
+ createTable("CREATE TABLE %s (k int, c int, a int, b int, e int, f int, PRIMARY KEY (k, c))");
+ createView("mv",
+ "CREATE MATERIALIZED VIEW %s AS SELECT a, b, c, k FROM %%s WHERE k IS NOT NULL AND c IS NOT NULL PRIMARY KEY (k,c)",
+ version,
+ this,
+ views);
+ Keyspace ks = Keyspace.open(keyspace());
+ ks.getColumnFamilyStore("mv").disableAutoCompaction();
+
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 10 SET b=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, 1, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, 1));
+
+ updateViewWithFlush("DELETE b FROM %s USING TIMESTAMP 11 WHERE k=1 AND c=1", flush, version, this);
+ assertEmpty(execute("SELECT * from %s"));
+ assertEmpty(execute("SELECT * from mv"));
+
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 1 SET a=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, 1, null, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, 1, null));
+
+ updateViewWithFlush("DELETE a FROM %s USING TIMESTAMP 1 WHERE k=1 AND c=1", flush, version, this);
+ assertEmpty(execute("SELECT * from %s"));
+ assertEmpty(execute("SELECT * from mv"));
+
+ // view livenessInfo should not be affected by selected column ts or tb
+ updateViewWithFlush("INSERT INTO %s(k,c) VALUES(1,1) USING TIMESTAMP 0", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, null, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, null));
+
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 12 SET b=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, 1, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, 1));
+
+ updateViewWithFlush("DELETE b FROM %s USING TIMESTAMP 13 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, null, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, null));
+
+ updateViewWithFlush("DELETE FROM %s USING TIMESTAMP 14 WHERE k=1 AND c=1", flush, version, this);
+ assertEmpty(execute("SELECT * from %s"));
+ assertEmpty(execute("SELECT * from mv"));
+
+ updateViewWithFlush("INSERT INTO %s(k,c) VALUES(1,1) USING TIMESTAMP 15", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, null, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, null));
+
+ updateViewWithFlush("UPDATE %s USING TTL 3 SET b=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, 1, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, 1));
+
+ TimeUnit.SECONDS.sleep(4);
+
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, null, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, null));
+
+ updateViewWithFlush("DELETE FROM %s USING TIMESTAMP 15 WHERE k=1 AND c=1", flush, version, this);
+ assertEmpty(execute("SELECT * from %s"));
+ assertEmpty(execute("SELECT * from mv"));
+
+ execute("truncate %s;");
+
+ // removal generated by unselected column should not shadow selected column with smaller timestamp
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 18 SET e=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, null, 1, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, null));
+
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 18 SET e=null WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"));
+ assertRows(execute("SELECT * from mv"));
+
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 16 SET a=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, 1, null, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, 1, null));
+ }
+
+ @Test
+ public void testRangeDeletionWithFlush() throws Throwable
+ {
+ testRangeDeletion(true);
+ }
+
+ @Test
+ public void testRangeDeletionWithoutFlush() throws Throwable
+ {
+ testRangeDeletion(false);
+ }
+
+ public void testRangeDeletion(boolean flush) throws Throwable
+ {
+ // for partition range deletion, need to know that existing row is shadowed instead of not existed.
+ createTable("CREATE TABLE %s (a int, b int, c int, d int, PRIMARY KEY (a))");
+
+ execute("USE " + keyspace());
+ executeNet(version, "USE " + keyspace());
+
+ createView("mv_test1",
+ "CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a IS NOT NULL AND b IS NOT NULL PRIMARY KEY (a, b)",
+ version,
+ this,
+ views);
+
+ Keyspace ks = Keyspace.open(keyspace());
+ ks.getColumnFamilyStore("mv_test1").disableAutoCompaction();
+
+ execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?) using timestamp 0", 1, 1, 1, 1);
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT * FROM mv_test1"), row(1, 1, 1, 1));
+
+ // remove view row
+ updateView("UPDATE %s using timestamp 1 set b = null WHERE a=1", version, this);
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT * FROM mv_test1"));
+ // remove base row, no view updated generated.
+ updateView("DELETE FROM %s using timestamp 2 where a=1", version, this);
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT * FROM mv_test1"));
+
+ // restor view row with b,c column. d is still tombstone
+ updateView("UPDATE %s using timestamp 3 set b = 1,c = 1 where a=1", version, this); // upsert
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT * FROM mv_test1"), row(1, 1, 1, null));
+ }
+
+ @Test
+ public void testCommutativeRowDeletionFlush() throws Throwable
+ {
+ // CASSANDRA-13409
+ testCommutativeRowDeletion(true);
+ }
+
+ @Test
+ public void testCommutativeRowDeletionWithoutFlush() throws Throwable
+ {
+ // CASSANDRA-13409
+ testCommutativeRowDeletion(false);
+ }
+
+ private void testCommutativeRowDeletion(boolean flush) throws Throwable
+ {
+ // CASSANDRA-13409 new update should not resurrect previous deleted data in view
+ createTable("create table %s (p int primary key, v1 int, v2 int)");
+
+ execute("USE " + keyspace());
+ executeNet(version, "USE " + keyspace());
+ Keyspace ks = Keyspace.open(keyspace());
+
+ createView("mv",
+ "create materialized view %s as select * from %%s where p is not null and v1 is not null primary key (v1, p);",
+ version,
+ this,
+ views);
+ ks.getColumnFamilyStore("mv").disableAutoCompaction();
+
+ // sstable-1, Set initial values TS=1
+ updateView("Insert into %s (p, v1, v2) values (3, 1, 3) using timestamp 1;", version, this);
+
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT v2, WRITETIME(v2) from mv WHERE v1 = ? AND p = ?", 1, 3), row(3, 1L));
+ // sstable-2
+ updateView("Delete from %s using timestamp 2 where p = 3;", version, this);
+
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT v1, p, v2, WRITETIME(v2) from mv"));
+ // sstable-3
+ updateView("Insert into %s (p, v1) values (3, 1) using timestamp 3;", version, this);
+
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT v1, p, v2, WRITETIME(v2) from mv"), row(1, 3, null, null));
+ // sstable-4
+ updateView("UPdate %s using timestamp 4 set v1 = 2 where p = 3;", version, this);
+
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT v1, p, v2, WRITETIME(v2) from mv"), row(2, 3, null, null));
+ // sstable-5
+ updateView("UPdate %s using timestamp 5 set v1 = 1 where p = 3;", version, this);
+
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT v1, p, v2, WRITETIME(v2) from mv"), row(1, 3, null, null));
+
+ if (flush)
+ {
+ // compact sstable 2 and 4, 5;
+ ColumnFamilyStore cfs = ks.getColumnFamilyStore("mv");
+ List<String> sstables = cfs.getLiveSSTables()
+ .stream()
+ .sorted((s1, s2) -> s1.descriptor.generation - s2.descriptor.generation)
+ .map(s -> s.getFilename())
+ .collect(Collectors.toList());
+ String dataFiles = String.join(",", Arrays.asList(sstables.get(1), sstables.get(3), sstables.get(4)));
+ CompactionManager.instance.forceUserDefinedCompaction(dataFiles);
+ assertEquals(3, cfs.getLiveSSTables().size());
+ }
+ // regular tombstone should be retained after compaction
+ assertRowsIgnoringOrder(execute("SELECT v1, p, v2, WRITETIME(v2) from mv"), row(1, 3, null, null));
+ }
+
+ @Test
+ public void complexTimestampDeletionTestWithFlush() throws Throwable
+ {
Review comment:
This is a copy-paste from the original. Should we add it? I think yes :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng commented on a change in pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng commented on a change in pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#discussion_r638469470
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexTest.java
##########
@@ -0,0 +1,605 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.google.common.base.Objects;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import com.datastax.driver.core.exceptions.OperationTimedOutException;
+import org.apache.cassandra.concurrent.SEPExecutor;
+import org.apache.cassandra.concurrent.Stage;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+
+/* This class been split into multiple one bc of timeout issues CASSANDRA-16670
+ * Any changes here check if they apply to the other classes
+ */
+@RunWith(Parameterized.class)
+public class ViewComplexTest extends CQLTester
+{
+ @Parameterized.Parameter
+ public ProtocolVersion version;
+
+ @Parameterized.Parameters()
+ public static Collection<Object[]> versions()
+ {
+ return ProtocolVersion.SUPPORTED.stream()
+ .map(v -> new Object[]{v})
+ .collect(Collectors.toList());
+ }
+
+ private final List<String> views = new ArrayList<>();
+
+ @BeforeClass
+ public static void startup()
+ {
+ requireNetwork();
+ }
+
+ @Before
+ public void begin()
+ {
+ beginSetup(views);
+ }
+
+ public static void beginSetup(List<String> views)
+ {
+ views.clear();
+ }
+
Review comment:
Bc you need to call it from this class and from the 'splits'.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng commented on pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng commented on pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#issuecomment-851320036
Rebased:
- CI [j11](https://app.circleci.com/pipelines/github/bereng/cassandra/322/workflows/aa95b758-412d-4055-9065-34c7958a9021) unrelated failures
- CI [j8](https://app.circleci.com/pipelines/github/bereng/cassandra/322/workflows/2df6e714-5886-4946-abc2-f0e4c7eaf6be)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng commented on a change in pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng commented on a change in pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#discussion_r638465005
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexTestDeletions.java
##########
@@ -0,0 +1,494 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.apache.cassandra.cql3.ViewComplexTest.createView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateViewWithFlush;
+import static org.junit.Assert.assertEquals;
+
+@RunWith(Parameterized.class)
+public class ViewComplexTestDeletions extends CQLTester
+{
+ @Parameterized.Parameter
+ public ProtocolVersion version;
+
+ @Parameterized.Parameters()
+ public static Collection<Object[]> versions()
+ {
+ return ViewComplexTest.versions();
+ }
+
+ private final List<String> views = new ArrayList<>();
+
+ @BeforeClass
+ public static void startup()
+ {
+ ViewComplexTest.startup();
+ }
+
+ @Before
+ public void begin()
+ {
+ ViewComplexTest.beginSetup(views);
+ }
Review comment:
You want all 'child' tests to use the same method bc: code duplicity, consistency, in case a bug arises you fic it in one place and making it obvious they are the same test only split into several files right? I fact I would be hard -1 on not doing sthg along the lines to reuse the original method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] ekaterinadimitrova2 commented on a change in pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
ekaterinadimitrova2 commented on a change in pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#discussion_r635560642
##########
File path: test/long/org/apache/cassandra/cql3/ViewComplexTest.java
##########
@@ -57,6 +57,9 @@
@RunWith(Parameterized.class)
public class ViewComplexTest extends CQLTester
{
+ @ClassRule
+ public static Timeout classTimeout = new Timeout(20, TimeUnit.MINUTES);
Review comment:
Isn't 20 too much? I see the long tests have by default 8 minutes timeout, is this one running that long?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng commented on pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng commented on pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#issuecomment-847555479
@ekaterinadimitrova2 I renamed the `ViewComplexTest*` files to `ViewComplex*Test` to conform to the rest of the test file name pattern #justfyi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng edited a comment on pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng edited a comment on pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#issuecomment-851320036
Rebased:
- CI [j11](https://app.circleci.com/pipelines/github/bereng/cassandra/324/workflows/b400e1d1-7d92-43fc-85a7-238749bd23ef)
- CI [j8](https://app.circleci.com/pipelines/github/bereng/cassandra/322/workflows/2df6e714-5886-4946-abc2-f0e4c7eaf6be)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng edited a comment on pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng edited a comment on pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#issuecomment-851320036
Rebased:
- CI [j11](https://app.circleci.com/pipelines/github/bereng/cassandra/324/workflows/b400e1d1-7d92-43fc-85a7-238749bd23ef)
- CI [j8](https://app.circleci.com/pipelines/github/bereng/cassandra/324/workflows/08b58e67-38bf-40e2-9595-4a04ac4ab387)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] bereng commented on pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
bereng commented on pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#issuecomment-843896131
CI [j11](https://app.circleci.com/pipelines/github/bereng/cassandra/301/workflows/5bfb4e80-02a5-4df5-b9ca-768fe6c73f5a)
CI [j8](https://app.circleci.com/pipelines/github/bereng/cassandra/301/workflows/e05ccab7-0bd0-4920-ac2e-c394077a910b)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] ekaterinadimitrova2 commented on a change in pull request #1012: Flaky ViewComplexTest
Posted by GitBox <gi...@apache.org>.
ekaterinadimitrova2 commented on a change in pull request #1012:
URL: https://github.com/apache/cassandra/pull/1012#discussion_r638281413
##########
File path: test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
##########
@@ -46,6 +42,9 @@
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
+/* This class been split into multiple one bc of timeout issues CASSANDRA-16670
+ * Any changes here check if they apply to the other classes
+ */
Review comment:
nit: Can we add which are those other classes so people don't have to search in the ticket?
##########
File path: test/unit/org/apache/cassandra/cql3/ViewFilteringClustering.java
##########
@@ -0,0 +1,656 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+import org.junit.After;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import org.apache.cassandra.db.SystemKeyspace;
+import org.apache.cassandra.transport.ProtocolVersion;
+
+@RunWith(Parameterized.class)
Review comment:
Again the comment
##########
File path: test/unit/org/apache/cassandra/cql3/ViewFilteringPK.java
##########
@@ -0,0 +1,650 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+import org.junit.After;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import org.apache.cassandra.db.SystemKeyspace;
+import org.apache.cassandra.transport.ProtocolVersion;
+
+@RunWith(Parameterized.class)
Review comment:
Again what about the comment
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexTestTTL.java
##########
@@ -0,0 +1,270 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.apache.cassandra.cql3.ViewComplexTest.createView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateViewWithFlush;
+import static org.junit.Assert.assertTrue;
+
+@RunWith(Parameterized.class)
Review comment:
Do we want to add also here the comment about the split? Similar to other places
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexTest.java
##########
@@ -0,0 +1,605 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.google.common.base.Objects;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import com.datastax.driver.core.exceptions.OperationTimedOutException;
+import org.apache.cassandra.concurrent.SEPExecutor;
+import org.apache.cassandra.concurrent.Stage;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+
+/* This class been split into multiple one bc of timeout issues CASSANDRA-16670
+ * Any changes here check if they apply to the other classes
+ */
Review comment:
Same comment applies here and at all other places, the typo and the addition of those other classes that we might have to check
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexTestUpdates.java
##########
@@ -0,0 +1,365 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.apache.cassandra.cql3.ViewComplexTest.createView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateView;
+
+@RunWith(Parameterized.class)
Review comment:
Again, can we add the same comment here too?
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexTest.java
##########
@@ -0,0 +1,605 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.google.common.base.Objects;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import com.datastax.driver.core.exceptions.OperationTimedOutException;
+import org.apache.cassandra.concurrent.SEPExecutor;
+import org.apache.cassandra.concurrent.Stage;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+
+/* This class been split into multiple one bc of timeout issues CASSANDRA-16670
+ * Any changes here check if they apply to the other classes
+ */
+@RunWith(Parameterized.class)
+public class ViewComplexTest extends CQLTester
+{
+ @Parameterized.Parameter
+ public ProtocolVersion version;
+
+ @Parameterized.Parameters()
+ public static Collection<Object[]> versions()
+ {
+ return ProtocolVersion.SUPPORTED.stream()
+ .map(v -> new Object[]{v})
+ .collect(Collectors.toList());
+ }
+
+ private final List<String> views = new ArrayList<>();
+
+ @BeforeClass
+ public static void startup()
+ {
+ requireNetwork();
+ }
+
+ @Before
+ public void begin()
+ {
+ beginSetup(views);
+ }
+
+ public static void beginSetup(List<String> views)
+ {
+ views.clear();
+ }
+
+ @After
+ public void end() throws Throwable
+ {
+ endTearDown(views, version, this);
+ }
+
+ public static void endTearDown(List<String> views, ProtocolVersion version, CQLTester cqlTester) throws Throwable
+ {
+ for (String viewName : views)
+ cqlTester.executeNet(version, "DROP MATERIALIZED VIEW " + viewName);
+ }
Review comment:
And same for this method
##########
File path: test/unit/org/apache/cassandra/cql3/ViewFilteringClustering.java
##########
@@ -0,0 +1,656 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+import org.junit.After;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import org.apache.cassandra.db.SystemKeyspace;
+import org.apache.cassandra.transport.ProtocolVersion;
+
+@RunWith(Parameterized.class)
+public class ViewFilteringClustering extends CQLTester
+{
Review comment:
`ViewFilteringClusteringTest`?
##########
File path: test/unit/org/apache/cassandra/cql3/ViewFilteringPK.java
##########
@@ -0,0 +1,650 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+import org.junit.After;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import org.apache.cassandra.db.SystemKeyspace;
+import org.apache.cassandra.transport.ProtocolVersion;
+
+@RunWith(Parameterized.class)
+public class ViewFilteringPK extends CQLTester
+{
Review comment:
`ViewFilteringPKTest`?
##########
File path: test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
##########
@@ -46,6 +42,9 @@
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
+/* This class been split into multiple one bc of timeout issues CASSANDRA-16670
+ * Any changes here check if they apply to the other classes
+ */
Review comment:
typo: multiple one**s**
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexTestDeletions.java
##########
@@ -0,0 +1,494 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.apache.cassandra.cql3.ViewComplexTest.createView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateViewWithFlush;
+import static org.junit.Assert.assertEquals;
+
+@RunWith(Parameterized.class)
+public class ViewComplexTestDeletions extends CQLTester
+{
+ @Parameterized.Parameter
+ public ProtocolVersion version;
+
+ @Parameterized.Parameters()
+ public static Collection<Object[]> versions()
+ {
+ return ViewComplexTest.versions();
+ }
+
+ private final List<String> views = new ArrayList<>();
+
+ @BeforeClass
+ public static void startup()
+ {
+ ViewComplexTest.startup();
+ }
+
+ @Before
+ public void begin()
+ {
+ ViewComplexTest.beginSetup(views);
+ }
+
+ @After
+ public void end() throws Throwable
+ {
+ ViewComplexTest.endTearDown(views, version, this);
+ }
+
+ // for now, unselected column cannot be fully supported, SEE CASSANDRA-11500
+ @Ignore
+ @Test
+ public void testPartialDeleteUnselectedColumn() throws Throwable
+ {
+ boolean flush = true;
+ execute("USE " + keyspace());
+ executeNet(version, "USE " + keyspace());
+ createTable("CREATE TABLE %s (k int, c int, a int, b int, PRIMARY KEY (k, c))");
+ createView("mv",
+ "CREATE MATERIALIZED VIEW %s AS SELECT k,c FROM %%s WHERE k IS NOT NULL AND c IS NOT NULL PRIMARY KEY (k,c)",
+ version,
+ this,
+ views);
+ Keyspace ks = Keyspace.open(keyspace());
+ ks.getColumnFamilyStore("mv").disableAutoCompaction();
+
+ updateView("UPDATE %s USING TIMESTAMP 10 SET b=1 WHERE k=1 AND c=1", version, this);
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, 1));
+ assertRows(execute("SELECT * from mv"), row(1, 1));
+ updateView("DELETE b FROM %s USING TIMESTAMP 11 WHERE k=1 AND c=1", version, this);
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+ assertEmpty(execute("SELECT * from %s"));
+ assertEmpty(execute("SELECT * from mv"));
+ updateView("UPDATE %s USING TIMESTAMP 1 SET a=1 WHERE k=1 AND c=1", version, this);
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+ assertRows(execute("SELECT * from %s"), row(1, 1, 1, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1));
+
+ execute("truncate %s;");
+
+ // removal generated by unselected column should not shadow PK update with smaller timestamp
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 18 SET a=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, 1, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1));
+
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 20 SET a=null WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"));
+ assertRows(execute("SELECT * from mv"));
+
+ updateViewWithFlush("INSERT INTO %s(k,c) VALUES(1,1) USING TIMESTAMP 15", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1));
+ }
+
+ @Test
+ public void testPartialDeleteSelectedColumnWithFlush() throws Throwable
+ {
+ testPartialDeleteSelectedColumn(true);
+ }
+
+ @Test
+ public void testPartialDeleteSelectedColumnWithoutFlush() throws Throwable
+ {
+ testPartialDeleteSelectedColumn(false);
+ }
+
+ private void testPartialDeleteSelectedColumn(boolean flush) throws Throwable
+ {
+ execute("USE " + keyspace());
+ executeNet(version, "USE " + keyspace());
+ createTable("CREATE TABLE %s (k int, c int, a int, b int, e int, f int, PRIMARY KEY (k, c))");
+ createView("mv",
+ "CREATE MATERIALIZED VIEW %s AS SELECT a, b, c, k FROM %%s WHERE k IS NOT NULL AND c IS NOT NULL PRIMARY KEY (k,c)",
+ version,
+ this,
+ views);
+ Keyspace ks = Keyspace.open(keyspace());
+ ks.getColumnFamilyStore("mv").disableAutoCompaction();
+
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 10 SET b=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, 1, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, 1));
+
+ updateViewWithFlush("DELETE b FROM %s USING TIMESTAMP 11 WHERE k=1 AND c=1", flush, version, this);
+ assertEmpty(execute("SELECT * from %s"));
+ assertEmpty(execute("SELECT * from mv"));
+
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 1 SET a=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, 1, null, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, 1, null));
+
+ updateViewWithFlush("DELETE a FROM %s USING TIMESTAMP 1 WHERE k=1 AND c=1", flush, version, this);
+ assertEmpty(execute("SELECT * from %s"));
+ assertEmpty(execute("SELECT * from mv"));
+
+ // view livenessInfo should not be affected by selected column ts or tb
+ updateViewWithFlush("INSERT INTO %s(k,c) VALUES(1,1) USING TIMESTAMP 0", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, null, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, null));
+
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 12 SET b=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, 1, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, 1));
+
+ updateViewWithFlush("DELETE b FROM %s USING TIMESTAMP 13 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, null, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, null));
+
+ updateViewWithFlush("DELETE FROM %s USING TIMESTAMP 14 WHERE k=1 AND c=1", flush, version, this);
+ assertEmpty(execute("SELECT * from %s"));
+ assertEmpty(execute("SELECT * from mv"));
+
+ updateViewWithFlush("INSERT INTO %s(k,c) VALUES(1,1) USING TIMESTAMP 15", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, null, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, null));
+
+ updateViewWithFlush("UPDATE %s USING TTL 3 SET b=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, 1, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, 1));
+
+ TimeUnit.SECONDS.sleep(4);
+
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, null, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, null));
+
+ updateViewWithFlush("DELETE FROM %s USING TIMESTAMP 15 WHERE k=1 AND c=1", flush, version, this);
+ assertEmpty(execute("SELECT * from %s"));
+ assertEmpty(execute("SELECT * from mv"));
+
+ execute("truncate %s;");
+
+ // removal generated by unselected column should not shadow selected column with smaller timestamp
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 18 SET e=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, null, null, 1, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, null, null));
+
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 18 SET e=null WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"));
+ assertRows(execute("SELECT * from mv"));
+
+ updateViewWithFlush("UPDATE %s USING TIMESTAMP 16 SET a=1 WHERE k=1 AND c=1", flush, version, this);
+ assertRows(execute("SELECT * from %s"), row(1, 1, 1, null, null, null));
+ assertRows(execute("SELECT * from mv"), row(1, 1, 1, null));
+ }
+
+ @Test
+ public void testRangeDeletionWithFlush() throws Throwable
+ {
+ testRangeDeletion(true);
+ }
+
+ @Test
+ public void testRangeDeletionWithoutFlush() throws Throwable
+ {
+ testRangeDeletion(false);
+ }
+
+ public void testRangeDeletion(boolean flush) throws Throwable
+ {
+ // for partition range deletion, need to know that existing row is shadowed instead of not existed.
+ createTable("CREATE TABLE %s (a int, b int, c int, d int, PRIMARY KEY (a))");
+
+ execute("USE " + keyspace());
+ executeNet(version, "USE " + keyspace());
+
+ createView("mv_test1",
+ "CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a IS NOT NULL AND b IS NOT NULL PRIMARY KEY (a, b)",
+ version,
+ this,
+ views);
+
+ Keyspace ks = Keyspace.open(keyspace());
+ ks.getColumnFamilyStore("mv_test1").disableAutoCompaction();
+
+ execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?) using timestamp 0", 1, 1, 1, 1);
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT * FROM mv_test1"), row(1, 1, 1, 1));
+
+ // remove view row
+ updateView("UPDATE %s using timestamp 1 set b = null WHERE a=1", version, this);
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT * FROM mv_test1"));
+ // remove base row, no view updated generated.
+ updateView("DELETE FROM %s using timestamp 2 where a=1", version, this);
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT * FROM mv_test1"));
+
+ // restor view row with b,c column. d is still tombstone
+ updateView("UPDATE %s using timestamp 3 set b = 1,c = 1 where a=1", version, this); // upsert
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT * FROM mv_test1"), row(1, 1, 1, null));
+ }
+
+ @Test
+ public void testCommutativeRowDeletionFlush() throws Throwable
+ {
+ // CASSANDRA-13409
+ testCommutativeRowDeletion(true);
+ }
+
+ @Test
+ public void testCommutativeRowDeletionWithoutFlush() throws Throwable
+ {
+ // CASSANDRA-13409
+ testCommutativeRowDeletion(false);
+ }
+
+ private void testCommutativeRowDeletion(boolean flush) throws Throwable
+ {
+ // CASSANDRA-13409 new update should not resurrect previous deleted data in view
+ createTable("create table %s (p int primary key, v1 int, v2 int)");
+
+ execute("USE " + keyspace());
+ executeNet(version, "USE " + keyspace());
+ Keyspace ks = Keyspace.open(keyspace());
+
+ createView("mv",
+ "create materialized view %s as select * from %%s where p is not null and v1 is not null primary key (v1, p);",
+ version,
+ this,
+ views);
+ ks.getColumnFamilyStore("mv").disableAutoCompaction();
+
+ // sstable-1, Set initial values TS=1
+ updateView("Insert into %s (p, v1, v2) values (3, 1, 3) using timestamp 1;", version, this);
+
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT v2, WRITETIME(v2) from mv WHERE v1 = ? AND p = ?", 1, 3), row(3, 1L));
+ // sstable-2
+ updateView("Delete from %s using timestamp 2 where p = 3;", version, this);
+
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT v1, p, v2, WRITETIME(v2) from mv"));
+ // sstable-3
+ updateView("Insert into %s (p, v1) values (3, 1) using timestamp 3;", version, this);
+
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT v1, p, v2, WRITETIME(v2) from mv"), row(1, 3, null, null));
+ // sstable-4
+ updateView("UPdate %s using timestamp 4 set v1 = 2 where p = 3;", version, this);
+
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT v1, p, v2, WRITETIME(v2) from mv"), row(2, 3, null, null));
+ // sstable-5
+ updateView("UPdate %s using timestamp 5 set v1 = 1 where p = 3;", version, this);
+
+ if (flush)
+ FBUtilities.waitOnFutures(ks.flush());
+
+ assertRowsIgnoringOrder(execute("SELECT v1, p, v2, WRITETIME(v2) from mv"), row(1, 3, null, null));
+
+ if (flush)
+ {
+ // compact sstable 2 and 4, 5;
+ ColumnFamilyStore cfs = ks.getColumnFamilyStore("mv");
+ List<String> sstables = cfs.getLiveSSTables()
+ .stream()
+ .sorted((s1, s2) -> s1.descriptor.generation - s2.descriptor.generation)
+ .map(s -> s.getFilename())
+ .collect(Collectors.toList());
+ String dataFiles = String.join(",", Arrays.asList(sstables.get(1), sstables.get(3), sstables.get(4)));
+ CompactionManager.instance.forceUserDefinedCompaction(dataFiles);
+ assertEquals(3, cfs.getLiveSSTables().size());
+ }
+ // regular tombstone should be retained after compaction
+ assertRowsIgnoringOrder(execute("SELECT v1, p, v2, WRITETIME(v2) from mv"), row(1, 3, null, null));
+ }
+
+ @Test
+ public void complexTimestampDeletionTestWithFlush() throws Throwable
+ {
Review comment:
I guess the `test` prefix was not added to the method name so it doesn't become too long?
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexTest.java
##########
@@ -0,0 +1,605 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.google.common.base.Objects;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import com.datastax.driver.core.exceptions.OperationTimedOutException;
+import org.apache.cassandra.concurrent.SEPExecutor;
+import org.apache.cassandra.concurrent.Stage;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+
+/* This class been split into multiple one bc of timeout issues CASSANDRA-16670
+ * Any changes here check if they apply to the other classes
+ */
+@RunWith(Parameterized.class)
+public class ViewComplexTest extends CQLTester
+{
+ @Parameterized.Parameter
+ public ProtocolVersion version;
+
+ @Parameterized.Parameters()
+ public static Collection<Object[]> versions()
+ {
+ return ProtocolVersion.SUPPORTED.stream()
+ .map(v -> new Object[]{v})
+ .collect(Collectors.toList());
+ }
+
+ private final List<String> views = new ArrayList<>();
+
+ @BeforeClass
+ public static void startup()
+ {
+ requireNetwork();
+ }
+
+ @Before
+ public void begin()
+ {
+ beginSetup(views);
+ }
+
+ public static void beginSetup(List<String> views)
+ {
+ views.clear();
+ }
+
Review comment:
I am not sure why did you decide to add this new method?
##########
File path: test/unit/org/apache/cassandra/cql3/ViewComplexTestDeletions.java
##########
@@ -0,0 +1,494 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.apache.cassandra.cql3.ViewComplexTest.createView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateView;
+import static org.apache.cassandra.cql3.ViewComplexTest.updateViewWithFlush;
+import static org.junit.Assert.assertEquals;
+
+@RunWith(Parameterized.class)
+public class ViewComplexTestDeletions extends CQLTester
+{
+ @Parameterized.Parameter
+ public ProtocolVersion version;
+
+ @Parameterized.Parameters()
+ public static Collection<Object[]> versions()
+ {
+ return ViewComplexTest.versions();
+ }
+
+ private final List<String> views = new ArrayList<>();
+
+ @BeforeClass
+ public static void startup()
+ {
+ ViewComplexTest.startup();
+ }
+
+ @Before
+ public void begin()
+ {
+ ViewComplexTest.beginSetup(views);
+ }
Review comment:
oh I see now, you reuse the method. Not sure how much value this reuse of method from the other test brings you here. What do I miss?
##########
File path: test/unit/org/apache/cassandra/cql3/ViewFilteringTest.java
##########
@@ -0,0 +1,1041 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.*;
+import java.util.stream.Collectors;
+
+import org.junit.After;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Ignore;
+import org.junit.Test;
+
+import com.datastax.driver.core.exceptions.InvalidQueryException;
+import org.junit.Assert;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import com.datastax.driver.core.exceptions.OperationTimedOutException;
+import org.apache.cassandra.concurrent.SEPExecutor;
+import org.apache.cassandra.concurrent.Stage;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.db.SystemKeyspace;
+import org.apache.cassandra.transport.ProtocolVersion;
+import org.apache.cassandra.utils.FBUtilities;
+
+/* This class been split into multiple one bc of timeout issues CASSANDRA-16670
+ * Any changes here check if they apply to the other classes
+ */
Review comment:
Same as the other places
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org