[GitHub] [ignite] Mmuzaf commented on a change in pull request #7934: IGNITE-11970 CQ buffers cleanup after acknowledge receiving implemented

[GitBox <gi...@apache.org>]

Mmuzaf commented on a change in pull request #7934:
URL: https://github.com/apache/ignite/pull/7934#discussion_r450912021



##########
File path: modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryEventBuffer.java
##########
@@ -325,6 +338,27 @@ private Batch initBatch(AffinityTopologyVersion topVer, boolean backup) {
             endCntr = startCntr + BUF_SIZE - 1;
         }
 
+        /**
+         * @param updateCntr Acknowledged counter.
+         */
+        synchronized void cleanupEntries(Long updateCntr) {
+            if (entries == null)
+                return;
+
+            int next = lastCleaned + 1;
+
+            for (int i = next; i < entries.length; i++) {
+                CacheContinuousQueryEntry e = entries[i];
+
+                if (e != null && e.updateCounter() <= updateCntr) {

Review comment:
       Do we need checking `e` for `non-null` value? I think we need to clean up entries that have update counter less than acked counter each time from the beginning of `entries` array. 
   We can have out of order events on backup due to cache messages reordering, that's why an entry which comes after the acked counter (this entry has less counter than acked counter from the client) will never be cleaned.

##########
File path: modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryHandler.java
##########
@@ -1334,7 +1346,9 @@ public boolean isMarshalled() {
     @Override public void onBatchAcknowledged(final UUID routineId,
         GridContinuousBatch batch,
         final GridKernalContext ctx) {
-        sendBackupAcknowledge(ackBuf.onAcknowledged(batch), routineId, ctx);
+        sendBackupAcknowledge(ackBufBackup.onAcknowledged(batch), routineId, ctx);
+
+        cleanupBuffers(ackBuf.onAcknowledged(batch));

Review comment:
       Can we achieve the same condition by calling `lsnr.cleanupBackupQueue` here with an acknowledged collection of counters? Why do we need a separate buffer for clearing partition buffers?
   
   ```
   ackedTup = ackBufBackup.onAcknowledged(batch);
   
   sendBackupAcknowledge(ackedTup, routineId, ctx);
   
   lsnr.cleanupBackupQueue(ackedTup.get1());
   ```

##########
File path: modules/core/src/test/java/org/apache/ignite/internal/processors/cache/query/continuous/ContinuousQueryBufferCleanupAbstractTest.java
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.query.continuous;
+
+import java.util.UUID;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteCache;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cache.query.AbstractContinuousQuery;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.processors.continuous.GridContinuousProcessor;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
+import org.junit.Test;
+
+import static org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.DFLT_ACK_SND_THRESHOLD;
+
+/**
+ * Test for continuous query buffer cleanup.
+ */
+public abstract class ContinuousQueryBufferCleanupAbstractTest extends GridCommonAbstractTest {
+    /** */
+    private static final int RECORDS_CNT = 10000;
+
+    /** */
+    private static final int ACK_THRESHOLD = 100;
+
+    /** */
+    private static final String REMOTE_ROUTINE_INFO_CLASS_NAME = "org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$RemoteRoutineInfo";
+
+    /** */
+    private static final String LOCAL_ROUTINE_INFO_CLASS_NAME = "org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$LocalRoutineInfo";
+
+    /** */
+    private static final String BATCH_CLASS_NAME = "org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryEventBuffer$Batch";
+
+    /** Continuous query updates counter */
+    protected final AtomicInteger updCntr = new AtomicInteger();
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        stopAllGrids();
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithSingleNode() throws Exception {
+        checkBuffer(1, 0, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodes() throws Exception {
+        checkBuffer(2, 0, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodesWithBackups() throws Exception {
+        checkBuffer(2, 1, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodesWithBackupsWithoutClient() throws Exception {
+        checkBuffer(2, 1, false);
+    }
+
+    /** Provides continuous query for the test. */
+    protected abstract AbstractContinuousQuery<Integer, String> getContinuousQuery();
+
+    /** */
+    private void checkBuffer(int srvCnt, int backupsCnt, boolean useClient) throws Exception {
+        System.setProperty("IGNITE_CONTINUOUS_QUERY_ACK_THRESHOLD", Integer.toString(ACK_THRESHOLD));
+
+        IgniteEx[] srvs = new IgniteEx[srvCnt];
+
+        for (int i = 0; i < srvCnt; i++)
+            srvs[i] = startGrid("srv" + i);
+
+        Ignite qryNode = useClient
+            ? startGrid(optimize(getConfiguration("client").setClientMode(true)))
+            : srvs[0];
+
+        CacheConfiguration<Integer, String> cacheCfg = new CacheConfiguration<Integer, String>("testCache")
+            .setBackups(backupsCnt)
+            .setAffinity(new RendezvousAffinityFunction(32, null));
+
+        IgniteCache<Integer, String> cache = qryNode.getOrCreateCache(cacheCfg);
+
+        AbstractContinuousQuery<Integer, String> qry = getContinuousQuery();
+
+        cache.query(qry);
+
+        for (int i = 0; i < RECORDS_CNT; i++)
+            cache.put(i, Integer.toString(i));
+
+        Thread.sleep(2000);

Review comment:
       Why do we need `sleep` here? I think we should use the `waitForCondition` here that all of the entries are processed by CQ.

##########
File path: modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryEventBuffer.java
##########
@@ -325,6 +338,27 @@ private Batch initBatch(AffinityTopologyVersion topVer, boolean backup) {
             endCntr = startCntr + BUF_SIZE - 1;
         }
 
+        /**
+         * @param updateCntr Acknowledged counter.
+         */
+        synchronized void cleanupEntries(Long updateCntr) {
+            if (entries == null)
+                return;
+
+            int next = lastCleaned + 1;
+
+            for (int i = next; i < entries.length; i++) {
+                CacheContinuousQueryEntry e = entries[i];
+
+                if (e != null && e.updateCounter() <= updateCntr) {
+                    entries[i] = null;
+
+                    lastCleaned = i;

Review comment:
       Due to my comment above we can remove `lastCleaned` position.

##########
File path: modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryHandler.java
##########
@@ -483,8 +492,11 @@ public void keepBinary(boolean keepBinary) {
                 for (Map.Entry<Integer, Long> e : updateCntrs.entrySet()) {

Review comment:
       It seems the `cleanupBackupQueue` becomes not the right name for this case. Let's rename it to something like `cleanupPartitionBuffers`.

##########
File path: modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryEventBuffer.java
##########
@@ -325,6 +338,27 @@ private Batch initBatch(AffinityTopologyVersion topVer, boolean backup) {
             endCntr = startCntr + BUF_SIZE - 1;
         }
 
+        /**
+         * @param updateCntr Acknowledged counter.
+         */
+        synchronized void cleanupEntries(Long updateCntr) {

Review comment:
       Let's use `long` primitive type here to avoid unboxing each time comparing update counters.

##########
File path: modules/core/src/test/java/org/apache/ignite/internal/processors/cache/query/continuous/ContinuousQueryBufferCleanupAbstractTest.java
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.query.continuous;
+
+import java.util.UUID;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteCache;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cache.query.AbstractContinuousQuery;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.processors.continuous.GridContinuousProcessor;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
+import org.junit.Test;
+
+import static org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.DFLT_ACK_SND_THRESHOLD;
+
+/**
+ * Test for continuous query buffer cleanup.
+ */
+public abstract class ContinuousQueryBufferCleanupAbstractTest extends GridCommonAbstractTest {
+    /** */
+    private static final int RECORDS_CNT = 10000;
+
+    /** */
+    private static final int ACK_THRESHOLD = 100;
+
+    /** */
+    private static final String REMOTE_ROUTINE_INFO_CLASS_NAME = "org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$RemoteRoutineInfo";
+
+    /** */
+    private static final String LOCAL_ROUTINE_INFO_CLASS_NAME = "org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$LocalRoutineInfo";
+
+    /** */
+    private static final String BATCH_CLASS_NAME = "org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryEventBuffer$Batch";
+
+    /** Continuous query updates counter */
+    protected final AtomicInteger updCntr = new AtomicInteger();
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        stopAllGrids();
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithSingleNode() throws Exception {
+        checkBuffer(1, 0, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodes() throws Exception {
+        checkBuffer(2, 0, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodesWithBackups() throws Exception {
+        checkBuffer(2, 1, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodesWithBackupsWithoutClient() throws Exception {
+        checkBuffer(2, 1, false);
+    }
+
+    /** Provides continuous query for the test. */
+    protected abstract AbstractContinuousQuery<Integer, String> getContinuousQuery();
+
+    /** */
+    private void checkBuffer(int srvCnt, int backupsCnt, boolean useClient) throws Exception {
+        System.setProperty("IGNITE_CONTINUOUS_QUERY_ACK_THRESHOLD", Integer.toString(ACK_THRESHOLD));

Review comment:
       The ACK_THRESHOLD is a static property. If the `CacheContinuousQueryHandler` class is loaded first by this test, other tests in the same JVM will use this value and may fail. If the class already loaded and used before, the property will have no effect and this case can fail.
   I'd suggest using the default `CacheContinuousQueryHandler.ACK_THRESHOLD`.

##########
File path: modules/core/src/test/java/org/apache/ignite/internal/processors/cache/query/continuous/ContinuousQueryBufferCleanupAbstractTest.java
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.query.continuous;
+
+import java.util.UUID;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteCache;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cache.query.AbstractContinuousQuery;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.processors.continuous.GridContinuousProcessor;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
+import org.junit.Test;
+
+import static org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.DFLT_ACK_SND_THRESHOLD;
+
+/**
+ * Test for continuous query buffer cleanup.
+ */
+public abstract class ContinuousQueryBufferCleanupAbstractTest extends GridCommonAbstractTest {
+    /** */
+    private static final int RECORDS_CNT = 10000;
+
+    /** */
+    private static final int ACK_THRESHOLD = 100;
+
+    /** */
+    private static final String REMOTE_ROUTINE_INFO_CLASS_NAME = "org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$RemoteRoutineInfo";
+
+    /** */
+    private static final String LOCAL_ROUTINE_INFO_CLASS_NAME = "org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$LocalRoutineInfo";
+
+    /** */
+    private static final String BATCH_CLASS_NAME = "org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryEventBuffer$Batch";
+
+    /** Continuous query updates counter */
+    protected final AtomicInteger updCntr = new AtomicInteger();
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        stopAllGrids();
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithSingleNode() throws Exception {
+        checkBuffer(1, 0, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodes() throws Exception {
+        checkBuffer(2, 0, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodesWithBackups() throws Exception {
+        checkBuffer(2, 1, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodesWithBackupsWithoutClient() throws Exception {
+        checkBuffer(2, 1, false);
+    }
+
+    /** Provides continuous query for the test. */
+    protected abstract AbstractContinuousQuery<Integer, String> getContinuousQuery();
+
+    /** */
+    private void checkBuffer(int srvCnt, int backupsCnt, boolean useClient) throws Exception {
+        System.setProperty("IGNITE_CONTINUOUS_QUERY_ACK_THRESHOLD", Integer.toString(ACK_THRESHOLD));
+
+        IgniteEx[] srvs = new IgniteEx[srvCnt];
+
+        for (int i = 0; i < srvCnt; i++)
+            srvs[i] = startGrid("srv" + i);
+
+        Ignite qryNode = useClient
+            ? startGrid(optimize(getConfiguration("client").setClientMode(true)))

Review comment:
       Let's use `startClientGrid()`.

##########
File path: modules/core/src/test/java/org/apache/ignite/internal/processors/cache/query/continuous/ContinuousQueryBufferCleanupAbstractTest.java
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.query.continuous;
+
+import java.util.UUID;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteCache;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cache.query.AbstractContinuousQuery;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.processors.continuous.GridContinuousProcessor;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
+import org.junit.Test;
+
+import static org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.DFLT_ACK_SND_THRESHOLD;
+
+/**
+ * Test for continuous query buffer cleanup.
+ */
+public abstract class ContinuousQueryBufferCleanupAbstractTest extends GridCommonAbstractTest {
+    /** */
+    private static final int RECORDS_CNT = 10000;
+
+    /** */
+    private static final int ACK_THRESHOLD = 100;
+
+    /** */
+    private static final String REMOTE_ROUTINE_INFO_CLASS_NAME = "org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$RemoteRoutineInfo";
+
+    /** */
+    private static final String LOCAL_ROUTINE_INFO_CLASS_NAME = "org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$LocalRoutineInfo";
+
+    /** */
+    private static final String BATCH_CLASS_NAME = "org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryEventBuffer$Batch";
+
+    /** Continuous query updates counter */
+    protected final AtomicInteger updCntr = new AtomicInteger();
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        stopAllGrids();
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithSingleNode() throws Exception {
+        checkBuffer(1, 0, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodes() throws Exception {
+        checkBuffer(2, 0, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodesWithBackups() throws Exception {
+        checkBuffer(2, 1, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodesWithBackupsWithoutClient() throws Exception {
+        checkBuffer(2, 1, false);
+    }
+
+    /** Provides continuous query for the test. */
+    protected abstract AbstractContinuousQuery<Integer, String> getContinuousQuery();
+
+    /** */
+    private void checkBuffer(int srvCnt, int backupsCnt, boolean useClient) throws Exception {

Review comment:
       Can the `@Parameterized` be used here instead of creating each test per value?

##########
File path: modules/core/src/test/java/org/apache/ignite/internal/processors/cache/query/continuous/ContinuousQueryBufferCleanupAbstractTest.java
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.query.continuous;
+
+import java.util.UUID;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteCache;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cache.query.AbstractContinuousQuery;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.processors.continuous.GridContinuousProcessor;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
+import org.junit.Test;
+
+import static org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.DFLT_ACK_SND_THRESHOLD;
+
+/**
+ * Test for continuous query buffer cleanup.
+ */
+public abstract class ContinuousQueryBufferCleanupAbstractTest extends GridCommonAbstractTest {
+    /** */
+    private static final int RECORDS_CNT = 10000;
+
+    /** */
+    private static final int ACK_THRESHOLD = 100;

Review comment:
       I think the constant from the `CacheContinuousQueryHandler` should be used here.

##########
File path: modules/core/src/test/java/org/apache/ignite/internal/processors/cache/query/continuous/ContinuousQueryBufferCleanupAbstractTest.java
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.query.continuous;
+
+import java.util.UUID;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteCache;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cache.query.AbstractContinuousQuery;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.processors.continuous.GridContinuousProcessor;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
+import org.junit.Test;
+
+import static org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.DFLT_ACK_SND_THRESHOLD;
+
+/**
+ * Test for continuous query buffer cleanup.
+ */
+public abstract class ContinuousQueryBufferCleanupAbstractTest extends GridCommonAbstractTest {
+    /** */
+    private static final int RECORDS_CNT = 10000;
+
+    /** */
+    private static final int ACK_THRESHOLD = 100;
+
+    /** */
+    private static final String REMOTE_ROUTINE_INFO_CLASS_NAME = "org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$RemoteRoutineInfo";
+
+    /** */
+    private static final String LOCAL_ROUTINE_INFO_CLASS_NAME = "org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$LocalRoutineInfo";
+
+    /** */
+    private static final String BATCH_CLASS_NAME = "org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryEventBuffer$Batch";
+
+    /** Continuous query updates counter */
+    protected final AtomicInteger updCntr = new AtomicInteger();
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        stopAllGrids();
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithSingleNode() throws Exception {
+        checkBuffer(1, 0, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodes() throws Exception {
+        checkBuffer(2, 0, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodesWithBackups() throws Exception {
+        checkBuffer(2, 1, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodesWithBackupsWithoutClient() throws Exception {
+        checkBuffer(2, 1, false);
+    }
+
+    /** Provides continuous query for the test. */
+    protected abstract AbstractContinuousQuery<Integer, String> getContinuousQuery();
+
+    /** */
+    private void checkBuffer(int srvCnt, int backupsCnt, boolean useClient) throws Exception {
+        System.setProperty("IGNITE_CONTINUOUS_QUERY_ACK_THRESHOLD", Integer.toString(ACK_THRESHOLD));
+
+        IgniteEx[] srvs = new IgniteEx[srvCnt];
+
+        for (int i = 0; i < srvCnt; i++)
+            srvs[i] = startGrid("srv" + i);
+
+        Ignite qryNode = useClient
+            ? startGrid(optimize(getConfiguration("client").setClientMode(true)))
+            : srvs[0];
+
+        CacheConfiguration<Integer, String> cacheCfg = new CacheConfiguration<Integer, String>("testCache")
+            .setBackups(backupsCnt)
+            .setAffinity(new RendezvousAffinityFunction(32, null));
+
+        IgniteCache<Integer, String> cache = qryNode.getOrCreateCache(cacheCfg);
+
+        AbstractContinuousQuery<Integer, String> qry = getContinuousQuery();

Review comment:
       It seems the `@Parameterized` as a continuous query can also be used here. So we can reduce the number of tests added to `IgniteCacheQuerySelfTestSuite6`. Can you, please, take a look?

##########
File path: modules/core/src/test/java/org/apache/ignite/internal/processors/cache/query/continuous/ContinuousQueryBufferCleanupAbstractTest.java
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.query.continuous;
+
+import java.util.UUID;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteCache;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cache.query.AbstractContinuousQuery;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.processors.continuous.GridContinuousProcessor;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
+import org.junit.Test;
+
+import static org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.DFLT_ACK_SND_THRESHOLD;
+
+/**
+ * Test for continuous query buffer cleanup.
+ */
+public abstract class ContinuousQueryBufferCleanupAbstractTest extends GridCommonAbstractTest {
+    /** */
+    private static final int RECORDS_CNT = 10000;
+
+    /** */
+    private static final int ACK_THRESHOLD = 100;
+
+    /** */
+    private static final String REMOTE_ROUTINE_INFO_CLASS_NAME = "org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$RemoteRoutineInfo";
+
+    /** */
+    private static final String LOCAL_ROUTINE_INFO_CLASS_NAME = "org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$LocalRoutineInfo";
+
+    /** */
+    private static final String BATCH_CLASS_NAME = "org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryEventBuffer$Batch";
+
+    /** Continuous query updates counter */
+    protected final AtomicInteger updCntr = new AtomicInteger();
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        stopAllGrids();
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithSingleNode() throws Exception {
+        checkBuffer(1, 0, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodes() throws Exception {
+        checkBuffer(2, 0, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodesWithBackups() throws Exception {
+        checkBuffer(2, 1, true);
+    }
+
+    /**
+     * @throws Exception If fail.
+     */
+    @Test
+    public void checkCacheWithMultipleNodesWithBackupsWithoutClient() throws Exception {
+        checkBuffer(2, 1, false);
+    }
+
+    /** Provides continuous query for the test. */
+    protected abstract AbstractContinuousQuery<Integer, String> getContinuousQuery();
+
+    /** */
+    private void checkBuffer(int srvCnt, int backupsCnt, boolean useClient) throws Exception {
+        System.setProperty("IGNITE_CONTINUOUS_QUERY_ACK_THRESHOLD", Integer.toString(ACK_THRESHOLD));
+
+        IgniteEx[] srvs = new IgniteEx[srvCnt];

Review comment:
       It seems this array is not needed. Let's use `grid(0)` instead of `srv[0]`.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org